2024-03-20T07:42:14.115 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-20T07:42:14.115 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_yuriw@teuthology --name yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi --description rados/upgrade/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} --archive /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023 -- /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023/orig.config.yaml 2024-03-20T07:42:14.166 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-20T07:42:14.260 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023 branch: wip-yuri5-testing-2024-03-18-1144 description: rados/upgrade/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} email: yweinste@redhat.com first_in_suite: false job_id: '7610023' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'run run randomized correctness test for rados operations on an erasure-coded pool ' - desc: 'object class functional tests ' - desc: 'generate read/write load with rados objects ranging from 1MB to 25MB ' - desc: 'run basic import/export cli tests for rbd ' - desc: 'librbd C and C++ api tests ' - desc: 'librbd python api tests ' name: yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri5-testing-2024-03-18-1144 ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true create_rbd_pool: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_system_packages: - python3-pytest flavor: default sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-yuri5-testing-2024-03-18-1144 sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae owner: scheduled_yuriw@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mds.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b seed: 7763 sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri5-testing-2024-03-18-1144 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae targets: smithi118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbyBWiqm9K3vb0bEIwsorV4bqxBbWYER5MR15mpEnGkV035m+ro90i9c3nuZGwRXMZCpZJ0zrou/7PV2IohK0U= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIc2vb0XkU8YfXuf01b0sKg+BKbgcC0lr0xFiWuFUX/l/bXX7AbozlnxqWD2T1Urpsb4Jtwm3hjKnH9JrnfGt4= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - print: '**** done start installing reef cephadm ...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef - print: '**** done end installing reef cephadm ...' - print: '**** done start cephadm.shell ceph config set mgr...' - cephadm.shell: mon.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - print: '**** done start telemetry reef...' - workunit: clients: client.0: - test_telemetry_reef.sh - print: '**** done end telemetry reef...' - print: '**** done start parallel' - parallel: - workload - upgrade-sequence - print: '**** done end parallel' - print: '**** done start telemetry x...' - workunit: clients: client.0: - test_telemetry_reef_x.sh - print: '**** done end telemetry x...' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-19_00:09:45 tube: smithi upgrade-sequence: sequential: - print: '**** done start upgrade, wait' - cephadm.shell: env: - sha1 mon.a: - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - while ceph orch upgrade status | jq '.in_progress' | grep true ; do ceph orch ps ; ceph versions ; sleep 30 ; done - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 - print: '**** done end upgrade, wait...' user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload: full_sequential: - print: '**** done start ec-rados-default.yaml' - rados: clients: - client.0 ec_pool: true objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 4000 write_append_excl: false - print: '**** done end ec-rados-default.yaml' - print: '**** done start rados_api.yaml' - workunit: branch: reef clients: client.0: - cls - print: '**** done end rados_api.yaml' - print: '**** done start rados_loadgenbig.yaml' - workunit: branch: reef clients: client.0: - rados/load-gen-big.sh - print: '**** done end rados_loadgenbig.yaml' - print: '**** done start rbd_import_export.yaml' - workunit: branch: reef clients: client.1: - rbd/import_export.sh env: RBD_CREATE_ARGS: --new-format - print: '**** done end rbd_import_export.yaml' - print: '**** done start test_rbd_api.yaml' - workunit: branch: reef clients: client.0: - rbd/test_librbd.sh env: RBD_FEATURES: '61' - print: '**** done end test_rbd_api.yaml' - print: '**** done start test_rbd_python.yaml' - workunit: branch: reef clients: client.0: - rbd/test_librbd_python.sh env: RBD_FEATURES: '61' - print: '**** done end test_rbd_python.yaml' 2024-03-20T07:42:14.260 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa; will attempt to use it 2024-03-20T07:42:14.261 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks 2024-03-20T07:42:14.261 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-20T07:42:14.263 INFO:teuthology.task.internal:Checking packages... 2024-03-20T07:42:14.286 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e7e80a6ac9849f0f23432d0d9832f096730308ae' 2024-03-20T07:42:14.286 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-20T07:42:14.287 INFO:teuthology.packaging:ref: None 2024-03-20T07:42:14.287 INFO:teuthology.packaging:tag: None 2024-03-20T07:42:14.287 INFO:teuthology.packaging:branch: wip-yuri5-testing-2024-03-18-1144 2024-03-20T07:42:14.287 INFO:teuthology.packaging:sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T07:42:14.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri5-testing-2024-03-18-1144 2024-03-20T07:42:14.432 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2153.ge7e80a6a 2024-03-20T07:42:14.433 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-20T07:42:14.443 INFO:teuthology.task.internal:no buildpackages task found 2024-03-20T07:42:14.443 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-20T07:42:14.543 INFO:teuthology.task.internal:Saving configuration 2024-03-20T07:42:14.563 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-20T07:42:14.571 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-20T07:42:14.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-20 07:36:40.677933', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbyBWiqm9K3vb0bEIwsorV4bqxBbWYER5MR15mpEnGkV035m+ro90i9c3nuZGwRXMZCpZJ0zrou/7PV2IohK0U='} 2024-03-20T07:42:14.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-20 07:36:40.679405', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIc2vb0XkU8YfXuf01b0sKg+BKbgcC0lr0xFiWuFUX/l/bXX7AbozlnxqWD2T1Urpsb4Jtwm3hjKnH9JrnfGt4='} 2024-03-20T07:42:14.619 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-20T07:42:14.652 INFO:teuthology.task.internal:roles: ubuntu@smithi118.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2024-03-20T07:42:14.652 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mds.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-03-20T07:42:14.653 INFO:teuthology.run_tasks:Running task console_log... 2024-03-20T07:42:14.818 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4d1937d160>, signals=[15]) 2024-03-20T07:42:14.819 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-20T07:42:14.827 INFO:teuthology.task.internal:Opening connections... 2024-03-20T07:42:14.828 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi118.front.sepia.ceph.com 2024-03-20T07:42:14.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:42:14.917 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-03-20T07:42:14.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:42:15.004 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-20T07:42:15.013 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-03-20T07:42:15.030 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-03-20T07:42:15.030 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-20T07:42:15.087 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-03-20T07:42:15.088 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-20T07:42:15.088 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-20T07:42:15.088 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-20T07:42:15.088 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-03-20T07:42:15.115 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-03-20T07:42:15.132 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-03-20T07:42:15.133 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-20T07:42:15.190 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-03-20T07:42:15.191 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-20T07:42:15.191 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-20T07:42:15.191 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-20T07:42:15.191 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-03-20T07:42:15.212 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-20T07:42:15.228 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-20T07:42:15.239 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-20T07:42:15.240 DEBUG:teuthology.orchestra.run.smithi118:> test '!' -e /home/ubuntu/cephtest 2024-03-20T07:42:15.242 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-03-20T07:42:15.257 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-20T07:42:15.268 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-20T07:42:15.268 DEBUG:teuthology.orchestra.run.smithi118:> test -z $(ls -A /var/lib/ceph) 2024-03-20T07:42:15.298 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-03-20T07:42:15.438 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-20T07:42:15.498 INFO:teuthology.run_tasks:Running task kernel... 2024-03-20T07:42:15.513 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-20T07:42:15.513 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-20T07:42:15.513 DEBUG:teuthology.orchestra.run.smithi118:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-20T07:42:15.513 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-20T07:42:15.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T07:42:15.529 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-03-20T07:42:15.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T07:42:15.530 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-03-20T07:42:15.587 INFO:teuthology.orchestra.run.smithi118.stdout:5.14.0-361.el9.x86_64 2024-03-20T07:42:15.587 INFO:teuthology.task.kernel:Running kernel on smithi118: 5.14.0-361.el9.x86_64 2024-03-20T07:42:15.587 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum install -y kernel 2024-03-20T07:42:15.588 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-361.el9.x86_64 2024-03-20T07:42:15.588 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.14.0-361.el9.x86_64 2024-03-20T07:42:15.588 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-03-20T07:42:17.783 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-20T07:42:19.320 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS-9 - AppStream 6.6 MB/s | 6.7 MB 00:01 2024-03-20T07:42:19.441 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-03-20T07:42:20.995 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-20T07:42:21.861 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-20T07:42:24.703 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-20T07:42:29.012 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.1 MB 00:00 2024-03-20T07:42:30.746 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-03-20T07:42:33.308 INFO:teuthology.orchestra.run.smithi118.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-03-20T07:42:38.189 INFO:teuthology.orchestra.run.smithi118.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-20T07:42:40.362 INFO:teuthology.orchestra.run.smithi118.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-20T07:42:40.362 INFO:teuthology.orchestra.run.smithi118.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-20T07:42:40.459 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-03-20T07:42:40.470 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-03-20T07:42:40.470 INFO:teuthology.orchestra.run.smithi118.stdout: Package Architecture Version Repository Size 2024-03-20T07:42:40.470 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-03-20T07:42:40.470 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-03-20T07:42:40.470 INFO:teuthology.orchestra.run.smithi118.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout:Install 4 Packages 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 76 M 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 123 M 2024-03-20T07:42:40.471 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-03-20T07:42:40.698 INFO:teuthology.orchestra.run.smithi118.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 745 kB/s | 44 kB 00:00 2024-03-20T07:42:41.607 INFO:teuthology.orchestra.run.smithi118.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 15 MB/s | 15 MB 00:00 2024-03-20T07:42:42.307 INFO:teuthology.orchestra.run.smithi118.stdout:(3/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 20 MB/s | 33 MB 00:01 2024-03-20T07:42:42.428 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 363 kB/s | 8.0 MB 00:22 2024-03-20T07:42:42.549 INFO:teuthology.orchestra.run.smithi118.stdout:(4/4): kernel-modules-core-5.14.0-430.el9.x86_6 15 MB/s | 27 MB 00:01 2024-03-20T07:42:42.549 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-03-20T07:42:42.549 INFO:teuthology.orchestra.run.smithi118.stdout:Total 36 MB/s | 76 MB 00:02 2024-03-20T07:42:43.220 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-03-20T07:42:43.563 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-03-20T07:42:43.563 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-03-20T07:42:44.337 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-20T07:42:44.595 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-03-20T07:42:44.595 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-03-20T07:42:47.071 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-03-20T07:42:47.661 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-20T07:42:47.757 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-20T07:42:48.480 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.1 MB 00:00 2024-03-20T07:42:49.997 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-03-20T07:42:50.466 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-20T07:42:50.601 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-20T07:42:51.226 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-20T07:42:55.334 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-20T07:42:55.368 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:42:56.149 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-20T07:42:58.230 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-20T07:42:58.230 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-20T07:42:58.324 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-20T07:42:58.335 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-03-20T07:42:58.336 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:42:58.336 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 76 M 2024-03-20T07:42:58.336 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 123 M 2024-03-20T07:42:58.336 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-20T07:42:58.691 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 190 kB/s | 44 kB 00:00 2024-03-20T07:43:00.099 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:01.000 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 5.9 MB/s | 15 MB 00:02 2024-03-20T07:43:01.658 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 9.3 MB/s | 27 MB 00:02 2024-03-20T07:43:01.959 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 9.6 MB/s | 33 MB 00:03 2024-03-20T07:43:01.959 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-20T07:43:01.959 INFO:teuthology.orchestra.run.smithi196.stdout:Total 21 MB/s | 76 MB 00:03 2024-03-20T07:43:02.624 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-20T07:43:03.006 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-20T07:43:03.006 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-20T07:43:04.014 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-20T07:43:04.014 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-20T07:43:06.490 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-20T07:43:07.135 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-20T07:43:07.224 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-20T07:43:09.957 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-20T07:43:10.094 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-20T07:43:14.785 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-20T07:43:14.893 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:19.597 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:30.119 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:30.119 INFO:teuthology.orchestra.run.smithi118.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-20T07:43:30.119 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:43:30.163 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:30.508 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:30.508 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-20T07:43:30.508 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-20T07:43:30.508 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:43:30.917 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-03-20T07:43:31.261 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q kernel | sort -rV | head -n 1 2024-03-20T07:43:31.371 INFO:teuthology.orchestra.run.smithi118.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-20T07:43:31.371 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-20T07:43:31.372 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-20T07:43:31.372 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-20T07:43:31.372 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-20T07:43:31.372 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-20T07:43:31.372 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi118.front.sepia.ceph.com, path=None, version=distro) 2024-03-20T07:43:31.372 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q kernel | sort -rV | head -n 1 2024-03-20T07:43:31.452 INFO:teuthology.orchestra.run.smithi118.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-20T07:43:31.452 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-20T07:43:31.452 DEBUG:teuthology.orchestra.run.smithi118:> sudo rpm -qi grub2-tools 2024-03-20T07:43:31.493 INFO:teuthology.orchestra.run.smithi118.stdout:Name : grub2-tools 2024-03-20T07:43:31.493 INFO:teuthology.orchestra.run.smithi118.stdout:Epoch : 1 2024-03-20T07:43:31.493 INFO:teuthology.orchestra.run.smithi118.stdout:Version : 2.06 2024-03-20T07:43:31.493 INFO:teuthology.orchestra.run.smithi118.stdout:Release : 68.el9 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Architecture: x86_64 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Group : Unspecified 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Size : 8264254 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:License : GPLv3+ 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Packager : builder@centos.org 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Vendor : CentOS 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Summary : Support tools for GRUB. 2024-03-20T07:43:31.494 INFO:teuthology.orchestra.run.smithi118.stdout:Description : 2024-03-20T07:43:31.495 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:43:31.495 INFO:teuthology.orchestra.run.smithi118.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-20T07:43:31.495 INFO:teuthology.orchestra.run.smithi118.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-20T07:43:31.495 INFO:teuthology.orchestra.run.smithi118.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-20T07:43:31.495 INFO:teuthology.orchestra.run.smithi118.stdout:hardware devices. 2024-03-20T07:43:31.495 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:43:31.495 INFO:teuthology.orchestra.run.smithi118.stdout:This subpackage provides tools for support of all platforms. 2024-03-20T07:43:31.496 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-20T07:43:31.496 INFO:teuthology.task.kernel:Updating grub on smithi118 to boot 5.14.0-430.el9.x86_64 2024-03-20T07:43:31.496 DEBUG:teuthology.orchestra.run.smithi118:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-20T07:43:32.124 INFO:teuthology.orchestra.run.smithi118.stderr:Generating grub configuration file ... 2024-03-20T07:43:33.534 INFO:teuthology.orchestra.run.smithi118.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-20T07:43:33.575 INFO:teuthology.orchestra.run.smithi118.stderr:done 2024-03-20T07:43:33.577 DEBUG:teuthology.orchestra.run.smithi118:> mktemp 2024-03-20T07:43:33.598 INFO:teuthology.orchestra.run.smithi118.stdout:/tmp/tmp.yorzbyC1Uz 2024-03-20T07:43:33.599 DEBUG:teuthology.orchestra.run.smithi118:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yorzbyC1Uz 2024-03-20T07:43:33.668 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0666 /tmp/tmp.yorzbyC1Uz 2024-03-20T07:43:33.807 DEBUG:teuthology.orchestra.remote:smithi118:/tmp/tmp.yorzbyC1Uz is 6KB 2024-03-20T07:43:33.826 DEBUG:teuthology.orchestra.run.smithi118:> rm -fr /tmp/tmp.yorzbyC1Uz 2024-03-20T07:43:33.840 DEBUG:teuthology.orchestra.run.smithi118:> sudo /bin/ls /boot/loader/entries || true 2024-03-20T07:43:33.904 INFO:teuthology.orchestra.run.smithi118.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-20T07:43:33.904 INFO:teuthology.orchestra.run.smithi118.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-20T07:43:33.905 INFO:teuthology.orchestra.run.smithi118.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-20T07:43:33.905 INFO:teuthology.orchestra.run.smithi118.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-20T07:43:33.906 DEBUG:teuthology.orchestra.run.smithi118:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-20T07:43:34.056 DEBUG:teuthology.orchestra.run.smithi118:> sudo shutdown -r now 2024-03-20T07:43:48.845 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:48.845 INFO:teuthology.orchestra.run.smithi196.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-20T07:43:48.846 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:43:48.889 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:49.302 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:49.302 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-20T07:43:49.302 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-20T07:43:49.302 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-20T07:43:49.714 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-20T07:43:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:43:49.715 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-20T07:43:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-20T07:43:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-20T07:43:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-20T07:43:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-20T07:43:49.715 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:43:49.715 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-20T07:43:50.114 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-03-20T07:43:50.204 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-20T07:43:50.205 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-20T07:43:50.205 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-20T07:43:50.205 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-20T07:43:50.205 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-20T07:43:50.205 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-20T07:43:50.205 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-03-20T07:43:50.205 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-03-20T07:43:50.286 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-20T07:43:50.286 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-20T07:43:50.286 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 68.el9 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8264254 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-20T07:43:50.324 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-20T07:43:50.325 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-03-20T07:43:50.326 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:43:50.326 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-03-20T07:43:50.327 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-20T07:43:50.327 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 5.14.0-430.el9.x86_64 2024-03-20T07:43:50.327 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-20T07:43:50.958 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-03-20T07:43:52.331 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-20T07:43:52.366 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-03-20T07:43:52.369 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-03-20T07:43:52.384 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.aaTqS8PXp3 2024-03-20T07:43:52.385 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aaTqS8PXp3 2024-03-20T07:43:52.453 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.aaTqS8PXp3 2024-03-20T07:43:52.591 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.aaTqS8PXp3 is 6KB 2024-03-20T07:43:52.640 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.aaTqS8PXp3 2024-03-20T07:43:52.654 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-03-20T07:43:52.721 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-20T07:43:52.721 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-20T07:43:52.721 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-20T07:43:52.721 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-20T07:43:52.722 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-20T07:43:52.880 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-03-20T07:44:04.070 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-20T07:44:04.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-20T07:44:04.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:44:22.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-20T07:44:22.883 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-20T07:44:22.884 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-20T07:44:22.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:44:31.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-20T07:44:31.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:44:34.626 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-20T07:44:41.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-20T07:44:46.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-20T07:44:46.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:44:49.695 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-20T07:44:50.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-20T07:44:50.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:44:53.314 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-20T07:45:04.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-20T07:45:04.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:45:05.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-20T07:45:05.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:45:23.743 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-20T07:45:36.735 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-03-20T07:45:37.149 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-20T07:45:37.149 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-430.el9.x86_64"... 2024-03-20T07:45:37.149 DEBUG:teuthology.orchestra.run.smithi118:> uname -r 2024-03-20T07:45:37.164 INFO:teuthology.orchestra.run.smithi118.stdout:5.14.0-430.el9.x86_64 2024-03-20T07:45:37.165 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-20T07:45:37.165 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-20T07:45:37.165 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-20T07:45:38.165 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-20T07:45:38.166 DEBUG:teuthology.orchestra.run.smithi118:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-20T07:45:38.240 INFO:teuthology.orchestra.run.smithi118.stdout:ttyS1 2024-03-20T07:45:38.271 DEBUG:teuthology.parallel:result is None 2024-03-20T07:45:38.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-20T07:45:38.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:45:39.123 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-03-20T07:45:39.578 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-20T07:45:39.578 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-430.el9.x86_64"... 2024-03-20T07:45:39.578 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-03-20T07:45:39.592 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-430.el9.x86_64 2024-03-20T07:45:39.592 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-20T07:45:39.592 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-20T07:45:39.592 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-20T07:45:40.593 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-20T07:45:40.593 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-20T07:45:40.684 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-03-20T07:45:40.736 DEBUG:teuthology.parallel:result is None 2024-03-20T07:45:40.737 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-20T07:45:40.745 INFO:teuthology.task.internal:Creating test directory... 2024-03-20T07:45:40.745 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-20T07:45:40.748 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-20T07:45:40.772 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-20T07:45:40.778 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-20T07:45:40.788 INFO:teuthology.task.internal:Creating archive directory... 2024-03-20T07:45:40.789 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-20T07:45:40.806 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-20T07:45:40.854 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-20T07:45:40.861 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-20T07:45:40.862 DEBUG:teuthology.orchestra.run.smithi118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-20T07:45:40.887 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-20T07:45:40.920 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-20T07:45:40.935 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-20T07:45:41.118 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-20T07:45:41.138 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-20T07:45:41.140 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-20T07:45:41.147 INFO:teuthology.task.internal:Configuring sudo... 2024-03-20T07:45:41.148 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-20T07:45:41.150 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-20T07:45:41.196 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-20T07:45:41.206 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-20T07:45:41.206 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-20T07:45:41.222 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-20T07:45:41.244 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-20T07:45:41.321 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-20T07:45:41.368 DEBUG:teuthology.orchestra.run.smithi118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-20T07:45:41.451 DEBUG:teuthology.orchestra.run.smithi118:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-20T07:45:41.494 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:45:41.495 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-20T07:45:41.563 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-20T07:45:41.608 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-20T07:45:41.656 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-20T07:45:41.735 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-20T07:45:41.782 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:45:41.782 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-20T07:45:41.848 DEBUG:teuthology.orchestra.run.smithi118:> sudo service rsyslog restart 2024-03-20T07:45:41.851 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-03-20T07:45:41.898 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-20T07:45:42.130 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-20T07:45:42.644 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-20T07:45:42.652 INFO:teuthology.task.internal:Starting timer... 2024-03-20T07:45:42.652 INFO:teuthology.run_tasks:Running task pcp... 2024-03-20T07:45:42.662 INFO:teuthology.run_tasks:Running task selinux... 2024-03-20T07:45:42.670 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-20T07:45:42.670 DEBUG:teuthology.orchestra.run.smithi118:> sudo service auditd rotate 2024-03-20T07:45:42.727 INFO:teuthology.orchestra.run.smithi118.stdout:Rotating logs: 2024-03-20T07:45:42.729 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-03-20T07:45:42.789 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-03-20T07:45:42.791 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-20T07:45:42.791 DEBUG:teuthology.orchestra.run.smithi118:> /usr/sbin/getenforce 2024-03-20T07:45:42.816 INFO:teuthology.orchestra.run.smithi118.stdout:Permissive 2024-03-20T07:45:42.816 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-03-20T07:45:42.840 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-03-20T07:45:42.840 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi118.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-03-20T07:45:42.841 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-20T07:45:42.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T07:45:42.887 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-20T07:45:42.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T07:45:42.915 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-20T07:45:42.915 DEBUG:teuthology.orchestra.run.smithi118:> sudo /usr/sbin/setenforce permissive 2024-03-20T07:45:42.968 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-03-20T07:45:43.001 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-20T07:45:43.012 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-20T07:45:43.028 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-20T07:45:43.029 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi118.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-20T07:49:58.172 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi118.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-03-20T07:49:58.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-20T07:49:58.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:49:58.262 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-03-20T07:49:58.340 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-20T07:49:58.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-20T07:49:58.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-20T07:49:58.428 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-03-20T07:49:58.501 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-20T07:49:58.501 INFO:teuthology.run_tasks:Running task clock... 2024-03-20T07:49:58.520 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-20T07:49:58.521 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-20T07:49:58.521 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-20T07:49:58.524 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-20T07:49:58.524 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-20T07:49:58.552 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-20T07:49:58.568 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-20T07:49:58.572 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-20T07:49:58.589 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-20T07:49:58.598 INFO:teuthology.orchestra.run.smithi118.stderr:sudo: ntpd: command not found 2024-03-20T07:49:58.611 INFO:teuthology.orchestra.run.smithi118.stdout:506 Cannot talk to daemon 2024-03-20T07:49:58.618 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-03-20T07:49:58.628 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-20T07:49:58.631 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-03-20T07:49:58.644 INFO:teuthology.orchestra.run.smithi118.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-20T07:49:58.648 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-20T07:49:58.666 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-20T07:49:58.701 INFO:teuthology.orchestra.run.smithi118.stderr:bash: line 1: ntpq: command not found 2024-03-20T07:49:58.704 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-20T07:49:58.705 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2024-03-20T07:49:58.705 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T07:49:58.705 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T07:49:58.705 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T07:49:58.705 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T07:49:58.715 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-03-20T07:49:58.719 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-20T07:49:58.719 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-03-20T07:49:58.719 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T07:49:58.719 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T07:49:58.719 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T07:49:58.719 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T07:49:58.720 INFO:teuthology.run_tasks:Running task pexec... 2024-03-20T07:49:58.738 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-20T07:49:58.739 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi118.front.sepia.ceph.com 2024-03-20T07:49:58.739 DEBUG:teuthology.orchestra.run.smithi118:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-20T07:49:58.739 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi196.front.sepia.ceph.com 2024-03-20T07:49:58.739 DEBUG:teuthology.orchestra.run.smithi196:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-20T07:49:59.325 INFO:teuthology.orchestra.run.smithi118.stdout:Last metadata expiration check: 0:00:50 ago on Wed 20 Mar 2024 07:49:09 AM UTC. 2024-03-20T07:49:59.343 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:50 ago on Wed 20 Mar 2024 07:49:09 AM UTC. 2024-03-20T07:49:59.470 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-03-20T07:49:59.472 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-03-20T07:49:59.472 INFO:teuthology.orchestra.run.smithi118.stdout: Package Architecture Version Repository Size 2024-03-20T07:49:59.472 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-03-20T07:49:59.472 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-03-20T07:49:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-03-20T07:49:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:49:59.473 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-03-20T07:49:59.473 INFO:teuthology.orchestra.run.smithi118.stdout:================================================================================ 2024-03-20T07:49:59.473 INFO:teuthology.orchestra.run.smithi118.stdout:Install 1 Package 2024-03-20T07:49:59.473 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:49:59.483 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 3.1 M 2024-03-20T07:49:59.483 INFO:teuthology.orchestra.run.smithi118.stdout:Installed size: 9.8 M 2024-03-20T07:49:59.484 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-03-20T07:49:59.488 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-20T07:49:59.497 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-03-20T07:49:59.498 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:49:59.508 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 3.1 M 2024-03-20T07:49:59.508 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 9.8 M 2024-03-20T07:49:59.508 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-20T07:50:04.820 INFO:teuthology.orchestra.run.smithi118.stdout:runc-1.1.12-2.el9.x86_64.rpm 605 kB/s | 3.1 MB 00:05 2024-03-20T07:50:04.820 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-03-20T07:50:04.820 INFO:teuthology.orchestra.run.smithi118.stdout:Total 600 kB/s | 3.1 MB 00:05 2024-03-20T07:50:04.822 INFO:teuthology.orchestra.run.smithi196.stdout:runc-1.1.12-2.el9.x86_64.rpm 611 kB/s | 3.1 MB 00:05 2024-03-20T07:50:04.822 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-20T07:50:04.822 INFO:teuthology.orchestra.run.smithi196.stdout:Total 603 kB/s | 3.1 MB 00:05 2024-03-20T07:50:04.858 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-03-20T07:50:04.861 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-20T07:50:04.864 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-03-20T07:50:04.865 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-03-20T07:50:04.867 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-20T07:50:04.868 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-20T07:50:04.947 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-03-20T07:50:04.947 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-03-20T07:50:04.949 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-20T07:50:04.949 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-20T07:50:05.168 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-20T07:50:05.176 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-03-20T07:50:05.258 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-20T07:50:05.282 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-20T07:50:05.702 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-20T07:50:05.797 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-20T07:50:06.012 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-20T07:50:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:50:06.013 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-20T07:50:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: runc-4:1.1.12-2.el9.x86_64 2024-03-20T07:50:06.013 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:50:06.013 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-20T07:50:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-20T07:50:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:50:06.103 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-03-20T07:50:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: runc-4:1.1.12-2.el9.x86_64 2024-03-20T07:50:06.103 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:50:06.104 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-03-20T07:50:06.290 DEBUG:teuthology.parallel:result is None 2024-03-20T07:50:06.342 DEBUG:teuthology.parallel:result is None 2024-03-20T07:50:06.342 INFO:teuthology.run_tasks:Running task install... 2024-03-20T07:50:06.351 DEBUG:teuthology.task.install:project ceph 2024-03-20T07:50:06.351 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'default', 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae'}} 2024-03-20T07:50:06.351 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'extra_system_packages': ['python3-pytest'], 'flavor': 'default', 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae'} 2024-03-20T07:50:06.351 INFO:teuthology.task.install:Using flavor: default 2024-03-20T07:50:06.360 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-20T07:50:06.360 INFO:teuthology.task.install:extra packages: [] 2024-03-20T07:50:06.360 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae', 'tag': None, 'wait_for_package': False} 2024-03-20T07:50:06.360 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-20T07:50:06.361 INFO:teuthology.packaging:ref: None 2024-03-20T07:50:06.361 INFO:teuthology.packaging:tag: None 2024-03-20T07:50:06.361 INFO:teuthology.packaging:branch: reef 2024-03-20T07:50:06.361 INFO:teuthology.packaging:sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T07:50:06.362 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-20T07:50:06.364 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae', 'tag': None, 'wait_for_package': False} 2024-03-20T07:50:06.364 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-20T07:50:06.364 INFO:teuthology.packaging:ref: None 2024-03-20T07:50:06.364 INFO:teuthology.packaging:tag: None 2024-03-20T07:50:06.364 INFO:teuthology.packaging:branch: reef 2024-03-20T07:50:06.364 INFO:teuthology.packaging:sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T07:50:06.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-20T07:50:06.592 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/flavors/default/ 2024-03-20T07:50:06.593 INFO:teuthology.task.install.rpm:Package version is 18.2.2-904.g78dbc53a 2024-03-20T07:50:06.603 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/flavors/default/ 2024-03-20T07:50:06.603 INFO:teuthology.task.install.rpm:Package version is 18.2.2-904.g78dbc53a 2024-03-20T07:50:06.711 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-20T07:50:06.711 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:50:06.711 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-20T07:50:06.739 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest, python3-pytest, python3-pytest on remote rpm x86_64 2024-03-20T07:50:06.739 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-20T07:50:06.739 INFO:teuthology.packaging:ref: None 2024-03-20T07:50:06.740 INFO:teuthology.packaging:tag: None 2024-03-20T07:50:06.740 INFO:teuthology.packaging:branch: reef 2024-03-20T07:50:06.740 INFO:teuthology.packaging:sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T07:50:06.740 DEBUG:teuthology.orchestra.run.smithi118:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-20T07:50:06.762 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-20T07:50:06.763 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:50:06.763 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-20T07:50:06.791 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest, python3-pytest, python3-pytest, python3-pytest on remote rpm x86_64 2024-03-20T07:50:06.791 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-20T07:50:06.791 INFO:teuthology.packaging:ref: None 2024-03-20T07:50:06.791 INFO:teuthology.packaging:tag: None 2024-03-20T07:50:06.791 INFO:teuthology.packaging:branch: reef 2024-03-20T07:50:06.791 INFO:teuthology.packaging:sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T07:50:06.791 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-20T07:50:06.808 DEBUG:teuthology.orchestra.run.smithi118:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-20T07:50:06.859 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-20T07:50:06.888 DEBUG:teuthology.orchestra.run.smithi118:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-20T07:50:06.939 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-20T07:50:06.954 INFO:teuthology.orchestra.run.smithi118.stdout:check_obsoletes = 1 2024-03-20T07:50:06.955 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum clean all 2024-03-20T07:50:07.005 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-03-20T07:50:07.007 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-03-20T07:50:07.188 INFO:teuthology.orchestra.run.smithi118.stdout:74 files removed 2024-03-20T07:50:07.215 DEBUG:teuthology.orchestra.run.smithi118:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest 2024-03-20T07:50:07.243 INFO:teuthology.orchestra.run.smithi196.stdout:74 files removed 2024-03-20T07:50:07.270 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest 2024-03-20T07:50:07.868 INFO:teuthology.orchestra.run.smithi118.stdout:ceph packages for x86_64 219 kB/s | 77 kB 00:00 2024-03-20T07:50:07.936 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 212 kB/s | 77 kB 00:00 2024-03-20T07:50:08.260 INFO:teuthology.orchestra.run.smithi118.stdout:ceph noarch packages 29 kB/s | 9.9 kB 00:00 2024-03-20T07:50:08.361 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 26 kB/s | 9.9 kB 00:00 2024-03-20T07:50:08.645 INFO:teuthology.orchestra.run.smithi118.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-03-20T07:50:08.795 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-03-20T07:50:09.039 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS-9 - AppStream 19 MB/s | 6.7 MB 00:00 2024-03-20T07:50:09.431 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-20T07:50:10.414 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-03-20T07:50:10.823 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-20T07:50:15.588 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-03-20T07:50:15.696 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.0 MB 00:04 2024-03-20T07:50:17.966 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-03-20T07:50:18.049 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-03-20T07:50:22.965 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.1 MB 00:01 2024-03-20T07:50:24.624 INFO:teuthology.orchestra.run.smithi118.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-03-20T07:50:25.828 INFO:teuthology.orchestra.run.smithi118.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-20T07:50:30.272 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 721 kB/s | 6.1 MB 00:08 2024-03-20T07:50:30.792 INFO:teuthology.orchestra.run.smithi118.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-20T07:50:32.805 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-20T07:50:33.203 INFO:teuthology.orchestra.run.smithi118.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-20T07:50:33.204 INFO:teuthology.orchestra.run.smithi118.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-20T07:50:33.279 INFO:teuthology.orchestra.run.smithi118.stdout:Dependencies resolved. 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout:=========================================================================================== 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout: Package Arch Version Repository Size 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout:=========================================================================================== 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout:Installing: 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout: ceph x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 6.5 k 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 5.2 M 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 843 k 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 142 k 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 1.4 M 2024-03-20T07:50:33.287 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 138 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 1.7 M 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 7.4 M 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 49 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 7.7 M 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 40 M 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 224 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 31 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 702 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 127 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 162 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytest noarch 6.2.2-6.el9 CentOS-AppStream 474 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 320 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 298 k 2024-03-20T07:50:33.288 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 99 k 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 87 k 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 3.0 M 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 172 k 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrading: 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: librados2 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 3.3 M 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 3.0 M 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout:Installing dependencies: 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 18 M 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 23 k 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 2.1 M 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 245 k 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 4.4 M 2024-03-20T07:50:33.289 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 18 M 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 15 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 25 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 166 k 2024-03-20T07:50:33.290 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 474 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 4.5 M 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-20T07:50:33.291 INFO:teuthology.orchestra.run.smithi118.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 45 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 127 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-iniconfig noarch 1.1.1-7.el9 CentOS-AppStream 16 k 2024-03-20T07:50:33.292 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-20T07:50:33.293 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pluggy noarch 0.13.1-7.el9 CentOS-AppStream 38 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-py noarch 1.10.0-6.el9 CentOS-AppStream 427 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-20T07:50:33.294 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-20T07:50:33.295 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction Summary 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout:=========================================================================================== 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout:Install 118 Packages 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout:Upgrade 2 Packages 2024-03-20T07:50:33.296 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:50:33.297 INFO:teuthology.orchestra.run.smithi118.stdout:Total download size: 177 M 2024-03-20T07:50:33.297 INFO:teuthology.orchestra.run.smithi118.stdout:Downloading Packages: 2024-03-20T07:50:34.268 INFO:teuthology.orchestra.run.smithi118.stdout:(1/120): ceph-18.2.2-904.g78dbc53a.el9.x86_64.r 35 kB/s | 6.5 kB 00:00 2024-03-20T07:50:34.618 INFO:teuthology.orchestra.run.smithi118.stdout:(2/120): ceph-fuse-18.2.2-904.g78dbc53a.el9.x86 2.4 MB/s | 843 kB 00:00 2024-03-20T07:50:34.719 INFO:teuthology.orchestra.run.smithi118.stdout:(3/120): ceph-immutable-object-cache-18.2.2-904 1.4 MB/s | 142 kB 00:00 2024-03-20T07:50:34.994 INFO:teuthology.orchestra.run.smithi118.stdout:(4/120): ceph-mds-18.2.2-904.g78dbc53a.el9.x86_ 7.7 MB/s | 2.1 MB 00:00 2024-03-20T07:50:35.153 INFO:teuthology.orchestra.run.smithi118.stdout:(5/120): ceph-base-18.2.2-904.g78dbc53a.el9.x86 4.9 MB/s | 5.2 MB 00:01 2024-03-20T07:50:35.261 INFO:teuthology.orchestra.run.smithi118.stdout:(6/120): ceph-mgr-18.2.2-904.g78dbc53a.el9.x86_ 5.4 MB/s | 1.4 MB 00:00 2024-03-20T07:50:35.912 INFO:teuthology.orchestra.run.smithi118.stdout:(7/120): ceph-common-18.2.2-904.g78dbc53a.el9.x 10 MB/s | 18 MB 00:01 2024-03-20T07:50:35.978 INFO:teuthology.orchestra.run.smithi118.stdout:(8/120): ceph-mon-18.2.2-904.g78dbc53a.el9.x86_ 5.3 MB/s | 4.4 MB 00:00 2024-03-20T07:50:36.045 INFO:teuthology.orchestra.run.smithi118.stdout:(9/120): ceph-selinux-18.2.2-904.g78dbc53a.el9. 378 kB/s | 25 kB 00:00 2024-03-20T07:50:37.779 INFO:teuthology.orchestra.run.smithi118.stdout:(10/120): ceph-radosgw-18.2.2-904.g78dbc53a.el9 4.1 MB/s | 7.7 MB 00:01 2024-03-20T07:50:37.921 INFO:teuthology.orchestra.run.smithi118.stdout:(11/120): libcephfs-devel-18.2.2-904.g78dbc53a. 219 kB/s | 31 kB 00:00 2024-03-20T07:50:38.888 INFO:teuthology.orchestra.run.smithi118.stdout:(12/120): ceph-osd-18.2.2-904.g78dbc53a.el9.x86 4.9 MB/s | 18 MB 00:03 2024-03-20T07:50:38.930 INFO:teuthology.orchestra.run.smithi118.stdout:(13/120): libcephfs2-18.2.2-904.g78dbc53a.el9.x 697 kB/s | 702 kB 00:01 2024-03-20T07:50:38.972 INFO:teuthology.orchestra.run.smithi118.stdout:(14/120): libcephsqlite-18.2.2-904.g78dbc53a.el 1.9 MB/s | 166 kB 00:00 2024-03-20T07:50:39.063 INFO:teuthology.orchestra.run.smithi118.stdout:(15/120): librados-devel-18.2.2-904.g78dbc53a.e 948 kB/s | 127 kB 00:00 2024-03-20T07:50:39.147 INFO:teuthology.orchestra.run.smithi118.stdout:(16/120): libradosstriper1-18.2.2-904.g78dbc53a 2.6 MB/s | 474 kB 00:00 2024-03-20T07:50:39.472 INFO:teuthology.orchestra.run.smithi118.stdout:(17/120): python3-ceph-argparse-18.2.2-904.g78d 138 kB/s | 45 kB 00:00 2024-03-20T07:50:40.398 INFO:teuthology.orchestra.run.smithi118.stdout:(18/120): ceph-test-18.2.2-904.g78dbc53a.el9.x8 9.1 MB/s | 40 MB 00:04 2024-03-20T07:50:40.415 INFO:teuthology.orchestra.run.smithi118.stdout:(19/120): python3-ceph-common-18.2.2-904.g78dbc 135 kB/s | 127 kB 00:00 2024-03-20T07:50:40.482 INFO:teuthology.orchestra.run.smithi118.stdout:(20/120): librgw2-18.2.2-904.g78dbc53a.el9.x86_ 3.1 MB/s | 4.5 MB 00:01 2024-03-20T07:50:40.499 INFO:teuthology.orchestra.run.smithi118.stdout:(21/120): python3-cephfs-18.2.2-904.g78dbc53a.e 1.6 MB/s | 162 kB 00:00 2024-03-20T07:50:40.549 INFO:teuthology.orchestra.run.smithi118.stdout:(22/120): python3-rados-18.2.2-904.g78dbc53a.el 2.3 MB/s | 320 kB 00:00 2024-03-20T07:50:40.574 INFO:teuthology.orchestra.run.smithi118.stdout:(23/120): python3-rbd-18.2.2-904.g78dbc53a.el9. 3.2 MB/s | 298 kB 00:00 2024-03-20T07:50:40.591 INFO:teuthology.orchestra.run.smithi118.stdout:(24/120): python3-rgw-18.2.2-904.g78dbc53a.el9. 1.1 MB/s | 99 kB 00:00 2024-03-20T07:50:40.624 INFO:teuthology.orchestra.run.smithi118.stdout:(25/120): rbd-fuse-18.2.2-904.g78dbc53a.el9.x86 1.1 MB/s | 87 kB 00:00 2024-03-20T07:50:40.708 INFO:teuthology.orchestra.run.smithi118.stdout:(26/120): ceph-grafana-dashboards-18.2.2-904.g7 280 kB/s | 23 kB 00:00 2024-03-20T07:50:40.733 INFO:teuthology.orchestra.run.smithi118.stdout:(27/120): rbd-nbd-18.2.2-904.g78dbc53a.el9.x86_ 1.2 MB/s | 172 kB 00:00 2024-03-20T07:50:40.792 INFO:teuthology.orchestra.run.smithi118.stdout:(28/120): ceph-mgr-cephadm-18.2.2-904.g78dbc53a 1.6 MB/s | 138 kB 00:00 2024-03-20T07:50:40.859 INFO:teuthology.orchestra.run.smithi118.stdout:(29/120): rbd-mirror-18.2.2-904.g78dbc53a.el9.x 10 MB/s | 3.0 MB 00:00 2024-03-20T07:50:40.934 INFO:teuthology.orchestra.run.smithi118.stdout:(30/120): ceph-mgr-dashboard-18.2.2-904.g78dbc5 8.7 MB/s | 1.7 MB 00:00 2024-03-20T07:50:40.960 INFO:teuthology.orchestra.run.smithi118.stdout:(31/120): ceph-mgr-modules-core-18.2.2-904.g78d 2.4 MB/s | 245 kB 00:00 2024-03-20T07:50:41.002 INFO:teuthology.orchestra.run.smithi118.stdout:(32/120): ceph-mgr-rook-18.2.2-904.g78dbc53a.el 737 kB/s | 49 kB 00:00 2024-03-20T07:50:41.018 INFO:teuthology.orchestra.run.smithi118.stdout:(33/120): ceph-prometheus-alerts-18.2.2-904.g78 253 kB/s | 15 kB 00:00 2024-03-20T07:50:41.102 INFO:teuthology.orchestra.run.smithi118.stdout:(34/120): cephadm-18.2.2-904.g78dbc53a.el9.noar 2.2 MB/s | 224 kB 00:00 2024-03-20T07:50:41.128 INFO:teuthology.orchestra.run.smithi118.stdout:(35/120): boost-program-options-1.75.0-8.el9.x8 975 kB/s | 106 kB 00:00 2024-03-20T07:50:41.145 INFO:teuthology.orchestra.run.smithi118.stdout:(36/120): flexiblas-3.0.4-8.el9.x86_64.rpm 754 kB/s | 32 kB 00:00 2024-03-20T07:50:41.195 INFO:teuthology.orchestra.run.smithi118.stdout:(37/120): flexiblas-openblas-openmp-3.0.4-8.el9 344 kB/s | 17 kB 00:00 2024-03-20T07:50:41.270 INFO:teuthology.orchestra.run.smithi118.stdout:(38/120): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-03-20T07:50:41.296 INFO:teuthology.orchestra.run.smithi118.stdout:(39/120): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 159 kB 00:00 2024-03-20T07:50:41.313 INFO:teuthology.orchestra.run.smithi118.stdout:(40/120): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-20T07:50:41.338 INFO:teuthology.orchestra.run.smithi118.stdout:(41/120): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 661 kB 00:00 2024-03-20T07:50:41.363 INFO:teuthology.orchestra.run.smithi118.stdout:(42/120): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-03-20T07:50:41.389 INFO:teuthology.orchestra.run.smithi118.stdout:(43/120): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-20T07:50:41.414 INFO:teuthology.orchestra.run.smithi118.stdout:(44/120): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2024-03-20T07:50:41.812 INFO:teuthology.orchestra.run.smithi118.stdout:(45/120): ceph-mgr-diskprediction-local-18.2.2- 7.3 MB/s | 7.4 MB 00:01 2024-03-20T07:50:41.932 INFO:teuthology.orchestra.run.smithi118.stdout:(46/120): openblas-openmp-0.3.21-2.el9.x86_64.r 8.7 MB/s | 4.7 MB 00:00 2024-03-20T07:50:41.957 INFO:teuthology.orchestra.run.smithi118.stdout:(47/120): python3-attrs-20.3.0-7.el9.noarch.rpm 144 kB/s | 78 kB 00:00 2024-03-20T07:50:42.107 INFO:teuthology.orchestra.run.smithi118.stdout:(48/120): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-20T07:50:42.132 INFO:teuthology.orchestra.run.smithi118.stdout:(49/120): python3-jsonpatch-1.21-16.el9.noarch. 141 kB/s | 24 kB 00:00 2024-03-20T07:50:42.158 INFO:teuthology.orchestra.run.smithi118.stdout:(50/120): python3-iniconfig-1.1.1-7.el9.noarch. 83 kB/s | 16 kB 00:00 2024-03-20T07:50:42.183 INFO:teuthology.orchestra.run.smithi118.stdout:(51/120): python3-jsonpointer-2.0-4.el9.noarch. 241 kB/s | 18 kB 00:00 2024-03-20T07:50:42.233 INFO:teuthology.orchestra.run.smithi118.stdout:(52/120): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 144 kB 00:00 2024-03-20T07:50:42.258 INFO:teuthology.orchestra.run.smithi118.stdout:(53/120): python3-markupsafe-1.1.1-12.el9.x86_6 422 kB/s | 32 kB 00:00 2024-03-20T07:50:42.350 INFO:teuthology.orchestra.run.smithi118.stdout:(54/120): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-03-20T07:50:42.367 INFO:teuthology.orchestra.run.smithi118.stdout:(55/120): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-20T07:50:42.392 INFO:teuthology.orchestra.run.smithi118.stdout:(56/120): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-20T07:50:42.418 INFO:teuthology.orchestra.run.smithi118.stdout:(57/120): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-20T07:50:42.443 INFO:teuthology.orchestra.run.smithi118.stdout:(58/120): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-03-20T07:50:42.470 INFO:teuthology.orchestra.run.smithi118.stdout:(59/120): python3-pluggy-0.13.1-7.el9.noarch.rp 501 kB/s | 38 kB 00:00 2024-03-20T07:50:42.493 INFO:teuthology.orchestra.run.smithi118.stdout:(60/120): python3-py-1.10.0-6.el9.noarch.rpm 5.5 MB/s | 427 kB 00:00 2024-03-20T07:50:42.519 INFO:teuthology.orchestra.run.smithi118.stdout:(61/120): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-20T07:50:42.544 INFO:teuthology.orchestra.run.smithi118.stdout:(62/120): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-20T07:50:42.569 INFO:teuthology.orchestra.run.smithi118.stdout:(63/120): python3-pytest-6.2.2-6.el9.noarch.rpm 6.2 MB/s | 474 kB 00:00 2024-03-20T07:50:42.586 INFO:teuthology.orchestra.run.smithi118.stdout:(64/120): python3-pytz-2021.1-5.el9.noarch.rpm 704 kB/s | 47 kB 00:00 2024-03-20T07:50:42.603 INFO:teuthology.orchestra.run.smithi118.stdout:(65/120): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-20T07:50:42.737 INFO:teuthology.orchestra.run.smithi118.stdout:(66/120): python3-toml-0.10.2-6.el9.noarch.rpm 258 kB/s | 39 kB 00:00 2024-03-20T07:50:42.979 INFO:teuthology.orchestra.run.smithi118.stdout:(67/120): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-20T07:50:43.004 INFO:teuthology.orchestra.run.smithi118.stdout:(68/120): socat-1.7.4.1-5.el9.x86_64.rpm 752 kB/s | 300 kB 00:00 2024-03-20T07:50:43.029 INFO:teuthology.orchestra.run.smithi118.stdout:(69/120): xmlstarlet-1.6.1-20.el9.x86_64.rpm 216 kB/s | 63 kB 00:00 2024-03-20T07:50:43.055 INFO:teuthology.orchestra.run.smithi118.stdout:(70/120): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-20T07:50:43.080 INFO:teuthology.orchestra.run.smithi118.stdout:(71/120): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-20T07:50:43.113 INFO:teuthology.orchestra.run.smithi118.stdout:(72/120): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-20T07:50:43.139 INFO:teuthology.orchestra.run.smithi118.stdout:(73/120): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-20T07:50:43.164 INFO:teuthology.orchestra.run.smithi118.stdout:(74/120): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-20T07:50:43.206 INFO:teuthology.orchestra.run.smithi118.stdout:(75/120): python3-urllib3-1.26.5-5.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2024-03-20T07:50:43.456 INFO:teuthology.orchestra.run.smithi118.stdout:(76/120): libquadmath-11.4.1-3.el9.x86_64.rpm 781 kB/s | 195 kB 00:00 2024-03-20T07:50:43.481 INFO:teuthology.orchestra.run.smithi118.stdout:(77/120): jq-1.6-16.el9.x86_64.rpm 547 kB/s | 187 kB 00:00 2024-03-20T07:50:43.507 INFO:teuthology.orchestra.run.smithi118.stdout:(78/120): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-03-20T07:50:43.540 INFO:teuthology.orchestra.run.smithi118.stdout:(79/120): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-03-20T07:50:43.566 INFO:teuthology.orchestra.run.smithi118.stdout:(80/120): python3-devel-3.9.17-2.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-03-20T07:50:43.616 INFO:teuthology.orchestra.run.smithi118.stdout:(81/120): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-03-20T07:50:43.725 INFO:teuthology.orchestra.run.smithi118.stdout:(82/120): fmt-8.1.1-5.el9.x86_64.rpm 602 kB/s | 111 kB 00:00 2024-03-20T07:50:43.833 INFO:teuthology.orchestra.run.smithi118.stdout:(83/120): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.2 MB/s | 309 kB 00:00 2024-03-20T07:50:43.909 INFO:teuthology.orchestra.run.smithi118.stdout:(84/120): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-03-20T07:50:43.933 INFO:teuthology.orchestra.run.smithi118.stdout:(85/120): libarrow-doc-9.0.0-10.el9.noarch.rpm 124 kB/s | 26 kB 00:00 2024-03-20T07:50:43.959 INFO:teuthology.orchestra.run.smithi118.stdout:(86/120): liboath-2.6.7-2.el9.x86_64.rpm 394 kB/s | 49 kB 00:00 2024-03-20T07:50:43.984 INFO:teuthology.orchestra.run.smithi118.stdout:(87/120): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-03-20T07:50:44.018 INFO:teuthology.orchestra.run.smithi118.stdout:(88/120): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-03-20T07:50:44.043 INFO:teuthology.orchestra.run.smithi118.stdout:(89/120): python3-asyncssh-2.13.2-5.el9.noarch. 6.4 MB/s | 548 kB 00:00 2024-03-20T07:50:44.068 INFO:teuthology.orchestra.run.smithi118.stdout:(90/120): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 525 kB/s | 43 kB 00:00 2024-03-20T07:50:44.094 INFO:teuthology.orchestra.run.smithi118.stdout:(91/120): python3-cachetools-4.2.4-1.el9.noarch 427 kB/s | 32 kB 00:00 2024-03-20T07:50:44.119 INFO:teuthology.orchestra.run.smithi118.stdout:(92/120): python3-certifi-2021.10.8-2.el9.noarc 300 kB/s | 15 kB 00:00 2024-03-20T07:50:44.144 INFO:teuthology.orchestra.run.smithi118.stdout:(93/120): python3-cheroot-8.6.0-4.el9.noarch.rp 2.2 MB/s | 172 kB 00:00 2024-03-20T07:50:44.170 INFO:teuthology.orchestra.run.smithi118.stdout:(94/120): python3-cherrypy-18.6.1-2.el9.noarch. 4.6 MB/s | 358 kB 00:00 2024-03-20T07:50:44.195 INFO:teuthology.orchestra.run.smithi118.stdout:(95/120): python3-google-auth-2.28.2-1.el9.noar 4.0 MB/s | 203 kB 00:00 2024-03-20T07:50:44.220 INFO:teuthology.orchestra.run.smithi118.stdout:(96/120): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2024-03-20T07:50:44.245 INFO:teuthology.orchestra.run.smithi118.stdout:(97/120): python3-jaraco-classes-3.2.1-5.el9.no 235 kB/s | 18 kB 00:00 2024-03-20T07:50:44.271 INFO:teuthology.orchestra.run.smithi118.stdout:(98/120): python3-jaraco-collections-3.0.0-8.el 461 kB/s | 23 kB 00:00 2024-03-20T07:50:44.296 INFO:teuthology.orchestra.run.smithi118.stdout:(99/120): python3-jaraco-functools-3.5.0-2.el9. 258 kB/s | 19 kB 00:00 2024-03-20T07:50:44.321 INFO:teuthology.orchestra.run.smithi118.stdout:(100/120): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-03-20T07:50:44.355 INFO:teuthology.orchestra.run.smithi118.stdout:(101/120): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-03-20T07:50:44.372 INFO:teuthology.orchestra.run.smithi118.stdout:(102/120): python3-logutils-0.3.5-21.el9.noarch 612 kB/s | 46 kB 00:00 2024-03-20T07:50:44.388 INFO:teuthology.orchestra.run.smithi118.stdout:(103/120): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-03-20T07:50:44.406 INFO:teuthology.orchestra.run.smithi118.stdout:(104/120): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-20T07:50:44.431 INFO:teuthology.orchestra.run.smithi118.stdout:(105/120): python3-portend-3.1.0-2.el9.noarch.r 395 kB/s | 16 kB 00:00 2024-03-20T07:50:44.456 INFO:teuthology.orchestra.run.smithi118.stdout:(106/120): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-03-20T07:50:44.481 INFO:teuthology.orchestra.run.smithi118.stdout:(107/120): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-20T07:50:44.507 INFO:teuthology.orchestra.run.smithi118.stdout:(108/120): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-03-20T07:50:44.532 INFO:teuthology.orchestra.run.smithi118.stdout:(109/120): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-20T07:50:44.557 INFO:teuthology.orchestra.run.smithi118.stdout:(110/120): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-20T07:50:44.582 INFO:teuthology.orchestra.run.smithi118.stdout:(111/120): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-03-20T07:50:44.607 INFO:teuthology.orchestra.run.smithi118.stdout:(112/120): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-20T07:50:44.633 INFO:teuthology.orchestra.run.smithi118.stdout:(113/120): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-03-20T07:50:44.658 INFO:teuthology.orchestra.run.smithi118.stdout:(114/120): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-20T07:50:44.692 INFO:teuthology.orchestra.run.smithi118.stdout:(115/120): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-20T07:50:44.717 INFO:teuthology.orchestra.run.smithi118.stdout:(116/120): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-03-20T07:50:44.742 INFO:teuthology.orchestra.run.smithi118.stdout:(117/120): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-03-20T07:50:44.784 INFO:teuthology.orchestra.run.smithi118.stdout:(118/120): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-20T07:50:45.043 INFO:teuthology.orchestra.run.smithi118.stdout:(119/120): librados2-18.2.2-904.g78dbc53a.el9.x 10 MB/s | 3.3 MB 00:00 2024-03-20T07:50:45.185 INFO:teuthology.orchestra.run.smithi118.stdout:(120/120): librbd1-18.2.2-904.g78dbc53a.el9.x86 6.8 MB/s | 3.0 MB 00:00 2024-03-20T07:50:45.191 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-03-20T07:50:45.191 INFO:teuthology.orchestra.run.smithi118.stdout:Total 15 MB/s | 177 MB 00:11 2024-03-20T07:50:45.258 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction check 2024-03-20T07:50:45.347 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction check succeeded. 2024-03-20T07:50:45.348 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction test 2024-03-20T07:50:46.819 INFO:teuthology.orchestra.run.smithi118.stdout:Transaction test succeeded. 2024-03-20T07:50:46.820 INFO:teuthology.orchestra.run.smithi118.stdout:Running transaction 2024-03-20T07:50:51.477 INFO:teuthology.orchestra.run.smithi118.stdout: Preparing : 1/1 2024-03-20T07:50:51.554 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/122 2024-03-20T07:50:51.579 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/122 2024-03-20T07:50:51.641 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/122 2024-03-20T07:50:51.960 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/122 2024-03-20T07:50:52.014 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librados2-2:18.2.2-904.g78dbc53a.el9.x86_64 5/122 2024-03-20T07:50:52.116 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:18.2.2-904.g78dbc53a.el9.x86_64 5/122 2024-03-20T07:50:52.148 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs2-2:18.2.2-904.g78dbc53a.el9.x86_64 6/122 2024-03-20T07:50:52.231 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephfs2-2:18.2.2-904.g78dbc53a.el9.x86_64 6/122 2024-03-20T07:50:52.457 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rados-2:18.2.2-904.g78dbc53a.el9.x86_64 7/122 2024-03-20T07:50:52.495 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/122 2024-03-20T07:50:52.531 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/122 2024-03-20T07:50:52.559 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephsqlite-2:18.2.2-904.g78dbc53a.el9.x86_64 10/122 2024-03-20T07:50:52.653 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libcephsqlite-2:18.2.2-904.g78dbc53a.el9.x86_64 10/122 2024-03-20T07:50:52.676 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libradosstriper1-2:18.2.2-904.g78dbc53a.el9.x86_ 11/122 2024-03-20T07:50:52.797 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: libradosstriper1-2:18.2.2-904.g78dbc53a.el9.x86_ 11/122 2024-03-20T07:50:52.831 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 12/122 2024-03-20T07:50:52.867 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : liboath-2.6.7-2.el9.x86_64 13/122 2024-03-20T07:50:52.919 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 14/122 2024-03-20T07:50:52.993 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 15/122 2024-03-20T07:50:53.047 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 16/122 2024-03-20T07:50:53.091 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 17/122 2024-03-20T07:50:53.121 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 18/122 2024-03-20T07:50:53.165 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-03-20T07:50:53.168 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 19/122 2024-03-20T07:50:53.233 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/122 2024-03-20T07:50:53.995 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/122 2024-03-20T07:50:54.104 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/122 2024-03-20T07:50:54.134 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 23/122 2024-03-20T07:50:54.178 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-argparse-2:18.2.2-904.g78dbc53a.el9 24/122 2024-03-20T07:50:54.254 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cephfs-2:18.2.2-904.g78dbc53a.el9.x86_64 25/122 2024-03-20T07:50:54.396 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/122 2024-03-20T07:50:54.446 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 27/122 2024-03-20T07:50:54.984 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rsa-4.9-2.el9.noarch 28/122 2024-03-20T07:50:55.068 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 29/122 2024-03-20T07:50:55.163 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 30/122 2024-03-20T07:50:55.189 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 31/122 2024-03-20T07:50:55.215 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 32/122 2024-03-20T07:50:55.248 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 33/122 2024-03-20T07:50:55.286 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librados-devel-2:18.2.2-904.g78dbc53a.el9.x86_64 34/122 2024-03-20T07:50:55.317 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 35/122 2024-03-20T07:50:55.351 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 36/122 2024-03-20T07:50:55.458 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/122 2024-03-20T07:50:55.496 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 38/122 2024-03-20T07:50:55.552 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/122 2024-03-20T07:50:55.626 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/122 2024-03-20T07:50:55.669 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/122 2024-03-20T07:50:55.706 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/122 2024-03-20T07:50:55.753 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/122 2024-03-20T07:50:55.798 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/122 2024-03-20T07:50:55.840 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/122 2024-03-20T07:50:55.986 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/122 2024-03-20T07:50:56.036 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/122 2024-03-20T07:50:56.078 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/122 2024-03-20T07:50:56.172 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/122 2024-03-20T07:50:56.211 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 50/122 2024-03-20T07:50:56.254 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/122 2024-03-20T07:50:56.284 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/122 2024-03-20T07:50:56.312 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 53/122 2024-03-20T07:50:56.416 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/122 2024-03-20T07:50:56.463 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 55/122 2024-03-20T07:50:56.512 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ply-3.11-14.el9.noarch 56/122 2024-03-20T07:50:56.568 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 57/122 2024-03-20T07:50:56.768 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 58/122 2024-03-20T07:50:56.826 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 59/122 2024-03-20T07:50:56.896 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 60/122 2024-03-20T07:50:56.984 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 61/122 2024-03-20T07:50:57.126 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 62/122 2024-03-20T07:50:57.167 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 63/122 2024-03-20T07:50:57.200 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 64/122 2024-03-20T07:50:57.233 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : fuse-2.9.9-15.el9.x86_64 65/122 2024-03-20T07:50:57.277 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 66/122 2024-03-20T07:50:57.401 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/122 2024-03-20T07:50:57.436 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-py-1.10.0-6.el9.noarch 68/122 2024-03-20T07:50:57.475 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 69/122 2024-03-20T07:50:57.609 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 70/122 2024-03-20T07:50:57.707 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/122 2024-03-20T07:50:57.764 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 72/122 2024-03-20T07:50:58.112 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-03-20T07:50:58.549 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/122 2024-03-20T07:50:58.596 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 74/122 2024-03-20T07:50:58.630 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 75/122 2024-03-20T07:50:58.664 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 76/122 2024-03-20T07:50:58.721 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 77/122 2024-03-20T07:50:58.756 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 78/122 2024-03-20T07:50:58.939 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-0.3.21-2.el9.x86_64 79/122 2024-03-20T07:50:58.970 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 80/122 2024-03-20T07:50:59.050 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/122 2024-03-20T07:50:59.734 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 82/122 2024-03-20T07:50:59.853 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 83/122 2024-03-20T07:51:00.558 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-20T07:51:00.561 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-20T07:51:00.640 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout:=========================================================================================== 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout:=========================================================================================== 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 6.5 k 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 5.2 M 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 843 k 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 142 k 2024-03-20T07:51:00.647 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 1.4 M 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 138 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 1.7 M 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 7.4 M 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 49 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 7.7 M 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 40 M 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 224 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 31 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 702 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 127 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 162 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytest noarch 6.2.2-6.el9 CentOS-AppStream 474 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 320 k 2024-03-20T07:51:00.648 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 298 k 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 99 k 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 87 k 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 3.0 M 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 172 k 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 3.3 M 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 3.0 M 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 18 M 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 23 k 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 2.1 M 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 245 k 2024-03-20T07:51:00.649 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 4.4 M 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 18 M 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:18.2.2-904.g78dbc53a.el9 ceph-noarch 15 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 25 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-20T07:51:00.650 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 166 k 2024-03-20T07:51:00.651 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-20T07:51:00.651 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-20T07:51:00.651 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-20T07:51:00.651 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-20T07:51:00.651 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-20T07:51:00.651 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 474 k 2024-03-20T07:51:00.651 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-20T07:51:00.652 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 4.5 M 2024-03-20T07:51:00.652 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-20T07:51:00.652 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-20T07:51:00.652 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-20T07:51:00.652 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-20T07:51:00.653 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-20T07:51:00.653 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-20T07:51:00.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-20T07:51:00.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-20T07:51:00.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-20T07:51:00.653 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 45 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:18.2.2-904.g78dbc53a.el9 ceph 127 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-iniconfig noarch 1.1.1-7.el9 CentOS-AppStream 16 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-20T07:51:00.654 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-20T07:51:00.655 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pluggy noarch 0.13.1-7.el9 CentOS-AppStream 38 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-py noarch 1.10.0-6.el9 CentOS-AppStream 427 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-20T07:51:00.656 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-20T07:51:00.657 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-20T07:51:00.658 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-20T07:51:00.658 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:00.658 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-20T07:51:00.658 INFO:teuthology.orchestra.run.smithi196.stdout:=========================================================================================== 2024-03-20T07:51:00.658 INFO:teuthology.orchestra.run.smithi196.stdout:Install 118 Packages 2024-03-20T07:51:00.658 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 2 Packages 2024-03-20T07:51:00.658 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:00.659 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 177 M 2024-03-20T07:51:00.659 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-20T07:51:01.114 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/122 2024-03-20T07:51:01.129 INFO:teuthology.orchestra.run.smithi196.stdout:(1/120): ceph-18.2.2-904.g78dbc53a.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2024-03-20T07:51:01.222 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 85/122 2024-03-20T07:51:01.254 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 86/122 2024-03-20T07:51:01.286 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : jq-1.6-16.el9.x86_64 87/122 2024-03-20T07:51:01.593 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 88/122 2024-03-20T07:51:01.616 INFO:teuthology.orchestra.run.smithi118.stdout: Upgrading : librbd1-2:18.2.2-904.g78dbc53a.el9.x86_64 89/122 2024-03-20T07:51:01.706 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:18.2.2-904.g78dbc53a.el9.x86_64 89/122 2024-03-20T07:51:01.737 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rbd-2:18.2.2-904.g78dbc53a.el9.x86_64 90/122 2024-03-20T07:51:01.774 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 91/122 2024-03-20T07:51:01.938 INFO:teuthology.orchestra.run.smithi196.stdout:(2/120): ceph-base-18.2.2-904.g78dbc53a.el9.x86 5.1 MB/s | 5.2 MB 00:01 2024-03-20T07:51:02.013 INFO:teuthology.orchestra.run.smithi196.stdout:(3/120): ceph-immutable-object-cache-18.2.2-904 1.9 MB/s | 142 kB 00:00 2024-03-20T07:51:02.175 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 92/122 2024-03-20T07:51:02.188 INFO:teuthology.orchestra.run.smithi196.stdout:(4/120): ceph-common-18.2.2-904.g78dbc53a.el9.x 14 MB/s | 18 MB 00:01 2024-03-20T07:51:02.202 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : librgw2-2:18.2.2-904.g78dbc53a.el9.x86_64 93/122 2024-03-20T07:51:02.222 INFO:teuthology.orchestra.run.smithi196.stdout:(5/120): ceph-fuse-18.2.2-904.g78dbc53a.el9.x86 772 kB/s | 843 kB 00:01 2024-03-20T07:51:02.278 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librgw2-2:18.2.2-904.g78dbc53a.el9.x86_64 93/122 2024-03-20T07:51:02.306 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-rgw-2:18.2.2-904.g78dbc53a.el9.x86_64 94/122 2024-03-20T07:51:02.314 INFO:teuthology.orchestra.run.smithi196.stdout:(6/120): ceph-mgr-18.2.2-904.g78dbc53a.el9.x86_ 12 MB/s | 1.4 MB 00:00 2024-03-20T07:51:02.356 INFO:teuthology.orchestra.run.smithi196.stdout:(7/120): ceph-mds-18.2.2-904.g78dbc53a.el9.x86_ 6.2 MB/s | 2.1 MB 00:00 2024-03-20T07:51:02.986 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 95/122 2024-03-20T07:51:03.010 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 95/122 2024-03-20T07:51:03.040 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-904.g78dbc53a.el 96/122 2024-03-20T07:51:03.115 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-904.g78dbc53a.e 97/122 2024-03-20T07:51:03.143 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-ceph-common-2:18.2.2-904.g78dbc53a.el9.x 98/122 2024-03-20T07:51:03.306 INFO:teuthology.orchestra.run.smithi196.stdout:(8/120): ceph-osd-18.2.2-904.g78dbc53a.el9.x86_ 18 MB/s | 18 MB 00:00 2024-03-20T07:51:03.348 INFO:teuthology.orchestra.run.smithi196.stdout:(9/120): ceph-selinux-18.2.2-904.g78dbc53a.el9. 608 kB/s | 25 kB 00:00 2024-03-20T07:51:03.498 INFO:teuthology.orchestra.run.smithi196.stdout:(10/120): ceph-radosgw-18.2.2-904.g78dbc53a.el9 6.8 MB/s | 7.7 MB 00:01 2024-03-20T07:51:03.557 INFO:teuthology.orchestra.run.smithi196.stdout:(11/120): libcephfs-devel-18.2.2-904.g78dbc53a. 534 kB/s | 31 kB 00:00 2024-03-20T07:51:03.715 INFO:teuthology.orchestra.run.smithi196.stdout:(12/120): libcephfs2-18.2.2-904.g78dbc53a.el9.x 4.3 MB/s | 702 kB 00:00 2024-03-20T07:51:03.808 INFO:teuthology.orchestra.run.smithi196.stdout:(13/120): libcephsqlite-18.2.2-904.g78dbc53a.el 1.8 MB/s | 166 kB 00:00 2024-03-20T07:51:03.891 INFO:teuthology.orchestra.run.smithi196.stdout:(14/120): librados-devel-18.2.2-904.g78dbc53a.e 1.5 MB/s | 127 kB 00:00 2024-03-20T07:51:03.991 INFO:teuthology.orchestra.run.smithi196.stdout:(15/120): libradosstriper1-18.2.2-904.g78dbc53a 4.6 MB/s | 474 kB 00:00 2024-03-20T07:51:04.750 INFO:teuthology.orchestra.run.smithi196.stdout:(16/120): librgw2-18.2.2-904.g78dbc53a.el9.x86_ 5.9 MB/s | 4.5 MB 00:00 2024-03-20T07:51:04.833 INFO:teuthology.orchestra.run.smithi196.stdout:(17/120): python3-ceph-argparse-18.2.2-904.g78d 542 kB/s | 45 kB 00:00 2024-03-20T07:51:04.942 INFO:teuthology.orchestra.run.smithi196.stdout:(18/120): python3-ceph-common-18.2.2-904.g78dbc 1.1 MB/s | 127 kB 00:00 2024-03-20T07:51:05.204 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 99/122 2024-03-20T07:51:05.257 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 99/122 2024-03-20T07:51:05.334 INFO:teuthology.orchestra.run.smithi196.stdout:(19/120): ceph-test-18.2.2-904.g78dbc53a.el9.x8 20 MB/s | 40 MB 00:01 2024-03-20T07:51:05.359 INFO:teuthology.orchestra.run.smithi196.stdout:(20/120): python3-cephfs-18.2.2-904.g78dbc53a.e 388 kB/s | 162 kB 00:00 2024-03-20T07:51:05.384 INFO:teuthology.orchestra.run.smithi196.stdout:(21/120): python3-rados-18.2.2-904.g78dbc53a.el 6.3 MB/s | 320 kB 00:00 2024-03-20T07:51:05.427 INFO:teuthology.orchestra.run.smithi196.stdout:(22/120): python3-rbd-18.2.2-904.g78dbc53a.el9. 4.3 MB/s | 298 kB 00:00 2024-03-20T07:51:05.452 INFO:teuthology.orchestra.run.smithi196.stdout:(23/120): python3-rgw-18.2.2-904.g78dbc53a.el9. 1.4 MB/s | 99 kB 00:00 2024-03-20T07:51:05.477 INFO:teuthology.orchestra.run.smithi196.stdout:(24/120): rbd-fuse-18.2.2-904.g78dbc53a.el9.x86 1.7 MB/s | 87 kB 00:00 2024-03-20T07:51:05.552 INFO:teuthology.orchestra.run.smithi196.stdout:(25/120): rbd-nbd-18.2.2-904.g78dbc53a.el9.x86_ 2.3 MB/s | 172 kB 00:00 2024-03-20T07:51:05.644 INFO:teuthology.orchestra.run.smithi196.stdout:(26/120): rbd-mirror-18.2.2-904.g78dbc53a.el9.x 16 MB/s | 3.0 MB 00:00 2024-03-20T07:51:05.669 INFO:teuthology.orchestra.run.smithi196.stdout:(27/120): ceph-grafana-dashboards-18.2.2-904.g7 200 kB/s | 23 kB 00:00 2024-03-20T07:51:05.695 INFO:teuthology.orchestra.run.smithi196.stdout:(28/120): ceph-mgr-cephadm-18.2.2-904.g78dbc53a 2.7 MB/s | 138 kB 00:00 2024-03-20T07:51:05.796 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 99/122 2024-03-20T07:51:05.843 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-base-2:18.2.2-904.g78dbc53a.el9.x86_64 100/122 2024-03-20T07:51:05.904 INFO:teuthology.orchestra.run.smithi196.stdout:(29/120): ceph-mgr-dashboard-18.2.2-904.g78dbc5 7.5 MB/s | 1.7 MB 00:00 2024-03-20T07:51:05.943 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-base-2:18.2.2-904.g78dbc53a.el9.x86_64 100/122 2024-03-20T07:51:05.944 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-20T07:51:05.944 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-20T07:51:05.944 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:05.978 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-selinux-2:18.2.2-904.g78dbc53a.el9.x86_64 101/122 2024-03-20T07:51:06.004 INFO:teuthology.orchestra.run.smithi196.stdout:(30/120): ceph-mgr-modules-core-18.2.2-904.g78d 2.4 MB/s | 245 kB 00:00 2024-03-20T07:51:06.112 INFO:teuthology.orchestra.run.smithi196.stdout:(31/120): ceph-mgr-diskprediction-local-18.2.2- 18 MB/s | 7.4 MB 00:00 2024-03-20T07:51:06.138 INFO:teuthology.orchestra.run.smithi196.stdout:(32/120): ceph-mgr-rook-18.2.2-904.g78dbc53a.el 369 kB/s | 49 kB 00:00 2024-03-20T07:51:06.163 INFO:teuthology.orchestra.run.smithi196.stdout:(33/120): ceph-prometheus-alerts-18.2.2-904.g78 298 kB/s | 15 kB 00:00 2024-03-20T07:51:06.188 INFO:teuthology.orchestra.run.smithi196.stdout:(34/120): cephadm-18.2.2-904.g78dbc53a.el9.noar 4.4 MB/s | 224 kB 00:00 2024-03-20T07:51:06.272 INFO:teuthology.orchestra.run.smithi196.stdout:(35/120): ceph-mon-18.2.2-904.g78dbc53a.el9.x86 1.1 MB/s | 4.4 MB 00:04 2024-03-20T07:51:06.330 INFO:teuthology.orchestra.run.smithi196.stdout:(36/120): boost-program-options-1.75.0-8.el9.x8 633 kB/s | 106 kB 00:00 2024-03-20T07:51:06.356 INFO:teuthology.orchestra.run.smithi196.stdout:(37/120): flexiblas-3.0.4-8.el9.x86_64.rpm 223 kB/s | 32 kB 00:00 2024-03-20T07:51:06.472 INFO:teuthology.orchestra.run.smithi196.stdout:(38/120): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-03-20T07:51:06.498 INFO:teuthology.orchestra.run.smithi196.stdout:(39/120): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2024-03-20T07:51:06.623 INFO:teuthology.orchestra.run.smithi196.stdout:(40/120): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.2 MB/s | 3.0 MB 00:00 2024-03-20T07:51:06.648 INFO:teuthology.orchestra.run.smithi196.stdout:(41/120): librabbitmq-0.11.0-7.el9.x86_64.rpm 252 kB/s | 44 kB 00:00 2024-03-20T07:51:06.673 INFO:teuthology.orchestra.run.smithi196.stdout:(42/120): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 661 kB 00:00 2024-03-20T07:51:06.699 INFO:teuthology.orchestra.run.smithi196.stdout:(43/120): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-20T07:51:06.716 INFO:teuthology.orchestra.run.smithi196.stdout:(44/120): openblas-0.3.21-2.el9.x86_64.rpm 826 kB/s | 34 kB 00:00 2024-03-20T07:51:06.741 INFO:teuthology.orchestra.run.smithi196.stdout:(45/120): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-03-20T07:51:06.874 INFO:teuthology.orchestra.run.smithi196.stdout:(46/120): python3-attrs-20.3.0-7.el9.noarch.rpm 493 kB/s | 78 kB 00:00 2024-03-20T07:51:06.941 INFO:teuthology.orchestra.run.smithi196.stdout:(47/120): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-03-20T07:51:07.042 INFO:teuthology.orchestra.run.smithi196.stdout:(48/120): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-20T07:51:07.067 INFO:teuthology.orchestra.run.smithi196.stdout:(49/120): python3-iniconfig-1.1.1-7.el9.noarch. 87 kB/s | 16 kB 00:00 2024-03-20T07:51:07.092 INFO:teuthology.orchestra.run.smithi196.stdout:(50/120): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-20T07:51:07.126 INFO:teuthology.orchestra.run.smithi196.stdout:(51/120): python3-mako-1.1.4-6.el9.noarch.rpm 4.3 MB/s | 144 kB 00:00 2024-03-20T07:51:07.151 INFO:teuthology.orchestra.run.smithi196.stdout:(52/120): python3-markupsafe-1.1.1-12.el9.x86_6 543 kB/s | 32 kB 00:00 2024-03-20T07:51:07.176 INFO:teuthology.orchestra.run.smithi196.stdout:(53/120): python3-jsonpointer-2.0-4.el9.noarch. 216 kB/s | 18 kB 00:00 2024-03-20T07:51:07.360 INFO:teuthology.orchestra.run.smithi196.stdout:(54/120): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-20T07:51:07.385 INFO:teuthology.orchestra.run.smithi196.stdout:(55/120): python3-numpy-f2py-1.20.1-5.el9.x86_6 933 kB/s | 217 kB 00:00 2024-03-20T07:51:07.410 INFO:teuthology.orchestra.run.smithi196.stdout:(56/120): python3-oauthlib-3.1.1-5.el9.noarch.r 750 kB/s | 174 kB 00:00 2024-03-20T07:51:07.444 INFO:teuthology.orchestra.run.smithi196.stdout:(57/120): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-03-20T07:51:07.469 INFO:teuthology.orchestra.run.smithi196.stdout:(58/120): python3-prettytable-0.7.2-27.el9.noar 708 kB/s | 41 kB 00:00 2024-03-20T07:51:07.495 INFO:teuthology.orchestra.run.smithi196.stdout:(59/120): python3-pluggy-0.13.1-7.el9.noarch.rp 452 kB/s | 38 kB 00:00 2024-03-20T07:51:07.519 INFO:teuthology.orchestra.run.smithi196.stdout:(60/120): python3-py-1.10.0-6.el9.noarch.rpm 5.6 MB/s | 427 kB 00:00 2024-03-20T07:51:07.545 INFO:teuthology.orchestra.run.smithi196.stdout:(61/120): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-20T07:51:07.570 INFO:teuthology.orchestra.run.smithi196.stdout:(62/120): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-20T07:51:07.595 INFO:teuthology.orchestra.run.smithi196.stdout:(63/120): python3-pytest-6.2.2-6.el9.noarch.rpm 6.2 MB/s | 474 kB 00:00 2024-03-20T07:51:07.620 INFO:teuthology.orchestra.run.smithi196.stdout:(64/120): python3-pytz-2021.1-5.el9.noarch.rpm 639 kB/s | 47 kB 00:00 2024-03-20T07:51:07.646 INFO:teuthology.orchestra.run.smithi196.stdout:(65/120): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2024-03-20T07:51:07.812 INFO:teuthology.orchestra.run.smithi196.stdout:(66/120): python3-toml-0.10.2-6.el9.noarch.rpm 204 kB/s | 39 kB 00:00 2024-03-20T07:51:07.863 INFO:teuthology.orchestra.run.smithi196.stdout:(67/120): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-20T07:51:08.021 INFO:teuthology.orchestra.run.smithi196.stdout:(68/120): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-20T07:51:08.046 INFO:teuthology.orchestra.run.smithi196.stdout:(69/120): xmlstarlet-1.6.1-20.el9.x86_64.rpm 271 kB/s | 63 kB 00:00 2024-03-20T07:51:08.072 INFO:teuthology.orchestra.run.smithi196.stdout:(70/120): fuse-2.9.9-15.el9.x86_64.rpm 380 kB/s | 79 kB 00:00 2024-03-20T07:51:08.097 INFO:teuthology.orchestra.run.smithi196.stdout:(71/120): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-20T07:51:08.131 INFO:teuthology.orchestra.run.smithi196.stdout:(72/120): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-20T07:51:08.165 INFO:teuthology.orchestra.run.smithi196.stdout:(73/120): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-20T07:51:08.190 INFO:teuthology.orchestra.run.smithi196.stdout:(74/120): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-20T07:51:08.215 INFO:teuthology.orchestra.run.smithi196.stdout:(75/120): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-20T07:51:09.131 INFO:teuthology.orchestra.run.smithi196.stdout:(76/120): jq-1.6-16.el9.x86_64.rpm 194 kB/s | 187 kB 00:00 2024-03-20T07:51:09.165 INFO:teuthology.orchestra.run.smithi196.stdout:(77/120): libquadmath-11.4.1-3.el9.x86_64.rpm 206 kB/s | 195 kB 00:00 2024-03-20T07:51:09.366 INFO:teuthology.orchestra.run.smithi196.stdout:(78/120): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-20T07:51:09.507 INFO:teuthology.orchestra.run.smithi196.stdout:(79/120): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-03-20T07:51:09.616 INFO:teuthology.orchestra.run.smithi196.stdout:(80/120): fmt-8.1.1-5.el9.x86_64.rpm 1.0 MB/s | 111 kB 00:00 2024-03-20T07:51:09.717 INFO:teuthology.orchestra.run.smithi196.stdout:(81/120): gperftools-libs-2.9.1-2.el9.x86_64.rp 3.0 MB/s | 309 kB 00:00 2024-03-20T07:51:09.975 INFO:teuthology.orchestra.run.smithi196.stdout:(82/120): libgfortran-11.4.1-3.el9.x86_64.rpm 450 kB/s | 803 kB 00:01 2024-03-20T07:51:10.000 INFO:teuthology.orchestra.run.smithi196.stdout:(83/120): python3-requests-2.25.1-8.el9.noarch. 144 kB/s | 125 kB 00:00 2024-03-20T07:51:10.067 INFO:teuthology.orchestra.run.smithi196.stdout:(84/120): libarrow-doc-9.0.0-10.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-03-20T07:51:10.092 INFO:teuthology.orchestra.run.smithi196.stdout:(85/120): liboath-2.6.7-2.el9.x86_64.rpm 531 kB/s | 49 kB 00:00 2024-03-20T07:51:10.142 INFO:teuthology.orchestra.run.smithi196.stdout:(86/120): libunwind-1.6.2-1.el9.x86_64.rpm 895 kB/s | 67 kB 00:00 2024-03-20T07:51:10.293 INFO:teuthology.orchestra.run.smithi196.stdout:(87/120): python3-asyncssh-2.13.2-5.el9.noarch. 3.6 MB/s | 548 kB 00:00 2024-03-20T07:51:10.318 INFO:teuthology.orchestra.run.smithi196.stdout:(88/120): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-03-20T07:51:10.335 INFO:teuthology.orchestra.run.smithi196.stdout:(89/120): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2024-03-20T07:51:10.369 INFO:teuthology.orchestra.run.smithi196.stdout:(90/120): python3-cachetools-4.2.4-1.el9.noarch 644 kB/s | 32 kB 00:00 2024-03-20T07:51:10.385 INFO:teuthology.orchestra.run.smithi196.stdout:(91/120): python3-certifi-2021.10.8-2.el9.noarc 299 kB/s | 15 kB 00:00 2024-03-20T07:51:10.486 INFO:teuthology.orchestra.run.smithi196.stdout:(92/120): python3-cheroot-8.6.0-4.el9.noarch.rp 1.7 MB/s | 172 kB 00:00 2024-03-20T07:51:10.545 INFO:teuthology.orchestra.run.smithi196.stdout:(93/120): python3-cherrypy-18.6.1-2.el9.noarch. 2.2 MB/s | 358 kB 00:00 2024-03-20T07:51:10.570 INFO:teuthology.orchestra.run.smithi196.stdout:(94/120): python3-google-auth-2.28.2-1.el9.noar 2.4 MB/s | 203 kB 00:00 2024-03-20T07:51:10.595 INFO:teuthology.orchestra.run.smithi196.stdout:(95/120): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-03-20T07:51:10.620 INFO:teuthology.orchestra.run.smithi196.stdout:(96/120): python3-jaraco-classes-3.2.1-5.el9.no 352 kB/s | 18 kB 00:00 2024-03-20T07:51:10.646 INFO:teuthology.orchestra.run.smithi196.stdout:(97/120): python3-jaraco-collections-3.0.0-8.el 461 kB/s | 23 kB 00:00 2024-03-20T07:51:10.671 INFO:teuthology.orchestra.run.smithi196.stdout:(98/120): python3-jaraco-functools-3.5.0-2.el9. 387 kB/s | 19 kB 00:00 2024-03-20T07:51:10.696 INFO:teuthology.orchestra.run.smithi196.stdout:(99/120): python3-jaraco-text-3.2.0-6.el9.noarc 393 kB/s | 20 kB 00:00 2024-03-20T07:51:10.772 INFO:teuthology.orchestra.run.smithi196.stdout:(100/120): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-03-20T07:51:10.797 INFO:teuthology.orchestra.run.smithi196.stdout:(101/120): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-03-20T07:51:10.830 INFO:teuthology.orchestra.run.smithi196.stdout:(102/120): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-20T07:51:10.856 INFO:teuthology.orchestra.run.smithi196.stdout:(103/120): python3-natsort-7.1.1-5.el9.noarch.r 984 kB/s | 58 kB 00:00 2024-03-20T07:51:10.897 INFO:teuthology.orchestra.run.smithi196.stdout:(104/120): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-03-20T07:51:10.923 INFO:teuthology.orchestra.run.smithi196.stdout:(105/120): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-20T07:51:10.939 INFO:teuthology.orchestra.run.smithi196.stdout:(106/120): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-03-20T07:51:10.982 INFO:teuthology.orchestra.run.smithi196.stdout:(107/120): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-03-20T07:51:11.007 INFO:teuthology.orchestra.run.smithi196.stdout:(108/120): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-03-20T07:51:11.032 INFO:teuthology.orchestra.run.smithi196.stdout:(109/120): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-03-20T07:51:11.057 INFO:teuthology.orchestra.run.smithi196.stdout:(110/120): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-03-20T07:51:11.082 INFO:teuthology.orchestra.run.smithi196.stdout:(111/120): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-03-20T07:51:11.108 INFO:teuthology.orchestra.run.smithi196.stdout:(112/120): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-20T07:51:11.158 INFO:teuthology.orchestra.run.smithi196.stdout:(113/120): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-03-20T07:51:11.183 INFO:teuthology.orchestra.run.smithi196.stdout:(114/120): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-20T07:51:11.208 INFO:teuthology.orchestra.run.smithi196.stdout:(115/120): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-03-20T07:51:11.234 INFO:teuthology.orchestra.run.smithi196.stdout:(116/120): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-20T07:51:11.268 INFO:teuthology.orchestra.run.smithi196.stdout:(117/120): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-20T07:51:11.360 INFO:teuthology.orchestra.run.smithi196.stdout:(118/120): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-20T07:51:11.568 INFO:teuthology.orchestra.run.smithi196.stdout:(119/120): librbd1-18.2.2-904.g78dbc53a.el9.x86 10 MB/s | 3.0 MB 00:00 2024-03-20T07:51:12.535 INFO:teuthology.orchestra.run.smithi196.stdout:(120/120): librados2-18.2.2-904.g78dbc53a.el9.x 2.5 MB/s | 3.3 MB 00:01 2024-03-20T07:51:12.539 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-20T07:51:12.539 INFO:teuthology.orchestra.run.smithi196.stdout:Total 15 MB/s | 177 MB 00:11 2024-03-20T07:51:12.613 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-20T07:51:12.706 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-20T07:51:12.706 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-20T07:51:14.204 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-20T07:51:14.204 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-selinux-2:18.2.2-904.g78dbc53a.el9.x86_64 101/122 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /sys 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /proc 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /mnt 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /var/tmp 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /home 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /root 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout:skipping the directory /tmp 2024-03-20T07:51:18.058 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:18.131 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-904.g78dbc53a.el9.noar 102/122 2024-03-20T07:51:18.357 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-904.g78dbc53a.el9.noar 102/122 2024-03-20T07:51:18.389 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-904.g78dbc53a.el9.no 103/122 2024-03-20T07:51:18.865 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-20T07:51:18.945 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/122 2024-03-20T07:51:18.970 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/122 2024-03-20T07:51:19.030 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/122 2024-03-20T07:51:19.354 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/122 2024-03-20T07:51:19.432 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:18.2.2-904.g78dbc53a.el9.x86_64 5/122 2024-03-20T07:51:19.501 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-904.g78dbc53a.el9.no 103/122 2024-03-20T07:51:19.534 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-904.g78db 104/122 2024-03-20T07:51:19.548 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:18.2.2-904.g78dbc53a.el9.x86_64 5/122 2024-03-20T07:51:19.572 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:18.2.2-904.g78dbc53a.el9.x86_64 6/122 2024-03-20T07:51:19.632 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-904.g78db 104/122 2024-03-20T07:51:19.662 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:18.2.2-904.g78dbc53a.el9.x86_64 6/122 2024-03-20T07:51:19.712 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:18.2.2-904.g78dbc53a.el9.x86_64 7/122 2024-03-20T07:51:19.743 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/122 2024-03-20T07:51:19.780 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/122 2024-03-20T07:51:19.787 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-904.g78dbc53a.el9 105/122 2024-03-20T07:51:19.808 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:18.2.2-904.g78dbc53a.el9.x86_64 10/122 2024-03-20T07:51:19.820 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-2:18.2.2-904.g78dbc53a.el9.x86_64 106/122 2024-03-20T07:51:19.844 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-2:18.2.2-904.g78dbc53a.el9.x86_64 106/122 2024-03-20T07:51:19.844 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:19.844 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-20T07:51:19.844 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-20T07:51:19.844 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-20T07:51:19.844 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:19.887 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mgr-rook-2:18.2.2-904.g78dbc53a.el9.noarch 107/122 2024-03-20T07:51:19.917 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:18.2.2-904.g78dbc53a.el9.x86_64 10/122 2024-03-20T07:51:19.942 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:18.2.2-904.g78dbc53a.el9.x86_ 11/122 2024-03-20T07:51:20.068 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:18.2.2-904.g78dbc53a.el9.x86_ 11/122 2024-03-20T07:51:20.076 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-904.g78dbc53a.el9.noarch 107/122 2024-03-20T07:51:20.100 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 12/122 2024-03-20T07:51:20.104 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mds-2:18.2.2-904.g78dbc53a.el9.x86_64 108/122 2024-03-20T07:51:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mds-2:18.2.2-904.g78dbc53a.el9.x86_64 108/122 2024-03-20T07:51:20.127 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:20.127 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-20T07:51:20.127 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-20T07:51:20.127 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-20T07:51:20.127 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:20.137 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 13/122 2024-03-20T07:51:20.187 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 14/122 2024-03-20T07:51:20.262 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 15/122 2024-03-20T07:51:20.315 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 16/122 2024-03-20T07:51:20.351 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 17/122 2024-03-20T07:51:20.381 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 18/122 2024-03-20T07:51:20.413 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 19/122 2024-03-20T07:51:20.478 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/122 2024-03-20T07:51:20.512 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/122 2024-03-20T07:51:20.540 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/122 2024-03-20T07:51:20.570 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 23/122 2024-03-20T07:51:20.572 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-mon-2:18.2.2-904.g78dbc53a.el9.x86_64 109/122 2024-03-20T07:51:20.594 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-mon-2:18.2.2-904.g78dbc53a.el9.x86_64 109/122 2024-03-20T07:51:20.594 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:20.594 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-20T07:51:20.595 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-20T07:51:20.595 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-20T07:51:20.595 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:20.613 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:18.2.2-904.g78dbc53a.el9 24/122 2024-03-20T07:51:20.687 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:18.2.2-904.g78dbc53a.el9.x86_64 25/122 2024-03-20T07:51:20.824 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/122 2024-03-20T07:51:20.874 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 27/122 2024-03-20T07:51:21.405 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 28/122 2024-03-20T07:51:21.478 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 29/122 2024-03-20T07:51:21.523 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 30/122 2024-03-20T07:51:21.557 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 31/122 2024-03-20T07:51:21.592 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 32/122 2024-03-20T07:51:21.633 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 33/122 2024-03-20T07:51:21.682 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:18.2.2-904.g78dbc53a.el9.x86_64 34/122 2024-03-20T07:51:21.712 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 35/122 2024-03-20T07:51:21.747 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 36/122 2024-03-20T07:51:21.854 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/122 2024-03-20T07:51:21.892 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 38/122 2024-03-20T07:51:21.947 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/122 2024-03-20T07:51:22.021 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/122 2024-03-20T07:51:22.059 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/122 2024-03-20T07:51:22.081 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-osd-2:18.2.2-904.g78dbc53a.el9.x86_64 110/122 2024-03-20T07:51:22.096 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/122 2024-03-20T07:51:22.106 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-osd-2:18.2.2-904.g78dbc53a.el9.x86_64 110/122 2024-03-20T07:51:22.107 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:22.107 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-20T07:51:22.107 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-20T07:51:22.107 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-20T07:51:22.107 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:22.143 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/122 2024-03-20T07:51:22.196 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/122 2024-03-20T07:51:22.239 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/122 2024-03-20T07:51:22.374 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/122 2024-03-20T07:51:22.424 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/122 2024-03-20T07:51:22.465 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/122 2024-03-20T07:51:22.560 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/122 2024-03-20T07:51:22.599 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 50/122 2024-03-20T07:51:22.647 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/122 2024-03-20T07:51:22.680 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/122 2024-03-20T07:51:22.707 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 53/122 2024-03-20T07:51:22.803 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/122 2024-03-20T07:51:22.814 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-2:18.2.2-904.g78dbc53a.el9.x86_64 111/122 2024-03-20T07:51:22.841 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-radosgw-2:18.2.2-904.g78dbc53a.el9.x86_64 112/122 2024-03-20T07:51:22.851 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 55/122 2024-03-20T07:51:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-904.g78dbc53a.el9.x86_64 112/122 2024-03-20T07:51:22.864 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:22.864 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-20T07:51:22.864 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-20T07:51:22.864 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-20T07:51:22.864 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:22.908 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 56/122 2024-03-20T07:51:22.908 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-904.g78dbc5 113/122 2024-03-20T07:51:22.930 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-904.g78dbc5 113/122 2024-03-20T07:51:22.930 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:22.930 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-20T07:51:22.930 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:22.964 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 57/122 2024-03-20T07:51:23.165 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 58/122 2024-03-20T07:51:23.213 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 59/122 2024-03-20T07:51:23.217 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-mirror-2:18.2.2-904.g78dbc53a.el9.x86_64 114/122 2024-03-20T07:51:23.240 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: rbd-mirror-2:18.2.2-904.g78dbc53a.el9.x86_64 114/122 2024-03-20T07:51:23.240 INFO:teuthology.orchestra.run.smithi118.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:23.240 INFO:teuthology.orchestra.run.smithi118.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-20T07:51:23.240 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-20T07:51:23.240 INFO:teuthology.orchestra.run.smithi118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-20T07:51:23.241 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:23.292 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 60/122 2024-03-20T07:51:23.381 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 61/122 2024-03-20T07:51:23.521 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 62/122 2024-03-20T07:51:23.555 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 63/122 2024-03-20T07:51:23.587 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 64/122 2024-03-20T07:51:23.620 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 65/122 2024-03-20T07:51:23.664 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 66/122 2024-03-20T07:51:23.789 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/122 2024-03-20T07:51:23.823 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-py-1.10.0-6.el9.noarch 68/122 2024-03-20T07:51:23.862 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 69/122 2024-03-20T07:51:23.904 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 70/122 2024-03-20T07:51:24.002 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/122 2024-03-20T07:51:24.084 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 72/122 2024-03-20T07:51:24.828 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/122 2024-03-20T07:51:24.872 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 74/122 2024-03-20T07:51:24.906 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 75/122 2024-03-20T07:51:24.940 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 76/122 2024-03-20T07:51:24.988 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 77/122 2024-03-20T07:51:25.023 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 78/122 2024-03-20T07:51:25.202 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 79/122 2024-03-20T07:51:25.230 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 80/122 2024-03-20T07:51:25.308 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/122 2024-03-20T07:51:26.001 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 82/122 2024-03-20T07:51:26.111 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 83/122 2024-03-20T07:51:26.913 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-test-2:18.2.2-904.g78dbc53a.el9.x86_64 115/122 2024-03-20T07:51:26.959 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 116/122 2024-03-20T07:51:27.069 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : rbd-nbd-2:18.2.2-904.g78dbc53a.el9.x86_64 117/122 2024-03-20T07:51:27.177 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : python3-pytest-6.2.2-6.el9.noarch 118/122 2024-03-20T07:51:27.212 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : ceph-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 119/122 2024-03-20T07:51:27.234 INFO:teuthology.orchestra.run.smithi118.stdout: Installing : libcephfs-devel-2:18.2.2-904.g78dbc53a.el9.x86_6 120/122 2024-03-20T07:51:27.235 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 121/122 2024-03-20T07:51:27.335 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 121/122 2024-03-20T07:51:27.335 INFO:teuthology.orchestra.run.smithi118.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 122/122 2024-03-20T07:51:27.383 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/122 2024-03-20T07:51:27.471 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 85/122 2024-03-20T07:51:27.503 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 86/122 2024-03-20T07:51:27.536 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : jq-1.6-16.el9.x86_64 87/122 2024-03-20T07:51:27.826 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 88/122 2024-03-20T07:51:27.857 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:18.2.2-904.g78dbc53a.el9.x86_64 89/122 2024-03-20T07:51:27.947 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:18.2.2-904.g78dbc53a.el9.x86_64 89/122 2024-03-20T07:51:27.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:18.2.2-904.g78dbc53a.el9.x86_64 90/122 2024-03-20T07:51:28.015 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 91/122 2024-03-20T07:51:28.416 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 92/122 2024-03-20T07:51:28.443 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:18.2.2-904.g78dbc53a.el9.x86_64 93/122 2024-03-20T07:51:28.515 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:18.2.2-904.g78dbc53a.el9.x86_64 93/122 2024-03-20T07:51:28.544 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:18.2.2-904.g78dbc53a.el9.x86_64 94/122 2024-03-20T07:51:29.671 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 95/122 2024-03-20T07:51:29.700 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 95/122 2024-03-20T07:51:29.731 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-904.g78dbc53a.el 96/122 2024-03-20T07:51:29.808 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-904.g78dbc53a.e 97/122 2024-03-20T07:51:29.867 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:18.2.2-904.g78dbc53a.el9.x 98/122 2024-03-20T07:51:31.883 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 99/122 2024-03-20T07:51:31.921 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 99/122 2024-03-20T07:51:32.458 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 99/122 2024-03-20T07:51:32.491 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:18.2.2-904.g78dbc53a.el9.x86_64 100/122 2024-03-20T07:51:32.520 INFO:teuthology.orchestra.run.smithi118.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 122/122 2024-03-20T07:51:32.520 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-2:18.2.2-904.g78dbc53a.el9.x86_64 1/122 2024-03-20T07:51:32.520 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-base-2:18.2.2-904.g78dbc53a.el9.x86_64 2/122 2024-03-20T07:51:32.520 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 3/122 2024-03-20T07:51:32.520 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 4/122 2024-03-20T07:51:32.520 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-904.g78dbc5 5/122 2024-03-20T07:51:32.520 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mds-2:18.2.2-904.g78dbc53a.el9.x86_64 6/122 2024-03-20T07:51:32.520 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-2:18.2.2-904.g78dbc53a.el9.x86_64 7/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mon-2:18.2.2-904.g78dbc53a.el9.x86_64 8/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-osd-2:18.2.2-904.g78dbc53a.el9.x86_64 9/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-radosgw-2:18.2.2-904.g78dbc53a.el9.x86_64 10/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-selinux-2:18.2.2-904.g78dbc53a.el9.x86_64 11/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-test-2:18.2.2-904.g78dbc53a.el9.x86_64 12/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs-devel-2:18.2.2-904.g78dbc53a.el9.x86_6 13/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephfs2-2:18.2.2-904.g78dbc53a.el9.x86_64 14/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libcephsqlite-2:18.2.2-904.g78dbc53a.el9.x86_64 15/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados-devel-2:18.2.2-904.g78dbc53a.el9.x86_64 16/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libradosstriper1-2:18.2.2-904.g78dbc53a.el9.x86_ 17/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librgw2-2:18.2.2-904.g78dbc53a.el9.x86_64 18/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-argparse-2:18.2.2-904.g78dbc53a.el9 19/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ceph-common-2:18.2.2-904.g78dbc53a.el9.x 20/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cephfs-2:18.2.2-904.g78dbc53a.el9.x86_64 21/122 2024-03-20T07:51:32.521 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rados-2:18.2.2-904.g78dbc53a.el9.x86_64 22/122 2024-03-20T07:51:32.522 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rbd-2:18.2.2-904.g78dbc53a.el9.x86_64 23/122 2024-03-20T07:51:32.522 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rgw-2:18.2.2-904.g78dbc53a.el9.x86_64 24/122 2024-03-20T07:51:32.522 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 25/122 2024-03-20T07:51:32.522 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-mirror-2:18.2.2-904.g78dbc53a.el9.x86_64 26/122 2024-03-20T07:51:32.522 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : rbd-nbd-2:18.2.2-904.g78dbc53a.el9.x86_64 27/122 2024-03-20T07:51:32.522 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-904.g78dbc53a.e 28/122 2024-03-20T07:51:32.522 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-904.g78dbc53a.el9.noar 29/122 2024-03-20T07:51:32.522 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-904.g78dbc53a.el9.no 30/122 2024-03-20T07:51:32.530 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-904.g78db 31/122 2024-03-20T07:51:32.530 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-904.g78dbc53a.el9 32/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-mgr-rook-2:18.2.2-904.g78dbc53a.el9.noarch 33/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-904.g78dbc53a.el 34/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 35/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 36/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 37/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 38/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 39/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 40/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 41/122 2024-03-20T07:51:32.531 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 42/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 43/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 44/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 45/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 46/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 47/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 48/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 49/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 50/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 51/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 52/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 53/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 54/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 55/122 2024-03-20T07:51:32.532 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 56/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 57/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 58/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 59/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 60/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 61/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 62/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pytest-6.2.2-6.el9.noarch 63/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 64/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 65/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 66/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 67/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 68/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 69/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 70/122 2024-03-20T07:51:32.533 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 71/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 72/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-ply-3.11-14.el9.noarch 73/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 74/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 75/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : jq-1.6-16.el9.x86_64 76/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 77/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 78/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 79/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 80/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 81/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 82/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 83/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 84/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 85/122 2024-03-20T07:51:32.534 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 86/122 2024-03-20T07:51:32.535 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 87/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 88/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 89/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 90/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 91/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 92/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 93/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 94/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 95/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 96/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 97/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 98/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 99/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 100/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 101/122 2024-03-20T07:51:32.536 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 102/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 103/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 104/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 105/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 106/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 107/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 108/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 109/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 110/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 111/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 112/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 113/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 114/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 115/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 116/122 2024-03-20T07:51:32.537 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : re2-1:20211101-3.el9.x86_64 117/122 2024-03-20T07:51:32.538 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 118/122 2024-03-20T07:51:32.538 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:18.2.2-904.g78dbc53a.el9.x86_64 119/122 2024-03-20T07:51:32.538 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 120/122 2024-03-20T07:51:32.538 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:18.2.2-904.g78dbc53a.el9.x86_64 121/122 2024-03-20T07:51:32.574 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:18.2.2-904.g78dbc53a.el9.x86_64 100/122 2024-03-20T07:51:32.574 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-20T07:51:32.574 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-20T07:51:32.574 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:32.609 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:18.2.2-904.g78dbc53a.el9.x86_64 101/122 2024-03-20T07:51:34.508 INFO:teuthology.orchestra.run.smithi118.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 122/122 2024-03-20T07:51:34.508 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:34.508 INFO:teuthology.orchestra.run.smithi118.stdout:Upgraded: 2024-03-20T07:51:34.508 INFO:teuthology.orchestra.run.smithi118.stdout: librados2-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.508 INFO:teuthology.orchestra.run.smithi118.stdout: librbd1-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.509 INFO:teuthology.orchestra.run.smithi118.stdout:Installed: 2024-03-20T07:51:34.509 INFO:teuthology.orchestra.run.smithi118.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-20T07:51:34.509 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.509 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-base-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.509 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.509 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.510 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-grafana-dashboards-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:34.510 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-immutable-object-cache-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.510 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mds-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.510 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.510 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:34.510 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-dashboard-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:34.511 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-diskprediction-local-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:34.511 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-modules-core-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:34.511 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mgr-rook-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:34.511 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-mon-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.511 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-osd-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.511 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-prometheus-alerts-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:34.512 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-radosgw-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.512 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-selinux-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.512 INFO:teuthology.orchestra.run.smithi118.stdout: ceph-test-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.512 INFO:teuthology.orchestra.run.smithi118.stdout: cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:34.512 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-20T07:51:34.512 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-20T07:51:34.513 INFO:teuthology.orchestra.run.smithi118.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-20T07:51:34.513 INFO:teuthology.orchestra.run.smithi118.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-20T07:51:34.513 INFO:teuthology.orchestra.run.smithi118.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-20T07:51:34.513 INFO:teuthology.orchestra.run.smithi118.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-20T07:51:34.513 INFO:teuthology.orchestra.run.smithi118.stdout: jq-1.6-16.el9.x86_64 2024-03-20T07:51:34.513 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-20T07:51:34.514 INFO:teuthology.orchestra.run.smithi118.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-20T07:51:34.514 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs-devel-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.514 INFO:teuthology.orchestra.run.smithi118.stdout: libcephfs2-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.514 INFO:teuthology.orchestra.run.smithi118.stdout: libcephsqlite-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.514 INFO:teuthology.orchestra.run.smithi118.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-20T07:51:34.515 INFO:teuthology.orchestra.run.smithi118.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-20T07:51:34.515 INFO:teuthology.orchestra.run.smithi118.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-20T07:51:34.515 INFO:teuthology.orchestra.run.smithi118.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-20T07:51:34.515 INFO:teuthology.orchestra.run.smithi118.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-20T07:51:34.515 INFO:teuthology.orchestra.run.smithi118.stdout: librados-devel-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.516 INFO:teuthology.orchestra.run.smithi118.stdout: libradosstriper1-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.516 INFO:teuthology.orchestra.run.smithi118.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-20T07:51:34.516 INFO:teuthology.orchestra.run.smithi118.stdout: librgw2-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.516 INFO:teuthology.orchestra.run.smithi118.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-20T07:51:34.516 INFO:teuthology.orchestra.run.smithi118.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-20T07:51:34.516 INFO:teuthology.orchestra.run.smithi118.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-20T07:51:34.517 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-20T07:51:34.517 INFO:teuthology.orchestra.run.smithi118.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-20T07:51:34.517 INFO:teuthology.orchestra.run.smithi118.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-20T07:51:34.517 INFO:teuthology.orchestra.run.smithi118.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-20T07:51:34.517 INFO:teuthology.orchestra.run.smithi118.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-20T07:51:34.517 INFO:teuthology.orchestra.run.smithi118.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-20T07:51:34.518 INFO:teuthology.orchestra.run.smithi118.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-20T07:51:34.518 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-20T07:51:34.518 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-argparse-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.518 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.519 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cephfs-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.519 INFO:teuthology.orchestra.run.smithi118.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-20T07:51:34.519 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-20T07:51:34.519 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-20T07:51:34.519 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-20T07:51:34.519 INFO:teuthology.orchestra.run.smithi118.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-20T07:51:34.520 INFO:teuthology.orchestra.run.smithi118.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-20T07:51:34.520 INFO:teuthology.orchestra.run.smithi118.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-20T07:51:34.520 INFO:teuthology.orchestra.run.smithi118.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2024-03-20T07:51:34.520 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-20T07:51:34.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-20T07:51:34.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-20T07:51:34.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-20T07:51:34.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-20T07:51:34.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-20T07:51:34.521 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-20T07:51:34.522 INFO:teuthology.orchestra.run.smithi118.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-20T07:51:34.522 INFO:teuthology.orchestra.run.smithi118.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-20T07:51:34.522 INFO:teuthology.orchestra.run.smithi118.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-20T07:51:34.522 INFO:teuthology.orchestra.run.smithi118.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-20T07:51:34.522 INFO:teuthology.orchestra.run.smithi118.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-20T07:51:34.522 INFO:teuthology.orchestra.run.smithi118.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-20T07:51:34.523 INFO:teuthology.orchestra.run.smithi118.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-20T07:51:34.523 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-20T07:51:34.523 INFO:teuthology.orchestra.run.smithi118.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-20T07:51:34.523 INFO:teuthology.orchestra.run.smithi118.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-20T07:51:34.523 INFO:teuthology.orchestra.run.smithi118.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-20T07:51:34.524 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-20T07:51:34.524 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pluggy-0.13.1-7.el9.noarch 2024-03-20T07:51:34.524 INFO:teuthology.orchestra.run.smithi118.stdout: python3-ply-3.11-14.el9.noarch 2024-03-20T07:51:34.524 INFO:teuthology.orchestra.run.smithi118.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-20T07:51:34.524 INFO:teuthology.orchestra.run.smithi118.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-20T07:51:34.524 INFO:teuthology.orchestra.run.smithi118.stdout: python3-py-1.10.0-6.el9.noarch 2024-03-20T07:51:34.525 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-20T07:51:34.525 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-20T07:51:34.525 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-20T07:51:34.525 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-20T07:51:34.525 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytest-6.2.2-6.el9.noarch 2024-03-20T07:51:34.525 INFO:teuthology.orchestra.run.smithi118.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-20T07:51:34.525 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rados-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rbd-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-20T07:51:34.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-20T07:51:34.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-20T07:51:34.526 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rgw-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-20T07:51:34.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-20T07:51:34.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-20T07:51:34.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-20T07:51:34.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-20T07:51:34.527 INFO:teuthology.orchestra.run.smithi118.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-20T07:51:34.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-20T07:51:34.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-20T07:51:34.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-20T07:51:34.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-20T07:51:34.528 INFO:teuthology.orchestra.run.smithi118.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-20T07:51:34.528 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.529 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-mirror-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.529 INFO:teuthology.orchestra.run.smithi118.stdout: rbd-nbd-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:34.529 INFO:teuthology.orchestra.run.smithi118.stdout: re2-1:20211101-3.el9.x86_64 2024-03-20T07:51:34.529 INFO:teuthology.orchestra.run.smithi118.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-20T07:51:34.529 INFO:teuthology.orchestra.run.smithi118.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-20T07:51:34.529 INFO:teuthology.orchestra.run.smithi118.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-20T07:51:34.530 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:51:34.530 INFO:teuthology.orchestra.run.smithi118.stdout:Complete! 2024-03-20T07:51:34.813 DEBUG:teuthology.parallel:result is None 2024-03-20T07:51:44.019 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:18.2.2-904.g78dbc53a.el9.x86_64 101/122 2024-03-20T07:51:44.019 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-03-20T07:51:44.020 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-03-20T07:51:44.020 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-03-20T07:51:44.020 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-03-20T07:51:44.020 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-03-20T07:51:44.020 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-03-20T07:51:44.020 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-03-20T07:51:44.020 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:44.098 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-904.g78dbc53a.el9.noar 102/122 2024-03-20T07:51:44.326 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-904.g78dbc53a.el9.noar 102/122 2024-03-20T07:51:44.357 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-904.g78dbc53a.el9.no 103/122 2024-03-20T07:51:45.335 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-904.g78dbc53a.el9.no 103/122 2024-03-20T07:51:45.360 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-904.g78db 104/122 2024-03-20T07:51:45.459 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-904.g78db 104/122 2024-03-20T07:51:45.622 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-904.g78dbc53a.el9 105/122 2024-03-20T07:51:45.645 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:18.2.2-904.g78dbc53a.el9.x86_64 106/122 2024-03-20T07:51:45.668 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:18.2.2-904.g78dbc53a.el9.x86_64 106/122 2024-03-20T07:51:45.668 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:45.668 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-20T07:51:45.668 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-20T07:51:45.668 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-20T07:51:45.668 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:45.712 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:18.2.2-904.g78dbc53a.el9.noarch 107/122 2024-03-20T07:51:45.900 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-904.g78dbc53a.el9.noarch 107/122 2024-03-20T07:51:45.921 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:18.2.2-904.g78dbc53a.el9.x86_64 108/122 2024-03-20T07:51:45.944 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:18.2.2-904.g78dbc53a.el9.x86_64 108/122 2024-03-20T07:51:45.944 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:45.944 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-20T07:51:45.944 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-20T07:51:45.944 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-20T07:51:45.944 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:46.397 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:18.2.2-904.g78dbc53a.el9.x86_64 109/122 2024-03-20T07:51:46.419 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:18.2.2-904.g78dbc53a.el9.x86_64 109/122 2024-03-20T07:51:46.419 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:46.419 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-20T07:51:46.420 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-20T07:51:46.420 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-20T07:51:46.420 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:47.914 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:18.2.2-904.g78dbc53a.el9.x86_64 110/122 2024-03-20T07:51:47.940 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:18.2.2-904.g78dbc53a.el9.x86_64 110/122 2024-03-20T07:51:47.940 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:47.940 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-20T07:51:47.940 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-20T07:51:47.940 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-20T07:51:47.940 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:48.661 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:18.2.2-904.g78dbc53a.el9.x86_64 111/122 2024-03-20T07:51:48.699 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:18.2.2-904.g78dbc53a.el9.x86_64 112/122 2024-03-20T07:51:48.722 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-904.g78dbc53a.el9.x86_64 112/122 2024-03-20T07:51:48.722 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:48.722 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-20T07:51:48.722 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-20T07:51:48.723 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-20T07:51:48.723 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:48.767 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-904.g78dbc5 113/122 2024-03-20T07:51:48.789 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-904.g78dbc5 113/122 2024-03-20T07:51:48.789 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:48.789 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-20T07:51:48.789 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:49.084 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:18.2.2-904.g78dbc53a.el9.x86_64 114/122 2024-03-20T07:51:49.106 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:18.2.2-904.g78dbc53a.el9.x86_64 114/122 2024-03-20T07:51:49.106 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-20T07:51:49.106 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-20T07:51:49.106 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-20T07:51:49.106 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-20T07:51:49.107 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:52.805 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:18.2.2-904.g78dbc53a.el9.x86_64 115/122 2024-03-20T07:51:52.856 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 116/122 2024-03-20T07:51:52.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:18.2.2-904.g78dbc53a.el9.x86_64 117/122 2024-03-20T07:51:53.085 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytest-6.2.2-6.el9.noarch 118/122 2024-03-20T07:51:53.128 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 119/122 2024-03-20T07:51:53.159 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:18.2.2-904.g78dbc53a.el9.x86_6 120/122 2024-03-20T07:51:53.159 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 121/122 2024-03-20T07:51:53.243 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 121/122 2024-03-20T07:51:53.243 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 122/122 2024-03-20T07:51:57.416 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 122/122 2024-03-20T07:51:57.416 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:18.2.2-904.g78dbc53a.el9.x86_64 1/122 2024-03-20T07:51:57.416 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:18.2.2-904.g78dbc53a.el9.x86_64 2/122 2024-03-20T07:51:57.416 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 3/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 4/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-904.g78dbc5 5/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:18.2.2-904.g78dbc53a.el9.x86_64 6/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:18.2.2-904.g78dbc53a.el9.x86_64 7/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:18.2.2-904.g78dbc53a.el9.x86_64 8/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:18.2.2-904.g78dbc53a.el9.x86_64 9/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:18.2.2-904.g78dbc53a.el9.x86_64 10/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:18.2.2-904.g78dbc53a.el9.x86_64 11/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:18.2.2-904.g78dbc53a.el9.x86_64 12/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:18.2.2-904.g78dbc53a.el9.x86_6 13/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:18.2.2-904.g78dbc53a.el9.x86_64 14/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:18.2.2-904.g78dbc53a.el9.x86_64 15/122 2024-03-20T07:51:57.417 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:18.2.2-904.g78dbc53a.el9.x86_64 16/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:18.2.2-904.g78dbc53a.el9.x86_ 17/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:18.2.2-904.g78dbc53a.el9.x86_64 18/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:18.2.2-904.g78dbc53a.el9 19/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:18.2.2-904.g78dbc53a.el9.x 20/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:18.2.2-904.g78dbc53a.el9.x86_64 21/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:18.2.2-904.g78dbc53a.el9.x86_64 22/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:18.2.2-904.g78dbc53a.el9.x86_64 23/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:18.2.2-904.g78dbc53a.el9.x86_64 24/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 25/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:18.2.2-904.g78dbc53a.el9.x86_64 26/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:18.2.2-904.g78dbc53a.el9.x86_64 27/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-904.g78dbc53a.e 28/122 2024-03-20T07:51:57.418 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-904.g78dbc53a.el9.noar 29/122 2024-03-20T07:51:57.427 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-904.g78dbc53a.el9.no 30/122 2024-03-20T07:51:57.427 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-904.g78db 31/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-904.g78dbc53a.el9 32/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:18.2.2-904.g78dbc53a.el9.noarch 33/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-904.g78dbc53a.el 34/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 35/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 36/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 37/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 38/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 39/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 40/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 41/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 42/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 43/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 44/122 2024-03-20T07:51:57.428 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 45/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 46/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 47/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 48/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 49/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 50/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 51/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 52/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 53/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 54/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 55/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 56/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 57/122 2024-03-20T07:51:57.429 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 58/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 59/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 60/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 61/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 62/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytest-6.2.2-6.el9.noarch 63/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 64/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 65/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 66/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 67/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 68/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 69/122 2024-03-20T07:51:57.430 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 70/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 71/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 72/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 73/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 74/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 75/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : jq-1.6-16.el9.x86_64 76/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 77/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 78/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 79/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 80/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 81/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 82/122 2024-03-20T07:51:57.431 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 83/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 84/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 85/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 86/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 87/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 88/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 89/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 90/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 91/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 92/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 93/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 94/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 95/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 96/122 2024-03-20T07:51:57.432 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 97/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 98/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 99/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 100/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 101/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 102/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 103/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 104/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 105/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 106/122 2024-03-20T07:51:57.433 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 107/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 108/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 109/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 110/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 111/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 112/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 113/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 114/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 115/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 116/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 117/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 118/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:18.2.2-904.g78dbc53a.el9.x86_64 119/122 2024-03-20T07:51:57.434 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 120/122 2024-03-20T07:51:57.435 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:18.2.2-904.g78dbc53a.el9.x86_64 121/122 2024-03-20T07:51:59.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 122/122 2024-03-20T07:51:59.387 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.388 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:18.2.2-904.g78dbc53a.el9.noarch 2024-03-20T07:51:59.389 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: jq-1.6-16.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.390 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-20T07:51:59.391 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-20T07:51:59.392 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-20T07:51:59.393 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-20T07:51:59.394 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pluggy-0.13.1-7.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-py-1.10.0-6.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytest-6.2.2-6.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-20T07:51:59.395 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-20T07:51:59.396 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:18.2.2-904.g78dbc53a.el9.x86_64 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:51:59.397 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-20T07:51:59.715 DEBUG:teuthology.parallel:result is None 2024-03-20T07:51:59.716 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-20T07:51:59.716 INFO:teuthology.packaging:ref: None 2024-03-20T07:51:59.716 INFO:teuthology.packaging:tag: None 2024-03-20T07:51:59.716 INFO:teuthology.packaging:branch: reef 2024-03-20T07:51:59.716 INFO:teuthology.packaging:sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T07:51:59.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-20T07:52:00.054 DEBUG:teuthology.orchestra.run.smithi118:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-20T07:52:00.080 INFO:teuthology.orchestra.run.smithi118.stdout:18.2.2-904.g78dbc53a.el9 2024-03-20T07:52:00.081 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-904.g78dbc53a.el9 2024-03-20T07:52:00.081 INFO:teuthology.task.install:The correct ceph version 18.2.2-904.g78dbc53a is installed. 2024-03-20T07:52:00.083 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-20T07:52:00.083 INFO:teuthology.packaging:ref: None 2024-03-20T07:52:00.083 INFO:teuthology.packaging:tag: None 2024-03-20T07:52:00.083 INFO:teuthology.packaging:branch: reef 2024-03-20T07:52:00.083 INFO:teuthology.packaging:sha1: e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T07:52:00.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-20T07:52:00.307 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-20T07:52:00.332 INFO:teuthology.orchestra.run.smithi196.stdout:18.2.2-904.g78dbc53a.el9 2024-03-20T07:52:00.333 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-904.g78dbc53a.el9 2024-03-20T07:52:00.333 INFO:teuthology.task.install:The correct ceph version 18.2.2-904.g78dbc53a is installed. 2024-03-20T07:52:00.335 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-20T07:52:00.335 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:52:00.335 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-20T07:52:00.370 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:52:00.370 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-20T07:52:00.405 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-20T07:52:00.405 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:52:00.405 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/daemon-helper 2024-03-20T07:52:00.439 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-20T07:52:00.507 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:52:00.507 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-03-20T07:52:00.537 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-20T07:52:00.605 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-20T07:52:00.606 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:52:00.606 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-20T07:52:00.634 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-20T07:52:00.702 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:52:00.702 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-20T07:52:00.731 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-20T07:52:00.801 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-20T07:52:00.801 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:52:00.801 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/usr/bin/stdin-killer 2024-03-20T07:52:00.828 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-20T07:52:00.896 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:52:00.896 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-03-20T07:52:00.924 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-20T07:52:00.993 INFO:teuthology.run_tasks:Running task print... 2024-03-20T07:52:01.021 INFO:teuthology.task.print:**** done install task... 2024-03-20T07:52:01.021 INFO:teuthology.run_tasks:Running task print... 2024-03-20T07:52:01.030 INFO:teuthology.task.print:**** done start installing reef cephadm ... 2024-03-20T07:52:01.030 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-20T07:52:01.136 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'create_rbd_pool': True, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae'} 2024-03-20T07:52:01.137 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-03-20T07:52:01.137 INFO:tasks.cephadm:Cluster fsid is bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:01.137 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-20T07:52:01.138 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.118', 'mon.c': '[v2:172.21.15.118:3301,v1:172.21.15.118:6790]', 'mon.b': '172.21.15.196'} 2024-03-20T07:52:01.138 INFO:tasks.cephadm:First mon is mon.a on smithi118 2024-03-20T07:52:01.138 INFO:tasks.cephadm:First mgr is y 2024-03-20T07:52:01.138 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-20T07:52:01.138 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname $(hostname -s) 2024-03-20T07:52:01.167 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-03-20T07:52:01.195 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-03-20T07:52:01.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T07:52:01.342 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77739/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2153-ge7e80a6a', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2153.ge7e80a6a'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-yuri5-testing-2024-03-18-1144/e7e80a6ac9849f0f23432d0d9832f096730308ae/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-18 20:22:32.637670', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-yuri5-testing-2024-03-18-1144', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-yuri5-testing-2024-03-18-1144/e7e80a6ac9849f0f23432d0d9832f096730308ae/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-20T07:52:01.574 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref reef, sha1 78dbc53aa74481debe12189646c683a43626f88b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-03-20T07:52:01.576 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/x86_64/flavors/default/cephadm 2024-03-20T07:52:01.576 DEBUG:teuthology.orchestra.run.smithi118:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-20T07:52:02.048 INFO:teuthology.orchestra.run.smithi118.stdout:-rw-r--r--. 1 ubuntu ubuntu 215104 Mar 20 07:52 /home/ubuntu/cephtest/cephadm 2024-03-20T07:52:02.048 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/reef/78dbc53aa74481debe12189646c683a43626f88b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-20T07:52:02.596 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 215104 Mar 20 07:52 /home/ubuntu/cephtest/cephadm 2024-03-20T07:52:02.596 DEBUG:teuthology.orchestra.run.smithi118:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-20T07:52:02.615 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-20T07:52:02.665 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-03-20T07:52:02.665 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-03-20T07:52:02.668 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-03-20T07:52:03.397 INFO:teuthology.orchestra.run.smithi118.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-20T07:52:03.523 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-20T07:52:28.404 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-03-20T07:52:28.405 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)", 2024-03-20T07:52:28.405 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94", 2024-03-20T07:52:28.405 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-03-20T07:52:28.405 INFO:teuthology.orchestra.run.smithi196.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b" 2024-03-20T07:52:28.405 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-03-20T07:52:28.405 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-03-20T07:52:29.930 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T07:52:29.930 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph_version": "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)", 2024-03-20T07:52:29.930 INFO:teuthology.orchestra.run.smithi118.stdout: "image_id": "3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94", 2024-03-20T07:52:29.931 INFO:teuthology.orchestra.run.smithi118.stdout: "repo_digests": [ 2024-03-20T07:52:29.931 INFO:teuthology.orchestra.run.smithi118.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b" 2024-03-20T07:52:29.931 INFO:teuthology.orchestra.run.smithi118.stdout: ] 2024-03-20T07:52:29.931 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T07:52:29.949 DEBUG:teuthology.orchestra.run.smithi118:> sudo mkdir -p /etc/ceph 2024-03-20T07:52:29.995 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-03-20T07:52:30.030 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 777 /etc/ceph 2024-03-20T07:52:30.057 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-03-20T07:52:30.100 INFO:tasks.cephadm:Writing seed config... 2024-03-20T07:52:30.101 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-20T07:52:30.101 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-20T07:52:30.101 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-20T07:52:30.101 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-20T07:52:30.102 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-20T07:52:30.102 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-03-20T07:52:30.102 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-20T07:52:30.102 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-20T07:52:30.102 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-20T07:52:30.102 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-20T07:52:30.102 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-20T07:52:30.103 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:52:30.103 DEBUG:teuthology.orchestra.run.smithi118:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-20T07:52:30.118 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bc2566e8-e68e-11ee-95c9-87774f69a715 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-03-20T07:52:30.119 DEBUG:teuthology.orchestra.run.smithi118:mon.a> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a.service 2024-03-20T07:52:30.161 DEBUG:teuthology.orchestra.run.smithi118:mgr.y> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y.service 2024-03-20T07:52:30.204 INFO:tasks.cephadm:Bootstrapping... 2024-03-20T07:52:30.204 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.118 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-20T07:52:30.361 INFO:teuthology.orchestra.run.smithi118.stdout:-------------------------------------------------------------------------------- 2024-03-20T07:52:30.361 INFO:teuthology.orchestra.run.smithi118.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', 'bc2566e8-e68e-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.118', '--skip-admin-label'] 2024-03-20T07:52:30.386 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-20T07:52:30.386 INFO:teuthology.orchestra.run.smithi118.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-20T07:52:30.386 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying podman|docker is present... 2024-03-20T07:52:30.409 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-20T07:52:30.409 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying lvm2 is present... 2024-03-20T07:52:30.409 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying time synchronization is in place... 2024-03-20T07:52:30.420 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-20T07:52:30.420 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-20T07:52:30.422 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-20T07:52:30.422 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-03-20T07:52:30.428 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout enabled 2024-03-20T07:52:30.434 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout active 2024-03-20T07:52:30.434 INFO:teuthology.orchestra.run.smithi118.stdout:Unit chronyd.service is enabled and running 2024-03-20T07:52:30.434 INFO:teuthology.orchestra.run.smithi118.stdout:Repeating the final host check... 2024-03-20T07:52:30.457 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-20T07:52:30.457 INFO:teuthology.orchestra.run.smithi118.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-20T07:52:30.457 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl is present 2024-03-20T07:52:30.457 INFO:teuthology.orchestra.run.smithi118.stdout:lvcreate is present 2024-03-20T07:52:30.463 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-20T07:52:30.463 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-20T07:52:30.468 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-20T07:52:30.468 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-03-20T07:52:30.474 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout enabled 2024-03-20T07:52:30.480 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout active 2024-03-20T07:52:30.480 INFO:teuthology.orchestra.run.smithi118.stdout:Unit chronyd.service is enabled and running 2024-03-20T07:52:30.480 INFO:teuthology.orchestra.run.smithi118.stdout:Host looks OK 2024-03-20T07:52:30.480 INFO:teuthology.orchestra.run.smithi118.stdout:Cluster fsid: bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:30.481 INFO:teuthology.orchestra.run.smithi118.stdout:Acquiring lock 140662232843936 on /run/cephadm/bc2566e8-e68e-11ee-95c9-87774f69a715.lock 2024-03-20T07:52:30.481 INFO:teuthology.orchestra.run.smithi118.stdout:Lock 140662232843936 acquired on /run/cephadm/bc2566e8-e68e-11ee-95c9-87774f69a715.lock 2024-03-20T07:52:30.481 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying IP 172.21.15.118 port 3300 ... 2024-03-20T07:52:30.481 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying IP 172.21.15.118 port 6789 ... 2024-03-20T07:52:30.481 INFO:teuthology.orchestra.run.smithi118.stdout:Base mon IP(s) is [172.21.15.118:3300, 172.21.15.118:6789], mon addrv is [v2:172.21.15.118:3300,v1:172.21.15.118:6789] 2024-03-20T07:52:30.484 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.118 metric 100 2024-03-20T07:52:30.484 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.118 metric 100 2024-03-20T07:52:30.486 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-20T07:52:30.488 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-20T07:52:30.488 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-03-20T07:52:30.491 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-20T07:52:30.491 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-20T07:52:30.491 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-20T07:52:30.491 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-20T07:52:30.491 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d725/64 scope link 2024-03-20T07:52:30.492 INFO:teuthology.orchestra.run.smithi118.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-20T07:52:30.492 INFO:teuthology.orchestra.run.smithi118.stdout:Mon IP `172.21.15.118` is in CIDR network `172.21.0.0/20` 2024-03-20T07:52:30.492 INFO:teuthology.orchestra.run.smithi118.stdout:Mon IP `172.21.15.118` is in CIDR network `172.21.0.0/20` 2024-03-20T07:52:30.492 INFO:teuthology.orchestra.run.smithi118.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-20T07:52:30.492 INFO:teuthology.orchestra.run.smithi118.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-20T07:52:30.492 INFO:teuthology.orchestra.run.smithi118.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-20T07:52:30.883 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stdout 3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 2024-03-20T07:52:30.883 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-03-20T07:52:30.883 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Getting image source signatures 2024-03-20T07:52:30.883 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Copying blob sha256:5d8213e7d777a86d7ed8872f5928591d359626eef106d514e74277d3aad8d543 2024-03-20T07:52:30.883 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-20T07:52:30.883 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Copying config sha256:3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 2024-03-20T07:52:30.883 INFO:teuthology.orchestra.run.smithi118.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-20T07:52:31.863 INFO:teuthology.orchestra.run.smithi118.stdout:ceph: stdout ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable) 2024-03-20T07:52:31.864 INFO:teuthology.orchestra.run.smithi118.stdout:Ceph version: ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable) 2024-03-20T07:52:31.864 INFO:teuthology.orchestra.run.smithi118.stdout:Extracting ceph user uid/gid from container image... 2024-03-20T07:52:32.894 INFO:teuthology.orchestra.run.smithi118.stdout:stat: stdout 167 167 2024-03-20T07:52:32.895 INFO:teuthology.orchestra.run.smithi118.stdout:Creating initial keys... 2024-03-20T07:52:33.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph-authtool: stdout AQDBlfplGL3QDhAAHYHLm1uuoP1SUiV4oyYDjA== 2024-03-20T07:52:34.780 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph-authtool: stdout AQDClfplCF2yAxAAKnQBnV+jZMT2e3hUdMK98A== 2024-03-20T07:52:35.619 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph-authtool: stdout AQDDlfpll5eRBRAA00/lRFSzK9ktS5MGriB5+Q== 2024-03-20T07:52:35.620 INFO:teuthology.orchestra.run.smithi118.stdout:Creating initial monmap... 2024-03-20T07:52:36.625 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout:monmaptool for a [v2:172.21.15.118:3300,v1:172.21.15.118:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout:setting min_mon_release = pacific 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: set fsid to bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:52:36.626 INFO:teuthology.orchestra.run.smithi118.stdout:Creating mon... 2024-03-20T07:52:37.706 INFO:teuthology.orchestra.run.smithi118.stdout:create mon.a on 2024-03-20T07:52:37.915 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-20T07:52:38.190 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-20T07:52:38.586 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715.target. 2024-03-20T07:52:38.586 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715.target. 2024-03-20T07:52:39.462 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a 2024-03-20T07:52:39.463 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-20T07:52:39.728 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715.target.wants/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@.service. 2024-03-20T07:52:40.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:40 smithi118 podman[26864]: 2024-03-20 07:52:39.903539192 +0000 UTC m=+0.018319250 image pull 3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 quay.ceph.io/ceph-ci/ceph:reef 2024-03-20T07:52:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:42 smithi118 podman[26864]: 2024-03-20 07:52:42.28131253 +0000 UTC m=+2.396092583 container create ebbe36039dd4dafbe34a2d3db31dead78c365588e3e01d0d1204e7f92deed9b2 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, RELEASE=reef-78dbc53, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-20T07:52:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:44 smithi118 podman[26864]: 2024-03-20 07:52:44.068237268 +0000 UTC m=+4.183017321 container init ebbe36039dd4dafbe34a2d3db31dead78c365588e3e01d0d1204e7f92deed9b2 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=reef-78dbc53, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-20T07:52:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:44 smithi118 podman[26864]: 2024-03-20 07:52:44.075448876 +0000 UTC m=+4.190228929 container start ebbe36039dd4dafbe34a2d3db31dead78c365588e3e01d0d1204e7f92deed9b2 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=reef-78dbc53, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-20T07:52:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:44 smithi118 ceph-mon[26901]: mkfs bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:44 smithi118 ceph-mon[26901]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-20T07:52:44.893 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-20T07:52:44.893 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout disabled 2024-03-20T07:52:44.900 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-20T07:52:44.900 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-03-20T07:52:44.900 INFO:teuthology.orchestra.run.smithi118.stdout:firewalld.service is not enabled 2024-03-20T07:52:44.900 INFO:teuthology.orchestra.run.smithi118.stdout:Not possible to enable service . firewalld.service is not available 2024-03-20T07:52:44.900 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mon to start... 2024-03-20T07:52:44.901 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mon... 2024-03-20T07:52:45.133 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:44 smithi118 bash[26864]: ebbe36039dd4dafbe34a2d3db31dead78c365588e3e01d0d1204e7f92deed9b2 2024-03-20T07:52:45.133 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:44 smithi118 systemd[1]: Started Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:52:45.133 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:45 smithi118 ceph-mon[26901]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-20T07:52:45.133 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:45 smithi118 ceph-mon[26901]: monmap e1: 1 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T07:52:45.133 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:45 smithi118 ceph-mon[26901]: fsmap 2024-03-20T07:52:45.133 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:45 smithi118 ceph-mon[26901]: osdmap e1: 0 total, 0 up, 0 in 2024-03-20T07:52:45.134 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:45 smithi118 ceph-mon[26901]: mgrmap e1: no daemons active 2024-03-20T07:52:48.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:48 smithi118 ceph-mon[26901]: from='client.? 172.21.15.118:0/2681569137' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-20T07:52:51.849 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout cluster: 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout id: bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout services: 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 3s) 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout data: 2024-03-20T07:52:51.851 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-20T07:52:51.852 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-20T07:52:51.852 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-20T07:52:51.852 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout pgs: 2024-03-20T07:52:51.852 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:52:51.852 INFO:teuthology.orchestra.run.smithi118.stdout:mon is available 2024-03-20T07:52:51.852 INFO:teuthology.orchestra.run.smithi118.stdout:Assimilating anything we can from ceph.conf... 2024-03-20T07:52:52.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:52 smithi118 ceph-mon[26901]: from='client.? 172.21.15.118:0/2824816596' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-20T07:52:52.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:52 smithi118 ceph-mon[26901]: from='client.? 172.21.15.118:0/2824816596' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [global] 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout fsid = bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.118:3300,v1:172.21.15.118:6789] 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-20T07:52:53.248 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:52:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-20T07:52:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-20T07:52:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:52:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [osd] 2024-03-20T07:52:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-20T07:52:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-20T07:52:53.249 INFO:teuthology.orchestra.run.smithi118.stdout:Generating new minimal ceph.conf... 2024-03-20T07:52:53.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:53 smithi118 ceph-mon[26901]: from='client.? 172.21.15.118:0/3708979774' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:52:54.262 INFO:teuthology.orchestra.run.smithi118.stdout:Restarting the monitor... 2024-03-20T07:52:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:54 smithi118 systemd[1]: Stopping Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:52:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[26878]: 2024-03-20T07:52:54.507+0000 7f0bac7db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T07:52:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[26878]: 2024-03-20T07:52:54.507+0000 7f0bac7db700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-20T07:52:55.020 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:54 smithi118 podman[27235]: 2024-03-20 07:52:54.703054005 +0000 UTC m=+0.280567772 container died ebbe36039dd4dafbe34a2d3db31dead78c365588e3e01d0d1204e7f92deed9b2 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=reef-78dbc53, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-20T07:52:55.020 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 podman[27235]: 2024-03-20 07:52:55.021612443 +0000 UTC m=+0.599126263 container cleanup ebbe36039dd4dafbe34a2d3db31dead78c365588e3e01d0d1204e7f92deed9b2 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=reef-78dbc53, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T07:52:55.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 bash[27235]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a 2024-03-20T07:52:55.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 podman[27254]: 2024-03-20 07:52:55.205857549 +0000 UTC m=+0.501037990 container remove ebbe36039dd4dafbe34a2d3db31dead78c365588e3e01d0d1204e7f92deed9b2 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=reef-78dbc53) 2024-03-20T07:52:55.646 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-20T07:52:55.646 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 systemd[1]: Stopped Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:52:55.646 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 systemd[1]: Starting Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:52:55.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 podman[27353]: 2024-03-20 07:52:55.55009263 +0000 UTC m=+0.018004856 image pull 3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 quay.ceph.io/ceph-ci/ceph:reef 2024-03-20T07:52:55.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 podman[27353]: 2024-03-20 07:52:55.648095119 +0000 UTC m=+0.116007329 container create 89e74e5abc841d5afb9c39a870540e6a5a79c4a0e8ad433c679d700f58ac2f54 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=reef-78dbc53, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-20T07:52:55.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 podman[27353]: 2024-03-20 07:52:55.850441405 +0000 UTC m=+0.318353621 container init 89e74e5abc841d5afb9c39a870540e6a5a79c4a0e8ad433c679d700f58ac2f54 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=reef-78dbc53, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-20T07:52:55.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 podman[27353]: 2024-03-20 07:52:55.855443606 +0000 UTC m=+0.323355823 container start 89e74e5abc841d5afb9c39a870540e6a5a79c4a0e8ad433c679d700f58ac2f54 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-78dbc53, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-20T07:52:55.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: set uid:gid to 167:167 (ceph:ceph) 2024-03-20T07:52:55.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable), process ceph-mon, pid 7 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: pidfile_write: ignore empty --pid-file 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: load: jerasure load: lrc 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: RocksDB version: 7.9.2 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Git sha 0 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Compile date 2024-03-19 18:37:42 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: DB SUMMARY 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: DB Session ID: KDEAZGD8ORXG3TJA339T 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: CURRENT file: CURRENT 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: IDENTITY file: IDENTITY 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 85857 ; 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.error_if_exists: 0 2024-03-20T07:52:55.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.create_if_missing: 0 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.paranoid_checks: 1 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.env: 0x55b194ad3720 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.fs: PosixFileSystem 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.info_log: 0x55b1963cd4e0 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_file_opening_threads: 16 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.statistics: (nil) 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.use_fsync: 0 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_log_file_size: 0 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-20T07:52:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.keep_log_file_num: 1000 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.recycle_log_file_num: 0 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.allow_fallocate: 1 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.allow_mmap_reads: 0 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.allow_mmap_writes: 0 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.use_direct_reads: 0 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.create_missing_column_families: 0 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.db_log_dir: 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.wal_dir: 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-20T07:52:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.advise_random_on_open: 1 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.db_write_buffer_size: 0 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.write_buffer_manager: 0x55b195646140 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.rate_limiter: (nil) 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.wal_recovery_mode: 2 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.enable_thread_tracking: 0 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.enable_pipelined_write: 0 2024-03-20T07:52:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.unordered_write: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.row_cache: None 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.wal_filter: None 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.allow_ingest_behind: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.two_write_queues: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.manual_wal_flush: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.wal_compression: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.atomic_flush: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-20T07:52:55.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.log_readahead_size: 0 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.best_efforts_recovery: 0 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.allow_data_in_errors: 0 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.db_host_id: __hostname__ 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_background_jobs: 2 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_background_compactions: -1 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_subcompactions: 1 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-20T07:52:55.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_total_wal_size: 0 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_open_files: -1 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bytes_per_sync: 0 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_readahead_size: 0 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_background_flushes: -1 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Compression algorithms supported: 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-20T07:52:55.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: kZSTD supported: 0 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: kXpressCompression supported: 0 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: kLZ4HCCompression supported: 1 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: kZlibCompression supported: 1 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: kSnappyCompression supported: 1 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: kLZ4Compression supported: 1 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: kBZip2Compression supported: 0 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.merge_operator: 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_filter: None 2024-03-20T07:52:55.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_filter_factory: None 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.sst_partitioner_factory: None 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b1963cd5e0) 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: cache_index_and_filter_blocks: 1 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: pin_top_level_index_and_filter: 1 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_type: 0 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: data_block_index_type: 0 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_shortening: 1 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: checksum: 4 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: no_block_cache: 0 2024-03-20T07:52:55.932 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache: 0x55b1956e71f0 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_name: BinnedLRUCache 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_options: 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: capacity : 536870912 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: num_shard_bits : 4 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: strict_capacity_limit : 0 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: high_pri_pool_ratio: 0.000 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_compressed: (nil) 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: persistent_cache: (nil) 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_size: 4096 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_size_deviation: 10 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_restart_interval: 16 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_block_restart_interval: 1 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: metadata_block_size: 4096 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: partition_filters: 0 2024-03-20T07:52:55.933 INFO:journalctl@ceph.mon.a.smithi118.stdout: use_delta_encoding: 1 2024-03-20T07:52:55.934 INFO:journalctl@ceph.mon.a.smithi118.stdout: filter_policy: bloomfilter 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: whole_key_filtering: 1 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: verify_compression: 0 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: read_amp_bytes_per_bit: 0 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: format_version: 5 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: enable_index_compression: 1 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_align: 0 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: max_auto_readahead_size: 262144 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: prepopulate_block_cache: 0 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: initial_auto_readahead_size: 8192 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout: num_file_reads_for_auto_readahead: 2 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.write_buffer_size: 33554432 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_write_buffer_number: 2 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression: NoCompression 2024-03-20T07:52:55.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression: Disabled 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.prefix_extractor: nullptr 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.num_levels: 7 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-20T07:52:55.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.level: 32767 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.strategy: 0 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.enabled: false 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.target_file_size_base: 67108864 2024-03-20T07:52:55.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.arena_block_size: 1048576 2024-03-20T07:52:55.938 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.disable_auto_compactions: 0 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.table_properties_collectors: 2024-03-20T07:52:55.939 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.inplace_update_support: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.bloom_locality: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.max_successive_merges: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.paranoid_file_checks: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.force_consistency_checks: 1 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.report_bg_io_stats: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.ttl: 2592000 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.enable_blob_files: false 2024-03-20T07:52:55.940 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.min_blob_size: 0 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.blob_file_size: 268435456 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.blob_file_starting_level: 0 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 337d34b6-ad27-444c-b877-fa393755793d 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921175884485, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-20T07:52:55.941 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921175885335, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81784, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 280, "table_properties": {"data_size": 79941, "index_size": 253, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10329, "raw_average_key_size": 47, "raw_value_size": 74070, "raw_average_value_size": 338, "num_data_blocks": 11, "num_entries": 219, "num_filter_entries": 219, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710921175, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "337d34b6-ad27-444c-b877-fa393755793d", "db_session_id": "KDEAZGD8ORXG3TJA339T", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921175885415, "job": 1, "event": "recovery_finished"} 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b195784000 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: DB pointer 0x55b195772000 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** DB Stats ** 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-20T07:52:55.942 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** Compaction Stats [default] ** 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: L0 2/0 81.70 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: Sum 2/0 81.70 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** Compaction Stats [default] ** 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-20T07:52:55.943 INFO:journalctl@ceph.mon.a.smithi118.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative compaction: 0.00 GB write, 28.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval compaction: 0.00 GB write, 28.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: Block cache BinnedLRUCache@0x55b1956e71f0#7 capacity: 512.00 MB usage: 1.20 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-03-20T07:52:55.944 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: starting mon.a rank 0 at public addrs [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] at bind addrs [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a@-1(???) e1 preinit fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a@-1(???).mds e1 new map 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a@-1(???).mds e1 print_map 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout: e1 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout: legacy client fscid: -1 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout: No filesystems configured 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:52:55.945 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:52:55.946 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-20T07:52:55.961 INFO:teuthology.orchestra.run.smithi118.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-20T07:52:56.208 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-20T07:52:56.209 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: monmap e1: 1 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T07:52:56.209 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: fsmap 2024-03-20T07:52:56.209 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: osdmap e1: 0 total, 0 up, 0 in 2024-03-20T07:52:56.209 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 ceph-mon[27391]: mgrmap e1: no daemons active 2024-03-20T07:52:56.209 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 bash[27353]: 89e74e5abc841d5afb9c39a870540e6a5a79c4a0e8ad433c679d700f58ac2f54 2024-03-20T07:52:56.209 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:55 smithi118 systemd[1]: Started Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:52:57.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:52:57 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1640132985' entity='client.admin' 2024-03-20T07:52:59.667 INFO:teuthology.orchestra.run.smithi118.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-20T07:52:59.679 INFO:teuthology.orchestra.run.smithi118.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-20T07:52:59.679 INFO:teuthology.orchestra.run.smithi118.stdout:Creating mgr... 2024-03-20T07:52:59.680 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-20T07:52:59.680 INFO:teuthology.orchestra.run.smithi118.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-20T07:52:59.933 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y 2024-03-20T07:52:59.933 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y.service: Unit ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y.service not loaded. 2024-03-20T07:53:00.075 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715.target.wants/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@.service. 2024-03-20T07:53:00.307 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:00 smithi118 systemd[1]: Starting Ceph mgr.y for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:53:00.617 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:00 smithi118 podman[27631]: 2024-03-20 07:53:00.308489934 +0000 UTC m=+0.111558864 container create 64dace5519ed3ce907358ac484af4ed5a9035b244e9ecf041a247c392b5a8c3a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-78dbc53, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T07:53:00.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:00 smithi118 podman[27631]: 2024-03-20 07:53:00.213756275 +0000 UTC m=+0.016825174 image pull 3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 quay.ceph.io/ceph-ci/ceph:reef 2024-03-20T07:53:00.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:00 smithi118 podman[27631]: 2024-03-20 07:53:00.511125707 +0000 UTC m=+0.314194605 container init 64dace5519ed3ce907358ac484af4ed5a9035b244e9ecf041a247c392b5a8c3a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-78dbc53, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T07:53:00.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:00 smithi118 podman[27631]: 2024-03-20 07:53:00.516893801 +0000 UTC m=+0.319962696 container start 64dace5519ed3ce907358ac484af4ed5a9035b244e9ecf041a247c392b5a8c3a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=reef-78dbc53, ceph=True) 2024-03-20T07:53:00.618 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:00 smithi118 bash[27631]: 64dace5519ed3ce907358ac484af4ed5a9035b244e9ecf041a247c392b5a8c3a 2024-03-20T07:53:00.632 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-20T07:53:00.632 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout disabled 2024-03-20T07:53:00.639 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-20T07:53:00.639 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-03-20T07:53:00.639 INFO:teuthology.orchestra.run.smithi118.stdout:firewalld.service is not enabled 2024-03-20T07:53:00.639 INFO:teuthology.orchestra.run.smithi118.stdout:Not possible to enable service . firewalld.service is not available 2024-03-20T07:53:00.648 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-20T07:53:00.648 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout disabled 2024-03-20T07:53:00.655 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-20T07:53:00.656 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-03-20T07:53:00.656 INFO:teuthology.orchestra.run.smithi118.stdout:firewalld.service is not enabled 2024-03-20T07:53:00.656 INFO:teuthology.orchestra.run.smithi118.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-20T07:53:00.656 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mgr to start... 2024-03-20T07:53:00.656 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mgr... 2024-03-20T07:53:00.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:00 smithi118 systemd[1]: Started Ceph mgr.y for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:53:01.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:01.074+0000 7f18ef9fc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T07:53:01.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:01.171+0000 7f18ef9fc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T07:53:01.765 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsid": "bc2566e8-e68e-11ee-95c9-87774f69a715", 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "health": { 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-20T07:53:01.766 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 0 2024-03-20T07:53:01.767 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:01.767 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-20T07:53:01.767 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "a" 2024-03-20T07:53:01.767 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:01.769 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-20T07:53:01.770 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "restful" 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:01.771 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modified": "2024-03-20T07:52:44.130664+0000", 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-03-20T07:53:01.772 INFO:teuthology.orchestra.run.smithi118.stdout:mgr not available, waiting (1/15)... 2024-03-20T07:53:02.043 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:01 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2424841944' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-20T07:53:02.043 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:01.812+0000 7f18ef9fc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T07:53:02.043 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:01.903+0000 7f18ef9fc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T07:53:02.043 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:02.043+0000 7f18ef9fc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T07:53:02.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:02.138+0000 7f18ef9fc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T07:53:02.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:02.221+0000 7f18ef9fc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T07:53:02.799 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:02.497+0000 7f18ef9fc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T07:53:02.799 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:02.614+0000 7f18ef9fc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T07:53:02.800 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:02.702+0000 7f18ef9fc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T07:53:02.800 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:02.799+0000 7f18ef9fc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T07:53:03.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:02.978+0000 7f18ef9fc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T07:53:04.143 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:03.966+0000 7f18ef9fc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T07:53:04.144 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:04.102+0000 7f18ef9fc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T07:53:04.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:04 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1377957597' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-20T07:53:04.727 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:04.441+0000 7f18ef9fc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T07:53:04.728 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:04.531+0000 7f18ef9fc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T07:53:04.728 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:04.728+0000 7f18ef9fc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:53:04.908 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsid": "bc2566e8-e68e-11ee-95c9-87774f69a715", 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "health": { 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 0 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-20T07:53:04.914 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "a" 2024-03-20T07:53:04.915 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:04.915 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-20T07:53:04.915 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-20T07:53:04.915 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:04.915 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-20T07:53:04.915 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-20T07:53:04.915 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:04.915 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-20T07:53:04.916 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-20T07:53:04.917 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "restful" 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modified": "2024-03-20T07:52:44.130664+0000", 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-03-20T07:53:04.918 INFO:teuthology.orchestra.run.smithi118.stdout:mgr not available, waiting (2/15)... 2024-03-20T07:53:05.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:04.911+0000 7f18ef9fc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T07:53:05.446 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:05.191+0000 7f18ef9fc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:53:05.446 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:05.446+0000 7f18ef9fc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T07:53:05.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:05.678+0000 7f18ef9fc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T07:53:06.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:06.470+0000 7f18ef9fc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T07:53:07.284 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:07.028+0000 7f18ef9fc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T07:53:07.284 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:07.125+0000 7f18ef9fc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T07:53:07.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:07.354+0000 7f18ef9fc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T07:53:07.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: Activating manager daemon y 2024-03-20T07:53:07.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: mgrmap e2: y(active, starting, since 0.00388731s) 2024-03-20T07:53:07.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: Manager daemon y is now available 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' 2024-03-20T07:53:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:07 smithi118 ceph-mon[27391]: from='mgr.14100 172.21.15.118:0/1487068427' entity='mgr.y' 2024-03-20T07:53:07.982 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsid": "bc2566e8-e68e-11ee-95c9-87774f69a715", 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "health": { 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 0 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "a" 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:07.983 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-20T07:53:07.984 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-20T07:53:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-20T07:53:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-20T07:53:07.988 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-20T07:53:07.989 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "restful" 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modified": "2024-03-20T07:52:44.130664+0000", 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-20T07:53:07.990 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:07.991 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-20T07:53:07.991 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-03-20T07:53:07.991 INFO:teuthology.orchestra.run.smithi118.stdout:mgr not available, waiting (3/15)... 2024-03-20T07:53:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:08 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2693983365' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-20T07:53:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:08 smithi118 ceph-mon[27391]: mgrmap e3: y(active, since 1.00726s) 2024-03-20T07:53:10.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:10 smithi118 ceph-mon[27391]: mgrmap e4: y(active, since 2s) 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsid": "bc2566e8-e68e-11ee-95c9-87774f69a715", 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "health": { 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-20T07:53:11.332 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 0 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "a" 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-20T07:53:11.333 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:11.334 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "restful" 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ], 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-20T07:53:11.335 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:11.336 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-20T07:53:11.336 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-20T07:53:11.336 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "modified": "2024-03-20T07:52:44.130664+0000", 2024-03-20T07:53:11.336 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-20T07:53:11.336 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout }, 2024-03-20T07:53:11.336 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-20T07:53:11.336 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-03-20T07:53:11.336 INFO:teuthology.orchestra.run.smithi118.stdout:mgr is available 2024-03-20T07:53:11.586 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:11 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3244919163' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-20T07:53:12.563 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:53:12.563 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [global] 2024-03-20T07:53:12.563 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout fsid = bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:53:12.563 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-20T07:53:12.563 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout [osd] 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-20T07:53:12.564 INFO:teuthology.orchestra.run.smithi118.stdout:Enabling cephadm module... 2024-03-20T07:53:12.669 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:12 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3207069451' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-20T07:53:13.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:13 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2612738609' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-20T07:53:13.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ignoring --setuser ceph since I am not root 2024-03-20T07:53:13.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ignoring --setgroup ceph since I am not root 2024-03-20T07:53:14.068 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:13.977+0000 7f2c9b2c5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T07:53:14.385 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:14.070+0000 7f2c9b2c5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T07:53:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:14 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2612738609' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-20T07:53:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:14 smithi118 ceph-mon[27391]: mgrmap e5: y(active, since 6s) 2024-03-20T07:53:15.056 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:14.724+0000 7f2c9b2c5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T07:53:15.056 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:14.819+0000 7f2c9b2c5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T07:53:15.056 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:14.958+0000 7f2c9b2c5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T07:53:15.056 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:15.057+0000 7f2c9b2c5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T07:53:15.347 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:15.139+0000 7f2c9b2c5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T07:53:15.350 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-03-20T07:53:15.350 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-20T07:53:15.350 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-20T07:53:15.350 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-20T07:53:15.350 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-20T07:53:15.350 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-03-20T07:53:15.350 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for the mgr to restart... 2024-03-20T07:53:15.351 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mgr epoch 5... 2024-03-20T07:53:15.601 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:15 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/156224643' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-20T07:53:15.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:15.408+0000 7f2c9b2c5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T07:53:15.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:15.519+0000 7f2c9b2c5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T07:53:15.883 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:15.609+0000 7f2c9b2c5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T07:53:15.883 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:15.704+0000 7f2c9b2c5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T07:53:16.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:15.884+0000 7f2c9b2c5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T07:53:17.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:16.827+0000 7f2c9b2c5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T07:53:17.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:16.954+0000 7f2c9b2c5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T07:53:17.561 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:17.280+0000 7f2c9b2c5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T07:53:17.561 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:17.369+0000 7f2c9b2c5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T07:53:17.561 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:17.561+0000 7f2c9b2c5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:53:17.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:17.742+0000 7f2c9b2c5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T07:53:18.275 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:18.022+0000 7f2c9b2c5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:53:18.276 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:18.275+0000 7f2c9b2c5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T07:53:18.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:18.506+0000 7f2c9b2c5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T07:53:19.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:19.290+0000 7f2c9b2c5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T07:53:20.157 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:19.848+0000 7f2c9b2c5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T07:53:20.157 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:19.945+0000 7f2c9b2c5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T07:53:20.157 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:20.157+0000 7f2c9b2c5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T07:53:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: Active manager daemon y restarted 2024-03-20T07:53:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: Activating manager daemon y 2024-03-20T07:53:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: osdmap e2: 0 total, 0 up, 0 in 2024-03-20T07:53:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: mgrmap e6: y(active, starting, since 0.00316131s) 2024-03-20T07:53:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:53:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T07:53:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T07:53:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T07:53:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T07:53:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: Manager daemon y is now available 2024-03-20T07:53:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:53:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:53:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:20 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:53:21.554 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:21 smithi118 ceph-mon[27391]: Found migration_current of "None". Setting to last migration. 2024-03-20T07:53:21.554 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:21 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:53:21.554 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:21 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:53:21.554 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:21 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:21.554 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:21 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:21.554 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:21 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:53:21.554 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:21 smithi118 ceph-mon[27391]: mgrmap e7: y(active, since 1.00538s) 2024-03-20T07:53:21.807 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-03-20T07:53:21.807 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-20T07:53:21.807 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-20T07:53:21.807 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-03-20T07:53:21.807 INFO:teuthology.orchestra.run.smithi118.stdout:mgr epoch 5 is available 2024-03-20T07:53:21.807 INFO:teuthology.orchestra.run.smithi118.stdout:Setting orchestrator backend to cephadm... 2024-03-20T07:53:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:22 smithi118 ceph-mon[27391]: [20/Mar/2024:07:53:20] ENGINE Bus STARTING 2024-03-20T07:53:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:22 smithi118 ceph-mon[27391]: [20/Mar/2024:07:53:20] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T07:53:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:22 smithi118 ceph-mon[27391]: [20/Mar/2024:07:53:20] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T07:53:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:22 smithi118 ceph-mon[27391]: [20/Mar/2024:07:53:20] ENGINE Bus STARTED 2024-03-20T07:53:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:22 smithi118 ceph-mon[27391]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-20T07:53:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:22 smithi118 ceph-mon[27391]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-20T07:53:23.300 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:23 smithi118 ceph-mon[27391]: mgrmap e8: y(active, since 2s) 2024-03-20T07:53:23.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:23 smithi118 ceph-mon[27391]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:23.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:23 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:23.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:23 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:53:24.370 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-20T07:53:24.370 INFO:teuthology.orchestra.run.smithi118.stdout:Generating ssh key... 2024-03-20T07:53:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:24 smithi118 ceph-mon[27391]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:25.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: Generating public/private rsa key pair. 2024-03-20T07:53:25.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: Your identification has been saved in /tmp/tmpx722pikv/key. 2024-03-20T07:53:25.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: Your public key has been saved in /tmp/tmpx722pikv/key.pub. 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: The key fingerprint is: 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: SHA256:oWWwqKnLCmIISVhrvqvku3auHWlwG7mSG7825gZgFW4 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: The key's randomart image is: 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: +---[RSA 3072]----+ 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: | ... . | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: |...o . o | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: |..+E. . + | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: |o=.o. + . | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: |=.++ . S | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: |oo+.= | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: |=*o* | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: |BoBB. | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: |*O%*o | 2024-03-20T07:53:25.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: +----[SHA256]-----+ 2024-03-20T07:53:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:26 smithi118 ceph-mon[27391]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:26 smithi118 ceph-mon[27391]: Generating ssh key... 2024-03-20T07:53:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:26 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:26 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:27.040 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjStTj7/QR61UPQ2Gxdze+c5naJjsvXVQhKKRLKVNct74ra+fZl/9m8nGgI6orJ1eIIGL7UpRzFcDkG9Qru7zYv/SBMaw4wAu/zA+vbG4Klpx0CqC/AKRBvcq6VhQXYxZsAMp6lm1zpxZCXWWHpTot1ShMC74Vma+NjMW4qHsuPa7JShn/Jo2zWbukNC5y/OkVN3feOA+0/n4fcVJPOWK7G3l5pfW28YKiRtUiPr9iGFW1VG8tQQqVaMXpGwoSb7zi5w/hSTgyukdz4hpSt/Mq0CkOPd26NFmWZq+vgYcSxLbFVvP2cEIsSU2EJdhbCbe8urYTPM06SHlBL8D3LzN/OnbY90WoR4azwPl9PJ7PiZZvk6M3yn3SR0myU4vEGcYieEetkvqVo4M1A4RI5Oryo0uJsQ3z1gKqUpMJxQYMZJiGMB5kwIJiQuB3Rko3e/eC4wLCf0hQt9VYfIxXKNgPCsFb9S+3OCewx879oEP8voYqG4XcSnGoEoMiMuHLOCE= ceph-bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:53:27.040 INFO:teuthology.orchestra.run.smithi118.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-20T07:53:27.041 INFO:teuthology.orchestra.run.smithi118.stdout:Adding key to root@localhost authorized_keys... 2024-03-20T07:53:27.041 INFO:teuthology.orchestra.run.smithi118.stdout:Adding host smithi118... 2024-03-20T07:53:27.295 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:27 smithi118 ceph-mon[27391]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:28.671 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:28 smithi118 ceph-mon[27391]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi118", "addr": "172.21.15.118", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:28.671 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:28 smithi118 ceph-mon[27391]: Deploying cephadm binary to smithi118 2024-03-20T07:53:30.791 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout Added host 'smithi118' with addr '172.21.15.118' 2024-03-20T07:53:30.791 INFO:teuthology.orchestra.run.smithi118.stdout:Deploying unmanaged mon service... 2024-03-20T07:53:31.315 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:31 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:31.315 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:31 smithi118 ceph-mon[27391]: Added host smithi118 2024-03-20T07:53:31.315 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:31 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:53:32.892 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:32 smithi118 ceph-mon[27391]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:32.892 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:32 smithi118 ceph-mon[27391]: Saving service mon spec with placement count:5 2024-03-20T07:53:32.892 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:32 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:32.893 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:32 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:32.895 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-20T07:53:32.895 INFO:teuthology.orchestra.run.smithi118.stdout:Deploying unmanaged mgr service... 2024-03-20T07:53:34.359 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:34 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:34.359 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:34 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:34.361 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-20T07:53:35.162 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:35 smithi118 ceph-mon[27391]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:35.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:35 smithi118 ceph-mon[27391]: Saving service mgr spec with placement count:2 2024-03-20T07:53:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:36 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3101106187' entity='client.admin' 2024-03-20T07:53:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:36 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:37.035 INFO:teuthology.orchestra.run.smithi118.stdout:Enabling the dashboard module... 2024-03-20T07:53:37.597 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:37 smithi118 ceph-mon[27391]: from='mgr.14120 172.21.15.118:0/1248076464' entity='mgr.y' 2024-03-20T07:53:37.597 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:37 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2550611762' entity='client.admin' 2024-03-20T07:53:38.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:38 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/4272546981' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-20T07:53:38.584 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ignoring --setuser ceph since I am not root 2024-03-20T07:53:38.584 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ignoring --setgroup ceph since I am not root 2024-03-20T07:53:39.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:38.898+0000 7f4ab2fb6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T07:53:39.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:38.992+0000 7f4ab2fb6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T07:53:39.472 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:39 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/4272546981' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-20T07:53:39.472 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:39 smithi118 ceph-mon[27391]: mgrmap e9: y(active, since 18s) 2024-03-20T07:53:39.733 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:39.637+0000 7f4ab2fb6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T07:53:39.733 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:39.734+0000 7f4ab2fb6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T07:53:40.010 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:39.880+0000 7f4ab2fb6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T07:53:40.010 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:39.976+0000 7f4ab2fb6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T07:53:40.318 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:40.063+0000 7f4ab2fb6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T07:53:40.620 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:40 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3046196096' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-20T07:53:40.621 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:40.328+0000 7f4ab2fb6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T07:53:40.621 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:40.439+0000 7f4ab2fb6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T07:53:40.621 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:40.525+0000 7f4ab2fb6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T07:53:40.621 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:40.621+0000 7f4ab2fb6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T07:53:40.638 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-03-20T07:53:40.638 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-20T07:53:40.638 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-20T07:53:40.638 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-20T07:53:40.639 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-20T07:53:40.639 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-03-20T07:53:40.639 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for the mgr to restart... 2024-03-20T07:53:40.639 INFO:teuthology.orchestra.run.smithi118.stdout:Waiting for mgr epoch 9... 2024-03-20T07:53:40.875 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:40.807+0000 7f4ab2fb6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T07:53:41.917 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:41.790+0000 7f4ab2fb6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T07:53:42.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:41.917+0000 7f4ab2fb6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T07:53:42.529 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:42.247+0000 7f4ab2fb6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T07:53:42.529 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:42.336+0000 7f4ab2fb6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T07:53:42.529 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:42.528+0000 7f4ab2fb6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:53:42.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:42.715+0000 7f4ab2fb6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T07:53:43.255 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:43.000+0000 7f4ab2fb6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:53:43.255 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:43.256+0000 7f4ab2fb6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T07:53:43.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:43.490+0000 7f4ab2fb6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T07:53:44.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:44.292+0000 7f4ab2fb6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T07:53:45.169 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:44.857+0000 7f4ab2fb6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T07:53:45.170 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:44.956+0000 7f4ab2fb6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T07:53:45.170 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:53:45.170+0000 7f4ab2fb6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T07:53:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: Active manager daemon y restarted 2024-03-20T07:53:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: Activating manager daemon y 2024-03-20T07:53:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: osdmap e3: 0 total, 0 up, 0 in 2024-03-20T07:53:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: mgrmap e10: y(active, starting, since 0.00360054s) 2024-03-20T07:53:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:53:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T07:53:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T07:53:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T07:53:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T07:53:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: Manager daemon y is now available 2024-03-20T07:53:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:53:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:53:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:53:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:53:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:46 smithi118 ceph-mon[27391]: mgrmap e11: y(active, since 1.00594s) 2024-03-20T07:53:47.131 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout { 2024-03-20T07:53:47.131 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-20T07:53:47.131 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-20T07:53:47.131 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout } 2024-03-20T07:53:47.131 INFO:teuthology.orchestra.run.smithi118.stdout:mgr epoch 9 is available 2024-03-20T07:53:47.132 INFO:teuthology.orchestra.run.smithi118.stdout:Generating a dashboard self-signed certificate... 2024-03-20T07:53:47.384 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:47 smithi118 ceph-mon[27391]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-20T07:53:47.384 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:47 smithi118 ceph-mon[27391]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-20T07:53:47.384 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:47 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:53:48.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:48 smithi118 ceph-mon[27391]: [20/Mar/2024:07:53:46] ENGINE Bus STARTING 2024-03-20T07:53:48.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:48 smithi118 ceph-mon[27391]: [20/Mar/2024:07:53:46] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T07:53:48.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:48 smithi118 ceph-mon[27391]: [20/Mar/2024:07:53:46] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T07:53:48.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:48 smithi118 ceph-mon[27391]: [20/Mar/2024:07:53:46] ENGINE Bus STARTED 2024-03-20T07:53:48.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:48 smithi118 ceph-mon[27391]: mgrmap e12: y(active, since 2s) 2024-03-20T07:53:48.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:53:48.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:53:48.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:53:48.429 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-20T07:53:48.429 INFO:teuthology.orchestra.run.smithi118.stdout:Creating initial admin user... 2024-03-20T07:53:49.282 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:49 smithi118 ceph-mon[27391]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:50 smithi118 ceph-mon[27391]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:53:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:53:50.683 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$UglW4YcdToiQSRRnGiOZ.e0a0NBxBVgE6pLaypHV/dKWG7V3olEim", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710921229, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-20T07:53:50.683 INFO:teuthology.orchestra.run.smithi118.stdout:Fetching dashboard port number... 2024-03-20T07:53:51.890 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stdout 8443 2024-03-20T07:53:51.900 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-20T07:53:51.900 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout disabled 2024-03-20T07:53:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-20T07:53:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:systemctl: stdout inactive 2024-03-20T07:53:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:firewalld.service is not enabled 2024-03-20T07:53:51.908 INFO:teuthology.orchestra.run.smithi118.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-20T07:53:51.910 INFO:teuthology.orchestra.run.smithi118.stdout:Ceph Dashboard is now available at: 2024-03-20T07:53:51.910 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:51.910 INFO:teuthology.orchestra.run.smithi118.stdout: URL: https://smithi118.front.sepia.ceph.com:8443/ 2024-03-20T07:53:51.910 INFO:teuthology.orchestra.run.smithi118.stdout: User: admin 2024-03-20T07:53:51.911 INFO:teuthology.orchestra.run.smithi118.stdout: Password: px6msgf5qf 2024-03-20T07:53:51.911 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:51.911 INFO:teuthology.orchestra.run.smithi118.stdout:Saving cluster configuration to /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config directory 2024-03-20T07:53:51.911 INFO:teuthology.orchestra.run.smithi118.stdout:Enabling autotune for osd_memory_target 2024-03-20T07:53:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:51 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2261568671' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-20T07:53:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:52 smithi118 ceph-mon[27391]: mgrmap e13: y(active, since 6s) 2024-03-20T07:53:54.544 INFO:teuthology.orchestra.run.smithi118.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-20T07:53:54.544 INFO:teuthology.orchestra.run.smithi118.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-20T07:53:54.544 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:54.544 INFO:teuthology.orchestra.run.smithi118.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-20T07:53:54.544 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:54.544 INFO:teuthology.orchestra.run.smithi118.stdout:Or, if you are only running a single cluster on this host: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: ceph telemetry on 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout:For more information see: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:53:54.545 INFO:teuthology.orchestra.run.smithi118.stdout:Bootstrap complete. 2024-03-20T07:53:54.581 INFO:tasks.cephadm:Fetching config... 2024-03-20T07:53:54.581 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:53:54.581 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-20T07:53:54.597 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-20T07:53:54.597 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:53:54.597 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-20T07:53:54.652 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-20T07:53:54.653 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:53:54.653 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-20T07:53:54.720 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-20T07:53:54.720 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:53:54.720 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-20T07:53:54.775 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-20T07:53:54.775 DEBUG:teuthology.orchestra.run.smithi118:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjStTj7/QR61UPQ2Gxdze+c5naJjsvXVQhKKRLKVNct74ra+fZl/9m8nGgI6orJ1eIIGL7UpRzFcDkG9Qru7zYv/SBMaw4wAu/zA+vbG4Klpx0CqC/AKRBvcq6VhQXYxZsAMp6lm1zpxZCXWWHpTot1ShMC74Vma+NjMW4qHsuPa7JShn/Jo2zWbukNC5y/OkVN3feOA+0/n4fcVJPOWK7G3l5pfW28YKiRtUiPr9iGFW1VG8tQQqVaMXpGwoSb7zi5w/hSTgyukdz4hpSt/Mq0CkOPd26NFmWZq+vgYcSxLbFVvP2cEIsSU2EJdhbCbe8urYTPM06SHlBL8D3LzN/OnbY90WoR4azwPl9PJ7PiZZvk6M3yn3SR0myU4vEGcYieEetkvqVo4M1A4RI5Oryo0uJsQ3z1gKqUpMJxQYMZJiGMB5kwIJiQuB3Rko3e/eC4wLCf0hQt9VYfIxXKNgPCsFb9S+3OCewx879oEP8voYqG4XcSnGoEoMiMuHLOCE= ceph-bc2566e8-e68e-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-20T07:53:54.853 INFO:teuthology.orchestra.run.smithi118.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjStTj7/QR61UPQ2Gxdze+c5naJjsvXVQhKKRLKVNct74ra+fZl/9m8nGgI6orJ1eIIGL7UpRzFcDkG9Qru7zYv/SBMaw4wAu/zA+vbG4Klpx0CqC/AKRBvcq6VhQXYxZsAMp6lm1zpxZCXWWHpTot1ShMC74Vma+NjMW4qHsuPa7JShn/Jo2zWbukNC5y/OkVN3feOA+0/n4fcVJPOWK7G3l5pfW28YKiRtUiPr9iGFW1VG8tQQqVaMXpGwoSb7zi5w/hSTgyukdz4hpSt/Mq0CkOPd26NFmWZq+vgYcSxLbFVvP2cEIsSU2EJdhbCbe8urYTPM06SHlBL8D3LzN/OnbY90WoR4azwPl9PJ7PiZZvk6M3yn3SR0myU4vEGcYieEetkvqVo4M1A4RI5Oryo0uJsQ3z1gKqUpMJxQYMZJiGMB5kwIJiQuB3Rko3e/eC4wLCf0hQt9VYfIxXKNgPCsFb9S+3OCewx879oEP8voYqG4XcSnGoEoMiMuHLOCE= ceph-bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:53:54.866 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjStTj7/QR61UPQ2Gxdze+c5naJjsvXVQhKKRLKVNct74ra+fZl/9m8nGgI6orJ1eIIGL7UpRzFcDkG9Qru7zYv/SBMaw4wAu/zA+vbG4Klpx0CqC/AKRBvcq6VhQXYxZsAMp6lm1zpxZCXWWHpTot1ShMC74Vma+NjMW4qHsuPa7JShn/Jo2zWbukNC5y/OkVN3feOA+0/n4fcVJPOWK7G3l5pfW28YKiRtUiPr9iGFW1VG8tQQqVaMXpGwoSb7zi5w/hSTgyukdz4hpSt/Mq0CkOPd26NFmWZq+vgYcSxLbFVvP2cEIsSU2EJdhbCbe8urYTPM06SHlBL8D3LzN/OnbY90WoR4azwPl9PJ7PiZZvk6M3yn3SR0myU4vEGcYieEetkvqVo4M1A4RI5Oryo0uJsQ3z1gKqUpMJxQYMZJiGMB5kwIJiQuB3Rko3e/eC4wLCf0hQt9VYfIxXKNgPCsFb9S+3OCewx879oEP8voYqG4XcSnGoEoMiMuHLOCE= ceph-bc2566e8-e68e-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-20T07:53:54.913 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjStTj7/QR61UPQ2Gxdze+c5naJjsvXVQhKKRLKVNct74ra+fZl/9m8nGgI6orJ1eIIGL7UpRzFcDkG9Qru7zYv/SBMaw4wAu/zA+vbG4Klpx0CqC/AKRBvcq6VhQXYxZsAMp6lm1zpxZCXWWHpTot1ShMC74Vma+NjMW4qHsuPa7JShn/Jo2zWbukNC5y/OkVN3feOA+0/n4fcVJPOWK7G3l5pfW28YKiRtUiPr9iGFW1VG8tQQqVaMXpGwoSb7zi5w/hSTgyukdz4hpSt/Mq0CkOPd26NFmWZq+vgYcSxLbFVvP2cEIsSU2EJdhbCbe8urYTPM06SHlBL8D3LzN/OnbY90WoR4azwPl9PJ7PiZZvk6M3yn3SR0myU4vEGcYieEetkvqVo4M1A4RI5Oryo0uJsQ3z1gKqUpMJxQYMZJiGMB5kwIJiQuB3Rko3e/eC4wLCf0hQt9VYfIxXKNgPCsFb9S+3OCewx879oEP8voYqG4XcSnGoEoMiMuHLOCE= ceph-bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:53:54.928 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-20T07:53:55.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:55 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/749504297' entity='client.admin' 2024-03-20T07:53:55.208 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:53:58.638 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-20T07:53:58.639 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-20T07:53:58.911 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:53:58 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1221524846' entity='client.admin' 2024-03-20T07:53:59.324 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:54:00.459 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:00.460 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:00.460 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi118", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:54:00.460 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:00.460 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:00.460 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:01.992 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-03-20T07:54:01.993 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:54:01.993 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-03-20T07:54:02.011 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:54:02.011 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:54:02.069 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-03-20T07:54:02.069 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch host add smithi196 2024-03-20T07:54:02.282 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:54:02.404 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:02.405 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:02.405 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:02.405 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:02.405 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:02.405 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T07:54:02.405 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:02.405 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:02.405 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:03.392 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:03 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:03.392 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:03 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:54:03.392 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:03 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:54:04.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:04 smithi118 ceph-mon[27391]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:06 smithi118 ceph-mon[27391]: Deploying cephadm binary to smithi196 2024-03-20T07:54:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:06 smithi118 ceph-mon[27391]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:06.691 INFO:teuthology.orchestra.run.smithi118.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-03-20T07:54:07.346 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-20T07:54:07.551 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:54:07.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:07 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:07.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:07 smithi118 ceph-mon[27391]: Added host smithi196 2024-03-20T07:54:07.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:07 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:07.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:07 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:07.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:07 smithi118 ceph-mon[27391]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:07.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:07 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:09.351 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:54:09.351 INFO:teuthology.orchestra.run.smithi118.stdout:[{"addr": "172.21.15.118", "hostname": "smithi118", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-03-20T07:54:09.669 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:09 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:09.964 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-20T07:54:09.964 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-20T07:54:10.161 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:54:10.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:10 smithi118 ceph-mon[27391]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:10.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:10 smithi118 ceph-mon[27391]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:54:12.192 INFO:teuthology.orchestra.run.smithi118.stderr:adjusted tunables profile to default 2024-03-20T07:54:12.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:12.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:12.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3032800217' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:12.819 INFO:tasks.cephadm:Adding mon.a on smithi118 2024-03-20T07:54:12.819 INFO:tasks.cephadm:Adding mon.c on smithi118 2024-03-20T07:54:12.819 INFO:tasks.cephadm:Adding mon.b on smithi196 2024-03-20T07:54:12.819 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi196:172.21.15.196=b' 2024-03-20T07:54:13.020 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:13.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:13 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:13.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:13 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:54:13.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:13 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:54:13.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:13 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3032800217' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-20T07:54:13.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:13 smithi118 ceph-mon[27391]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:14 smithi118 ceph-mon[27391]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:14.914 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-03-20T07:54:15.561 DEBUG:teuthology.orchestra.run.smithi118:mon.c> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.c.service 2024-03-20T07:54:15.564 DEBUG:teuthology.orchestra.run.smithi196:mon.b> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.b.service 2024-03-20T07:54:15.566 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-20T07:54:15.566 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-20T07:54:16.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:16.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: Saving service mon spec with placement smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi196:172.21.15.196=b;count:3 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: Deploying daemon mon.b on smithi196 2024-03-20T07:54:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:15 smithi118 ceph-mon[27391]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:16.699 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:18.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:18 smithi118 ceph-mon[27391]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:18.817 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:54:18.817 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","modified":"2024-03-20T07:52:36.029987Z","created":"2024-03-20T07:52:36.029987Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-20T07:54:18.818 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-20T07:54:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:19 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/673385466' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:20.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).mds e1 new map 2024-03-20T07:54:20.614 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-20T07:54:20.614 INFO:journalctl@ceph.mon.b.smithi196.stdout: e1 2024-03-20T07:54:20.614 INFO:journalctl@ceph.mon.b.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-20T07:54:20.614 INFO:journalctl@ceph.mon.b.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T07:54:20.615 INFO:journalctl@ceph.mon.b.smithi196.stdout: legacy client fscid: -1 2024-03-20T07:54:20.615 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T07:54:20.615 INFO:journalctl@ceph.mon.b.smithi196.stdout: No filesystems configured 2024-03-20T07:54:20.615 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-20T07:54:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:20 smithi118 ceph-mon[27391]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:20.845 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-20T07:54:20.846 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-20T07:54:20.872 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:54:20.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:20.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3032800217' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3032800217' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:20.875 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: Saving service mon spec with placement smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi196:172.21.15.196=b;count:3 2024-03-20T07:54:20.876 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.876 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:20.876 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:20.876 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:20.876 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:20.876 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:20.876 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:20.876 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: Deploying daemon mon.b on smithi196 2024-03-20T07:54:20.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:20.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:20.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/673385466' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:20.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:20.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 ceph-mon[31033]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-20T07:54:20.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 bash[30995]: fd26544c725b0c066670c5030b50b72b7c8f39b7260cc731748b15af91f6aa7a 2024-03-20T07:54:20.877 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:20 smithi196 systemd[1]: Started Ceph mon.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:54:21.045 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:54:26.076 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:54:26.076 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","modified":"2024-03-20T07:54:20.620851Z","created":"2024-03-20T07:52:36.029987Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-20T07:54:26.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:54:26.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: mon.a calling monitor election 2024-03-20T07:54:26.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: mon.b calling monitor election 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: monmap e2: 2 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: fsmap 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:26.077 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: mgrmap e13: y(active, since 40s) 2024-03-20T07:54:26.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: overall HEALTH_OK 2024-03-20T07:54:26.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:26.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:26.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:26.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:26.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:26.078 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-03-20T07:54:26.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:54:26.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: mon.a calling monitor election 2024-03-20T07:54:26.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: mon.b calling monitor election 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: monmap e2: 2 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: fsmap 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: mgrmap e13: y(active, since 40s) 2024-03-20T07:54:26.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: overall HEALTH_OK 2024-03-20T07:54:26.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:26.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:26.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:26.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:26.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:27.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:26 smithi196 ceph-mon[31033]: Deploying daemon mon.c on smithi118 2024-03-20T07:54:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:26 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/3959162926' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:26 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:27.170 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:26 smithi118 ceph-mon[27391]: Deploying daemon mon.c on smithi118 2024-03-20T07:54:27.170 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:26 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/3959162926' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:27.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:26 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:27.687 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-20T07:54:27.687 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-20T07:54:27.898 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:54:28.019 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:27 smithi196 ceph-mon[31033]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:28.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:27 smithi118 ceph-mon[27391]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:28.860 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:28 smithi118 systemd[1]: Starting Ceph mon.c for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:28 smithi118 podman[38944]: 2024-03-20 07:54:28.764713634 +0000 UTC m=+0.017765777 image pull 3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:28 smithi118 podman[38944]: 2024-03-20 07:54:28.863762034 +0000 UTC m=+0.116814173 container create 8193e23658db475ccabcc876eca7a92d1fbfd39a01f8dca11b7e350657fab00e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-78dbc53, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 podman[38944]: 2024-03-20 07:54:29.074766087 +0000 UTC m=+0.327818238 container init 8193e23658db475ccabcc876eca7a92d1fbfd39a01f8dca11b7e350657fab00e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=reef-78dbc53, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 podman[38944]: 2024-03-20 07:54:29.080375579 +0000 UTC m=+0.333427718 container start 8193e23658db475ccabcc876eca7a92d1fbfd39a01f8dca11b7e350657fab00e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-78dbc53, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: set uid:gid to 167:167 (ceph:ceph) 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable), process ceph-mon, pid 7 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pidfile_write: ignore empty --pid-file 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: load: jerasure load: lrc 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: RocksDB version: 7.9.2 2024-03-20T07:54:29.111 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Git sha 0 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Compile date 2024-03-19 18:37:42 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: DB SUMMARY 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: DB Session ID: 9OEB7J13WQ5M4MRWF7PW 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: CURRENT file: CURRENT 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: IDENTITY file: IDENTITY 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.error_if_exists: 0 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.create_if_missing: 0 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.paranoid_checks: 1 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-20T07:54:29.112 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.env: 0x5644a5a42720 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.fs: PosixFileSystem 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.info_log: 0x5644a7d3f4e0 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_file_opening_threads: 16 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.statistics: (nil) 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.use_fsync: 0 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_log_file_size: 0 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.keep_log_file_num: 1000 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.recycle_log_file_num: 0 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.allow_fallocate: 1 2024-03-20T07:54:29.113 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.allow_mmap_reads: 0 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.allow_mmap_writes: 0 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.use_direct_reads: 0 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.create_missing_column_families: 0 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.db_log_dir: 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.wal_dir: 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.advise_random_on_open: 1 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.db_write_buffer_size: 0 2024-03-20T07:54:29.114 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.write_buffer_manager: 0x5644a6fb8140 2024-03-20T07:54:29.115 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-20T07:54:29.115 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-20T07:54:29.115 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-20T07:54:29.115 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.rate_limiter: (nil) 2024-03-20T07:54:29.115 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-20T07:54:29.116 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.wal_recovery_mode: 2 2024-03-20T07:54:29.116 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.enable_thread_tracking: 0 2024-03-20T07:54:29.116 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.enable_pipelined_write: 0 2024-03-20T07:54:29.116 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.unordered_write: 0 2024-03-20T07:54:29.116 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-20T07:54:29.116 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.row_cache: None 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.wal_filter: None 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.allow_ingest_behind: 0 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.two_write_queues: 0 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.manual_wal_flush: 0 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.wal_compression: 0 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.atomic_flush: 0 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-20T07:54:29.117 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.log_readahead_size: 0 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.best_efforts_recovery: 0 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.allow_data_in_errors: 0 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.db_host_id: __hostname__ 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_background_jobs: 2 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_background_compactions: -1 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_subcompactions: 1 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-20T07:54:29.118 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_total_wal_size: 0 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_open_files: -1 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bytes_per_sync: 0 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_readahead_size: 0 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_background_flushes: -1 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Compression algorithms supported: 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: kZSTD supported: 0 2024-03-20T07:54:29.119 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: kXpressCompression supported: 0 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: kLZ4HCCompression supported: 1 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: kZlibCompression supported: 1 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: kSnappyCompression supported: 1 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: kLZ4Compression supported: 1 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: kBZip2Compression supported: 0 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.merge_operator: 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_filter: None 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_filter_factory: None 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.sst_partitioner_factory: None 2024-03-20T07:54:29.120 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5644a7d3f5e0) 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: cache_index_and_filter_blocks: 1 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: pin_top_level_index_and_filter: 1 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_type: 0 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_block_index_type: 0 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_shortening: 1 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: checksum: 4 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: no_block_cache: 0 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache: 0x5644a70591f0 2024-03-20T07:54:29.121 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_name: BinnedLRUCache 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_options: 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: capacity : 536870912 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: num_shard_bits : 4 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: strict_capacity_limit : 0 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: high_pri_pool_ratio: 0.000 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_compressed: (nil) 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: persistent_cache: (nil) 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_size: 4096 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_size_deviation: 10 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_restart_interval: 16 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_block_restart_interval: 1 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: metadata_block_size: 4096 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: partition_filters: 0 2024-03-20T07:54:29.122 INFO:journalctl@ceph.mon.c.smithi118.stdout: use_delta_encoding: 1 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: filter_policy: bloomfilter 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: whole_key_filtering: 1 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: verify_compression: 0 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: read_amp_bytes_per_bit: 0 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: format_version: 5 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: enable_index_compression: 1 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_align: 0 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: max_auto_readahead_size: 262144 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: prepopulate_block_cache: 0 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: initial_auto_readahead_size: 8192 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout: num_file_reads_for_auto_readahead: 2 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.write_buffer_size: 33554432 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_write_buffer_number: 2 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression: NoCompression 2024-03-20T07:54:29.123 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression: Disabled 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.prefix_extractor: nullptr 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.num_levels: 7 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-20T07:54:29.124 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.level: 32767 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.strategy: 0 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.enabled: false 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.target_file_size_base: 67108864 2024-03-20T07:54:29.125 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-20T07:54:29.126 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.arena_block_size: 1048576 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.disable_auto_compactions: 0 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.table_properties_collectors: 2024-03-20T07:54:29.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.inplace_update_support: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.bloom_locality: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.max_successive_merges: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.paranoid_file_checks: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.force_consistency_checks: 1 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.report_bg_io_stats: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.ttl: 2592000 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-20T07:54:29.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.enable_blob_files: false 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.min_blob_size: 0 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.blob_file_size: 268435456 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.blob_file_starting_level: 0 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6410052f-dd47-4b1a-833f-2e4c11ba051f 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921269110257, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-20T07:54:29.129 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921269110628, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710921269, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6410052f-dd47-4b1a-833f-2e4c11ba051f", "db_session_id": "9OEB7J13WQ5M4MRWF7PW", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921269110706, "job": 1, "event": "recovery_finished"} 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5644a70f6000 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: DB pointer 0x5644a70e4000 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** DB Stats ** 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T07:54:29.130 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** Compaction Stats [default] ** 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** Compaction Stats [default] ** 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-20T07:54:29.131 INFO:journalctl@ceph.mon.c.smithi118.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: Block cache BinnedLRUCache@0x5644a70591f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-20T07:54:29.132 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T07:54:29.133 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-20T07:54:29.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-20T07:54:29.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: using public_addrv [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] 2024-03-20T07:54:29.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: starting mon.c rank -1 at public addrs [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] at bind addrs [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:54:29.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(???) e0 preinit fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).mds e1 new map 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout: e1 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout: legacy client fscid: -1 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout: No filesystems configured 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-20T07:54:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3032800217' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3032800217' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi196:172.21.15.196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: Saving service mon spec with placement smithi118:172.21.15.118=a;smithi118:[v2:172.21.15.118:3301,v1:172.21.15.118:6790]=c;smithi196:172.21.15.196=b;count:3 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: Deploying daemon mon.b on smithi196 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/673385466' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.a calling monitor election 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.b calling monitor election 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:29.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: monmap e2: 2 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: fsmap 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mgrmap e13: y(active, since 40s) 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: overall HEALTH_OK 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: Deploying daemon mon.c on smithi118 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/3959162926' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:29.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:29.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 ceph-mon[38981]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-20T07:54:29.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 bash[38944]: 8193e23658db475ccabcc876eca7a92d1fbfd39a01f8dca11b7e350657fab00e 2024-03-20T07:54:29.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:29 smithi118 systemd[1]: Started Ceph mon.c for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:54:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:54:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: mon.a calling monitor election 2024-03-20T07:54:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: mon.b calling monitor election 2024-03-20T07:54:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: mon.c calling monitor election 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: mon.a calling monitor election 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: fsmap 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: mgrmap e13: y(active, since 48s) 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: overall HEALTH_OK 2024-03-20T07:54:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:34.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:34.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:54:34.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: mon.a calling monitor election 2024-03-20T07:54:34.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: mon.b calling monitor election 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: mon.c calling monitor election 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: mon.a calling monitor election 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T07:54:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: fsmap 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: mgrmap e13: y(active, since 48s) 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: overall HEALTH_OK 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: mon.a calling monitor election 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: mon.b calling monitor election 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:54:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: mon.c calling monitor election 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: mon.a calling monitor election 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: fsmap 2024-03-20T07:54:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: osdmap e4: 0 total, 0 up, 0 in 2024-03-20T07:54:34.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: mgrmap e13: y(active, since 48s) 2024-03-20T07:54:34.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: overall HEALTH_OK 2024-03-20T07:54:34.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:34.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:35.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:54:35.762 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-20T07:54:35.763 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","modified":"2024-03-20T07:54:29.130208Z","created":"2024-03-20T07:52:36.029987Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3300","nonce":0},{"type":"v1","addr":"172.21.15.118:6789","nonce":0}]},"addr":"172.21.15.118:6789/0","public_addr":"172.21.15.118:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:3301","nonce":0},{"type":"v1","addr":"172.21.15.118:6790","nonce":0}]},"addr":"172.21.15.118:6790/0","public_addr":"172.21.15.118:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-20T07:54:35.765 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/2587162296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:36.405 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:36.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:36.424 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-20T07:54:36.425 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-20T07:54:36.653 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:36.653 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/2587162296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.654 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.655 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/2587162296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:36.656 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:37.209 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:54:37.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T07:54:37.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:54:37.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:37.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:37.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[27391]: Reconfiguring mon.a (unknown last config time)... 2024-03-20T07:54:37.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[27391]: Reconfiguring daemon mon.a on smithi118 2024-03-20T07:54:37.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T07:54:37.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:54:37.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:37.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:37.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[38981]: Reconfiguring mon.a (unknown last config time)... 2024-03-20T07:54:37.572 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:37 smithi118 ceph-mon[38981]: Reconfiguring daemon mon.a on smithi118 2024-03-20T07:54:37.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:37 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T07:54:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:37 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:54:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:37 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:37 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:54:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:37 smithi196 ceph-mon[31033]: Reconfiguring mon.a (unknown last config time)... 2024-03-20T07:54:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:37 smithi196 ceph-mon[31033]: Reconfiguring daemon mon.a on smithi118 2024-03-20T07:54:38.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[27391]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:38.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:38.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:38.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:38.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:38.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:38.506 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[38981]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:38.506 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:38.506 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:38.506 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:38.506 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:38.506 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:38 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:38.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:38 smithi196 ceph-mon[31033]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:38 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:38 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:38 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:38 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:38 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:39.344 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:39 smithi118 ceph-mon[27391]: Reconfiguring mon.c (monmap changed)... 2024-03-20T07:54:39.344 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:39 smithi118 ceph-mon[27391]: Reconfiguring daemon mon.c on smithi118 2024-03-20T07:54:39.344 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:39 smithi118 ceph-mon[38981]: Reconfiguring mon.c (monmap changed)... 2024-03-20T07:54:39.344 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:39 smithi118 ceph-mon[38981]: Reconfiguring daemon mon.c on smithi118 2024-03-20T07:54:39.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:39 smithi196 ceph-mon[31033]: Reconfiguring mon.c (monmap changed)... 2024-03-20T07:54:39.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:39 smithi196 ceph-mon[31033]: Reconfiguring daemon mon.c on smithi118 2024-03-20T07:54:40.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[38981]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:40.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:40.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:40.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:40.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:40.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:40.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[27391]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:40.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:40.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:40.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:40.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:40.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:40.488 INFO:teuthology.orchestra.run.smithi118.stdout:# minimal ceph.conf for bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:54:40.488 INFO:teuthology.orchestra.run.smithi118.stdout:[global] 2024-03-20T07:54:40.488 INFO:teuthology.orchestra.run.smithi118.stdout: fsid = bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T07:54:40.488 INFO:teuthology.orchestra.run.smithi118.stdout: mon_host = [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] 2024-03-20T07:54:40.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:40 smithi196 ceph-mon[31033]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:40.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:40.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:40.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T07:54:40.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T07:54:40.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:41.090 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-20T07:54:41.090 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:54:41.090 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.conf 2024-03-20T07:54:41.120 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:54:41.120 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:54:41.190 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:54:41.190 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-03-20T07:54:41.219 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:54:41.219 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:54:41.285 INFO:tasks.cephadm:Adding mgr.y on smithi118 2024-03-20T07:54:41.285 INFO:tasks.cephadm:Adding mgr.x on smithi196 2024-03-20T07:54:41.286 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch apply mgr '2;smithi118=y;smithi196=x' 2024-03-20T07:54:41.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: Reconfiguring mon.b (monmap changed)... 2024-03-20T07:54:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: Reconfiguring daemon mon.b on smithi196 2024-03-20T07:54:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1604106441' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: Reconfiguring mon.b (monmap changed)... 2024-03-20T07:54:41.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: Reconfiguring daemon mon.b on smithi196 2024-03-20T07:54:41.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1604106441' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: Reconfiguring mon.b (monmap changed)... 2024-03-20T07:54:41.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: Reconfiguring daemon mon.b on smithi196 2024-03-20T07:54:41.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1604106441' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:41.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:41.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:41.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:41.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:41.522 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:54:42.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:42 smithi196 ceph-mon[31033]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:42 smithi118 ceph-mon[27391]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:42.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:42 smithi118 ceph-mon[38981]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:43.267 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-03-20T07:54:43.868 DEBUG:teuthology.orchestra.run.smithi196:mgr.x> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x.service 2024-03-20T07:54:43.870 INFO:tasks.cephadm:Deploying OSDs... 2024-03-20T07:54:43.871 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:54:43.871 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/scratch_devs of=/dev/stdout 2024-03-20T07:54:43.889 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-20T07:54:43.890 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_1 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-03-20 07:54:28.590418666 +0000 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-03-20 07:49:34.992639266 +0000 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-03-20 07:49:34.992639266 +0000 2024-03-20T07:54:43.948 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-03-20 07:49:34.992639266 +0000 2024-03-20T07:54:43.949 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-20T07:54:44.018 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-03-20T07:54:44.018 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-03-20T07:54:44.018 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000134498 s, 3.8 MB/s 2024-03-20T07:54:44.020 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-20T07:54:44.080 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_2 2024-03-20T07:54:44.138 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-20T07:54:44.139 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-20T07:54:44.139 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-20T07:54:44.139 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-20T07:54:44.139 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-03-20T07:54:44.139 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-03-20 07:54:28.590418666 +0000 2024-03-20T07:54:44.139 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-03-20 07:49:35.331631703 +0000 2024-03-20T07:54:44.139 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-03-20 07:49:35.331631703 +0000 2024-03-20T07:54:44.139 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-03-20 07:49:35.331631703 +0000 2024-03-20T07:54:44.139 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-20T07:54:44.208 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-03-20T07:54:44.209 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-03-20T07:54:44.209 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000106844 s, 4.8 MB/s 2024-03-20T07:54:44.210 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-20T07:54:44.271 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_3 2024-03-20T07:54:44.328 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-20T07:54:44.329 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-20T07:54:44.329 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-20T07:54:44.329 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-20T07:54:44.329 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-03-20T07:54:44.329 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-03-20 07:54:28.590418666 +0000 2024-03-20T07:54:44.330 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-03-20 07:49:35.661624339 +0000 2024-03-20T07:54:44.330 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-03-20 07:49:35.661624339 +0000 2024-03-20T07:54:44.330 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-03-20 07:49:35.661624339 +0000 2024-03-20T07:54:44.330 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi118=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: Saving service mgr spec with placement smithi118=y;smithi196=x;count:2 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T07:54:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:44 smithi196 ceph-mon[31033]: Deploying daemon mgr.x on smithi196 2024-03-20T07:54:44.398 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-03-20T07:54:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-03-20T07:54:44.399 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.00011443 s, 4.5 MB/s 2024-03-20T07:54:44.400 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-20T07:54:44.460 DEBUG:teuthology.orchestra.run.smithi118:> stat /dev/vg_nvme/lv_4 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout:Context: system_u:object_r:device_t:s0 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout:Access: 2024-03-20 07:54:28.591418644 +0000 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout:Modify: 2024-03-20 07:49:35.994616909 +0000 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout:Change: 2024-03-20 07:49:35.994616909 +0000 2024-03-20T07:54:44.519 INFO:teuthology.orchestra.run.smithi118.stdout: Birth: 2024-03-20 07:49:35.994616909 +0000 2024-03-20T07:54:44.520 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-20T07:54:44.586 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi118=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: Saving service mgr spec with placement smithi118=y;smithi196=x;count:2 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:44.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[27391]: Deploying daemon mgr.x on smithi196 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi118=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: Saving service mgr spec with placement smithi118=y;smithi196=x;count:2 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T07:54:44.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:44.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:44 smithi118 ceph-mon[38981]: Deploying daemon mgr.x on smithi196 2024-03-20T07:54:44.589 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records in 2024-03-20T07:54:44.589 INFO:teuthology.orchestra.run.smithi118.stderr:1+0 records out 2024-03-20T07:54:44.589 INFO:teuthology.orchestra.run.smithi118.stderr:512 bytes copied, 0.000151901 s, 3.4 MB/s 2024-03-20T07:54:44.590 DEBUG:teuthology.orchestra.run.smithi118:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-20T07:54:44.649 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:54:44.649 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-03-20T07:54:44.666 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-20T07:54:44.666 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-20 07:54:20.069210315 +0000 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-20 07:49:34.996250368 +0000 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-20 07:49:34.996250368 +0000 2024-03-20T07:54:44.722 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-20 07:49:34.996250368 +0000 2024-03-20T07:54:44.723 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-20T07:54:44.787 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-20T07:54:44.787 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-20T07:54:44.787 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000203014 s, 2.5 MB/s 2024-03-20T07:54:44.788 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-20T07:54:44.844 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-03-20T07:54:44.899 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-20T07:54:44.900 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-20T07:54:44.900 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-20T07:54:44.900 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-20T07:54:44.900 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-20T07:54:44.900 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-20 07:54:20.069210315 +0000 2024-03-20T07:54:44.900 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-20 07:49:35.363241779 +0000 2024-03-20T07:54:44.900 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-20 07:49:35.363241779 +0000 2024-03-20T07:54:44.900 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-20 07:49:35.363241779 +0000 2024-03-20T07:54:44.900 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-20T07:54:44.965 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-20T07:54:44.965 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-20T07:54:44.965 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000287318 s, 1.8 MB/s 2024-03-20T07:54:44.967 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-20T07:54:45.024 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-03-20T07:54:45.079 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-20T07:54:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-20T07:54:45.080 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-20T07:54:45.080 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-20T07:54:45.080 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-20T07:54:45.080 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-20 07:54:20.069210315 +0000 2024-03-20T07:54:45.080 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-20 07:49:35.721233401 +0000 2024-03-20T07:54:45.080 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-20 07:49:35.721233401 +0000 2024-03-20T07:54:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-20 07:49:35.721233401 +0000 2024-03-20T07:54:45.080 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-20T07:54:45.145 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-20T07:54:45.145 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-20T07:54:45.145 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000220986 s, 2.3 MB/s 2024-03-20T07:54:45.146 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-20T07:54:45.203 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-03-20T07:54:45.258 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-20T07:54:45.258 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-20T07:54:45.258 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-20T07:54:45.258 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-20T07:54:45.258 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-20T07:54:45.259 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-20 07:54:20.070210292 +0000 2024-03-20T07:54:45.259 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-20 07:49:36.067225304 +0000 2024-03-20T07:54:45.259 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-20 07:49:36.067225304 +0000 2024-03-20T07:54:45.259 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-20 07:49:36.067225304 +0000 2024-03-20T07:54:45.259 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-20T07:54:45.351 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-20T07:54:45.351 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-20T07:54:45.351 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.00017969 s, 2.8 MB/s 2024-03-20T07:54:45.352 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-20T07:54:45.367 INFO:tasks.cephadm:Deploying osd.0 on smithi118 with /dev/vg_nvme/lv_4... 2024-03-20T07:54:45.367 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-20T07:54:45.574 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:54:45.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:45 smithi196 systemd[1]: Starting Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:54:46.018 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:45 smithi196 podman[33128]: 2024-03-20 07:54:45.748990258 +0000 UTC m=+0.113942910 container create ef306191c58d106e82e4862eb9524ff40994b0cceb09e5da29b7ae5aa64c5f0e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=reef-78dbc53, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T07:54:46.018 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:45 smithi196 podman[33128]: 2024-03-20 07:54:45.652924236 +0000 UTC m=+0.017876897 image pull 3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b 2024-03-20T07:54:46.018 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:46 smithi196 podman[33128]: 2024-03-20 07:54:46.01941924 +0000 UTC m=+0.384371893 container init ef306191c58d106e82e4862eb9524ff40994b0cceb09e5da29b7ae5aa64c5f0e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-78dbc53, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-20T07:54:46.269 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:46 smithi196 podman[33128]: 2024-03-20 07:54:46.025424791 +0000 UTC m=+0.390377452 container start ef306191c58d106e82e4862eb9524ff40994b0cceb09e5da29b7ae5aa64c5f0e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-78dbc53, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-20T07:54:46.269 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:46 smithi196 ceph-mgr[33166]: -- 172.21.15.196:0/3464301933 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5631308cc1a0 con 0x5631308a2800 2024-03-20T07:54:46.269 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:46 smithi196 bash[33128]: ef306191c58d106e82e4862eb9524ff40994b0cceb09e5da29b7ae5aa64c5f0e 2024-03-20T07:54:46.269 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:46 smithi196 systemd[1]: Started Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:54:46.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:46 smithi196 ceph-mon[31033]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:46.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.640 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[27391]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[38981]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.641 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:46.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:46.593+0000 7f1feaf2d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T07:54:46.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:46.698+0000 7f1feaf2d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T07:54:47.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:47 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:47.517 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-20T07:54:47.518 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-20T07:54:47.518 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2024-03-20T07:54:47.518 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2024-03-20T07:54:47.518 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00845337 s, 1.2 GB/s 2024-03-20T07:54:47.518 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2024-03-20T07:54:47.541 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:47 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:47.542 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:47 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:47.638 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:47.364+0000 7f1feaf2d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T07:54:47.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:47.455+0000 7f1feaf2d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T07:54:47.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:47.600+0000 7f1feaf2d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T07:54:48.045 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:47.702+0000 7f1feaf2d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T07:54:48.045 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:47.787+0000 7f1feaf2d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T07:54:48.045 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:48.046+0000 7f1feaf2d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T07:54:48.184 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi118:vg_nvme/lv_4 2024-03-20T07:54:48.332 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:48.154+0000 7f1feaf2d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:48.239+0000 7f1feaf2d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:48.333+0000 7f1feaf2d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T07:54:48.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T07:54:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T07:54:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:48.638 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:48.508+0000 7f1feaf2d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T07:54:48.882 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:54:49.599 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:49 smithi196 ceph-mon[31033]: Reconfiguring mgr.y (unknown last config time)... 2024-03-20T07:54:49.599 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:49 smithi196 ceph-mon[31033]: Reconfiguring daemon mgr.y on smithi118 2024-03-20T07:54:49.600 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:49.470+0000 7f1feaf2d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T07:54:49.600 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:49.600+0000 7f1feaf2d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T07:54:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:49 smithi118 ceph-mon[27391]: Reconfiguring mgr.y (unknown last config time)... 2024-03-20T07:54:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:49 smithi118 ceph-mon[27391]: Reconfiguring daemon mgr.y on smithi118 2024-03-20T07:54:49.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:49 smithi118 ceph-mon[38981]: Reconfiguring mgr.y (unknown last config time)... 2024-03-20T07:54:49.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:49 smithi118 ceph-mon[38981]: Reconfiguring daemon mgr.y on smithi118 2024-03-20T07:54:50.228 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:49.946+0000 7f1feaf2d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T07:54:50.228 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:50.035+0000 7f1feaf2d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T07:54:50.228 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:50.229+0000 7f1feaf2d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:54:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[27391]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[38981]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.638 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:50.417+0000 7f1feaf2d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T07:54:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-mon[31033]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:54:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:54:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:54:50.959 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:50.699+0000 7f1feaf2d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:54:50.959 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:50.960+0000 7f1feaf2d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T07:54:51.388 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:51.191+0000 7f1feaf2d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T07:54:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[27391]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:54:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:54:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:52.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[27391]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[38981]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:54:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:54:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:51 smithi118 ceph-mon[38981]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:52.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:51.985+0000 7f1feaf2d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T07:54:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:51 smithi196 ceph-mon[31033]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:54:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:51 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:54:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:51 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:54:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:51 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:54:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:51 smithi196 ceph-mon[31033]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:52.881 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:52.568+0000 7f1feaf2d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T07:54:52.881 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:52.667+0000 7f1feaf2d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T07:54:52.881 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:54:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:54:52.881+0000 7f1feaf2d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T07:54:53.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:52 smithi196 ceph-mon[31033]: Standby manager daemon x started 2024-03-20T07:54:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:52 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T07:54:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:52 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T07:54:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:52 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T07:54:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:52 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T07:54:53.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[27391]: Standby manager daemon x started 2024-03-20T07:54:53.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T07:54:53.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T07:54:53.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T07:54:53.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T07:54:53.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[38981]: Standby manager daemon x started 2024-03-20T07:54:53.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T07:54:53.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T07:54:53.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T07:54:53.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:52 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/2456527699' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T07:54:54.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[38981]: mgrmap e14: y(active, since 67s), standbys: x 2024-03-20T07:54:54.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T07:54:54.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[38981]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:54.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/88762239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c23409-c8bb-4018-9071-b8a8f1d2eed8"}]: dispatch 2024-03-20T07:54:54.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/88762239' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88c23409-c8bb-4018-9071-b8a8f1d2eed8"}]': finished 2024-03-20T07:54:54.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[38981]: osdmap e5: 1 total, 0 up, 1 in 2024-03-20T07:54:54.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:54:54.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[27391]: mgrmap e14: y(active, since 67s), standbys: x 2024-03-20T07:54:54.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T07:54:54.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[27391]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:54.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/88762239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c23409-c8bb-4018-9071-b8a8f1d2eed8"}]: dispatch 2024-03-20T07:54:54.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/88762239' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88c23409-c8bb-4018-9071-b8a8f1d2eed8"}]': finished 2024-03-20T07:54:54.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[27391]: osdmap e5: 1 total, 0 up, 1 in 2024-03-20T07:54:54.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:53 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:54:54.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:53 smithi196 ceph-mon[31033]: mgrmap e14: y(active, since 67s), standbys: x 2024-03-20T07:54:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:53 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T07:54:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:53 smithi196 ceph-mon[31033]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:53 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/88762239' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88c23409-c8bb-4018-9071-b8a8f1d2eed8"}]: dispatch 2024-03-20T07:54:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:53 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/88762239' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88c23409-c8bb-4018-9071-b8a8f1d2eed8"}]': finished 2024-03-20T07:54:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:53 smithi196 ceph-mon[31033]: osdmap e5: 1 total, 0 up, 1 in 2024-03-20T07:54:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:53 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:54:55.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:54 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1947556243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:54:55.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:54 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1947556243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:54:55.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:54 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1947556243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:54:56.290 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:55 smithi196 ceph-mon[31033]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:55 smithi118 ceph-mon[27391]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:56.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:55 smithi118 ceph-mon[38981]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:54:58 smithi196 ceph-mon[31033]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:58.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:54:58 smithi118 ceph-mon[27391]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:54:58.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:54:58 smithi118 ceph-mon[38981]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:00 smithi118 ceph-mon[27391]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T07:55:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:00 smithi118 ceph-mon[38981]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T07:55:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:00.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:00 smithi196 ceph-mon[31033]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T07:55:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:01.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:01 smithi118 ceph-mon[27391]: Deploying daemon osd.0 on smithi118 2024-03-20T07:55:01.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:01 smithi118 ceph-mon[38981]: Deploying daemon osd.0 on smithi118 2024-03-20T07:55:01.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:01 smithi196 ceph-mon[31033]: Deploying daemon osd.0 on smithi118 2024-03-20T07:55:02.620 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:02 smithi118 ceph-mon[27391]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:02.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:02 smithi118 ceph-mon[38981]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:02.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:02 smithi196 ceph-mon[31033]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:04.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:04 smithi196 ceph-mon[31033]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:04.669 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:04 smithi118 ceph-mon[27391]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:04.670 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:04 smithi118 ceph-mon[38981]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:06.604 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:06 smithi118 ceph-mon[38981]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:06.605 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:06.605 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:06.605 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:06.605 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:06 smithi118 ceph-mon[27391]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:06.605 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:06.605 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:06.605 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:06.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:06 smithi196 ceph-mon[31033]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[27391]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[38981]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:08.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:08.532 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:08.532 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:08.532 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:08.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:08 smithi196 ceph-mon[31033]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:08.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:08.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:08.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:08.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:08.544 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:09.182 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 0 on host 'smithi118' 2024-03-20T07:55:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:09 smithi118 ceph-mon[27391]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-20T07:55:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:09 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:09 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:09 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:09.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:09 smithi118 ceph-mon[38981]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-20T07:55:09.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:09 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:09.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:09 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:09 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:09.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:09 smithi196 ceph-mon[31033]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-20T07:55:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:09 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:09 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:09 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:09.719 DEBUG:teuthology.orchestra.run.smithi118:osd.0> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.0.service 2024-03-20T07:55:09.721 INFO:tasks.cephadm:Deploying osd.1 on smithi118 with /dev/vg_nvme/lv_3... 2024-03-20T07:55:09.721 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-20T07:55:10.351 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[38981]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[38981]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[38981]: osdmap e6: 1 total, 0 up, 1 in 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[38981]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[27391]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[27391]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[27391]: osdmap e6: 1 total, 0 up, 1 in 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:10.352 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-mon[27391]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:55:10.353 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[43417]: 2024-03-20T07:55:10.275+0000 7f7913ab0700 -1 osd.0 0 waiting for initial osdmap 2024-03-20T07:55:10.353 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:55:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[43417]: 2024-03-20T07:55:10.278+0000 7f790f8a5700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T07:55:10.577 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:55:10.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:10 smithi196 ceph-mon[31033]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:10 smithi196 ceph-mon[31033]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-20T07:55:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:10 smithi196 ceph-mon[31033]: osdmap e6: 1 total, 0 up, 1 in 2024-03-20T07:55:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:10 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:10 smithi196 ceph-mon[31033]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:55:11.485 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:11 smithi118 ceph-mon[27391]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:55:11.485 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:11 smithi118 ceph-mon[27391]: osdmap e7: 1 total, 0 up, 1 in 2024-03-20T07:55:11.485 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:11 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:11.485 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:11 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:11.486 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:11 smithi118 ceph-mon[38981]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:55:11.486 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:11 smithi118 ceph-mon[38981]: osdmap e7: 1 total, 0 up, 1 in 2024-03-20T07:55:11.486 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:11 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:11.486 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:11 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:11 smithi196 ceph-mon[31033]: from='osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:55:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:11 smithi196 ceph-mon[31033]: osdmap e7: 1 total, 0 up, 1 in 2024-03-20T07:55:11.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:11 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:11.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:11 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:12.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[27391]: purged_snaps scrub starts 2024-03-20T07:55:12.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[27391]: purged_snaps scrub ok 2024-03-20T07:55:12.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[27391]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:12.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[27391]: osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853] boot 2024-03-20T07:55:12.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[27391]: osdmap e8: 1 total, 1 up, 1 in 2024-03-20T07:55:12.573 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:12.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[38981]: purged_snaps scrub starts 2024-03-20T07:55:12.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[38981]: purged_snaps scrub ok 2024-03-20T07:55:12.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[38981]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:12.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[38981]: osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853] boot 2024-03-20T07:55:12.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[38981]: osdmap e8: 1 total, 1 up, 1 in 2024-03-20T07:55:12.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:12 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:12.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:12 smithi196 ceph-mon[31033]: purged_snaps scrub starts 2024-03-20T07:55:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:12 smithi196 ceph-mon[31033]: purged_snaps scrub ok 2024-03-20T07:55:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:12 smithi196 ceph-mon[31033]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-20T07:55:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:12 smithi196 ceph-mon[31033]: osd.0 [v2:172.21.15.118:6802/4171340853,v1:172.21.15.118:6803/4171340853] boot 2024-03-20T07:55:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:12 smithi196 ceph-mon[31033]: osdmap e8: 1 total, 1 up, 1 in 2024-03-20T07:55:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:12 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:55:13.130 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-20T07:55:13.130 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-20T07:55:13.130 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2024-03-20T07:55:13.130 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2024-03-20T07:55:13.130 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119513 s, 877 MB/s 2024-03-20T07:55:13.131 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2024-03-20T07:55:13.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:13 smithi118 ceph-mon[27391]: osdmap e9: 1 total, 1 up, 1 in 2024-03-20T07:55:13.623 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:13 smithi118 ceph-mon[38981]: osdmap e9: 1 total, 1 up, 1 in 2024-03-20T07:55:13.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:13 smithi196 ceph-mon[31033]: osdmap e9: 1 total, 1 up, 1 in 2024-03-20T07:55:13.782 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi118:vg_nvme/lv_3 2024-03-20T07:55:14.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:14 smithi118 ceph-mon[38981]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:14 smithi118 ceph-mon[27391]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:14.422 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:55:14.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:14 smithi196 ceph-mon[31033]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:15.570 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: Detected new or changed devices on smithi118 2024-03-20T07:55:15.570 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.570 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.570 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:15.570 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi118 to 16064M 2024-03-20T07:55:15.570 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: Detected new or changed devices on smithi118 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi118 to 16064M 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:15.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:15.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: Detected new or changed devices on smithi118 2024-03-20T07:55:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi118 to 16064M 2024-03-20T07:55:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:15 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:16.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:16 smithi118 ceph-mon[27391]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:16.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:16 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:55:16.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:16 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:55:16.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:16 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:16.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:16 smithi118 ceph-mon[38981]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:16.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:16 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:55:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:16 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:55:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:16 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:16.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:16 smithi196 ceph-mon[31033]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:16 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:55:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:16 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:55:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:16 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:17 smithi118 ceph-mon[27391]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:55:17.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:17 smithi118 ceph-mon[38981]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:55:17.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:17 smithi196 ceph-mon[31033]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:55:18.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:18 smithi118 ceph-mon[27391]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:18.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:18 smithi118 ceph-mon[38981]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:18.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:18 smithi196 ceph-mon[31033]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3957312362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3792a048-252f-44c3-ae87-fa7445418bc9"}]: dispatch 2024-03-20T07:55:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3957312362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3792a048-252f-44c3-ae87-fa7445418bc9"}]': finished 2024-03-20T07:55:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[27391]: osdmap e10: 2 total, 1 up, 2 in 2024-03-20T07:55:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[27391]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:19.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3957312362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3792a048-252f-44c3-ae87-fa7445418bc9"}]: dispatch 2024-03-20T07:55:19.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3957312362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3792a048-252f-44c3-ae87-fa7445418bc9"}]': finished 2024-03-20T07:55:19.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[38981]: osdmap e10: 2 total, 1 up, 2 in 2024-03-20T07:55:19.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:19.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:19 smithi118 ceph-mon[38981]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:19.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:19 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3957312362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3792a048-252f-44c3-ae87-fa7445418bc9"}]: dispatch 2024-03-20T07:55:19.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:19 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3957312362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3792a048-252f-44c3-ae87-fa7445418bc9"}]': finished 2024-03-20T07:55:19.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:19 smithi196 ceph-mon[31033]: osdmap e10: 2 total, 1 up, 2 in 2024-03-20T07:55:19.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:19.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:19 smithi196 ceph-mon[31033]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:20 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/589325060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:55:20.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:20 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/589325060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:55:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:20 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/589325060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:55:22.035 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:21 smithi118 ceph-mon[27391]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:22.036 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:21 smithi118 ceph-mon[38981]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:22.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:21 smithi196 ceph-mon[31033]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:24.269 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:24 smithi118 ceph-mon[38981]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:24.269 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:24 smithi118 ceph-mon[27391]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:24.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:24 smithi196 ceph-mon[31033]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:25.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T07:55:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:25.671 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:25 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T07:55:25.671 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:25 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:25.671 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T07:55:25.671 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:26.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:26 smithi118 ceph-mon[27391]: Deploying daemon osd.1 on smithi118 2024-03-20T07:55:26.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:26 smithi118 ceph-mon[27391]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:26.361 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:26 smithi118 ceph-mon[38981]: Deploying daemon osd.1 on smithi118 2024-03-20T07:55:26.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:26 smithi118 ceph-mon[38981]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:26.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:26 smithi196 ceph-mon[31033]: Deploying daemon osd.1 on smithi118 2024-03-20T07:55:26.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:26 smithi196 ceph-mon[31033]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:28.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:28 smithi196 ceph-mon[31033]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:28.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:28 smithi118 ceph-mon[27391]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:28.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:28 smithi118 ceph-mon[38981]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:30 smithi118 ceph-mon[27391]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:30.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:30 smithi118 ceph-mon[38981]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:30.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:30 smithi196 ceph-mon[31033]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:31.782 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:31 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:31.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:31 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:31.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:31 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:31.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:31 smithi118 ceph-mon[27391]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:31.783 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:31 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:31.783 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:31 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:31.783 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:31 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:31.783 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:31 smithi118 ceph-mon[38981]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:31.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:31 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:31.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:31 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:31.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:31 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:31.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:31 smithi196 ceph-mon[31033]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:34.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[27391]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:34.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:34.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:34.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.302 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[27391]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-20T07:55:34.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[38981]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:34.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:34.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:34.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.303 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:34 smithi118 ceph-mon[38981]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-20T07:55:34.348 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 1 on host 'smithi118' 2024-03-20T07:55:34.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:34 smithi196 ceph-mon[31033]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:34 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:34 smithi196 ceph-mon[31033]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-20T07:55:34.976 DEBUG:teuthology.orchestra.run.smithi118:osd.1> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.1.service 2024-03-20T07:55:34.979 INFO:tasks.cephadm:Deploying osd.2 on smithi118 with /dev/vg_nvme/lv_2... 2024-03-20T07:55:34.979 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-20T07:55:35.469 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[38981]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-20T07:55:35.469 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[38981]: osdmap e11: 2 total, 1 up, 2 in 2024-03-20T07:55:35.469 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:35.469 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[38981]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:55:35.469 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:35.469 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:35.470 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:35.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[27391]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-20T07:55:35.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[27391]: osdmap e11: 2 total, 1 up, 2 in 2024-03-20T07:55:35.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:35.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[27391]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:55:35.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:35.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:35.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:35.471 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[49360]: 2024-03-20T07:55:35.215+0000 7f48adb2f700 -1 osd.1 0 waiting for initial osdmap 2024-03-20T07:55:35.471 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:55:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[49360]: 2024-03-20T07:55:35.218+0000 7f48a711f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T07:55:35.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:35 smithi196 ceph-mon[31033]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-20T07:55:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:35 smithi196 ceph-mon[31033]: osdmap e11: 2 total, 1 up, 2 in 2024-03-20T07:55:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:35 smithi196 ceph-mon[31033]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:55:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:35.841 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:55:36.504 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[38981]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[38981]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[38981]: osdmap e12: 2 total, 1 up, 2 in 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[27391]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[27391]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[27391]: osdmap e12: 2 total, 1 up, 2 in 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:36.505 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:36.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:36 smithi196 ceph-mon[31033]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:36 smithi196 ceph-mon[31033]: from='osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:55:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:36 smithi196 ceph-mon[31033]: osdmap e12: 2 total, 1 up, 2 in 2024-03-20T07:55:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:37.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[38981]: purged_snaps scrub starts 2024-03-20T07:55:37.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[38981]: purged_snaps scrub ok 2024-03-20T07:55:37.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[38981]: osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816] boot 2024-03-20T07:55:37.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[38981]: osdmap e13: 2 total, 2 up, 2 in 2024-03-20T07:55:37.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:37.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[27391]: purged_snaps scrub starts 2024-03-20T07:55:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[27391]: purged_snaps scrub ok 2024-03-20T07:55:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[27391]: osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816] boot 2024-03-20T07:55:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[27391]: osdmap e13: 2 total, 2 up, 2 in 2024-03-20T07:55:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:37 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:37.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:37 smithi196 ceph-mon[31033]: purged_snaps scrub starts 2024-03-20T07:55:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:37 smithi196 ceph-mon[31033]: purged_snaps scrub ok 2024-03-20T07:55:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:37 smithi196 ceph-mon[31033]: osd.1 [v2:172.21.15.118:6810/4053982816,v1:172.21.15.118:6811/4053982816] boot 2024-03-20T07:55:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:37 smithi196 ceph-mon[31033]: osdmap e13: 2 total, 2 up, 2 in 2024-03-20T07:55:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:37 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:55:38.417 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:38 smithi118 ceph-mon[27391]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:38.417 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:38 smithi118 ceph-mon[27391]: osdmap e14: 2 total, 2 up, 2 in 2024-03-20T07:55:38.417 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:38 smithi118 ceph-mon[38981]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:38.418 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:38 smithi118 ceph-mon[38981]: osdmap e14: 2 total, 2 up, 2 in 2024-03-20T07:55:38.418 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-20T07:55:38.418 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-20T07:55:38.418 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2024-03-20T07:55:38.418 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2024-03-20T07:55:38.418 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00885454 s, 1.2 GB/s 2024-03-20T07:55:38.418 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2024-03-20T07:55:38.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:38 smithi196 ceph-mon[31033]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-20T07:55:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:38 smithi196 ceph-mon[31033]: osdmap e14: 2 total, 2 up, 2 in 2024-03-20T07:55:39.708 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi118:vg_nvme/lv_2 2024-03-20T07:55:39.909 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: Detected new or changed devices on smithi118 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi118 to 8032M 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: Detected new or changed devices on smithi118 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi118 to 8032M 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.290 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:40.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:40.291 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: Detected new or changed devices on smithi118 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi118 to 8032M 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:55:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:42.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:41 smithi118 ceph-mon[38981]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:42.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:55:42.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:55:42.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:42.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:41 smithi118 ceph-mon[27391]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:42.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:55:42.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:55:42.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:41 smithi196 ceph-mon[31033]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:42.431 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:55:42.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:55:42.432 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:43.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:42 smithi118 ceph-mon[27391]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:55:43.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:42 smithi118 ceph-mon[38981]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:55:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:42 smithi196 ceph-mon[31033]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:55:44.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:43 smithi196 ceph-mon[31033]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:43 smithi118 ceph-mon[27391]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:43 smithi118 ceph-mon[38981]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:45.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:44 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1517059576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54cacc96-d4a4-4c6a-b695-0edeb528c222"}]: dispatch 2024-03-20T07:55:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:44 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1517059576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54cacc96-d4a4-4c6a-b695-0edeb528c222"}]': finished 2024-03-20T07:55:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:44 smithi196 ceph-mon[31033]: osdmap e15: 3 total, 2 up, 3 in 2024-03-20T07:55:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:44 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:55:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:44 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1517059576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54cacc96-d4a4-4c6a-b695-0edeb528c222"}]: dispatch 2024-03-20T07:55:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:44 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1517059576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54cacc96-d4a4-4c6a-b695-0edeb528c222"}]': finished 2024-03-20T07:55:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:44 smithi118 ceph-mon[27391]: osdmap e15: 3 total, 2 up, 3 in 2024-03-20T07:55:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:44 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:55:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:44 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1517059576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54cacc96-d4a4-4c6a-b695-0edeb528c222"}]: dispatch 2024-03-20T07:55:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:44 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1517059576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54cacc96-d4a4-4c6a-b695-0edeb528c222"}]': finished 2024-03-20T07:55:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:44 smithi118 ceph-mon[38981]: osdmap e15: 3 total, 2 up, 3 in 2024-03-20T07:55:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:44 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:55:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:45 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2233261650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:55:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:45 smithi118 ceph-mon[27391]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:46.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:45 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2233261650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:55:46.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:45 smithi118 ceph-mon[38981]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:46.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:45 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2233261650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:55:46.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:45 smithi196 ceph-mon[31033]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:48 smithi118 ceph-mon[27391]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:48 smithi118 ceph-mon[38981]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:48.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:48 smithi196 ceph-mon[31033]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:50.562 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:50 smithi118 ceph-mon[27391]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:50.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:50 smithi118 ceph-mon[38981]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:50.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:50 smithi196 ceph-mon[31033]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:51.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:51 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T07:55:51.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:51 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:51.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:51 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T07:55:51.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:51 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:51.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:51 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T07:55:51.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:51 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:55:52.320 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:52 smithi118 ceph-mon[27391]: Deploying daemon osd.2 on smithi118 2024-03-20T07:55:52.320 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:52 smithi118 ceph-mon[27391]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:52.320 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:52 smithi118 ceph-mon[38981]: Deploying daemon osd.2 on smithi118 2024-03-20T07:55:52.320 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:52 smithi118 ceph-mon[38981]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:52.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:52 smithi196 ceph-mon[31033]: Deploying daemon osd.2 on smithi118 2024-03-20T07:55:52.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:52 smithi196 ceph-mon[31033]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:54 smithi118 ceph-mon[27391]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:54 smithi118 ceph-mon[38981]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:54.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:54 smithi196 ceph-mon[31033]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:56 smithi118 ceph-mon[27391]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:56 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:56 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:56 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:56 smithi118 ceph-mon[38981]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:56.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:56 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:56.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:56 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:56.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:56 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:56.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:56 smithi196 ceph-mon[31033]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:56.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:56 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:56.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:56 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:55:56.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:56 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:55:58.384 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:55:58 smithi118 ceph-mon[27391]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:58.385 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:55:58 smithi118 ceph-mon[38981]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:55:58.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:55:58 smithi196 ceph-mon[31033]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:56:00.064 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 2 on host 'smithi118' 2024-03-20T07:56:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T07:56:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T07:56:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:00 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:00.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:00 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:00.735 DEBUG:teuthology.orchestra.run.smithi118:osd.2> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.2.service 2024-03-20T07:56:00.737 INFO:tasks.cephadm:Deploying osd.3 on smithi118 with /dev/vg_nvme/lv_1... 2024-03-20T07:56:00.737 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-20T07:56:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[27391]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-20T07:56:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[27391]: osdmap e16: 3 total, 2 up, 3 in 2024-03-20T07:56:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[27391]: from='osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[27391]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[38981]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-20T07:56:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[38981]: osdmap e16: 3 total, 2 up, 3 in 2024-03-20T07:56:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[38981]: from='osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-mon[38981]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:01.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[55341]: 2024-03-20T07:56:01.160+0000 7feeacc43700 -1 osd.2 0 waiting for initial osdmap 2024-03-20T07:56:01.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[55341]: 2024-03-20T07:56:01.163+0000 7feea8237700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T07:56:01.501 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:56:01.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:01 smithi196 ceph-mon[31033]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-20T07:56:01.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:01 smithi196 ceph-mon[31033]: osdmap e16: 3 total, 2 up, 3 in 2024-03-20T07:56:01.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:01 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:01.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:01 smithi196 ceph-mon[31033]: from='osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:01.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:01 smithi196 ceph-mon[31033]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[27391]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:56:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[27391]: osdmap e17: 3 total, 2 up, 3 in 2024-03-20T07:56:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[27391]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:56:02.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[38981]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:56:02.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[38981]: osdmap e17: 3 total, 2 up, 3 in 2024-03-20T07:56:02.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:02.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:02.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:02 smithi118 ceph-mon[38981]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:56:02.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:02 smithi196 ceph-mon[31033]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:56:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:02 smithi196 ceph-mon[31033]: osdmap e17: 3 total, 2 up, 3 in 2024-03-20T07:56:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:02 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:02 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:02 smithi196 ceph-mon[31033]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-20T07:56:03.368 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[27391]: purged_snaps scrub starts 2024-03-20T07:56:03.369 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[27391]: purged_snaps scrub ok 2024-03-20T07:56:03.369 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[27391]: osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060] boot 2024-03-20T07:56:03.369 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[27391]: osdmap e18: 3 total, 3 up, 3 in 2024-03-20T07:56:03.369 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:03.369 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[38981]: purged_snaps scrub starts 2024-03-20T07:56:03.369 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[38981]: purged_snaps scrub ok 2024-03-20T07:56:03.369 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[38981]: osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060] boot 2024-03-20T07:56:03.369 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[38981]: osdmap e18: 3 total, 3 up, 3 in 2024-03-20T07:56:03.370 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:03 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:03.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:03 smithi196 ceph-mon[31033]: purged_snaps scrub starts 2024-03-20T07:56:03.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:03 smithi196 ceph-mon[31033]: purged_snaps scrub ok 2024-03-20T07:56:03.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:03 smithi196 ceph-mon[31033]: osd.2 [v2:172.21.15.118:6818/1856880060,v1:172.21.15.118:6819/1856880060] boot 2024-03-20T07:56:03.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:03 smithi196 ceph-mon[31033]: osdmap e18: 3 total, 3 up, 3 in 2024-03-20T07:56:03.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:03 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:56:04.054 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-20T07:56:04.054 INFO:teuthology.orchestra.run.smithi118.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-20T07:56:04.055 INFO:teuthology.orchestra.run.smithi118.stderr: stderr: 10+0 records in 2024-03-20T07:56:04.055 INFO:teuthology.orchestra.run.smithi118.stderr:10+0 records out 2024-03-20T07:56:04.055 INFO:teuthology.orchestra.run.smithi118.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00954376 s, 1.1 GB/s 2024-03-20T07:56:04.055 INFO:teuthology.orchestra.run.smithi118.stderr:--> Zapping successful for: 2024-03-20T07:56:04.341 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:04 smithi118 ceph-mon[38981]: osdmap e19: 3 total, 3 up, 3 in 2024-03-20T07:56:04.342 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:04 smithi118 ceph-mon[38981]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:04.342 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:04 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:56:04.342 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:04 smithi118 ceph-mon[27391]: osdmap e19: 3 total, 3 up, 3 in 2024-03-20T07:56:04.342 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:04 smithi118 ceph-mon[27391]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:04.342 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:04 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:56:04.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:04 smithi196 ceph-mon[31033]: osdmap e19: 3 total, 3 up, 3 in 2024-03-20T07:56:04.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:04 smithi196 ceph-mon[31033]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:04 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:56:04.882 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi118:vg_nvme/lv_1 2024-03-20T07:56:05.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:05 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-20T07:56:05.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:05 smithi118 ceph-mon[27391]: osdmap e20: 3 total, 3 up, 3 in 2024-03-20T07:56:05.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:05 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:56:05.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:05 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-20T07:56:05.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:05 smithi118 ceph-mon[38981]: osdmap e20: 3 total, 3 up, 3 in 2024-03-20T07:56:05.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:05 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:56:05.626 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:56:05.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:05 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-20T07:56:05.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:05 smithi196 ceph-mon[31033]: osdmap e20: 3 total, 3 up, 3 in 2024-03-20T07:56:05.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:05 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:56:05.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:56:05 smithi118 sudo[59725]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-20T07:56:05.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:56:05 smithi118 sudo[59725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:05.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:56:05 smithi118 sudo[59725]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:05.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:56:05 smithi118 sudo[59725]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:06.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:56:05 smithi118 sudo[59760]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-20T07:56:06.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:56:05 smithi118 sudo[59760]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:06.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:56:05 smithi118 sudo[59760]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:06.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 07:56:05 smithi118 sudo[59760]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: osdmap e21: 3 total, 3 up, 3 in 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: osdmap e21: 3 total, 3 up, 3 in 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.328 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:06.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:06.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.329 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59800]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-20T07:56:06.329 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59800]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:06.329 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59800]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:06.329 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59800]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: osdmap e21: 3 total, 3 up, 3 in 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:06.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59808]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-20T07:56:06.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59808]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:06.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59808]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:06.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59808]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:06.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-20T07:56:06.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:06.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59822]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:06.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59822]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:07.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59847]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-20T07:56:07.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59847]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:07.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59847]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:07.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 07:56:06 smithi118 sudo[59847]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:07.457 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 sudo[59862]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-20T07:56:07.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 sudo[59862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:07.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 sudo[59862]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:07.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 sudo[59862]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:07.734 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 ceph-mon[31033]: Detected new or changed devices on smithi118 2024-03-20T07:56:07.734 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi118 to 5354M 2024-03-20T07:56:07.734 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 ceph-mon[31033]: osdmap e22: 3 total, 3 up, 3 in 2024-03-20T07:56:07.734 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 ceph-mon[31033]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-20T07:56:07.734 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:07.734 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 sudo[33890]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-20T07:56:07.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 sudo[33890]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:07.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 sudo[33890]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:07.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 sudo[33890]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:07.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[38981]: Detected new or changed devices on smithi118 2024-03-20T07:56:07.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi118 to 5354M 2024-03-20T07:56:07.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[38981]: osdmap e22: 3 total, 3 up, 3 in 2024-03-20T07:56:07.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[38981]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-20T07:56:07.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:07.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 sudo[59899]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-20T07:56:07.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 sudo[59899]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:07.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 sudo[59899]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:07.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 sudo[59899]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:07.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[27391]: Detected new or changed devices on smithi118 2024-03-20T07:56:07.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi118 to 5354M 2024-03-20T07:56:07.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[27391]: osdmap e22: 3 total, 3 up, 3 in 2024-03-20T07:56:07.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[27391]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-20T07:56:07.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:07 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:08.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 sudo[33893]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-20T07:56:08.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 sudo[33893]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:08.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 sudo[33893]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:08.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:07 smithi196 sudo[33893]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:08.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 sudo[60025]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-20T07:56:08.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 sudo[60025]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:08.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 sudo[60025]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:08.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 sudo[60025]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:08.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:08.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:08.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:08.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:08.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:08.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:08.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:08.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:08.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:08.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 sudo[60065]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-20T07:56:08.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 sudo[60065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 sudo[60065]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 sudo[60065]: pam_unix(sudo:session): session closed for user root 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi118:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:08.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:08.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:08.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:08.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:56:08.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:56:08.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:56:09.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:09 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:09.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:09 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:09.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:09 smithi196 ceph-mon[31033]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:09.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:09 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:09.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:09 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:09.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:09 smithi118 ceph-mon[38981]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:09.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:09 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-20T07:56:09.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:09 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-20T07:56:09.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:09 smithi118 ceph-mon[27391]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:10.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:10 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2189091029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]: dispatch 2024-03-20T07:56:10.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:10 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]: dispatch 2024-03-20T07:56:10.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:10 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]': finished 2024-03-20T07:56:10.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:10 smithi196 ceph-mon[31033]: osdmap e23: 4 total, 3 up, 4 in 2024-03-20T07:56:10.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:10 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:10.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2189091029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]: dispatch 2024-03-20T07:56:10.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]: dispatch 2024-03-20T07:56:10.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]': finished 2024-03-20T07:56:10.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[38981]: osdmap e23: 4 total, 3 up, 4 in 2024-03-20T07:56:10.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2189091029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]: dispatch 2024-03-20T07:56:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]: dispatch 2024-03-20T07:56:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b5ff3b5-b736-4c14-ac49-be59960e3a5a"}]': finished 2024-03-20T07:56:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[27391]: osdmap e23: 4 total, 3 up, 4 in 2024-03-20T07:56:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:10 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:11 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3064009182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:56:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:11 smithi196 ceph-mon[31033]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:12.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:11 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3064009182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:56:12.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:11 smithi118 ceph-mon[27391]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:12.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:11 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3064009182' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:56:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:11 smithi118 ceph-mon[38981]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:14.516 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:14 smithi118 ceph-mon[27391]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-20T07:56:14.516 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:14 smithi118 ceph-mon[38981]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-20T07:56:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:14 smithi196 ceph-mon[31033]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-20T07:56:16.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:16 smithi118 ceph-mon[38981]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-20T07:56:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:16 smithi118 ceph-mon[27391]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-20T07:56:16.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:16 smithi196 ceph-mon[31033]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-20T07:56:17.621 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T07:56:17.621 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:17.622 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T07:56:17.622 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:17.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T07:56:17.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:18.391 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:18 smithi118 ceph-mon[27391]: Deploying daemon osd.3 on smithi118 2024-03-20T07:56:18.391 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:18 smithi118 ceph-mon[27391]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-20T07:56:18.391 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:18 smithi118 ceph-mon[38981]: Deploying daemon osd.3 on smithi118 2024-03-20T07:56:18.391 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:18 smithi118 ceph-mon[38981]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-20T07:56:18.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:18 smithi196 ceph-mon[31033]: Deploying daemon osd.3 on smithi118 2024-03-20T07:56:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:18 smithi196 ceph-mon[31033]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-20T07:56:20.635 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:20 smithi118 ceph-mon[27391]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-20T07:56:20.636 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:20 smithi118 ceph-mon[38981]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-20T07:56:20.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:20 smithi196 ceph-mon[31033]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-20T07:56:22.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:21 smithi118 ceph-mon[27391]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-03-20T07:56:22.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:21 smithi118 ceph-mon[38981]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-03-20T07:56:22.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:21 smithi196 ceph-mon[31033]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-03-20T07:56:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:23 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:23 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:23 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:23 smithi118 ceph-mon[27391]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-20T07:56:23.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:23 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:23.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:23 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:23.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:23 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:23.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:23 smithi118 ceph-mon[38981]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-20T07:56:23.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:23 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:23.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:23 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:23.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:23 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:23.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:23 smithi196 ceph-mon[31033]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-20T07:56:26.333 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[27391]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:26.333 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[27391]: from='osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T07:56:26.333 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[27391]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[38981]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[38981]: from='osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[38981]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:26.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:26.335 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:26 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:26.381 INFO:teuthology.orchestra.run.smithi118.stdout:Created osd(s) 3 on host 'smithi118' 2024-03-20T07:56:26.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:26 smithi196 ceph-mon[31033]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:26.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:26 smithi196 ceph-mon[31033]: from='osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T07:56:26.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:26 smithi196 ceph-mon[31033]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T07:56:26.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:26 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:26.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:26 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:26.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:26 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:26.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:26 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:26.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:26 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:27.006 DEBUG:teuthology.orchestra.run.smithi118:osd.3> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.3.service 2024-03-20T07:56:27.009 INFO:tasks.cephadm:Deploying osd.4 on smithi196 with /dev/vg_nvme/lv_4... 2024-03-20T07:56:27.009 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-20T07:56:27.219 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:56:27.341 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:27 smithi196 ceph-mon[31033]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-20T07:56:27.341 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:27 smithi196 ceph-mon[31033]: osdmap e24: 4 total, 3 up, 4 in 2024-03-20T07:56:27.341 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:27 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:27.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:27 smithi196 ceph-mon[31033]: from='osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:27.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:27 smithi196 ceph-mon[31033]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:27.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:27 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:27.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:27 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:27.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:27 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:27.606 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[38981]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-20T07:56:27.606 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[38981]: osdmap e24: 4 total, 3 up, 4 in 2024-03-20T07:56:27.606 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:27.606 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[38981]: from='osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:27.606 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[38981]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:27.606 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[27391]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[27391]: osdmap e24: 4 total, 3 up, 4 in 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[27391]: from='osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[27391]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:27.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:27.608 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[61973]: 2024-03-20T07:56:27.277+0000 7f5bcc10d700 -1 osd.3 0 waiting for initial osdmap 2024-03-20T07:56:27.608 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 07:56:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[61973]: 2024-03-20T07:56:27.281+0000 7f5bc7f02700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T07:56:28.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:28 smithi196 ceph-mon[31033]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:28.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:28 smithi196 ceph-mon[31033]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:56:28.382 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:28 smithi196 ceph-mon[31033]: osdmap e25: 4 total, 3 up, 4 in 2024-03-20T07:56:28.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:28 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:28.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:28 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:28.582 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[27391]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:28.582 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[27391]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:56:28.582 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[27391]: osdmap e25: 4 total, 3 up, 4 in 2024-03-20T07:56:28.583 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:28.583 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:28.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[38981]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:28.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[38981]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]': finished 2024-03-20T07:56:28.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[38981]: osdmap e25: 4 total, 3 up, 4 in 2024-03-20T07:56:28.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:28.583 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:28 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:29.214 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-20T07:56:29.214 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-20T07:56:29.214 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-20T07:56:29.214 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-20T07:56:29.214 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132287 s, 793 MB/s 2024-03-20T07:56:29.214 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-20T07:56:29.554 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:29 smithi196 ceph-mon[31033]: purged_snaps scrub starts 2024-03-20T07:56:29.555 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:29 smithi196 ceph-mon[31033]: purged_snaps scrub ok 2024-03-20T07:56:29.555 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:29 smithi196 ceph-mon[31033]: osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705] boot 2024-03-20T07:56:29.555 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:29 smithi196 ceph-mon[31033]: osdmap e26: 4 total, 4 up, 4 in 2024-03-20T07:56:29.555 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:29 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[38981]: purged_snaps scrub starts 2024-03-20T07:56:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[38981]: purged_snaps scrub ok 2024-03-20T07:56:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[38981]: osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705] boot 2024-03-20T07:56:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[38981]: osdmap e26: 4 total, 4 up, 4 in 2024-03-20T07:56:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[27391]: purged_snaps scrub starts 2024-03-20T07:56:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[27391]: purged_snaps scrub ok 2024-03-20T07:56:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[27391]: osd.3 [v2:172.21.15.118:6826/3211575705,v1:172.21.15.118:6827/3211575705] boot 2024-03-20T07:56:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[27391]: osdmap e26: 4 total, 4 up, 4 in 2024-03-20T07:56:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:29 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:56:29.801 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-03-20T07:56:30.000 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:56:30.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: osdmap e27: 4 total, 4 up, 4 in 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: osdmap e27: 4 total, 4 up, 4 in 2024-03-20T07:56:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: osdmap e27: 4 total, 4 up, 4 in 2024-03-20T07:56:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:31 smithi196 ceph-mon[31033]: Detected new or changed devices on smithi118 2024-03-20T07:56:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:31 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi118 to 4016M 2024-03-20T07:56:31.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:31 smithi118 ceph-mon[27391]: Detected new or changed devices on smithi118 2024-03-20T07:56:31.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:31 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi118 to 4016M 2024-03-20T07:56:31.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:31 smithi118 ceph-mon[38981]: Detected new or changed devices on smithi118 2024-03-20T07:56:31.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:31 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi118 to 4016M 2024-03-20T07:56:32.566 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:32 smithi196 ceph-mon[31033]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:32.566 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:32 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:32.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:32 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:32.567 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:32 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:32 smithi118 ceph-mon[27391]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:32 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:32 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:32 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:32.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:32 smithi118 ceph-mon[38981]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:32 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:32 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:32 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:33.583 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:33 smithi196 ceph-mon[31033]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:33.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:33 smithi118 ceph-mon[27391]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:33.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:33 smithi118 ceph-mon[38981]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:34.573 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:34 smithi118 ceph-mon[27391]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:34.574 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:34 smithi118 ceph-mon[38981]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:34.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:34 smithi196 ceph-mon[31033]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:35 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/1734907537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]: dispatch 2024-03-20T07:56:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:35 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]: dispatch 2024-03-20T07:56:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:35 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]': finished 2024-03-20T07:56:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:35 smithi196 ceph-mon[31033]: osdmap e28: 5 total, 4 up, 5 in 2024-03-20T07:56:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:35 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:35 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/1479457815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:56:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/1734907537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]: dispatch 2024-03-20T07:56:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]: dispatch 2024-03-20T07:56:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]': finished 2024-03-20T07:56:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[27391]: osdmap e28: 5 total, 4 up, 5 in 2024-03-20T07:56:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:35.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/1479457815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:56:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/1734907537' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]: dispatch 2024-03-20T07:56:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]: dispatch 2024-03-20T07:56:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e6db394-b7e5-48c7-9aae-dd1b9469552e"}]': finished 2024-03-20T07:56:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[38981]: osdmap e28: 5 total, 4 up, 5 in 2024-03-20T07:56:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:35 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/1479457815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:56:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:36 smithi118 ceph-mon[27391]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:36 smithi118 ceph-mon[38981]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:36.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:36 smithi196 ceph-mon[31033]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:38 smithi118 ceph-mon[38981]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:38 smithi118 ceph-mon[27391]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:38.706 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:38 smithi196 ceph-mon[31033]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:40 smithi118 ceph-mon[27391]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:40 smithi118 ceph-mon[38981]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:40.706 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:40 smithi196 ceph-mon[31033]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:41.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T07:56:41.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:41.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T07:56:41.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T07:56:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:42.599 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:42 smithi196 ceph-mon[31033]: Deploying daemon osd.4 on smithi196 2024-03-20T07:56:42.599 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:42 smithi196 ceph-mon[31033]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:42 smithi118 ceph-mon[27391]: Deploying daemon osd.4 on smithi196 2024-03-20T07:56:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:42 smithi118 ceph-mon[27391]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:42.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:42 smithi118 ceph-mon[38981]: Deploying daemon osd.4 on smithi196 2024-03-20T07:56:42.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:42 smithi118 ceph-mon[38981]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:44.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:44 smithi196 ceph-mon[31033]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:44.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:44 smithi118 ceph-mon[38981]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:44.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:44 smithi118 ceph-mon[27391]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:46 smithi118 ceph-mon[27391]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:46 smithi118 ceph-mon[38981]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:46.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:46 smithi196 ceph-mon[31033]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:46.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:46.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:46.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[27391]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[38981]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:48 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:48.840 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:48 smithi196 ceph-mon[31033]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:48.840 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:48.840 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:48.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:48.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:48.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:48 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:49.578 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 4 on host 'smithi196' 2024-03-20T07:56:49.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:49 smithi196 ceph-mon[31033]: from='osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T07:56:49.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:49 smithi196 ceph-mon[31033]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T07:56:49.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:49 smithi196 ceph-mon[31033]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:49.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:49 smithi118 ceph-mon[27391]: from='osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T07:56:49.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:49 smithi118 ceph-mon[27391]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T07:56:49.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:49 smithi118 ceph-mon[27391]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:49.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:49 smithi118 ceph-mon[38981]: from='osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T07:56:49.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:49 smithi118 ceph-mon[38981]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T07:56:49.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:49 smithi118 ceph-mon[38981]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:50.135 DEBUG:teuthology.orchestra.run.smithi196:osd.4> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.4.service 2024-03-20T07:56:50.138 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-03-20T07:56:50.138 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-20T07:56:50.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[36567]: 2024-03-20T07:56:50.436+0000 7fc6a3bda700 -1 osd.4 0 waiting for initial osdmap 2024-03-20T07:56:50.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[36567]: 2024-03-20T07:56:50.441+0000 7fc69d9cb700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T07:56:50.651 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-mon[31033]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-20T07:56:50.651 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-mon[31033]: osdmap e29: 5 total, 4 up, 5 in 2024-03-20T07:56:50.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:50.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-mon[31033]: from='osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:56:50.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-mon[31033]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:56:50.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:50.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:50.652 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:50.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[27391]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-20T07:56:50.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[27391]: osdmap e29: 5 total, 4 up, 5 in 2024-03-20T07:56:50.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:50.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[27391]: from='osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:56:50.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[27391]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[38981]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[38981]: osdmap e29: 5 total, 4 up, 5 in 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[38981]: from='osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[38981]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:50.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:56:51.408 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:56:51.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:51 smithi196 ceph-mon[31033]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:56:51.661 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:51 smithi196 ceph-mon[31033]: osdmap e30: 5 total, 4 up, 5 in 2024-03-20T07:56:51.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:51 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:51 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:51 smithi196 ceph-mon[31033]: osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663] boot 2024-03-20T07:56:51.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:51 smithi196 ceph-mon[31033]: osdmap e31: 5 total, 5 up, 5 in 2024-03-20T07:56:51.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:51 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.662 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:51 smithi196 ceph-mon[31033]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:51.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[38981]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:56:51.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[38981]: osdmap e30: 5 total, 4 up, 5 in 2024-03-20T07:56:51.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[38981]: osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663] boot 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[38981]: osdmap e31: 5 total, 5 up, 5 in 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[38981]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[27391]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[27391]: osdmap e30: 5 total, 4 up, 5 in 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[27391]: osd.4 [v2:172.21.15.196:6800/2757499663,v1:172.21.15.196:6801/2757499663] boot 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[27391]: osdmap e31: 5 total, 5 up, 5 in 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:56:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:51 smithi118 ceph-mon[27391]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-20T07:56:53.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:52 smithi196 ceph-mon[31033]: purged_snaps scrub starts 2024-03-20T07:56:53.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:52 smithi196 ceph-mon[31033]: purged_snaps scrub ok 2024-03-20T07:56:53.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:52 smithi196 ceph-mon[31033]: osdmap e32: 5 total, 5 up, 5 in 2024-03-20T07:56:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:52 smithi118 ceph-mon[27391]: purged_snaps scrub starts 2024-03-20T07:56:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:52 smithi118 ceph-mon[27391]: purged_snaps scrub ok 2024-03-20T07:56:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:52 smithi118 ceph-mon[27391]: osdmap e32: 5 total, 5 up, 5 in 2024-03-20T07:56:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:52 smithi118 ceph-mon[38981]: purged_snaps scrub starts 2024-03-20T07:56:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:52 smithi118 ceph-mon[38981]: purged_snaps scrub ok 2024-03-20T07:56:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:52 smithi118 ceph-mon[38981]: osdmap e32: 5 total, 5 up, 5 in 2024-03-20T07:56:53.703 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-20T07:56:53.703 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-20T07:56:53.703 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-20T07:56:53.703 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-20T07:56:53.703 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114085 s, 919 MB/s 2024-03-20T07:56:53.703 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-20T07:56:54.020 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:53 smithi196 ceph-mon[31033]: osdmap e33: 5 total, 5 up, 5 in 2024-03-20T07:56:54.020 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:53 smithi196 ceph-mon[31033]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:53 smithi118 ceph-mon[27391]: osdmap e33: 5 total, 5 up, 5 in 2024-03-20T07:56:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:53 smithi118 ceph-mon[27391]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:54.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:53 smithi118 ceph-mon[38981]: osdmap e33: 5 total, 5 up, 5 in 2024-03-20T07:56:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:53 smithi118 ceph-mon[38981]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:54.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-03-20T07:56:54.841 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: osdmap e34: 5 total, 5 up, 5 in 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: Detected new or changed devices on smithi196 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi196 to 17088M 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:55.209 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:54 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: osdmap e34: 5 total, 5 up, 5 in 2024-03-20T07:56:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: Detected new or changed devices on smithi196 2024-03-20T07:56:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi196 to 17088M 2024-03-20T07:56:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: osdmap e34: 5 total, 5 up, 5 in 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: Detected new or changed devices on smithi196 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi196 to 17088M 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:55.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:56:55.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:54 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:56:56.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:55 smithi196 ceph-mon[31033]: osdmap e35: 5 total, 5 up, 5 in 2024-03-20T07:56:56.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:55 smithi196 ceph-mon[31033]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:55 smithi118 ceph-mon[27391]: osdmap e35: 5 total, 5 up, 5 in 2024-03-20T07:56:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:55 smithi118 ceph-mon[27391]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:56.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:55 smithi118 ceph-mon[38981]: osdmap e35: 5 total, 5 up, 5 in 2024-03-20T07:56:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:55 smithi118 ceph-mon[38981]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:57.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:56 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:57.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:56 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:57.215 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:56 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:57.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:56 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:57.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:56 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:57.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:56 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:57.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:56 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:56:57.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:56 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:56:57.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:56 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:56:58.030 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:57 smithi196 ceph-mon[31033]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:58.030 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:57 smithi196 ceph-mon[31033]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:58.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:57 smithi118 ceph-mon[27391]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:58.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:57 smithi118 ceph-mon[27391]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:58.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:57 smithi118 ceph-mon[38981]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:56:58.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:57 smithi118 ceph-mon[38981]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:56:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/507386161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]: dispatch 2024-03-20T07:56:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]: dispatch 2024-03-20T07:56:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]': finished 2024-03-20T07:56:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[27391]: osdmap e36: 6 total, 5 up, 6 in 2024-03-20T07:56:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:56:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/507386161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]: dispatch 2024-03-20T07:56:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]: dispatch 2024-03-20T07:56:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]': finished 2024-03-20T07:56:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[38981]: osdmap e36: 6 total, 5 up, 6 in 2024-03-20T07:56:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:56:59 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:56:59.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:59 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/507386161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]: dispatch 2024-03-20T07:56:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:59 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]: dispatch 2024-03-20T07:56:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:59 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8d481e-0604-4613-b8fa-d138564be31d"}]': finished 2024-03-20T07:56:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:59 smithi196 ceph-mon[31033]: osdmap e36: 6 total, 5 up, 6 in 2024-03-20T07:56:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:56:59 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:00.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:00 smithi196 ceph-mon[31033]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:57:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:00 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/2640783446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:00.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:00 smithi118 ceph-mon[27391]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:57:00.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:00 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/2640783446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:00.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:00 smithi118 ceph-mon[38981]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:57:00.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:00 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/2640783446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:02.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:01 smithi196 ceph-mon[31033]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:57:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:01 smithi118 ceph-mon[27391]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:57:02.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:01 smithi118 ceph-mon[38981]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:57:04.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:04 smithi196 ceph-mon[31033]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:57:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:04 smithi118 ceph-mon[27391]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:57:04.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:04 smithi118 ceph-mon[38981]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:57:06.538 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:06 smithi196 ceph-mon[31033]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:06.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T07:57:06.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:06.539 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:06 smithi196 ceph-mon[31033]: Deploying daemon osd.5 on smithi196 2024-03-20T07:57:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:06 smithi118 ceph-mon[27391]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T07:57:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:06 smithi118 ceph-mon[27391]: Deploying daemon osd.5 on smithi196 2024-03-20T07:57:06.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:06 smithi118 ceph-mon[38981]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:06.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T07:57:06.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:06.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:06 smithi118 ceph-mon[38981]: Deploying daemon osd.5 on smithi196 2024-03-20T07:57:08.499 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:08 smithi196 ceph-mon[31033]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:08 smithi118 ceph-mon[27391]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:08.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:08 smithi118 ceph-mon[38981]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:10.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:10 smithi196 ceph-mon[31033]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:10.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:10 smithi118 ceph-mon[27391]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:10.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:10 smithi118 ceph-mon[38981]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:11.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:11 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:11 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:11 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:11 smithi196 ceph-mon[31033]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:11.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:11 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:11.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:11 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:11.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:11 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:11.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:11 smithi118 ceph-mon[27391]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:11.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:11 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:11.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:11 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:11.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:11 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:11.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:11 smithi118 ceph-mon[38981]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:13.962 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-03-20T07:57:14.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.561 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:14 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:14.580 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.5.service 2024-03-20T07:57:14.582 INFO:tasks.cephadm:Deploying osd.6 on smithi196 with /dev/vg_nvme/lv_2... 2024-03-20T07:57:14.582 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-20T07:57:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T07:57:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T07:57:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:14 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:15.388 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:57:15.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:15 smithi196 ceph-mon[31033]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-20T07:57:15.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:15 smithi196 ceph-mon[31033]: osdmap e37: 6 total, 5 up, 6 in 2024-03-20T07:57:15.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:15 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:15.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:15 smithi196 ceph-mon[31033]: from='osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:15.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:15 smithi196 ceph-mon[31033]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:15.618 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 07:57:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[42600]: 2024-03-20T07:57:15.362+0000 7f481d73d700 -1 osd.5 0 waiting for initial osdmap 2024-03-20T07:57:15.618 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 07:57:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[42600]: 2024-03-20T07:57:15.369+0000 7f481752e700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T07:57:15.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[27391]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-20T07:57:15.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[27391]: osdmap e37: 6 total, 5 up, 6 in 2024-03-20T07:57:15.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:15.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[27391]: from='osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:15.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[27391]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:15.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[38981]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-20T07:57:15.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[38981]: osdmap e37: 6 total, 5 up, 6 in 2024-03-20T07:57:15.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:15.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[38981]: from='osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:15.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:15 smithi118 ceph-mon[38981]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:16 smithi196 ceph-mon[31033]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:16 smithi196 ceph-mon[31033]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:57:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:16 smithi196 ceph-mon[31033]: osdmap e38: 6 total, 5 up, 6 in 2024-03-20T07:57:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:16 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:16 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:16 smithi196 ceph-mon[31033]: osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509] boot 2024-03-20T07:57:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:16 smithi196 ceph-mon[31033]: osdmap e39: 6 total, 6 up, 6 in 2024-03-20T07:57:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:16 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:16.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[38981]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:16.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[38981]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:57:16.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[38981]: osdmap e38: 6 total, 5 up, 6 in 2024-03-20T07:57:16.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[38981]: osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509] boot 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[38981]: osdmap e39: 6 total, 6 up, 6 in 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[27391]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[27391]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[27391]: osdmap e38: 6 total, 5 up, 6 in 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[27391]: osd.5 [v2:172.21.15.196:6808/3669975509,v1:172.21.15.196:6809/3669975509] boot 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[27391]: osdmap e39: 6 total, 6 up, 6 in 2024-03-20T07:57:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:16 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:57:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:17 smithi196 ceph-mon[31033]: purged_snaps scrub starts 2024-03-20T07:57:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:17 smithi196 ceph-mon[31033]: purged_snaps scrub ok 2024-03-20T07:57:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:17 smithi196 ceph-mon[31033]: osdmap e40: 6 total, 6 up, 6 in 2024-03-20T07:57:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:17 smithi118 ceph-mon[27391]: purged_snaps scrub starts 2024-03-20T07:57:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:17 smithi118 ceph-mon[27391]: purged_snaps scrub ok 2024-03-20T07:57:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:17 smithi118 ceph-mon[27391]: osdmap e40: 6 total, 6 up, 6 in 2024-03-20T07:57:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:17 smithi118 ceph-mon[38981]: purged_snaps scrub starts 2024-03-20T07:57:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:17 smithi118 ceph-mon[38981]: purged_snaps scrub ok 2024-03-20T07:57:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:17 smithi118 ceph-mon[38981]: osdmap e40: 6 total, 6 up, 6 in 2024-03-20T07:57:17.694 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-20T07:57:17.695 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-20T07:57:17.695 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-20T07:57:17.695 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-20T07:57:17.695 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126228 s, 831 MB/s 2024-03-20T07:57:17.695 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-20T07:57:18.458 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:18 smithi196 ceph-mon[31033]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:18.458 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:18 smithi196 ceph-mon[31033]: osdmap e41: 6 total, 6 up, 6 in 2024-03-20T07:57:18.480 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-03-20T07:57:18.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:18 smithi118 ceph-mon[27391]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:18.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:18 smithi118 ceph-mon[27391]: osdmap e41: 6 total, 6 up, 6 in 2024-03-20T07:57:18.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:18 smithi118 ceph-mon[38981]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:18.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:18 smithi118 ceph-mon[38981]: osdmap e41: 6 total, 6 up, 6 in 2024-03-20T07:57:19.210 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:57:20.310 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: osdmap e42: 6 total, 6 up, 6 in 2024-03-20T07:57:20.310 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: Detected new or changed devices on smithi196 2024-03-20T07:57:20.310 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.310 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.310 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:20.310 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:20.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi196 to 8544M 2024-03-20T07:57:20.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:20.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:20.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:19 smithi196 ceph-mon[31033]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: osdmap e42: 6 total, 6 up, 6 in 2024-03-20T07:57:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: Detected new or changed devices on smithi196 2024-03-20T07:57:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi196 to 8544M 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[27391]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: osdmap e42: 6 total, 6 up, 6 in 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: Detected new or changed devices on smithi196 2024-03-20T07:57:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi196 to 8544M 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:19 smithi118 ceph-mon[38981]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:21.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:20 smithi196 ceph-mon[31033]: osdmap e43: 6 total, 6 up, 6 in 2024-03-20T07:57:21.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:20 smithi196 ceph-mon[31033]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-20T07:57:21.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:20 smithi118 ceph-mon[27391]: osdmap e43: 6 total, 6 up, 6 in 2024-03-20T07:57:21.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:20 smithi118 ceph-mon[27391]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-20T07:57:21.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:20 smithi118 ceph-mon[38981]: osdmap e43: 6 total, 6 up, 6 in 2024-03-20T07:57:21.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:20 smithi118 ceph-mon[38981]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-20T07:57:22.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:21 smithi196 ceph-mon[31033]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:57:22.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:21 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:57:22.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:21 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:57:22.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:21 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:22.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:21 smithi196 ceph-mon[31033]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[27391]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:57:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:57:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:57:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[27391]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:22.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[38981]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:57:22.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:57:22.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:57:22.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:22.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:21 smithi118 ceph-mon[38981]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:23.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:23 smithi196 ceph-mon[31033]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-20T07:57:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:23 smithi196 ceph-mon[31033]: Cluster is now healthy 2024-03-20T07:57:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:23 smithi118 ceph-mon[27391]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-20T07:57:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:23 smithi118 ceph-mon[27391]: Cluster is now healthy 2024-03-20T07:57:23.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:23 smithi118 ceph-mon[38981]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-20T07:57:23.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:23 smithi118 ceph-mon[38981]: Cluster is now healthy 2024-03-20T07:57:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:24 smithi196 ceph-mon[31033]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:24 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/257844707' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]: dispatch 2024-03-20T07:57:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:24 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]: dispatch 2024-03-20T07:57:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:24 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]': finished 2024-03-20T07:57:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:24 smithi196 ceph-mon[31033]: osdmap e44: 7 total, 6 up, 7 in 2024-03-20T07:57:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:24 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/2643174391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[27391]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/257844707' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]: dispatch 2024-03-20T07:57:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]: dispatch 2024-03-20T07:57:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]': finished 2024-03-20T07:57:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[27391]: osdmap e44: 7 total, 6 up, 7 in 2024-03-20T07:57:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:24.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/2643174391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[38981]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/257844707' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]: dispatch 2024-03-20T07:57:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]: dispatch 2024-03-20T07:57:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69651e4b-3242-4177-9672-30e0979d33a0"}]': finished 2024-03-20T07:57:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[38981]: osdmap e44: 7 total, 6 up, 7 in 2024-03-20T07:57:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:24 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/2643174391' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:26.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:26 smithi196 ceph-mon[31033]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-20T07:57:26.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:26 smithi118 ceph-mon[27391]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-20T07:57:26.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:26 smithi118 ceph-mon[38981]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-20T07:57:28.543 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:28 smithi196 ceph-mon[31033]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-20T07:57:28.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:28 smithi118 ceph-mon[27391]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-20T07:57:28.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:28 smithi118 ceph-mon[38981]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-20T07:57:30.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:30 smithi196 ceph-mon[31033]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-20T07:57:30.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T07:57:30.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:30 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:30 smithi118 ceph-mon[27391]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-20T07:57:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T07:57:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:30 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:30.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:30 smithi118 ceph-mon[38981]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-20T07:57:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T07:57:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:30 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:31.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:31 smithi196 ceph-mon[31033]: Deploying daemon osd.6 on smithi196 2024-03-20T07:57:31.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:31 smithi118 ceph-mon[27391]: Deploying daemon osd.6 on smithi196 2024-03-20T07:57:31.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:31 smithi118 ceph-mon[38981]: Deploying daemon osd.6 on smithi196 2024-03-20T07:57:32.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:32 smithi196 ceph-mon[31033]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:32 smithi118 ceph-mon[27391]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:32.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:32 smithi118 ceph-mon[38981]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:34.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:34 smithi196 ceph-mon[31033]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:34 smithi118 ceph-mon[27391]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:34.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:34 smithi118 ceph-mon[38981]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:36 smithi196 ceph-mon[31033]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:36.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:36 smithi118 ceph-mon[27391]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:36 smithi118 ceph-mon[38981]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:38.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:38 smithi196 ceph-mon[31033]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:38.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:38 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:38.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:38 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:38 smithi118 ceph-mon[27391]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:38 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:38 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:38 smithi118 ceph-mon[38981]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:38 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:38 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:38.997 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 6 on host 'smithi196' 2024-03-20T07:57:39.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:39 smithi196 ceph-mon[31033]: from='osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T07:57:39.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:39 smithi196 ceph-mon[31033]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T07:57:39.558 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:39 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:39.558 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:39 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:39.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:39 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:39 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:39 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.559 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:39 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:39.578 DEBUG:teuthology.orchestra.run.smithi196:osd.6> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.6.service 2024-03-20T07:57:39.581 INFO:tasks.cephadm:Deploying osd.7 on smithi196 with /dev/vg_nvme/lv_1... 2024-03-20T07:57:39.581 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-20T07:57:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[27391]: from='osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T07:57:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[27391]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T07:57:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[38981]: from='osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[38981]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:39 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:57:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:40 smithi196 ceph-mon[31033]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:40 smithi196 ceph-mon[31033]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-20T07:57:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:40 smithi196 ceph-mon[31033]: osdmap e45: 7 total, 6 up, 7 in 2024-03-20T07:57:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:40 smithi196 ceph-mon[31033]: from='osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:40 smithi196 ceph-mon[31033]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:40.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 07:57:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[48573]: 2024-03-20T07:57:40.294+0000 7f97f3625700 -1 osd.6 0 waiting for initial osdmap 2024-03-20T07:57:40.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 07:57:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[48573]: 2024-03-20T07:57:40.301+0000 7f97ed416700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T07:57:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[27391]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[27391]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-20T07:57:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[27391]: osdmap e45: 7 total, 6 up, 7 in 2024-03-20T07:57:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[27391]: from='osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[27391]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[38981]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[38981]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-20T07:57:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[38981]: osdmap e45: 7 total, 6 up, 7 in 2024-03-20T07:57:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[38981]: from='osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:40 smithi118 ceph-mon[38981]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:57:41.180 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:57:41.587 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:41 smithi196 ceph-mon[31033]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:57:41.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:41 smithi196 ceph-mon[31033]: osdmap e46: 7 total, 6 up, 7 in 2024-03-20T07:57:41.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:41.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:41.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:41 smithi196 ceph-mon[31033]: osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477] boot 2024-03-20T07:57:41.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:41 smithi196 ceph-mon[31033]: osdmap e47: 7 total, 7 up, 7 in 2024-03-20T07:57:41.588 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:41 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[27391]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:57:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[27391]: osdmap e46: 7 total, 6 up, 7 in 2024-03-20T07:57:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[27391]: osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477] boot 2024-03-20T07:57:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[27391]: osdmap e47: 7 total, 7 up, 7 in 2024-03-20T07:57:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[38981]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:57:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[38981]: osdmap e46: 7 total, 6 up, 7 in 2024-03-20T07:57:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[38981]: osd.6 [v2:172.21.15.196:6816/1568164477,v1:172.21.15.196:6817/1568164477] boot 2024-03-20T07:57:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[38981]: osdmap e47: 7 total, 7 up, 7 in 2024-03-20T07:57:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:41 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:57:42.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:42 smithi196 ceph-mon[31033]: purged_snaps scrub starts 2024-03-20T07:57:42.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:42 smithi196 ceph-mon[31033]: purged_snaps scrub ok 2024-03-20T07:57:42.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:42 smithi196 ceph-mon[31033]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:42.526 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:42 smithi196 ceph-mon[31033]: osdmap e48: 7 total, 7 up, 7 in 2024-03-20T07:57:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:42 smithi118 ceph-mon[27391]: purged_snaps scrub starts 2024-03-20T07:57:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:42 smithi118 ceph-mon[27391]: purged_snaps scrub ok 2024-03-20T07:57:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:42 smithi118 ceph-mon[27391]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:42 smithi118 ceph-mon[27391]: osdmap e48: 7 total, 7 up, 7 in 2024-03-20T07:57:42.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:42 smithi118 ceph-mon[38981]: purged_snaps scrub starts 2024-03-20T07:57:42.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:42 smithi118 ceph-mon[38981]: purged_snaps scrub ok 2024-03-20T07:57:42.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:42 smithi118 ceph-mon[38981]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-20T07:57:42.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:42 smithi118 ceph-mon[38981]: osdmap e48: 7 total, 7 up, 7 in 2024-03-20T07:57:43.807 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-20T07:57:43.807 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-20T07:57:43.807 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-20T07:57:43.807 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-20T07:57:43.807 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112869 s, 929 MB/s 2024-03-20T07:57:43.807 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: osdmap e49: 7 total, 7 up, 7 in 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: Detected new or changed devices on smithi196 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi196 to 5696M 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:43 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: osdmap e49: 7 total, 7 up, 7 in 2024-03-20T07:57:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:57:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: Detected new or changed devices on smithi196 2024-03-20T07:57:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi196 to 5696M 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: osdmap e49: 7 total, 7 up, 7 in 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: Detected new or changed devices on smithi196 2024-03-20T07:57:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi196 to 5696M 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:57:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:43 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:57:44.525 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-03-20T07:57:44.730 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:57:45.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:44 smithi196 ceph-mon[31033]: osdmap e50: 7 total, 7 up, 7 in 2024-03-20T07:57:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:44 smithi118 ceph-mon[27391]: osdmap e50: 7 total, 7 up, 7 in 2024-03-20T07:57:45.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:44 smithi118 ceph-mon[38981]: osdmap e50: 7 total, 7 up, 7 in 2024-03-20T07:57:46.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:45 smithi196 ceph-mon[31033]: osdmap e51: 7 total, 7 up, 7 in 2024-03-20T07:57:46.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:45 smithi196 ceph-mon[31033]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:57:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:45 smithi118 ceph-mon[27391]: osdmap e51: 7 total, 7 up, 7 in 2024-03-20T07:57:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:45 smithi118 ceph-mon[27391]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:57:46.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:45 smithi118 ceph-mon[38981]: osdmap e51: 7 total, 7 up, 7 in 2024-03-20T07:57:46.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:45 smithi118 ceph-mon[38981]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:57:47.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:57:47.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:57:47.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:57:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:57:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:47.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-20T07:57:47.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-20T07:57:47.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:48.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:47 smithi196 ceph-mon[31033]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:57:48.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:47 smithi196 ceph-mon[31033]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-20T07:57:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:47 smithi118 ceph-mon[27391]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:57:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:47 smithi118 ceph-mon[27391]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-20T07:57:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:47 smithi118 ceph-mon[38981]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:57:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:47 smithi118 ceph-mon[38981]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-20T07:57:49.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:49 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/2637239934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]: dispatch 2024-03-20T07:57:49.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:49 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]: dispatch 2024-03-20T07:57:49.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:49 smithi196 ceph-mon[31033]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]': finished 2024-03-20T07:57:49.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:49 smithi196 ceph-mon[31033]: osdmap e52: 8 total, 7 up, 8 in 2024-03-20T07:57:49.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:49 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:57:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/2637239934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]: dispatch 2024-03-20T07:57:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]: dispatch 2024-03-20T07:57:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[27391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]': finished 2024-03-20T07:57:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[27391]: osdmap e52: 8 total, 7 up, 8 in 2024-03-20T07:57:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:57:49.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/2637239934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]: dispatch 2024-03-20T07:57:49.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]: dispatch 2024-03-20T07:57:49.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[38981]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "279db45c-13c3-4aaa-b9e2-f65751ae6beb"}]': finished 2024-03-20T07:57:49.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[38981]: osdmap e52: 8 total, 7 up, 8 in 2024-03-20T07:57:49.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:49 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:57:50.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:50 smithi196 ceph-mon[31033]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:57:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:50 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/1440304592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:50 smithi118 ceph-mon[27391]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:57:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:50 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/1440304592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:50.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:50 smithi118 ceph-mon[38981]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:57:50.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:50 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/1440304592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-20T07:57:52.349 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:51 smithi196 ceph-mon[31033]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:57:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:51 smithi118 ceph-mon[27391]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:57:52.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:51 smithi118 ceph-mon[38981]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:57:54.470 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:54 smithi196 ceph-mon[31033]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:57:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:54 smithi118 ceph-mon[27391]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:57:54.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:54 smithi118 ceph-mon[38981]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:57:56.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:56 smithi196 ceph-mon[31033]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:56 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T07:57:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:56 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:56.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:56 smithi118 ceph-mon[27391]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:56.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:56 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T07:57:56.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:56 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:56.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:56 smithi118 ceph-mon[38981]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:57:56.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:56 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T07:57:56.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:56 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:57:57.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:57 smithi196 ceph-mon[31033]: Deploying daemon osd.7 on smithi196 2024-03-20T07:57:57.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:57 smithi118 ceph-mon[27391]: Deploying daemon osd.7 on smithi196 2024-03-20T07:57:57.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:57 smithi118 ceph-mon[38981]: Deploying daemon osd.7 on smithi196 2024-03-20T07:57:58.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:57:58 smithi196 ceph-mon[31033]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:57:58.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:57:58 smithi118 ceph-mon[27391]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:57:58.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:57:58 smithi118 ceph-mon[38981]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:00.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:00 smithi196 ceph-mon[31033]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:00.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:00 smithi118 ceph-mon[27391]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:00.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:00 smithi118 ceph-mon[38981]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:02.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:01 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:02.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:01 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:02.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:01 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:02.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:01 smithi196 ceph-mon[31033]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:01 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:01 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:01 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:01 smithi118 ceph-mon[27391]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:02.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:01 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:02.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:01 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:01 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:01 smithi118 ceph-mon[38981]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:04.327 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:04 smithi196 ceph-mon[31033]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:04.327 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:04 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:04 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:04 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:04.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:04 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:04.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:04 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:04 smithi196 ceph-mon[31033]: from='osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T07:58:04.328 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:04 smithi196 ceph-mon[31033]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T07:58:04.408 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 7 on host 'smithi196' 2024-03-20T07:58:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[27391]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[27391]: from='osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[27391]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[38981]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[38981]: from='osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T07:58:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:04 smithi118 ceph-mon[38981]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T07:58:05.095 DEBUG:teuthology.orchestra.run.smithi196:osd.7> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.7.service 2024-03-20T07:58:05.097 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-03-20T07:58:05.098 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-20T07:58:05.312 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:58:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:05.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:05.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[27391]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-20T07:58:05.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[27391]: osdmap e53: 8 total, 7 up, 8 in 2024-03-20T07:58:05.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:05.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[27391]: from='osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:58:05.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[27391]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:58:05.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:05.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:05.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:05.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[38981]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-20T07:58:05.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[38981]: osdmap e53: 8 total, 7 up, 8 in 2024-03-20T07:58:05.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:05.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[38981]: from='osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:58:05.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:05 smithi118 ceph-mon[38981]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:58:05.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:05.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:05.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:05.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-mon[31033]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-20T07:58:05.617 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-mon[31033]: osdmap e53: 8 total, 7 up, 8 in 2024-03-20T07:58:05.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:05.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-mon[31033]: from='osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:58:05.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-mon[31033]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T07:58:05.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[54676]: 2024-03-20T07:58:05.686+0000 7f102944c700 -1 osd.7 0 waiting for initial osdmap 2024-03-20T07:58:05.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 07:58:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[54676]: 2024-03-20T07:58:05.693+0000 7f1022a3c700 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T07:58:06.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:06 smithi196 ceph-mon[31033]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:06.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:06 smithi196 ceph-mon[31033]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:58:06.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:06 smithi196 ceph-mon[31033]: osdmap e54: 8 total, 7 up, 8 in 2024-03-20T07:58:06.570 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:06.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:06.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:06 smithi196 ceph-mon[31033]: osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474] boot 2024-03-20T07:58:06.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:06 smithi196 ceph-mon[31033]: osdmap e55: 8 total, 8 up, 8 in 2024-03-20T07:58:06.571 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:06 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:06.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[27391]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:06.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[27391]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[27391]: osdmap e54: 8 total, 7 up, 8 in 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[27391]: osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474] boot 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[27391]: osdmap e55: 8 total, 8 up, 8 in 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[38981]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[38981]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[38981]: osdmap e54: 8 total, 7 up, 8 in 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:06.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[38981]: osd.7 [v2:172.21.15.196:6824/2088108474,v1:172.21.15.196:6825/2088108474] boot 2024-03-20T07:58:06.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[38981]: osdmap e55: 8 total, 8 up, 8 in 2024-03-20T07:58:06.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:06 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:58:07.343 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:07 smithi196 ceph-mon[31033]: purged_snaps scrub starts 2024-03-20T07:58:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:07 smithi196 ceph-mon[31033]: purged_snaps scrub ok 2024-03-20T07:58:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:07 smithi196 ceph-mon[31033]: osdmap e56: 8 total, 8 up, 8 in 2024-03-20T07:58:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:07 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3750493052' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-20T07:58:07.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:07 smithi118 ceph-mon[27391]: purged_snaps scrub starts 2024-03-20T07:58:07.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:07 smithi118 ceph-mon[27391]: purged_snaps scrub ok 2024-03-20T07:58:07.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:07 smithi118 ceph-mon[27391]: osdmap e56: 8 total, 8 up, 8 in 2024-03-20T07:58:07.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:07 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3750493052' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-20T07:58:07.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:07 smithi118 ceph-mon[38981]: purged_snaps scrub starts 2024-03-20T07:58:07.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:07 smithi118 ceph-mon[38981]: purged_snaps scrub ok 2024-03-20T07:58:07.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:07 smithi118 ceph-mon[38981]: osdmap e56: 8 total, 8 up, 8 in 2024-03-20T07:58:07.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:07 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3750493052' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-20T07:58:08.119 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1710921485,"num_in_osds":8,"osd_in_since":1710921469,"num_remapped_pgs":1} 2024-03-20T07:58:08.120 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-20T07:58:08.322 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: osdmap e57: 8 total, 8 up, 8 in 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:08 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: osdmap e57: 8 total, 8 up, 8 in 2024-03-20T07:58:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: osdmap e57: 8 total, 8 up, 8 in 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:58:08.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:08.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:08.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:08.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:08 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:09.539 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:09 smithi118 ceph-mon[27391]: Detected new or changed devices on smithi196 2024-03-20T07:58:09.540 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:09 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi196 to 4272M 2024-03-20T07:58:09.540 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:09 smithi118 ceph-mon[27391]: osdmap e58: 8 total, 8 up, 8 in 2024-03-20T07:58:09.540 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:09 smithi118 ceph-mon[38981]: Detected new or changed devices on smithi196 2024-03-20T07:58:09.540 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:09 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi196 to 4272M 2024-03-20T07:58:09.540 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:09 smithi118 ceph-mon[38981]: osdmap e58: 8 total, 8 up, 8 in 2024-03-20T07:58:09.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:09 smithi196 ceph-mon[31033]: Detected new or changed devices on smithi196 2024-03-20T07:58:09.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:09 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi196 to 4272M 2024-03-20T07:58:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:09 smithi196 ceph-mon[31033]: osdmap e58: 8 total, 8 up, 8 in 2024-03-20T07:58:10.312 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:10.313 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":59,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:58:09.955658+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:58:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:10 smithi196 ceph-mon[31033]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:10 smithi196 ceph-mon[31033]: osdmap e59: 8 total, 8 up, 8 in 2024-03-20T07:58:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:10 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2215790126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:10.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:10 smithi118 ceph-mon[38981]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:10.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:10 smithi118 ceph-mon[38981]: osdmap e59: 8 total, 8 up, 8 in 2024-03-20T07:58:10.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:10 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2215790126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:10.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:10 smithi118 ceph-mon[27391]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:10.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:10 smithi118 ceph-mon[27391]: osdmap e59: 8 total, 8 up, 8 in 2024-03-20T07:58:10.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:10 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2215790126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:11.102 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-20T07:56:03.504701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-20T07:58:11.103 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-20T07:58:11.306 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:58:12.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:11 smithi118 ceph-mon[27391]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:12.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:11 smithi118 ceph-mon[38981]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:12.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:11 smithi196 ceph-mon[31033]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:13.342 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-03-20T07:58:14.122 INFO:tasks.cephadm:Adding mds.a on smithi118 2024-03-20T07:58:14.122 INFO:tasks.cephadm:Adding mds.b on smithi196 2024-03-20T07:58:14.123 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch apply mds all '2;smithi118=a;smithi196=b' 2024-03-20T07:58:14.339 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:58:14.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:14 smithi118 ceph-mon[38981]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:58:14.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:14 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3653885543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:14 smithi118 ceph-mon[27391]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:58:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:14 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3653885543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:14.453 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:14 smithi196 ceph-mon[31033]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-20T07:58:14.453 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:14 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3653885543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:16.275 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-03-20T07:58:16.573 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:16 smithi196 ceph-mon[31033]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:58:16.573 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:16 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:16.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:16 smithi118 ceph-mon[38981]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:58:16.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:16 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:16.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:16 smithi118 ceph-mon[27391]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-20T07:58:16.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:16 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:16.878 DEBUG:teuthology.orchestra.run.smithi118:mds.a> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.a.service 2024-03-20T07:58:16.881 DEBUG:teuthology.orchestra.run.smithi196:mds.b> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.b.service 2024-03-20T07:58:16.883 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-20T07:58:16.883 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T07:58:17.228 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:17.228 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":59,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:58:09.955658+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:58:17.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-20T07:56:03.504701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-20T07:58:17.241 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-20T07:58:17.528 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "2;smithi118=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: Saving service mds.all spec with placement smithi118=a;smithi196=b;count:2 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/277112889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:17.529 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "2;smithi118=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: Saving service mds.all spec with placement smithi118=a;smithi196=b;count:2 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:17.530 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:17 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/277112889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:17.535 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-03-20T07:58:17.551 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "2;smithi118=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: Saving service mds.all spec with placement smithi118=a;smithi196=b;count:2 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:17 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/277112889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:17.870 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:17.870 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-20T07:58:17.870 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 1 2024-03-20T07:58:18.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:18 smithi196 ceph-mon[31033]: Deploying daemon mds.a on smithi118 2024-03-20T07:58:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:18 smithi196 ceph-mon[31033]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:58:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:18 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3254899900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:18 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2332647744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:18.660 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:18 smithi118 ceph-mon[27391]: Deploying daemon mds.a on smithi118 2024-03-20T07:58:18.660 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:18 smithi118 ceph-mon[27391]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:58:18.660 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:18 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3254899900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:18.660 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:18 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2332647744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:18.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:18 smithi118 ceph-mon[38981]: Deploying daemon mds.a on smithi118 2024-03-20T07:58:18.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:18 smithi118 ceph-mon[38981]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-20T07:58:18.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:18 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3254899900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:18.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:18 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2332647744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:18.922 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 07:58:18 smithi118 systemd[1]: Starting Ceph mds.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:58:19.301 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 07:58:19 smithi118 podman[67448]: 2024-03-20 07:58:18.954181201 +0000 UTC m=+0.018531344 image pull 3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b 2024-03-20T07:58:19.301 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 07:58:19 smithi118 podman[67448]: 2024-03-20 07:58:19.06159193 +0000 UTC m=+0.125942101 container create 2909c37f937131c8a703b63a3b882aebeae9e8f8a7d66634ac4808d442b21253 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=reef-78dbc53, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-20T07:58:19.301 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 07:58:19 smithi118 podman[67448]: 2024-03-20 07:58:19.29704512 +0000 UTC m=+0.361395258 container init 2909c37f937131c8a703b63a3b882aebeae9e8f8a7d66634ac4808d442b21253 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-20T07:58:19.301 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 07:58:19 smithi118 podman[67448]: 2024-03-20 07:58:19.302423949 +0000 UTC m=+0.366774084 container start 2909c37f937131c8a703b63a3b882aebeae9e8f8a7d66634ac4808d442b21253 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-78dbc53, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-20T07:58:19.672 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 07:58:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[67464]: starting mds.a at 2024-03-20T07:58:19.672 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 07:58:19 smithi118 bash[67448]: 2909c37f937131c8a703b63a3b882aebeae9e8f8a7d66634ac4808d442b21253 2024-03-20T07:58:19.672 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 07:58:19 smithi118 systemd[1]: Started Ceph mds.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:58:19.883 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T07:58:20.207 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:20.207 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3215808514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3215808514},{"type":"v1","addr":"172.21.15.118:6835","nonce":3215808514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-20T07:58:20.207 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 2 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: mds.? [v2:172.21.15.118:6834/3215808514,v1:172.21.15.118:6835/3215808514] up:boot 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: fsmap 1 up:standby 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:20 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1796082751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:20.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:58:20.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: mds.? [v2:172.21.15.118:6834/3215808514,v1:172.21.15.118:6835/3215808514] up:boot 2024-03-20T07:58:20.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: fsmap 1 up:standby 2024-03-20T07:58:20.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T07:58:20.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1796082751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: mds.? [v2:172.21.15.118:6834/3215808514,v1:172.21.15.118:6835/3215808514] up:boot 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: fsmap 1 up:standby 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T07:58:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-20T07:58:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:20 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1796082751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:21.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:21 smithi196 ceph-mon[31033]: Deploying daemon mds.b on smithi196 2024-03-20T07:58:21.633 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:21 smithi196 systemd[1]: Starting Ceph mds.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:58:21.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:21 smithi118 ceph-mon[27391]: Deploying daemon mds.b on smithi196 2024-03-20T07:58:21.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:21 smithi118 ceph-mon[38981]: Deploying daemon mds.b on smithi196 2024-03-20T07:58:21.898 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:21 smithi196 podman[58673]: 2024-03-20 07:58:21.537319464 +0000 UTC m=+0.018776212 image pull 3b5d1eaa4228196f9de5cc0fee191ec85c854f97f6cdeea98699ac74f8b76f94 quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b 2024-03-20T07:58:21.899 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:21 smithi196 podman[58673]: 2024-03-20 07:58:21.641058141 +0000 UTC m=+0.122514876 container create e5e6b027dc5893db1e3f2204d656aab142a8b00d9d63c471a645e6097fc36bbf (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-78dbc53, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-20T07:58:22.220 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T07:58:22.344 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:21 smithi196 podman[58673]: 2024-03-20 07:58:21.893526562 +0000 UTC m=+0.374983317 container init e5e6b027dc5893db1e3f2204d656aab142a8b00d9d63c471a645e6097fc36bbf (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-78dbc53, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-20T07:58:22.344 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:21 smithi196 podman[58673]: 2024-03-20 07:58:21.898557902 +0000 UTC m=+0.380014638 container start e5e6b027dc5893db1e3f2204d656aab142a8b00d9d63c471a645e6097fc36bbf (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, org.label-schema.schema-version=1.0, RELEASE=reef-78dbc53, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD) 2024-03-20T07:58:22.344 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b[58688]: starting mds.b at 2024-03-20T07:58:22.345 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:21 smithi196 bash[58673]: e5e6b027dc5893db1e3f2204d656aab142a8b00d9d63c471a645e6097fc36bbf 2024-03-20T07:58:22.345 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:21 smithi196 systemd[1]: Started Ceph mds.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:58:22.557 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:22.558 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3215808514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3215808514},{"type":"v1","addr":"172.21.15.118:6835","nonce":3215808514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24343,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/5972345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":5972345},{"type":"v1","addr":"172.21.15.196:6833","nonce":5972345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-20T07:58:22.558 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 3 2024-03-20T07:58:22.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[38981]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-20T07:58:22.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:22.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[27391]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-20T07:58:22.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:22 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:22.573 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T07:58:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:22 smithi196 ceph-mon[31033]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-20T07:58:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:22 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:22 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:22 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:22 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:22 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:22.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:22 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:22.874 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:22.874 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":59,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:58:09.955658+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:58:22.886 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-20T07:56:03.504701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-20T07:58:22.886 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-20T07:58:23.181 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-03-20T07:58:23.193 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-20T07:58:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[27391]: mds.? [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] up:boot 2024-03-20T07:58:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[27391]: fsmap 2 up:standby 2024-03-20T07:58:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3303592871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3117748382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1237705762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[38981]: mds.? [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] up:boot 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[38981]: fsmap 2 up:standby 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3303592871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3117748382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:23 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1237705762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:23.508 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:23.508 INFO:teuthology.orchestra.run.smithi118.stdout:[ 2024-03-20T07:58:23.508 INFO:teuthology.orchestra.run.smithi118.stdout: { 2024-03-20T07:58:23.508 INFO:teuthology.orchestra.run.smithi118.stdout: "poolnum": 1, 2024-03-20T07:58:23.508 INFO:teuthology.orchestra.run.smithi118.stdout: "poolname": ".mgr" 2024-03-20T07:58:23.508 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T07:58:23.508 INFO:teuthology.orchestra.run.smithi118.stdout:] 2024-03-20T07:58:23.522 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-20T07:58:23.522 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-20T07:58:23.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:23 smithi196 ceph-mon[31033]: mds.? [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] up:boot 2024-03-20T07:58:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:23 smithi196 ceph-mon[31033]: fsmap 2 up:standby 2024-03-20T07:58:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:23 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T07:58:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:23 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3303592871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:23 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3117748382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:23 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1237705762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/905926767' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/963422607' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-20T07:58:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:24.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:24.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:24.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:24 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-20T07:58:24.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/905926767' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-20T07:58:24.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/963422607' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/905926767' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/963422607' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:24.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:24 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:24.748 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_metadata' created 2024-03-20T07:58:24.763 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-20T07:58:25.752 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_data' created 2024-03-20T07:58:25.776 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-20T07:58:26.107 INFO:teuthology.orchestra.run.smithi118.stderr:new fs with metadata pool 2 and data pool 3 2024-03-20T07:58:26.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-20T07:58:26.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[27391]: osdmap e60: 8 total, 8 up, 8 in 2024-03-20T07:58:26.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2791016576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-20T07:58:26.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[27391]: pgmap v190: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:26.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:26.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-20T07:58:26.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[38981]: osdmap e60: 8 total, 8 up, 8 in 2024-03-20T07:58:26.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2791016576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-20T07:58:26.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[38981]: pgmap v190: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:26.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:25 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:26.134 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T07:58:26.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:25 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-20T07:58:26.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:25 smithi196 ceph-mon[31033]: osdmap e60: 8 total, 8 up, 8 in 2024-03-20T07:58:26.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:25 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2791016576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-20T07:58:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:25 smithi196 ceph-mon[31033]: pgmap v190: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:25 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:26.418 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:26.419 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":62,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:58:26.102468+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:58:26.431 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T07:58:26.712 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:26.713 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":62,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:58:26.102468+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:58:26.725 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2791016576' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: osdmap e61: 8 total, 8 up, 8 in 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1513209090' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: osdmap e62: 8 total, 8 up, 8 in 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-20T07:58:27.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: fsmap cephfs:1 {0=b=up:creating} 1 up:standby 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2556787677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1212922323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2791016576' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: osdmap e61: 8 total, 8 up, 8 in 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1513209090' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: osdmap e62: 8 total, 8 up, 8 in 2024-03-20T07:58:27.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-20T07:58:27.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: fsmap cephfs:1 {0=b=up:creating} 1 up:standby 2024-03-20T07:58:27.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-20T07:58:27.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2556787677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:27.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:26 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1212922323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:27.128 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-20T07:58:27.129 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-20T07:58:27.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T07:58:27.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2791016576' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-20T07:58:27.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: osdmap e61: 8 total, 8 up, 8 in 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1513209090' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: osdmap e62: 8 total, 8 up, 8 in 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: fsmap cephfs:1 {0=b=up:creating} 1 up:standby 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2556787677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:26 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1212922323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:27.638 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b[58688]: 2024-03-20T07:58:27.353+0000 7f358eca5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/4120402387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[38981]: osdmap e63: 8 total, 8 up, 8 in 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[38981]: mds.? [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] up:active 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[38981]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[38981]: pgmap v194: 97 pgs: 65 active+clean, 32 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2559395147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-20T07:58:28.131 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/4120402387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-20T07:58:28.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-20T07:58:28.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[27391]: osdmap e63: 8 total, 8 up, 8 in 2024-03-20T07:58:28.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[27391]: mds.? [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] up:active 2024-03-20T07:58:28.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-20T07:58:28.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[27391]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T07:58:28.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[27391]: pgmap v194: 97 pgs: 65 active+clean, 32 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-03-20T07:58:28.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:27 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2559395147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-20T07:58:28.133 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-20T07:58:28.133 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T07:58:28.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/4120402387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-20T07:58:28.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-20T07:58:28.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-mon[31033]: osdmap e63: 8 total, 8 up, 8 in 2024-03-20T07:58:28.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-mon[31033]: mds.? [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] up:active 2024-03-20T07:58:28.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-20T07:58:28.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-mon[31033]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T07:58:28.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-mon[31033]: pgmap v194: 97 pgs: 65 active+clean, 32 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 5 op/s 2024-03-20T07:58:28.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:27 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2559395147' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-20T07:58:28.458 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:28.459 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3215808514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3215808514},{"type":"v1","addr":"172.21.15.118:6835","nonce":3215808514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-20T07:58:26.102802+0000","modified":"2024-03-20T07:58:28.113441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24343":{"gid":24343,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/5972345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":5972345},{"type":"v1","addr":"172.21.15.196:6833","nonce":5972345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-20T07:58:28.459 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 6 2024-03-20T07:58:28.471 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T07:58:28.755 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:28.755 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":63,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:58:27.104251+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:58:28.768 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-20T07:58:28.768 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-20T07:58:29.057 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:29.058 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T07:58:29.066 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[27391]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T07:58:29.066 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[27391]: Cluster is now healthy 2024-03-20T07:58:29.066 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2559395147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-20T07:58:29.066 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[27391]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T07:58:29.066 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/24715312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:29.067 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[38981]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T07:58:29.067 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[38981]: Cluster is now healthy 2024-03-20T07:58:29.067 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2559395147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-20T07:58:29.067 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[38981]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T07:58:29.067 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:28 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/24715312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:29.068 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-03-20T07:58:27.236354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592114,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":181,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":749961216,"kb_used":222632,"kb_used_data":6448,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749738584,"statfs":{"total":767960285184,"available":767732310016,"internally_reserved":0,"allocated":6602752,"data_stored":3734391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1746,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000583"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.146486+0000","last_change":"2024-03-20T07:58:26.145662+0000","last_active":"2024-03-20T07:58:26.146486+0000","last_peered":"2024-03-20T07:58:26.146486+0000","last_clean":"2024-03-20T07:58:26.146486+0000","last_became_active":"2024-03-20T07:58:26.145286+0000","last_became_peered":"2024-03-20T07:58:26.145286+0000","last_unstale":"2024-03-20T07:58:26.146486+0000","last_undegraded":"2024-03-20T07:58:26.146486+0000","last_fullsized":"2024-03-20T07:58:26.146486+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108335+0000","last_change":"2024-03-20T07:58:26.129069+0000","last_active":"2024-03-20T07:58:27.108335+0000","last_peered":"2024-03-20T07:58:27.108335+0000","last_clean":"2024-03-20T07:58:27.108335+0000","last_became_active":"2024-03-20T07:58:26.128811+0000","last_became_peered":"2024-03-20T07:58:26.128811+0000","last_unstale":"2024-03-20T07:58:27.108335+0000","last_undegraded":"2024-03-20T07:58:27.108335+0000","last_fullsized":"2024-03-20T07:58:27.108335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139286+0000","last_change":"2024-03-20T07:58:26.138067+0000","last_active":"2024-03-20T07:58:26.139286+0000","last_peered":"2024-03-20T07:58:26.139286+0000","last_clean":"2024-03-20T07:58:26.139286+0000","last_became_active":"2024-03-20T07:58:26.137768+0000","last_became_peered":"2024-03-20T07:58:26.137768+0000","last_unstale":"2024-03-20T07:58:26.139286+0000","last_undegraded":"2024-03-20T07:58:26.139286+0000","last_fullsized":"2024-03-20T07:58:26.139286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134630+0000","last_change":"2024-03-20T07:58:26.131923+0000","last_active":"2024-03-20T07:58:26.134630+0000","last_peered":"2024-03-20T07:58:26.134630+0000","last_clean":"2024-03-20T07:58:26.134630+0000","last_became_active":"2024-03-20T07:58:26.131585+0000","last_became_peered":"2024-03-20T07:58:26.131585+0000","last_unstale":"2024-03-20T07:58:26.134630+0000","last_undegraded":"2024-03-20T07:58:26.134630+0000","last_fullsized":"2024-03-20T07:58:26.134630+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139194+0000","last_change":"2024-03-20T07:58:26.137308+0000","last_active":"2024-03-20T07:58:26.139194+0000","last_peered":"2024-03-20T07:58:26.139194+0000","last_clean":"2024-03-20T07:58:26.139194+0000","last_became_active":"2024-03-20T07:58:26.135727+0000","last_became_peered":"2024-03-20T07:58:26.135727+0000","last_unstale":"2024-03-20T07:58:26.139194+0000","last_undegraded":"2024-03-20T07:58:26.139194+0000","last_fullsized":"2024-03-20T07:58:26.139194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134374+0000","last_change":"2024-03-20T07:58:26.132941+0000","last_active":"2024-03-20T07:58:26.134374+0000","last_peered":"2024-03-20T07:58:26.134374+0000","last_clean":"2024-03-20T07:58:26.134374+0000","last_became_active":"2024-03-20T07:58:26.132230+0000","last_became_peered":"2024-03-20T07:58:26.132230+0000","last_unstale":"2024-03-20T07:58:26.134374+0000","last_undegraded":"2024-03-20T07:58:26.134374+0000","last_fullsized":"2024-03-20T07:58:26.134374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145322+0000","last_change":"2024-03-20T07:58:26.144631+0000","last_active":"2024-03-20T07:58:26.145322+0000","last_peered":"2024-03-20T07:58:26.145322+0000","last_clean":"2024-03-20T07:58:26.145322+0000","last_became_active":"2024-03-20T07:58:26.144302+0000","last_became_peered":"2024-03-20T07:58:26.144302+0000","last_unstale":"2024-03-20T07:58:26.145322+0000","last_undegraded":"2024-03-20T07:58:26.145322+0000","last_fullsized":"2024-03-20T07:58:26.145322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139423+0000","last_change":"2024-03-20T07:58:26.138657+0000","last_active":"2024-03-20T07:58:26.139423+0000","last_peered":"2024-03-20T07:58:26.139423+0000","last_clean":"2024-03-20T07:58:26.139423+0000","last_became_active":"2024-03-20T07:58:26.138348+0000","last_became_peered":"2024-03-20T07:58:26.138348+0000","last_unstale":"2024-03-20T07:58:26.139423+0000","last_undegraded":"2024-03-20T07:58:26.139423+0000","last_fullsized":"2024-03-20T07:58:26.139423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138763+0000","last_change":"2024-03-20T07:58:26.137961+0000","last_active":"2024-03-20T07:58:26.138763+0000","last_peered":"2024-03-20T07:58:26.138763+0000","last_clean":"2024-03-20T07:58:26.138763+0000","last_became_active":"2024-03-20T07:58:26.137672+0000","last_became_peered":"2024-03-20T07:58:26.137672+0000","last_unstale":"2024-03-20T07:58:26.138763+0000","last_undegraded":"2024-03-20T07:58:26.138763+0000","last_fullsized":"2024-03-20T07:58:26.138763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.137140+0000","last_change":"2024-03-20T07:58:26.136246+0000","last_active":"2024-03-20T07:58:26.137140+0000","last_peered":"2024-03-20T07:58:26.137140+0000","last_clean":"2024-03-20T07:58:26.137140+0000","last_became_active":"2024-03-20T07:58:26.135935+0000","last_became_peered":"2024-03-20T07:58:26.135935+0000","last_unstale":"2024-03-20T07:58:26.137140+0000","last_undegraded":"2024-03-20T07:58:26.137140+0000","last_fullsized":"2024-03-20T07:58:26.137140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107015+0000","last_change":"2024-03-20T07:58:26.127042+0000","last_active":"2024-03-20T07:58:27.107015+0000","last_peered":"2024-03-20T07:58:27.107015+0000","last_clean":"2024-03-20T07:58:27.107015+0000","last_became_active":"2024-03-20T07:58:26.126555+0000","last_became_peered":"2024-03-20T07:58:26.126555+0000","last_unstale":"2024-03-20T07:58:27.107015+0000","last_undegraded":"2024-03-20T07:58:27.107015+0000","last_fullsized":"2024-03-20T07:58:27.107015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.132048+0000","last_change":"2024-03-20T07:58:26.129086+0000","last_active":"2024-03-20T07:58:26.132048+0000","last_peered":"2024-03-20T07:58:26.132048+0000","last_clean":"2024-03-20T07:58:26.132048+0000","last_became_active":"2024-03-20T07:58:26.128111+0000","last_became_peered":"2024-03-20T07:58:26.128111+0000","last_unstale":"2024-03-20T07:58:26.132048+0000","last_undegraded":"2024-03-20T07:58:26.132048+0000","last_fullsized":"2024-03-20T07:58:26.132048+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.131910+0000","last_change":"2024-03-20T07:58:26.128812+0000","last_active":"2024-03-20T07:58:26.131910+0000","last_peered":"2024-03-20T07:58:26.131910+0000","last_clean":"2024-03-20T07:58:26.131910+0000","last_became_active":"2024-03-20T07:58:26.128501+0000","last_became_peered":"2024-03-20T07:58:26.128501+0000","last_unstale":"2024-03-20T07:58:26.131910+0000","last_undegraded":"2024-03-20T07:58:26.131910+0000","last_fullsized":"2024-03-20T07:58:26.131910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134420+0000","last_change":"2024-03-20T07:58:26.133112+0000","last_active":"2024-03-20T07:58:26.134420+0000","last_peered":"2024-03-20T07:58:26.134420+0000","last_clean":"2024-03-20T07:58:26.134420+0000","last_became_active":"2024-03-20T07:58:26.132794+0000","last_became_peered":"2024-03-20T07:58:26.132794+0000","last_unstale":"2024-03-20T07:58:26.134420+0000","last_undegraded":"2024-03-20T07:58:26.134420+0000","last_fullsized":"2024-03-20T07:58:26.134420+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145432+0000","last_change":"2024-03-20T07:58:25.778043+0000","last_active":"2024-03-20T07:58:26.145432+0000","last_peered":"2024-03-20T07:58:26.145432+0000","last_clean":"2024-03-20T07:58:26.145432+0000","last_became_active":"2024-03-20T07:58:25.777710+0000","last_became_peered":"2024-03-20T07:58:25.777710+0000","last_unstale":"2024-03-20T07:58:26.145432+0000","last_undegraded":"2024-03-20T07:58:26.145432+0000","last_fullsized":"2024-03-20T07:58:26.145432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.123506+0000","last_change":"2024-03-20T07:58:26.122440+0000","last_active":"2024-03-20T07:58:26.123506+0000","last_peered":"2024-03-20T07:58:26.123506+0000","last_clean":"2024-03-20T07:58:26.123506+0000","last_became_active":"2024-03-20T07:58:26.122136+0000","last_became_peered":"2024-03-20T07:58:26.122136+0000","last_unstale":"2024-03-20T07:58:26.123506+0000","last_undegraded":"2024-03-20T07:58:26.123506+0000","last_fullsized":"2024-03-20T07:58:26.123506+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138961+0000","last_change":"2024-03-20T07:58:25.776107+0000","last_active":"2024-03-20T07:58:26.138961+0000","last_peered":"2024-03-20T07:58:26.138961+0000","last_clean":"2024-03-20T07:58:26.138961+0000","last_became_active":"2024-03-20T07:58:25.775756+0000","last_became_peered":"2024-03-20T07:58:25.775756+0000","last_unstale":"2024-03-20T07:58:26.138961+0000","last_undegraded":"2024-03-20T07:58:26.138961+0000","last_fullsized":"2024-03-20T07:58:26.138961+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138778+0000","last_change":"2024-03-20T07:58:26.137236+0000","last_active":"2024-03-20T07:58:26.138778+0000","last_peered":"2024-03-20T07:58:26.138778+0000","last_clean":"2024-03-20T07:58:26.138778+0000","last_became_active":"2024-03-20T07:58:26.136660+0000","last_became_peered":"2024-03-20T07:58:26.136660+0000","last_unstale":"2024-03-20T07:58:26.138778+0000","last_undegraded":"2024-03-20T07:58:26.138778+0000","last_fullsized":"2024-03-20T07:58:26.138778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:36.426353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.119043+0000","last_change":"2024-03-20T07:58:26.118793+0000","last_active":"2024-03-20T07:58:26.119043+0000","last_peered":"2024-03-20T07:58:26.119043+0000","last_clean":"2024-03-20T07:58:26.119043+0000","last_became_active":"2024-03-20T07:58:26.118430+0000","last_became_peered":"2024-03-20T07:58:26.118430+0000","last_unstale":"2024-03-20T07:58:26.119043+0000","last_undegraded":"2024-03-20T07:58:26.119043+0000","last_fullsized":"2024-03-20T07:58:26.119043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140552+0000","last_change":"2024-03-20T07:58:25.765918+0000","last_active":"2024-03-20T07:58:26.140552+0000","last_peered":"2024-03-20T07:58:26.140552+0000","last_clean":"2024-03-20T07:58:26.140552+0000","last_became_active":"2024-03-20T07:58:25.765598+0000","last_became_peered":"2024-03-20T07:58:25.765598+0000","last_unstale":"2024-03-20T07:58:26.140552+0000","last_undegraded":"2024-03-20T07:58:26.140552+0000","last_fullsized":"2024-03-20T07:58:26.140552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111175+0000","last_change":"2024-03-20T07:58:25.769007+0000","last_active":"2024-03-20T07:58:26.111175+0000","last_peered":"2024-03-20T07:58:26.111175+0000","last_clean":"2024-03-20T07:58:26.111175+0000","last_became_active":"2024-03-20T07:58:25.768716+0000","last_became_peered":"2024-03-20T07:58:25.768716+0000","last_unstale":"2024-03-20T07:58:26.111175+0000","last_undegraded":"2024-03-20T07:58:26.111175+0000","last_fullsized":"2024-03-20T07:58:26.111175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107247+0000","last_change":"2024-03-20T07:58:26.119518+0000","last_active":"2024-03-20T07:58:27.107247+0000","last_peered":"2024-03-20T07:58:27.107247+0000","last_clean":"2024-03-20T07:58:27.107247+0000","last_became_active":"2024-03-20T07:58:26.119184+0000","last_became_peered":"2024-03-20T07:58:26.119184+0000","last_unstale":"2024-03-20T07:58:27.107247+0000","last_undegraded":"2024-03-20T07:58:27.107247+0000","last_fullsized":"2024-03-20T07:58:27.107247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.128555+0000","last_change":"2024-03-20T07:58:25.769300+0000","last_active":"2024-03-20T07:58:26.128555+0000","last_peered":"2024-03-20T07:58:26.128555+0000","last_clean":"2024-03-20T07:58:26.128555+0000","last_became_active":"2024-03-20T07:58:25.768916+0000","last_became_peered":"2024-03-20T07:58:25.768916+0000","last_unstale":"2024-03-20T07:58:26.128555+0000","last_undegraded":"2024-03-20T07:58:26.128555+0000","last_fullsized":"2024-03-20T07:58:26.128555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.133696+0000","last_change":"2024-03-20T07:58:26.130616+0000","last_active":"2024-03-20T07:58:26.133696+0000","last_peered":"2024-03-20T07:58:26.133696+0000","last_clean":"2024-03-20T07:58:26.133696+0000","last_became_active":"2024-03-20T07:58:26.130332+0000","last_became_peered":"2024-03-20T07:58:26.130332+0000","last_unstale":"2024-03-20T07:58:26.133696+0000","last_undegraded":"2024-03-20T07:58:26.133696+0000","last_fullsized":"2024-03-20T07:58:26.133696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111307+0000","last_change":"2024-03-20T07:58:25.769790+0000","last_active":"2024-03-20T07:58:26.111307+0000","last_peered":"2024-03-20T07:58:26.111307+0000","last_clean":"2024-03-20T07:58:26.111307+0000","last_became_active":"2024-03-20T07:58:25.769492+0000","last_became_peered":"2024-03-20T07:58:25.769492+0000","last_unstale":"2024-03-20T07:58:26.111307+0000","last_undegraded":"2024-03-20T07:58:26.111307+0000","last_fullsized":"2024-03-20T07:58:26.111307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108079+0000","last_change":"2024-03-20T07:58:25.770988+0000","last_active":"2024-03-20T07:58:27.108079+0000","last_peered":"2024-03-20T07:58:27.108079+0000","last_clean":"2024-03-20T07:58:27.108079+0000","last_became_active":"2024-03-20T07:58:25.770657+0000","last_became_peered":"2024-03-20T07:58:25.770657+0000","last_unstale":"2024-03-20T07:58:27.108079+0000","last_undegraded":"2024-03-20T07:58:27.108079+0000","last_fullsized":"2024-03-20T07:58:27.108079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.131447+0000","last_change":"2024-03-20T07:58:26.128506+0000","last_active":"2024-03-20T07:58:26.131447+0000","last_peered":"2024-03-20T07:58:26.131447+0000","last_clean":"2024-03-20T07:58:26.131447+0000","last_became_active":"2024-03-20T07:58:26.128201+0000","last_became_peered":"2024-03-20T07:58:26.128201+0000","last_unstale":"2024-03-20T07:58:26.131447+0000","last_undegraded":"2024-03-20T07:58:26.131447+0000","last_fullsized":"2024-03-20T07:58:26.131447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.147717+0000","last_change":"2024-03-20T07:58:25.767830+0000","last_active":"2024-03-20T07:58:26.147717+0000","last_peered":"2024-03-20T07:58:26.147717+0000","last_clean":"2024-03-20T07:58:26.147717+0000","last_became_active":"2024-03-20T07:58:25.767533+0000","last_became_peered":"2024-03-20T07:58:25.767533+0000","last_unstale":"2024-03-20T07:58:26.147717+0000","last_undegraded":"2024-03-20T07:58:26.147717+0000","last_fullsized":"2024-03-20T07:58:26.147717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140470+0000","last_change":"2024-03-20T07:58:26.140127+0000","last_active":"2024-03-20T07:58:26.140470+0000","last_peered":"2024-03-20T07:58:26.140470+0000","last_clean":"2024-03-20T07:58:26.140470+0000","last_became_active":"2024-03-20T07:58:26.139362+0000","last_became_peered":"2024-03-20T07:58:26.139362+0000","last_unstale":"2024-03-20T07:58:26.140470+0000","last_undegraded":"2024-03-20T07:58:26.140470+0000","last_fullsized":"2024-03-20T07:58:26.140470+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.112184+0000","last_change":"2024-03-20T07:58:25.771839+0000","last_active":"2024-03-20T07:58:26.112184+0000","last_peered":"2024-03-20T07:58:26.112184+0000","last_clean":"2024-03-20T07:58:26.112184+0000","last_became_active":"2024-03-20T07:58:25.771464+0000","last_became_peered":"2024-03-20T07:58:25.771464+0000","last_unstale":"2024-03-20T07:58:26.112184+0000","last_undegraded":"2024-03-20T07:58:26.112184+0000","last_fullsized":"2024-03-20T07:58:26.112184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.129894+0000","last_change":"2024-03-20T07:58:26.129094+0000","last_active":"2024-03-20T07:58:26.129894+0000","last_peered":"2024-03-20T07:58:26.129894+0000","last_clean":"2024-03-20T07:58:26.129894+0000","last_became_active":"2024-03-20T07:58:26.128675+0000","last_became_peered":"2024-03-20T07:58:26.128675+0000","last_unstale":"2024-03-20T07:58:26.129894+0000","last_undegraded":"2024-03-20T07:58:26.129894+0000","last_fullsized":"2024-03-20T07:58:26.129894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.124730+0000","last_change":"2024-03-20T07:58:25.767586+0000","last_active":"2024-03-20T07:58:26.124730+0000","last_peered":"2024-03-20T07:58:26.124730+0000","last_clean":"2024-03-20T07:58:26.124730+0000","last_became_active":"2024-03-20T07:58:25.767268+0000","last_became_peered":"2024-03-20T07:58:25.767268+0000","last_unstale":"2024-03-20T07:58:26.124730+0000","last_undegraded":"2024-03-20T07:58:26.124730+0000","last_fullsized":"2024-03-20T07:58:26.124730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.144101+0000","last_change":"2024-03-20T07:58:26.139925+0000","last_active":"2024-03-20T07:58:26.144101+0000","last_peered":"2024-03-20T07:58:26.144101+0000","last_clean":"2024-03-20T07:58:26.144101+0000","last_became_active":"2024-03-20T07:58:26.138286+0000","last_became_peered":"2024-03-20T07:58:26.138286+0000","last_unstale":"2024-03-20T07:58:26.144101+0000","last_undegraded":"2024-03-20T07:58:26.144101+0000","last_fullsized":"2024-03-20T07:58:26.144101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134394+0000","last_change":"2024-03-20T07:58:26.133140+0000","last_active":"2024-03-20T07:58:26.134394+0000","last_peered":"2024-03-20T07:58:26.134394+0000","last_clean":"2024-03-20T07:58:26.134394+0000","last_became_active":"2024-03-20T07:58:26.132630+0000","last_became_peered":"2024-03-20T07:58:26.132630+0000","last_unstale":"2024-03-20T07:58:26.134394+0000","last_undegraded":"2024-03-20T07:58:26.134394+0000","last_fullsized":"2024-03-20T07:58:26.134394+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.135400+0000","last_change":"2024-03-20T07:58:25.771322+0000","last_active":"2024-03-20T07:58:26.135400+0000","last_peered":"2024-03-20T07:58:26.135400+0000","last_clean":"2024-03-20T07:58:26.135400+0000","last_became_active":"2024-03-20T07:58:25.770984+0000","last_became_peered":"2024-03-20T07:58:25.770984+0000","last_unstale":"2024-03-20T07:58:26.135400+0000","last_undegraded":"2024-03-20T07:58:26.135400+0000","last_fullsized":"2024-03-20T07:58:26.135400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145263+0000","last_change":"2024-03-20T07:58:26.144254+0000","last_active":"2024-03-20T07:58:26.145263+0000","last_peered":"2024-03-20T07:58:26.145263+0000","last_clean":"2024-03-20T07:58:26.145263+0000","last_became_active":"2024-03-20T07:58:26.143683+0000","last_became_peered":"2024-03-20T07:58:26.143683+0000","last_unstale":"2024-03-20T07:58:26.145263+0000","last_undegraded":"2024-03-20T07:58:26.145263+0000","last_fullsized":"2024-03-20T07:58:26.145263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107428+0000","last_change":"2024-03-20T07:58:25.773911+0000","last_active":"2024-03-20T07:58:27.107428+0000","last_peered":"2024-03-20T07:58:27.107428+0000","last_clean":"2024-03-20T07:58:27.107428+0000","last_became_active":"2024-03-20T07:58:25.773526+0000","last_became_peered":"2024-03-20T07:58:25.773526+0000","last_unstale":"2024-03-20T07:58:27.107428+0000","last_undegraded":"2024-03-20T07:58:27.107428+0000","last_fullsized":"2024-03-20T07:58:27.107428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.123504+0000","last_change":"2024-03-20T07:58:26.122909+0000","last_active":"2024-03-20T07:58:26.123504+0000","last_peered":"2024-03-20T07:58:26.123504+0000","last_clean":"2024-03-20T07:58:26.123504+0000","last_became_active":"2024-03-20T07:58:26.121608+0000","last_became_peered":"2024-03-20T07:58:26.121608+0000","last_unstale":"2024-03-20T07:58:26.123504+0000","last_undegraded":"2024-03-20T07:58:26.123504+0000","last_fullsized":"2024-03-20T07:58:26.123504+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111928+0000","last_change":"2024-03-20T07:58:25.770569+0000","last_active":"2024-03-20T07:58:26.111928+0000","last_peered":"2024-03-20T07:58:26.111928+0000","last_clean":"2024-03-20T07:58:26.111928+0000","last_became_active":"2024-03-20T07:58:25.770298+0000","last_became_peered":"2024-03-20T07:58:25.770298+0000","last_unstale":"2024-03-20T07:58:26.111928+0000","last_undegraded":"2024-03-20T07:58:26.111928+0000","last_fullsized":"2024-03-20T07:58:26.111928+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.137205+0000","last_change":"2024-03-20T07:58:26.136200+0000","last_active":"2024-03-20T07:58:26.137205+0000","last_peered":"2024-03-20T07:58:26.137205+0000","last_clean":"2024-03-20T07:58:26.137205+0000","last_became_active":"2024-03-20T07:58:26.135909+0000","last_became_peered":"2024-03-20T07:58:26.135909+0000","last_unstale":"2024-03-20T07:58:26.137205+0000","last_undegraded":"2024-03-20T07:58:26.137205+0000","last_fullsized":"2024-03-20T07:58:26.137205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107104+0000","last_change":"2024-03-20T07:58:26.130079+0000","last_active":"2024-03-20T07:58:27.107104+0000","last_peered":"2024-03-20T07:58:27.107104+0000","last_clean":"2024-03-20T07:58:27.107104+0000","last_became_active":"2024-03-20T07:58:26.129798+0000","last_became_peered":"2024-03-20T07:58:26.129798+0000","last_unstale":"2024-03-20T07:58:27.107104+0000","last_undegraded":"2024-03-20T07:58:27.107104+0000","last_fullsized":"2024-03-20T07:58:27.107104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.110613+0000","last_change":"2024-03-20T07:58:25.766643+0000","last_active":"2024-03-20T07:58:26.110613+0000","last_peered":"2024-03-20T07:58:26.110613+0000","last_clean":"2024-03-20T07:58:26.110613+0000","last_became_active":"2024-03-20T07:58:25.766341+0000","last_became_peered":"2024-03-20T07:58:25.766341+0000","last_unstale":"2024-03-20T07:58:26.110613+0000","last_undegraded":"2024-03-20T07:58:26.110613+0000","last_fullsized":"2024-03-20T07:58:26.110613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"22'117","reported_seq":218,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-20T07:58:25.769715+0000","last_change":"2024-03-20T07:58:09.964395+0000","last_active":"2024-03-20T07:58:25.769715+0000","last_peered":"2024-03-20T07:58:25.769715+0000","last_clean":"2024-03-20T07:58:25.769715+0000","last_became_active":"2024-03-20T07:58:09.963927+0000","last_became_peered":"2024-03-20T07:58:09.963927+0000","last_unstale":"2024-03-20T07:58:25.769715+0000","last_undegraded":"2024-03-20T07:58:25.769715+0000","last_fullsized":"2024-03-20T07:58:25.769715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.118188+0000","last_change":"2024-03-20T07:58:25.779129+0000","last_active":"2024-03-20T07:58:26.118188+0000","last_peered":"2024-03-20T07:58:26.118188+0000","last_clean":"2024-03-20T07:58:26.118188+0000","last_became_active":"2024-03-20T07:58:25.778258+0000","last_became_peered":"2024-03-20T07:58:25.778258+0000","last_unstale":"2024-03-20T07:58:26.118188+0000","last_undegraded":"2024-03-20T07:58:26.118188+0000","last_fullsized":"2024-03-20T07:58:26.118188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.130223+0000","last_change":"2024-03-20T07:58:25.777973+0000","last_active":"2024-03-20T07:58:26.130223+0000","last_peered":"2024-03-20T07:58:26.130223+0000","last_clean":"2024-03-20T07:58:26.130223+0000","last_became_active":"2024-03-20T07:58:25.777666+0000","last_became_peered":"2024-03-20T07:58:25.777666+0000","last_unstale":"2024-03-20T07:58:26.130223+0000","last_undegraded":"2024-03-20T07:58:26.130223+0000","last_fullsized":"2024-03-20T07:58:26.130223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107508+0000","last_change":"2024-03-20T07:58:26.126692+0000","last_active":"2024-03-20T07:58:27.107508+0000","last_peered":"2024-03-20T07:58:27.107508+0000","last_clean":"2024-03-20T07:58:27.107508+0000","last_became_active":"2024-03-20T07:58:26.126397+0000","last_became_peered":"2024-03-20T07:58:26.126397+0000","last_unstale":"2024-03-20T07:58:27.107508+0000","last_undegraded":"2024-03-20T07:58:27.107508+0000","last_fullsized":"2024-03-20T07:58:27.107508+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108422+0000","last_change":"2024-03-20T07:58:25.769337+0000","last_active":"2024-03-20T07:58:27.108422+0000","last_peered":"2024-03-20T07:58:27.108422+0000","last_clean":"2024-03-20T07:58:27.108422+0000","last_became_active":"2024-03-20T07:58:25.769010+0000","last_became_peered":"2024-03-20T07:58:25.769010+0000","last_unstale":"2024-03-20T07:58:27.108422+0000","last_undegraded":"2024-03-20T07:58:27.108422+0000","last_fullsized":"2024-03-20T07:58:27.108422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.141871+0000","last_change":"2024-03-20T07:58:25.771097+0000","last_active":"2024-03-20T07:58:26.141871+0000","last_peered":"2024-03-20T07:58:26.141871+0000","last_clean":"2024-03-20T07:58:26.141871+0000","last_became_active":"2024-03-20T07:58:25.770737+0000","last_became_peered":"2024-03-20T07:58:25.770737+0000","last_unstale":"2024-03-20T07:58:26.141871+0000","last_undegraded":"2024-03-20T07:58:26.141871+0000","last_fullsized":"2024-03-20T07:58:26.141871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.117985+0000","last_change":"2024-03-20T07:58:25.777961+0000","last_active":"2024-03-20T07:58:26.117985+0000","last_peered":"2024-03-20T07:58:26.117985+0000","last_clean":"2024-03-20T07:58:26.117985+0000","last_became_active":"2024-03-20T07:58:25.777649+0000","last_became_peered":"2024-03-20T07:58:25.777649+0000","last_unstale":"2024-03-20T07:58:26.117985+0000","last_undegraded":"2024-03-20T07:58:26.117985+0000","last_fullsized":"2024-03-20T07:58:26.117985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.124307+0000","last_change":"2024-03-20T07:58:26.120305+0000","last_active":"2024-03-20T07:58:26.124307+0000","last_peered":"2024-03-20T07:58:26.124307+0000","last_clean":"2024-03-20T07:58:26.124307+0000","last_became_active":"2024-03-20T07:58:26.119966+0000","last_became_peered":"2024-03-20T07:58:26.119966+0000","last_unstale":"2024-03-20T07:58:26.124307+0000","last_undegraded":"2024-03-20T07:58:26.124307+0000","last_fullsized":"2024-03-20T07:58:26.124307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107099+0000","last_change":"2024-03-20T07:58:25.765548+0000","last_active":"2024-03-20T07:58:27.107099+0000","last_peered":"2024-03-20T07:58:27.107099+0000","last_clean":"2024-03-20T07:58:27.107099+0000","last_became_active":"2024-03-20T07:58:25.765273+0000","last_became_peered":"2024-03-20T07:58:25.765273+0000","last_unstale":"2024-03-20T07:58:27.107099+0000","last_undegraded":"2024-03-20T07:58:27.107099+0000","last_fullsized":"2024-03-20T07:58:27.107099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107547+0000","last_change":"2024-03-20T07:58:25.773811+0000","last_active":"2024-03-20T07:58:27.107547+0000","last_peered":"2024-03-20T07:58:27.107547+0000","last_clean":"2024-03-20T07:58:27.107547+0000","last_became_active":"2024-03-20T07:58:25.773520+0000","last_became_peered":"2024-03-20T07:58:25.773520+0000","last_unstale":"2024-03-20T07:58:27.107547+0000","last_undegraded":"2024-03-20T07:58:27.107547+0000","last_fullsized":"2024-03-20T07:58:27.107547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138933+0000","last_change":"2024-03-20T07:58:25.776066+0000","last_active":"2024-03-20T07:58:26.138933+0000","last_peered":"2024-03-20T07:58:26.138933+0000","last_clean":"2024-03-20T07:58:26.138933+0000","last_became_active":"2024-03-20T07:58:25.775708+0000","last_became_peered":"2024-03-20T07:58:25.775708+0000","last_unstale":"2024-03-20T07:58:26.138933+0000","last_undegraded":"2024-03-20T07:58:26.138933+0000","last_fullsized":"2024-03-20T07:58:26.138933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:36:20.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134688+0000","last_change":"2024-03-20T07:58:26.134067+0000","last_active":"2024-03-20T07:58:26.134688+0000","last_peered":"2024-03-20T07:58:26.134688+0000","last_clean":"2024-03-20T07:58:26.134688+0000","last_became_active":"2024-03-20T07:58:26.133784+0000","last_became_peered":"2024-03-20T07:58:26.133784+0000","last_unstale":"2024-03-20T07:58:26.134688+0000","last_undegraded":"2024-03-20T07:58:26.134688+0000","last_fullsized":"2024-03-20T07:58:26.134688+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.142797+0000","last_change":"2024-03-20T07:58:26.142445+0000","last_active":"2024-03-20T07:58:26.142797+0000","last_peered":"2024-03-20T07:58:26.142797+0000","last_clean":"2024-03-20T07:58:26.142797+0000","last_became_active":"2024-03-20T07:58:26.142148+0000","last_became_peered":"2024-03-20T07:58:26.142148+0000","last_unstale":"2024-03-20T07:58:26.142797+0000","last_undegraded":"2024-03-20T07:58:26.142797+0000","last_fullsized":"2024-03-20T07:58:26.142797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"62'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107988+0000","last_change":"2024-03-20T07:58:25.766427+0000","last_active":"2024-03-20T07:58:27.107988+0000","last_peered":"2024-03-20T07:58:27.107988+0000","last_clean":"2024-03-20T07:58:27.107988+0000","last_became_active":"2024-03-20T07:58:25.766111+0000","last_became_peered":"2024-03-20T07:58:25.766111+0000","last_unstale":"2024-03-20T07:58:27.107988+0000","last_undegraded":"2024-03-20T07:58:27.107988+0000","last_fullsized":"2024-03-20T07:58:27.107988+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.136811+0000","last_change":"2024-03-20T07:58:26.135339+0000","last_active":"2024-03-20T07:58:26.136811+0000","last_peered":"2024-03-20T07:58:26.136811+0000","last_clean":"2024-03-20T07:58:26.136811+0000","last_became_active":"2024-03-20T07:58:26.134903+0000","last_became_peered":"2024-03-20T07:58:26.134903+0000","last_unstale":"2024-03-20T07:58:26.136811+0000","last_undegraded":"2024-03-20T07:58:26.136811+0000","last_fullsized":"2024-03-20T07:58:26.136811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145073+0000","last_change":"2024-03-20T07:58:26.144008+0000","last_active":"2024-03-20T07:58:26.145073+0000","last_peered":"2024-03-20T07:58:26.145073+0000","last_clean":"2024-03-20T07:58:26.145073+0000","last_became_active":"2024-03-20T07:58:26.143696+0000","last_became_peered":"2024-03-20T07:58:26.143696+0000","last_unstale":"2024-03-20T07:58:26.145073+0000","last_undegraded":"2024-03-20T07:58:26.145073+0000","last_fullsized":"2024-03-20T07:58:26.145073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140012+0000","last_change":"2024-03-20T07:58:26.138096+0000","last_active":"2024-03-20T07:58:26.140012+0000","last_peered":"2024-03-20T07:58:26.140012+0000","last_clean":"2024-03-20T07:58:26.140012+0000","last_became_active":"2024-03-20T07:58:26.137804+0000","last_became_peered":"2024-03-20T07:58:26.137804+0000","last_unstale":"2024-03-20T07:58:26.140012+0000","last_undegraded":"2024-03-20T07:58:26.140012+0000","last_fullsized":"2024-03-20T07:58:26.140012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.126540+0000","last_change":"2024-03-20T07:58:26.125004+0000","last_active":"2024-03-20T07:58:26.126540+0000","last_peered":"2024-03-20T07:58:26.126540+0000","last_clean":"2024-03-20T07:58:26.126540+0000","last_became_active":"2024-03-20T07:58:26.124689+0000","last_became_peered":"2024-03-20T07:58:26.124689+0000","last_unstale":"2024-03-20T07:58:26.126540+0000","last_undegraded":"2024-03-20T07:58:26.126540+0000","last_fullsized":"2024-03-20T07:58:26.126540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107012+0000","last_change":"2024-03-20T07:58:26.126113+0000","last_active":"2024-03-20T07:58:27.107012+0000","last_peered":"2024-03-20T07:58:27.107012+0000","last_clean":"2024-03-20T07:58:27.107012+0000","last_became_active":"2024-03-20T07:58:26.122635+0000","last_became_peered":"2024-03-20T07:58:26.122635+0000","last_unstale":"2024-03-20T07:58:27.107012+0000","last_undegraded":"2024-03-20T07:58:27.107012+0000","last_fullsized":"2024-03-20T07:58:27.107012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.144469+0000","last_change":"2024-03-20T07:58:26.141379+0000","last_active":"2024-03-20T07:58:26.144469+0000","last_peered":"2024-03-20T07:58:26.144469+0000","last_clean":"2024-03-20T07:58:26.144469+0000","last_became_active":"2024-03-20T07:58:26.141099+0000","last_became_peered":"2024-03-20T07:58:26.141099+0000","last_unstale":"2024-03-20T07:58:26.144469+0000","last_undegraded":"2024-03-20T07:58:26.144469+0000","last_fullsized":"2024-03-20T07:58:26.144469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138838+0000","last_change":"2024-03-20T07:58:26.137638+0000","last_active":"2024-03-20T07:58:26.138838+0000","last_peered":"2024-03-20T07:58:26.138838+0000","last_clean":"2024-03-20T07:58:26.138838+0000","last_became_active":"2024-03-20T07:58:26.137344+0000","last_became_peered":"2024-03-20T07:58:26.137344+0000","last_unstale":"2024-03-20T07:58:26.138838+0000","last_undegraded":"2024-03-20T07:58:26.138838+0000","last_fullsized":"2024-03-20T07:58:26.138838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138511+0000","last_change":"2024-03-20T07:58:26.137117+0000","last_active":"2024-03-20T07:58:26.138511+0000","last_peered":"2024-03-20T07:58:26.138511+0000","last_clean":"2024-03-20T07:58:26.138511+0000","last_became_active":"2024-03-20T07:58:26.136135+0000","last_became_peered":"2024-03-20T07:58:26.136135+0000","last_unstale":"2024-03-20T07:58:26.138511+0000","last_undegraded":"2024-03-20T07:58:26.138511+0000","last_fullsized":"2024-03-20T07:58:26.138511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134663+0000","last_change":"2024-03-20T07:58:26.133160+0000","last_active":"2024-03-20T07:58:26.134663+0000","last_peered":"2024-03-20T07:58:26.134663+0000","last_clean":"2024-03-20T07:58:26.134663+0000","last_became_active":"2024-03-20T07:58:26.132793+0000","last_became_peered":"2024-03-20T07:58:26.132793+0000","last_unstale":"2024-03-20T07:58:26.134663+0000","last_undegraded":"2024-03-20T07:58:26.134663+0000","last_fullsized":"2024-03-20T07:58:26.134663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":123,"acting":123,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":69,"acting":69,"num_store_stats":5}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201286,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":838705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462923,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1183744,"data_stored":831669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717512,"statfs":{"total":95995035648,"available":95966732288,"internally_reserved":0,"allocated":626688,"data_stored":255702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.33100000000000002,"15min":0.32300000000000001},"min":{"1min":0.26200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.35599999999999998,"15min":0.34399999999999997},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.313}]},{"osd":2,"last update":"Wed Mar 20 07:58:07 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.39300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.25},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.308,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.27800000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.37}]},{"osd":4,"last update":"Wed Mar 20 07:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738409,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1175552,"data_stored":825753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 20 07:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.26300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.32200000000000001,"15min":0.312},"min":{"1min":0.26300000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed Mar 20 07:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Mar 20 07:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.376},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.44600000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65100000000000002}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717564,"statfs":{"total":95995035648,"available":95966785536,"internally_reserved":0,"allocated":581632,"data_stored":235385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.36799999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.32800000000000001}]},{"osd":1,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.33900000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.38},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38100000000000001},"min":{"1min":0.33000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52000000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed Mar 20 07:57:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.313}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.47899999999999998}]},{"osd":5,"last update":"Wed Mar 20 07:58:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149721,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717556,"statfs":{"total":95995035648,"available":95966777344,"internally_reserved":0,"allocated":581632,"data_stored":235385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.36499999999999999}]},{"osd":1,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.626},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Mar 20 07:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986197,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":634880,"data_stored":256250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.374}]},{"osd":1,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.503}]},{"osd":2,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.432}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.35699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717528,"statfs":{"total":95995035648,"available":95966748672,"internally_reserved":0,"allocated":618496,"data_stored":255542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44}]},{"osd":1,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.495}]},{"osd":3,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.38400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T07:58:29.073 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-20T07:58:29.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:28 smithi196 ceph-mon[31033]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T07:58:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:28 smithi196 ceph-mon[31033]: Cluster is now healthy 2024-03-20T07:58:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:28 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2559395147' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-20T07:58:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:28 smithi196 ceph-mon[31033]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T07:58:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:28 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/24715312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T07:58:29.394 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:29.396 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T07:58:29.406 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-20T07:58:29.236703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":249,"acting":249,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":749961216,"kb_used":222732,"kb_used_data":6536,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749738484,"statfs":{"total":767960285184,"available":767732207616,"internally_reserved":0,"allocated":6692864,"data_stored":3775433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.491697"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.146486+0000","last_change":"2024-03-20T07:58:26.145662+0000","last_active":"2024-03-20T07:58:26.146486+0000","last_peered":"2024-03-20T07:58:26.146486+0000","last_clean":"2024-03-20T07:58:26.146486+0000","last_became_active":"2024-03-20T07:58:26.145286+0000","last_became_peered":"2024-03-20T07:58:26.145286+0000","last_unstale":"2024-03-20T07:58:26.146486+0000","last_undegraded":"2024-03-20T07:58:26.146486+0000","last_fullsized":"2024-03-20T07:58:26.146486+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108335+0000","last_change":"2024-03-20T07:58:26.129069+0000","last_active":"2024-03-20T07:58:27.108335+0000","last_peered":"2024-03-20T07:58:27.108335+0000","last_clean":"2024-03-20T07:58:27.108335+0000","last_became_active":"2024-03-20T07:58:26.128811+0000","last_became_peered":"2024-03-20T07:58:26.128811+0000","last_unstale":"2024-03-20T07:58:27.108335+0000","last_undegraded":"2024-03-20T07:58:27.108335+0000","last_fullsized":"2024-03-20T07:58:27.108335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239393+0000","last_change":"2024-03-20T07:58:26.119030+0000","last_active":"2024-03-20T07:58:27.239393+0000","last_peered":"2024-03-20T07:58:27.239393+0000","last_clean":"2024-03-20T07:58:27.239393+0000","last_became_active":"2024-03-20T07:58:26.118755+0000","last_became_peered":"2024-03-20T07:58:26.118755+0000","last_unstale":"2024-03-20T07:58:27.239393+0000","last_undegraded":"2024-03-20T07:58:27.239393+0000","last_fullsized":"2024-03-20T07:58:27.239393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:51:41.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139286+0000","last_change":"2024-03-20T07:58:26.138067+0000","last_active":"2024-03-20T07:58:26.139286+0000","last_peered":"2024-03-20T07:58:26.139286+0000","last_clean":"2024-03-20T07:58:26.139286+0000","last_became_active":"2024-03-20T07:58:26.137768+0000","last_became_peered":"2024-03-20T07:58:26.137768+0000","last_unstale":"2024-03-20T07:58:26.139286+0000","last_undegraded":"2024-03-20T07:58:26.139286+0000","last_fullsized":"2024-03-20T07:58:26.139286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134630+0000","last_change":"2024-03-20T07:58:26.131923+0000","last_active":"2024-03-20T07:58:26.134630+0000","last_peered":"2024-03-20T07:58:26.134630+0000","last_clean":"2024-03-20T07:58:26.134630+0000","last_became_active":"2024-03-20T07:58:26.131585+0000","last_became_peered":"2024-03-20T07:58:26.131585+0000","last_unstale":"2024-03-20T07:58:26.134630+0000","last_undegraded":"2024-03-20T07:58:26.134630+0000","last_fullsized":"2024-03-20T07:58:26.134630+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139194+0000","last_change":"2024-03-20T07:58:26.137308+0000","last_active":"2024-03-20T07:58:26.139194+0000","last_peered":"2024-03-20T07:58:26.139194+0000","last_clean":"2024-03-20T07:58:26.139194+0000","last_became_active":"2024-03-20T07:58:26.135727+0000","last_became_peered":"2024-03-20T07:58:26.135727+0000","last_unstale":"2024-03-20T07:58:26.139194+0000","last_undegraded":"2024-03-20T07:58:26.139194+0000","last_fullsized":"2024-03-20T07:58:26.139194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134374+0000","last_change":"2024-03-20T07:58:26.132941+0000","last_active":"2024-03-20T07:58:26.134374+0000","last_peered":"2024-03-20T07:58:26.134374+0000","last_clean":"2024-03-20T07:58:26.134374+0000","last_became_active":"2024-03-20T07:58:26.132230+0000","last_became_peered":"2024-03-20T07:58:26.132230+0000","last_unstale":"2024-03-20T07:58:26.134374+0000","last_undegraded":"2024-03-20T07:58:26.134374+0000","last_fullsized":"2024-03-20T07:58:26.134374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145322+0000","last_change":"2024-03-20T07:58:26.144631+0000","last_active":"2024-03-20T07:58:26.145322+0000","last_peered":"2024-03-20T07:58:26.145322+0000","last_clean":"2024-03-20T07:58:26.145322+0000","last_became_active":"2024-03-20T07:58:26.144302+0000","last_became_peered":"2024-03-20T07:58:26.144302+0000","last_unstale":"2024-03-20T07:58:26.145322+0000","last_undegraded":"2024-03-20T07:58:26.145322+0000","last_fullsized":"2024-03-20T07:58:26.145322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139423+0000","last_change":"2024-03-20T07:58:26.138657+0000","last_active":"2024-03-20T07:58:26.139423+0000","last_peered":"2024-03-20T07:58:26.139423+0000","last_clean":"2024-03-20T07:58:26.139423+0000","last_became_active":"2024-03-20T07:58:26.138348+0000","last_became_peered":"2024-03-20T07:58:26.138348+0000","last_unstale":"2024-03-20T07:58:26.139423+0000","last_undegraded":"2024-03-20T07:58:26.139423+0000","last_fullsized":"2024-03-20T07:58:26.139423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138763+0000","last_change":"2024-03-20T07:58:26.137961+0000","last_active":"2024-03-20T07:58:26.138763+0000","last_peered":"2024-03-20T07:58:26.138763+0000","last_clean":"2024-03-20T07:58:26.138763+0000","last_became_active":"2024-03-20T07:58:26.137672+0000","last_became_peered":"2024-03-20T07:58:26.137672+0000","last_unstale":"2024-03-20T07:58:26.138763+0000","last_undegraded":"2024-03-20T07:58:26.138763+0000","last_fullsized":"2024-03-20T07:58:26.138763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.137140+0000","last_change":"2024-03-20T07:58:26.136246+0000","last_active":"2024-03-20T07:58:26.137140+0000","last_peered":"2024-03-20T07:58:26.137140+0000","last_clean":"2024-03-20T07:58:26.137140+0000","last_became_active":"2024-03-20T07:58:26.135935+0000","last_became_peered":"2024-03-20T07:58:26.135935+0000","last_unstale":"2024-03-20T07:58:26.137140+0000","last_undegraded":"2024-03-20T07:58:26.137140+0000","last_fullsized":"2024-03-20T07:58:26.137140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238796+0000","last_change":"2024-03-20T07:58:26.123008+0000","last_active":"2024-03-20T07:58:27.238796+0000","last_peered":"2024-03-20T07:58:27.238796+0000","last_clean":"2024-03-20T07:58:27.238796+0000","last_became_active":"2024-03-20T07:58:26.122270+0000","last_became_peered":"2024-03-20T07:58:26.122270+0000","last_unstale":"2024-03-20T07:58:27.238796+0000","last_undegraded":"2024-03-20T07:58:27.238796+0000","last_fullsized":"2024-03-20T07:58:27.238796+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:37.385767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107015+0000","last_change":"2024-03-20T07:58:26.127042+0000","last_active":"2024-03-20T07:58:27.107015+0000","last_peered":"2024-03-20T07:58:27.107015+0000","last_clean":"2024-03-20T07:58:27.107015+0000","last_became_active":"2024-03-20T07:58:26.126555+0000","last_became_peered":"2024-03-20T07:58:26.126555+0000","last_unstale":"2024-03-20T07:58:27.107015+0000","last_undegraded":"2024-03-20T07:58:27.107015+0000","last_fullsized":"2024-03-20T07:58:27.107015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.132048+0000","last_change":"2024-03-20T07:58:26.129086+0000","last_active":"2024-03-20T07:58:26.132048+0000","last_peered":"2024-03-20T07:58:26.132048+0000","last_clean":"2024-03-20T07:58:26.132048+0000","last_became_active":"2024-03-20T07:58:26.128111+0000","last_became_peered":"2024-03-20T07:58:26.128111+0000","last_unstale":"2024-03-20T07:58:26.132048+0000","last_undegraded":"2024-03-20T07:58:26.132048+0000","last_fullsized":"2024-03-20T07:58:26.132048+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.131910+0000","last_change":"2024-03-20T07:58:26.128812+0000","last_active":"2024-03-20T07:58:26.131910+0000","last_peered":"2024-03-20T07:58:26.131910+0000","last_clean":"2024-03-20T07:58:26.131910+0000","last_became_active":"2024-03-20T07:58:26.128501+0000","last_became_peered":"2024-03-20T07:58:26.128501+0000","last_unstale":"2024-03-20T07:58:26.131910+0000","last_undegraded":"2024-03-20T07:58:26.131910+0000","last_fullsized":"2024-03-20T07:58:26.131910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238912+0000","last_change":"2024-03-20T07:58:26.128211+0000","last_active":"2024-03-20T07:58:27.238912+0000","last_peered":"2024-03-20T07:58:27.238912+0000","last_clean":"2024-03-20T07:58:27.238912+0000","last_became_active":"2024-03-20T07:58:26.127943+0000","last_became_peered":"2024-03-20T07:58:26.127943+0000","last_unstale":"2024-03-20T07:58:27.238912+0000","last_undegraded":"2024-03-20T07:58:27.238912+0000","last_fullsized":"2024-03-20T07:58:27.238912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:19.979568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134420+0000","last_change":"2024-03-20T07:58:26.133112+0000","last_active":"2024-03-20T07:58:26.134420+0000","last_peered":"2024-03-20T07:58:26.134420+0000","last_clean":"2024-03-20T07:58:26.134420+0000","last_became_active":"2024-03-20T07:58:26.132794+0000","last_became_peered":"2024-03-20T07:58:26.132794+0000","last_unstale":"2024-03-20T07:58:26.134420+0000","last_undegraded":"2024-03-20T07:58:26.134420+0000","last_fullsized":"2024-03-20T07:58:26.134420+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"62'4","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239419+0000","last_change":"2024-03-20T07:58:25.774846+0000","last_active":"2024-03-20T07:58:27.239419+0000","last_peered":"2024-03-20T07:58:27.239419+0000","last_clean":"2024-03-20T07:58:27.239419+0000","last_became_active":"2024-03-20T07:58:25.774570+0000","last_became_peered":"2024-03-20T07:58:25.774570+0000","last_unstale":"2024-03-20T07:58:27.239419+0000","last_undegraded":"2024-03-20T07:58:27.239419+0000","last_fullsized":"2024-03-20T07:58:27.239419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:00.513690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239316+0000","last_change":"2024-03-20T07:58:26.118404+0000","last_active":"2024-03-20T07:58:27.239316+0000","last_peered":"2024-03-20T07:58:27.239316+0000","last_clean":"2024-03-20T07:58:27.239316+0000","last_became_active":"2024-03-20T07:58:26.118107+0000","last_became_peered":"2024-03-20T07:58:26.118107+0000","last_unstale":"2024-03-20T07:58:27.239316+0000","last_undegraded":"2024-03-20T07:58:27.239316+0000","last_fullsized":"2024-03-20T07:58:27.239316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:51.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145432+0000","last_change":"2024-03-20T07:58:25.778043+0000","last_active":"2024-03-20T07:58:26.145432+0000","last_peered":"2024-03-20T07:58:26.145432+0000","last_clean":"2024-03-20T07:58:26.145432+0000","last_became_active":"2024-03-20T07:58:25.777710+0000","last_became_peered":"2024-03-20T07:58:25.777710+0000","last_unstale":"2024-03-20T07:58:26.145432+0000","last_undegraded":"2024-03-20T07:58:26.145432+0000","last_fullsized":"2024-03-20T07:58:26.145432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.123506+0000","last_change":"2024-03-20T07:58:26.122440+0000","last_active":"2024-03-20T07:58:26.123506+0000","last_peered":"2024-03-20T07:58:26.123506+0000","last_clean":"2024-03-20T07:58:26.123506+0000","last_became_active":"2024-03-20T07:58:26.122136+0000","last_became_peered":"2024-03-20T07:58:26.122136+0000","last_unstale":"2024-03-20T07:58:26.123506+0000","last_undegraded":"2024-03-20T07:58:26.123506+0000","last_fullsized":"2024-03-20T07:58:26.123506+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138961+0000","last_change":"2024-03-20T07:58:25.776107+0000","last_active":"2024-03-20T07:58:26.138961+0000","last_peered":"2024-03-20T07:58:26.138961+0000","last_clean":"2024-03-20T07:58:26.138961+0000","last_became_active":"2024-03-20T07:58:25.775756+0000","last_became_peered":"2024-03-20T07:58:25.775756+0000","last_unstale":"2024-03-20T07:58:26.138961+0000","last_undegraded":"2024-03-20T07:58:26.138961+0000","last_fullsized":"2024-03-20T07:58:26.138961+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138778+0000","last_change":"2024-03-20T07:58:26.137236+0000","last_active":"2024-03-20T07:58:26.138778+0000","last_peered":"2024-03-20T07:58:26.138778+0000","last_clean":"2024-03-20T07:58:26.138778+0000","last_became_active":"2024-03-20T07:58:26.136660+0000","last_became_peered":"2024-03-20T07:58:26.136660+0000","last_unstale":"2024-03-20T07:58:26.138778+0000","last_undegraded":"2024-03-20T07:58:26.138778+0000","last_fullsized":"2024-03-20T07:58:26.138778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:36.426353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.119043+0000","last_change":"2024-03-20T07:58:26.118793+0000","last_active":"2024-03-20T07:58:26.119043+0000","last_peered":"2024-03-20T07:58:26.119043+0000","last_clean":"2024-03-20T07:58:26.119043+0000","last_became_active":"2024-03-20T07:58:26.118430+0000","last_became_peered":"2024-03-20T07:58:26.118430+0000","last_unstale":"2024-03-20T07:58:26.119043+0000","last_undegraded":"2024-03-20T07:58:26.119043+0000","last_fullsized":"2024-03-20T07:58:26.119043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140552+0000","last_change":"2024-03-20T07:58:25.765918+0000","last_active":"2024-03-20T07:58:26.140552+0000","last_peered":"2024-03-20T07:58:26.140552+0000","last_clean":"2024-03-20T07:58:26.140552+0000","last_became_active":"2024-03-20T07:58:25.765598+0000","last_became_peered":"2024-03-20T07:58:25.765598+0000","last_unstale":"2024-03-20T07:58:26.140552+0000","last_undegraded":"2024-03-20T07:58:26.140552+0000","last_fullsized":"2024-03-20T07:58:26.140552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239315+0000","last_change":"2024-03-20T07:58:26.119313+0000","last_active":"2024-03-20T07:58:27.239315+0000","last_peered":"2024-03-20T07:58:27.239315+0000","last_clean":"2024-03-20T07:58:27.239315+0000","last_became_active":"2024-03-20T07:58:26.118645+0000","last_became_peered":"2024-03-20T07:58:26.118645+0000","last_unstale":"2024-03-20T07:58:27.239315+0000","last_undegraded":"2024-03-20T07:58:27.239315+0000","last_fullsized":"2024-03-20T07:58:27.239315+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:13.410704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111175+0000","last_change":"2024-03-20T07:58:25.769007+0000","last_active":"2024-03-20T07:58:26.111175+0000","last_peered":"2024-03-20T07:58:26.111175+0000","last_clean":"2024-03-20T07:58:26.111175+0000","last_became_active":"2024-03-20T07:58:25.768716+0000","last_became_peered":"2024-03-20T07:58:25.768716+0000","last_unstale":"2024-03-20T07:58:26.111175+0000","last_undegraded":"2024-03-20T07:58:26.111175+0000","last_fullsized":"2024-03-20T07:58:26.111175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107247+0000","last_change":"2024-03-20T07:58:26.119518+0000","last_active":"2024-03-20T07:58:27.107247+0000","last_peered":"2024-03-20T07:58:27.107247+0000","last_clean":"2024-03-20T07:58:27.107247+0000","last_became_active":"2024-03-20T07:58:26.119184+0000","last_became_peered":"2024-03-20T07:58:26.119184+0000","last_unstale":"2024-03-20T07:58:27.107247+0000","last_undegraded":"2024-03-20T07:58:27.107247+0000","last_fullsized":"2024-03-20T07:58:27.107247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.128555+0000","last_change":"2024-03-20T07:58:25.769300+0000","last_active":"2024-03-20T07:58:26.128555+0000","last_peered":"2024-03-20T07:58:26.128555+0000","last_clean":"2024-03-20T07:58:26.128555+0000","last_became_active":"2024-03-20T07:58:25.768916+0000","last_became_peered":"2024-03-20T07:58:25.768916+0000","last_unstale":"2024-03-20T07:58:26.128555+0000","last_undegraded":"2024-03-20T07:58:26.128555+0000","last_fullsized":"2024-03-20T07:58:26.128555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.133696+0000","last_change":"2024-03-20T07:58:26.130616+0000","last_active":"2024-03-20T07:58:26.133696+0000","last_peered":"2024-03-20T07:58:26.133696+0000","last_clean":"2024-03-20T07:58:26.133696+0000","last_became_active":"2024-03-20T07:58:26.130332+0000","last_became_peered":"2024-03-20T07:58:26.130332+0000","last_unstale":"2024-03-20T07:58:26.133696+0000","last_undegraded":"2024-03-20T07:58:26.133696+0000","last_fullsized":"2024-03-20T07:58:26.133696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111307+0000","last_change":"2024-03-20T07:58:25.769790+0000","last_active":"2024-03-20T07:58:26.111307+0000","last_peered":"2024-03-20T07:58:26.111307+0000","last_clean":"2024-03-20T07:58:26.111307+0000","last_became_active":"2024-03-20T07:58:25.769492+0000","last_became_peered":"2024-03-20T07:58:25.769492+0000","last_unstale":"2024-03-20T07:58:26.111307+0000","last_undegraded":"2024-03-20T07:58:26.111307+0000","last_fullsized":"2024-03-20T07:58:26.111307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239245+0000","last_change":"2024-03-20T07:58:26.116344+0000","last_active":"2024-03-20T07:58:27.239245+0000","last_peered":"2024-03-20T07:58:27.239245+0000","last_clean":"2024-03-20T07:58:27.239245+0000","last_became_active":"2024-03-20T07:58:26.116037+0000","last_became_peered":"2024-03-20T07:58:26.116037+0000","last_unstale":"2024-03-20T07:58:27.239245+0000","last_undegraded":"2024-03-20T07:58:27.239245+0000","last_fullsized":"2024-03-20T07:58:27.239245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:16.183856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108079+0000","last_change":"2024-03-20T07:58:25.770988+0000","last_active":"2024-03-20T07:58:27.108079+0000","last_peered":"2024-03-20T07:58:27.108079+0000","last_clean":"2024-03-20T07:58:27.108079+0000","last_became_active":"2024-03-20T07:58:25.770657+0000","last_became_peered":"2024-03-20T07:58:25.770657+0000","last_unstale":"2024-03-20T07:58:27.108079+0000","last_undegraded":"2024-03-20T07:58:27.108079+0000","last_fullsized":"2024-03-20T07:58:27.108079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.131447+0000","last_change":"2024-03-20T07:58:26.128506+0000","last_active":"2024-03-20T07:58:26.131447+0000","last_peered":"2024-03-20T07:58:26.131447+0000","last_clean":"2024-03-20T07:58:26.131447+0000","last_became_active":"2024-03-20T07:58:26.128201+0000","last_became_peered":"2024-03-20T07:58:26.128201+0000","last_unstale":"2024-03-20T07:58:26.131447+0000","last_undegraded":"2024-03-20T07:58:26.131447+0000","last_fullsized":"2024-03-20T07:58:26.131447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.147717+0000","last_change":"2024-03-20T07:58:25.767830+0000","last_active":"2024-03-20T07:58:26.147717+0000","last_peered":"2024-03-20T07:58:26.147717+0000","last_clean":"2024-03-20T07:58:26.147717+0000","last_became_active":"2024-03-20T07:58:25.767533+0000","last_became_peered":"2024-03-20T07:58:25.767533+0000","last_unstale":"2024-03-20T07:58:26.147717+0000","last_undegraded":"2024-03-20T07:58:26.147717+0000","last_fullsized":"2024-03-20T07:58:26.147717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140470+0000","last_change":"2024-03-20T07:58:26.140127+0000","last_active":"2024-03-20T07:58:26.140470+0000","last_peered":"2024-03-20T07:58:26.140470+0000","last_clean":"2024-03-20T07:58:26.140470+0000","last_became_active":"2024-03-20T07:58:26.139362+0000","last_became_peered":"2024-03-20T07:58:26.139362+0000","last_unstale":"2024-03-20T07:58:26.140470+0000","last_undegraded":"2024-03-20T07:58:26.140470+0000","last_fullsized":"2024-03-20T07:58:26.140470+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239407+0000","last_change":"2024-03-20T07:58:25.771772+0000","last_active":"2024-03-20T07:58:27.239407+0000","last_peered":"2024-03-20T07:58:27.239407+0000","last_clean":"2024-03-20T07:58:27.239407+0000","last_became_active":"2024-03-20T07:58:25.771499+0000","last_became_peered":"2024-03-20T07:58:25.771499+0000","last_unstale":"2024-03-20T07:58:27.239407+0000","last_undegraded":"2024-03-20T07:58:27.239407+0000","last_fullsized":"2024-03-20T07:58:27.239407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:19:45.833311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238700+0000","last_change":"2024-03-20T07:58:26.122549+0000","last_active":"2024-03-20T07:58:27.238700+0000","last_peered":"2024-03-20T07:58:27.238700+0000","last_clean":"2024-03-20T07:58:27.238700+0000","last_became_active":"2024-03-20T07:58:26.121780+0000","last_became_peered":"2024-03-20T07:58:26.121780+0000","last_unstale":"2024-03-20T07:58:27.238700+0000","last_undegraded":"2024-03-20T07:58:27.238700+0000","last_fullsized":"2024-03-20T07:58:27.238700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:37:01.164741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.112184+0000","last_change":"2024-03-20T07:58:25.771839+0000","last_active":"2024-03-20T07:58:26.112184+0000","last_peered":"2024-03-20T07:58:26.112184+0000","last_clean":"2024-03-20T07:58:26.112184+0000","last_became_active":"2024-03-20T07:58:25.771464+0000","last_became_peered":"2024-03-20T07:58:25.771464+0000","last_unstale":"2024-03-20T07:58:26.112184+0000","last_undegraded":"2024-03-20T07:58:26.112184+0000","last_fullsized":"2024-03-20T07:58:26.112184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.129894+0000","last_change":"2024-03-20T07:58:26.129094+0000","last_active":"2024-03-20T07:58:26.129894+0000","last_peered":"2024-03-20T07:58:26.129894+0000","last_clean":"2024-03-20T07:58:26.129894+0000","last_became_active":"2024-03-20T07:58:26.128675+0000","last_became_peered":"2024-03-20T07:58:26.128675+0000","last_unstale":"2024-03-20T07:58:26.129894+0000","last_undegraded":"2024-03-20T07:58:26.129894+0000","last_fullsized":"2024-03-20T07:58:26.129894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.124730+0000","last_change":"2024-03-20T07:58:25.767586+0000","last_active":"2024-03-20T07:58:26.124730+0000","last_peered":"2024-03-20T07:58:26.124730+0000","last_clean":"2024-03-20T07:58:26.124730+0000","last_became_active":"2024-03-20T07:58:25.767268+0000","last_became_peered":"2024-03-20T07:58:25.767268+0000","last_unstale":"2024-03-20T07:58:26.124730+0000","last_undegraded":"2024-03-20T07:58:26.124730+0000","last_fullsized":"2024-03-20T07:58:26.124730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.144101+0000","last_change":"2024-03-20T07:58:26.139925+0000","last_active":"2024-03-20T07:58:26.144101+0000","last_peered":"2024-03-20T07:58:26.144101+0000","last_clean":"2024-03-20T07:58:26.144101+0000","last_became_active":"2024-03-20T07:58:26.138286+0000","last_became_peered":"2024-03-20T07:58:26.138286+0000","last_unstale":"2024-03-20T07:58:26.144101+0000","last_undegraded":"2024-03-20T07:58:26.144101+0000","last_fullsized":"2024-03-20T07:58:26.144101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238203+0000","last_change":"2024-03-20T07:58:25.757217+0000","last_active":"2024-03-20T07:58:27.238203+0000","last_peered":"2024-03-20T07:58:27.238203+0000","last_clean":"2024-03-20T07:58:27.238203+0000","last_became_active":"2024-03-20T07:58:25.756879+0000","last_became_peered":"2024-03-20T07:58:25.756879+0000","last_unstale":"2024-03-20T07:58:27.238203+0000","last_undegraded":"2024-03-20T07:58:27.238203+0000","last_fullsized":"2024-03-20T07:58:27.238203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:28.214822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134394+0000","last_change":"2024-03-20T07:58:26.133140+0000","last_active":"2024-03-20T07:58:26.134394+0000","last_peered":"2024-03-20T07:58:26.134394+0000","last_clean":"2024-03-20T07:58:26.134394+0000","last_became_active":"2024-03-20T07:58:26.132630+0000","last_became_peered":"2024-03-20T07:58:26.132630+0000","last_unstale":"2024-03-20T07:58:26.134394+0000","last_undegraded":"2024-03-20T07:58:26.134394+0000","last_fullsized":"2024-03-20T07:58:26.134394+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.135400+0000","last_change":"2024-03-20T07:58:25.771322+0000","last_active":"2024-03-20T07:58:26.135400+0000","last_peered":"2024-03-20T07:58:26.135400+0000","last_clean":"2024-03-20T07:58:26.135400+0000","last_became_active":"2024-03-20T07:58:25.770984+0000","last_became_peered":"2024-03-20T07:58:25.770984+0000","last_unstale":"2024-03-20T07:58:26.135400+0000","last_undegraded":"2024-03-20T07:58:26.135400+0000","last_fullsized":"2024-03-20T07:58:26.135400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145263+0000","last_change":"2024-03-20T07:58:26.144254+0000","last_active":"2024-03-20T07:58:26.145263+0000","last_peered":"2024-03-20T07:58:26.145263+0000","last_clean":"2024-03-20T07:58:26.145263+0000","last_became_active":"2024-03-20T07:58:26.143683+0000","last_became_peered":"2024-03-20T07:58:26.143683+0000","last_unstale":"2024-03-20T07:58:26.145263+0000","last_undegraded":"2024-03-20T07:58:26.145263+0000","last_fullsized":"2024-03-20T07:58:26.145263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107428+0000","last_change":"2024-03-20T07:58:25.773911+0000","last_active":"2024-03-20T07:58:27.107428+0000","last_peered":"2024-03-20T07:58:27.107428+0000","last_clean":"2024-03-20T07:58:27.107428+0000","last_became_active":"2024-03-20T07:58:25.773526+0000","last_became_peered":"2024-03-20T07:58:25.773526+0000","last_unstale":"2024-03-20T07:58:27.107428+0000","last_undegraded":"2024-03-20T07:58:27.107428+0000","last_fullsized":"2024-03-20T07:58:27.107428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.123504+0000","last_change":"2024-03-20T07:58:26.122909+0000","last_active":"2024-03-20T07:58:26.123504+0000","last_peered":"2024-03-20T07:58:26.123504+0000","last_clean":"2024-03-20T07:58:26.123504+0000","last_became_active":"2024-03-20T07:58:26.121608+0000","last_became_peered":"2024-03-20T07:58:26.121608+0000","last_unstale":"2024-03-20T07:58:26.123504+0000","last_undegraded":"2024-03-20T07:58:26.123504+0000","last_fullsized":"2024-03-20T07:58:26.123504+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111928+0000","last_change":"2024-03-20T07:58:25.770569+0000","last_active":"2024-03-20T07:58:26.111928+0000","last_peered":"2024-03-20T07:58:26.111928+0000","last_clean":"2024-03-20T07:58:26.111928+0000","last_became_active":"2024-03-20T07:58:25.770298+0000","last_became_peered":"2024-03-20T07:58:25.770298+0000","last_unstale":"2024-03-20T07:58:26.111928+0000","last_undegraded":"2024-03-20T07:58:26.111928+0000","last_fullsized":"2024-03-20T07:58:26.111928+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.137205+0000","last_change":"2024-03-20T07:58:26.136200+0000","last_active":"2024-03-20T07:58:26.137205+0000","last_peered":"2024-03-20T07:58:26.137205+0000","last_clean":"2024-03-20T07:58:26.137205+0000","last_became_active":"2024-03-20T07:58:26.135909+0000","last_became_peered":"2024-03-20T07:58:26.135909+0000","last_unstale":"2024-03-20T07:58:26.137205+0000","last_undegraded":"2024-03-20T07:58:26.137205+0000","last_fullsized":"2024-03-20T07:58:26.137205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107104+0000","last_change":"2024-03-20T07:58:26.130079+0000","last_active":"2024-03-20T07:58:27.107104+0000","last_peered":"2024-03-20T07:58:27.107104+0000","last_clean":"2024-03-20T07:58:27.107104+0000","last_became_active":"2024-03-20T07:58:26.129798+0000","last_became_peered":"2024-03-20T07:58:26.129798+0000","last_unstale":"2024-03-20T07:58:27.107104+0000","last_undegraded":"2024-03-20T07:58:27.107104+0000","last_fullsized":"2024-03-20T07:58:27.107104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238173+0000","last_change":"2024-03-20T07:58:25.764736+0000","last_active":"2024-03-20T07:58:27.238173+0000","last_peered":"2024-03-20T07:58:27.238173+0000","last_clean":"2024-03-20T07:58:27.238173+0000","last_became_active":"2024-03-20T07:58:25.764390+0000","last_became_peered":"2024-03-20T07:58:25.764390+0000","last_unstale":"2024-03-20T07:58:27.238173+0000","last_undegraded":"2024-03-20T07:58:27.238173+0000","last_fullsized":"2024-03-20T07:58:27.238173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:27:58.881448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.110613+0000","last_change":"2024-03-20T07:58:25.766643+0000","last_active":"2024-03-20T07:58:26.110613+0000","last_peered":"2024-03-20T07:58:26.110613+0000","last_clean":"2024-03-20T07:58:26.110613+0000","last_became_active":"2024-03-20T07:58:25.766341+0000","last_became_peered":"2024-03-20T07:58:25.766341+0000","last_unstale":"2024-03-20T07:58:26.110613+0000","last_undegraded":"2024-03-20T07:58:26.110613+0000","last_fullsized":"2024-03-20T07:58:26.110613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"22'117","reported_seq":218,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-20T07:58:25.769715+0000","last_change":"2024-03-20T07:58:09.964395+0000","last_active":"2024-03-20T07:58:25.769715+0000","last_peered":"2024-03-20T07:58:25.769715+0000","last_clean":"2024-03-20T07:58:25.769715+0000","last_became_active":"2024-03-20T07:58:09.963927+0000","last_became_peered":"2024-03-20T07:58:09.963927+0000","last_unstale":"2024-03-20T07:58:25.769715+0000","last_undegraded":"2024-03-20T07:58:25.769715+0000","last_fullsized":"2024-03-20T07:58:25.769715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238574+0000","last_change":"2024-03-20T07:58:26.125436+0000","last_active":"2024-03-20T07:58:27.238574+0000","last_peered":"2024-03-20T07:58:27.238574+0000","last_clean":"2024-03-20T07:58:27.238574+0000","last_became_active":"2024-03-20T07:58:26.125161+0000","last_became_peered":"2024-03-20T07:58:26.125161+0000","last_unstale":"2024-03-20T07:58:27.238574+0000","last_undegraded":"2024-03-20T07:58:27.238574+0000","last_fullsized":"2024-03-20T07:58:27.238574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:53:55.085832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.118188+0000","last_change":"2024-03-20T07:58:25.779129+0000","last_active":"2024-03-20T07:58:26.118188+0000","last_peered":"2024-03-20T07:58:26.118188+0000","last_clean":"2024-03-20T07:58:26.118188+0000","last_became_active":"2024-03-20T07:58:25.778258+0000","last_became_peered":"2024-03-20T07:58:25.778258+0000","last_unstale":"2024-03-20T07:58:26.118188+0000","last_undegraded":"2024-03-20T07:58:26.118188+0000","last_fullsized":"2024-03-20T07:58:26.118188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.130223+0000","last_change":"2024-03-20T07:58:25.777973+0000","last_active":"2024-03-20T07:58:26.130223+0000","last_peered":"2024-03-20T07:58:26.130223+0000","last_clean":"2024-03-20T07:58:26.130223+0000","last_became_active":"2024-03-20T07:58:25.777666+0000","last_became_peered":"2024-03-20T07:58:25.777666+0000","last_unstale":"2024-03-20T07:58:26.130223+0000","last_undegraded":"2024-03-20T07:58:26.130223+0000","last_fullsized":"2024-03-20T07:58:26.130223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107508+0000","last_change":"2024-03-20T07:58:26.126692+0000","last_active":"2024-03-20T07:58:27.107508+0000","last_peered":"2024-03-20T07:58:27.107508+0000","last_clean":"2024-03-20T07:58:27.107508+0000","last_became_active":"2024-03-20T07:58:26.126397+0000","last_became_peered":"2024-03-20T07:58:26.126397+0000","last_unstale":"2024-03-20T07:58:27.107508+0000","last_undegraded":"2024-03-20T07:58:27.107508+0000","last_fullsized":"2024-03-20T07:58:27.107508+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108422+0000","last_change":"2024-03-20T07:58:25.769337+0000","last_active":"2024-03-20T07:58:27.108422+0000","last_peered":"2024-03-20T07:58:27.108422+0000","last_clean":"2024-03-20T07:58:27.108422+0000","last_became_active":"2024-03-20T07:58:25.769010+0000","last_became_peered":"2024-03-20T07:58:25.769010+0000","last_unstale":"2024-03-20T07:58:27.108422+0000","last_undegraded":"2024-03-20T07:58:27.108422+0000","last_fullsized":"2024-03-20T07:58:27.108422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.141871+0000","last_change":"2024-03-20T07:58:25.771097+0000","last_active":"2024-03-20T07:58:26.141871+0000","last_peered":"2024-03-20T07:58:26.141871+0000","last_clean":"2024-03-20T07:58:26.141871+0000","last_became_active":"2024-03-20T07:58:25.770737+0000","last_became_peered":"2024-03-20T07:58:25.770737+0000","last_unstale":"2024-03-20T07:58:26.141871+0000","last_undegraded":"2024-03-20T07:58:26.141871+0000","last_fullsized":"2024-03-20T07:58:26.141871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239163+0000","last_change":"2024-03-20T07:58:26.126230+0000","last_active":"2024-03-20T07:58:27.239163+0000","last_peered":"2024-03-20T07:58:27.239163+0000","last_clean":"2024-03-20T07:58:27.239163+0000","last_became_active":"2024-03-20T07:58:26.125980+0000","last_became_peered":"2024-03-20T07:58:26.125980+0000","last_unstale":"2024-03-20T07:58:27.239163+0000","last_undegraded":"2024-03-20T07:58:27.239163+0000","last_fullsized":"2024-03-20T07:58:27.239163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:18:03.971589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.117985+0000","last_change":"2024-03-20T07:58:25.777961+0000","last_active":"2024-03-20T07:58:26.117985+0000","last_peered":"2024-03-20T07:58:26.117985+0000","last_clean":"2024-03-20T07:58:26.117985+0000","last_became_active":"2024-03-20T07:58:25.777649+0000","last_became_peered":"2024-03-20T07:58:25.777649+0000","last_unstale":"2024-03-20T07:58:26.117985+0000","last_undegraded":"2024-03-20T07:58:26.117985+0000","last_fullsized":"2024-03-20T07:58:26.117985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.124307+0000","last_change":"2024-03-20T07:58:26.120305+0000","last_active":"2024-03-20T07:58:26.124307+0000","last_peered":"2024-03-20T07:58:26.124307+0000","last_clean":"2024-03-20T07:58:26.124307+0000","last_became_active":"2024-03-20T07:58:26.119966+0000","last_became_peered":"2024-03-20T07:58:26.119966+0000","last_unstale":"2024-03-20T07:58:26.124307+0000","last_undegraded":"2024-03-20T07:58:26.124307+0000","last_fullsized":"2024-03-20T07:58:26.124307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107099+0000","last_change":"2024-03-20T07:58:25.765548+0000","last_active":"2024-03-20T07:58:27.107099+0000","last_peered":"2024-03-20T07:58:27.107099+0000","last_clean":"2024-03-20T07:58:27.107099+0000","last_became_active":"2024-03-20T07:58:25.765273+0000","last_became_peered":"2024-03-20T07:58:25.765273+0000","last_unstale":"2024-03-20T07:58:27.107099+0000","last_undegraded":"2024-03-20T07:58:27.107099+0000","last_fullsized":"2024-03-20T07:58:27.107099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107547+0000","last_change":"2024-03-20T07:58:25.773811+0000","last_active":"2024-03-20T07:58:27.107547+0000","last_peered":"2024-03-20T07:58:27.107547+0000","last_clean":"2024-03-20T07:58:27.107547+0000","last_became_active":"2024-03-20T07:58:25.773520+0000","last_became_peered":"2024-03-20T07:58:25.773520+0000","last_unstale":"2024-03-20T07:58:27.107547+0000","last_undegraded":"2024-03-20T07:58:27.107547+0000","last_fullsized":"2024-03-20T07:58:27.107547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138933+0000","last_change":"2024-03-20T07:58:25.776066+0000","last_active":"2024-03-20T07:58:26.138933+0000","last_peered":"2024-03-20T07:58:26.138933+0000","last_clean":"2024-03-20T07:58:26.138933+0000","last_became_active":"2024-03-20T07:58:25.775708+0000","last_became_peered":"2024-03-20T07:58:25.775708+0000","last_unstale":"2024-03-20T07:58:26.138933+0000","last_undegraded":"2024-03-20T07:58:26.138933+0000","last_fullsized":"2024-03-20T07:58:26.138933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:36:20.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134688+0000","last_change":"2024-03-20T07:58:26.134067+0000","last_active":"2024-03-20T07:58:26.134688+0000","last_peered":"2024-03-20T07:58:26.134688+0000","last_clean":"2024-03-20T07:58:26.134688+0000","last_became_active":"2024-03-20T07:58:26.133784+0000","last_became_peered":"2024-03-20T07:58:26.133784+0000","last_unstale":"2024-03-20T07:58:26.134688+0000","last_undegraded":"2024-03-20T07:58:26.134688+0000","last_fullsized":"2024-03-20T07:58:26.134688+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238423+0000","last_change":"2024-03-20T07:58:25.763015+0000","last_active":"2024-03-20T07:58:27.238423+0000","last_peered":"2024-03-20T07:58:27.238423+0000","last_clean":"2024-03-20T07:58:27.238423+0000","last_became_active":"2024-03-20T07:58:25.762682+0000","last_became_peered":"2024-03-20T07:58:25.762682+0000","last_unstale":"2024-03-20T07:58:27.238423+0000","last_undegraded":"2024-03-20T07:58:27.238423+0000","last_fullsized":"2024-03-20T07:58:27.238423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:21:45.137669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.142797+0000","last_change":"2024-03-20T07:58:26.142445+0000","last_active":"2024-03-20T07:58:26.142797+0000","last_peered":"2024-03-20T07:58:26.142797+0000","last_clean":"2024-03-20T07:58:26.142797+0000","last_became_active":"2024-03-20T07:58:26.142148+0000","last_became_peered":"2024-03-20T07:58:26.142148+0000","last_unstale":"2024-03-20T07:58:26.142797+0000","last_undegraded":"2024-03-20T07:58:26.142797+0000","last_fullsized":"2024-03-20T07:58:26.142797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"62'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107988+0000","last_change":"2024-03-20T07:58:25.766427+0000","last_active":"2024-03-20T07:58:27.107988+0000","last_peered":"2024-03-20T07:58:27.107988+0000","last_clean":"2024-03-20T07:58:27.107988+0000","last_became_active":"2024-03-20T07:58:25.766111+0000","last_became_peered":"2024-03-20T07:58:25.766111+0000","last_unstale":"2024-03-20T07:58:27.107988+0000","last_undegraded":"2024-03-20T07:58:27.107988+0000","last_fullsized":"2024-03-20T07:58:27.107988+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.136811+0000","last_change":"2024-03-20T07:58:26.135339+0000","last_active":"2024-03-20T07:58:26.136811+0000","last_peered":"2024-03-20T07:58:26.136811+0000","last_clean":"2024-03-20T07:58:26.136811+0000","last_became_active":"2024-03-20T07:58:26.134903+0000","last_became_peered":"2024-03-20T07:58:26.134903+0000","last_unstale":"2024-03-20T07:58:26.136811+0000","last_undegraded":"2024-03-20T07:58:26.136811+0000","last_fullsized":"2024-03-20T07:58:26.136811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"62'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238866+0000","last_change":"2024-03-20T07:58:25.766233+0000","last_active":"2024-03-20T07:58:27.238866+0000","last_peered":"2024-03-20T07:58:27.238866+0000","last_clean":"2024-03-20T07:58:27.238866+0000","last_became_active":"2024-03-20T07:58:25.765948+0000","last_became_peered":"2024-03-20T07:58:25.765948+0000","last_unstale":"2024-03-20T07:58:27.238866+0000","last_undegraded":"2024-03-20T07:58:27.238866+0000","last_fullsized":"2024-03-20T07:58:27.238866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:34:01.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145073+0000","last_change":"2024-03-20T07:58:26.144008+0000","last_active":"2024-03-20T07:58:26.145073+0000","last_peered":"2024-03-20T07:58:26.145073+0000","last_clean":"2024-03-20T07:58:26.145073+0000","last_became_active":"2024-03-20T07:58:26.143696+0000","last_became_peered":"2024-03-20T07:58:26.143696+0000","last_unstale":"2024-03-20T07:58:26.145073+0000","last_undegraded":"2024-03-20T07:58:26.145073+0000","last_fullsized":"2024-03-20T07:58:26.145073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140012+0000","last_change":"2024-03-20T07:58:26.138096+0000","last_active":"2024-03-20T07:58:26.140012+0000","last_peered":"2024-03-20T07:58:26.140012+0000","last_clean":"2024-03-20T07:58:26.140012+0000","last_became_active":"2024-03-20T07:58:26.137804+0000","last_became_peered":"2024-03-20T07:58:26.137804+0000","last_unstale":"2024-03-20T07:58:26.140012+0000","last_undegraded":"2024-03-20T07:58:26.140012+0000","last_fullsized":"2024-03-20T07:58:26.140012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238668+0000","last_change":"2024-03-20T07:58:26.126085+0000","last_active":"2024-03-20T07:58:27.238668+0000","last_peered":"2024-03-20T07:58:27.238668+0000","last_clean":"2024-03-20T07:58:27.238668+0000","last_became_active":"2024-03-20T07:58:26.125793+0000","last_became_peered":"2024-03-20T07:58:26.125793+0000","last_unstale":"2024-03-20T07:58:27.238668+0000","last_undegraded":"2024-03-20T07:58:27.238668+0000","last_fullsized":"2024-03-20T07:58:27.238668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:11:45.470755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.126540+0000","last_change":"2024-03-20T07:58:26.125004+0000","last_active":"2024-03-20T07:58:26.126540+0000","last_peered":"2024-03-20T07:58:26.126540+0000","last_clean":"2024-03-20T07:58:26.126540+0000","last_became_active":"2024-03-20T07:58:26.124689+0000","last_became_peered":"2024-03-20T07:58:26.124689+0000","last_unstale":"2024-03-20T07:58:26.126540+0000","last_undegraded":"2024-03-20T07:58:26.126540+0000","last_fullsized":"2024-03-20T07:58:26.126540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239208+0000","last_change":"2024-03-20T07:58:26.120048+0000","last_active":"2024-03-20T07:58:27.239208+0000","last_peered":"2024-03-20T07:58:27.239208+0000","last_clean":"2024-03-20T07:58:27.239208+0000","last_became_active":"2024-03-20T07:58:26.119755+0000","last_became_peered":"2024-03-20T07:58:26.119755+0000","last_unstale":"2024-03-20T07:58:27.239208+0000","last_undegraded":"2024-03-20T07:58:27.239208+0000","last_fullsized":"2024-03-20T07:58:27.239208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:39:07.689571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107012+0000","last_change":"2024-03-20T07:58:26.126113+0000","last_active":"2024-03-20T07:58:27.107012+0000","last_peered":"2024-03-20T07:58:27.107012+0000","last_clean":"2024-03-20T07:58:27.107012+0000","last_became_active":"2024-03-20T07:58:26.122635+0000","last_became_peered":"2024-03-20T07:58:26.122635+0000","last_unstale":"2024-03-20T07:58:27.107012+0000","last_undegraded":"2024-03-20T07:58:27.107012+0000","last_fullsized":"2024-03-20T07:58:27.107012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.144469+0000","last_change":"2024-03-20T07:58:26.141379+0000","last_active":"2024-03-20T07:58:26.144469+0000","last_peered":"2024-03-20T07:58:26.144469+0000","last_clean":"2024-03-20T07:58:26.144469+0000","last_became_active":"2024-03-20T07:58:26.141099+0000","last_became_peered":"2024-03-20T07:58:26.141099+0000","last_unstale":"2024-03-20T07:58:26.144469+0000","last_undegraded":"2024-03-20T07:58:26.144469+0000","last_fullsized":"2024-03-20T07:58:26.144469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138838+0000","last_change":"2024-03-20T07:58:26.137638+0000","last_active":"2024-03-20T07:58:26.138838+0000","last_peered":"2024-03-20T07:58:26.138838+0000","last_clean":"2024-03-20T07:58:26.138838+0000","last_became_active":"2024-03-20T07:58:26.137344+0000","last_became_peered":"2024-03-20T07:58:26.137344+0000","last_unstale":"2024-03-20T07:58:26.138838+0000","last_undegraded":"2024-03-20T07:58:26.138838+0000","last_fullsized":"2024-03-20T07:58:26.138838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138511+0000","last_change":"2024-03-20T07:58:26.137117+0000","last_active":"2024-03-20T07:58:26.138511+0000","last_peered":"2024-03-20T07:58:26.138511+0000","last_clean":"2024-03-20T07:58:26.138511+0000","last_became_active":"2024-03-20T07:58:26.136135+0000","last_became_peered":"2024-03-20T07:58:26.136135+0000","last_unstale":"2024-03-20T07:58:26.138511+0000","last_undegraded":"2024-03-20T07:58:26.138511+0000","last_fullsized":"2024-03-20T07:58:26.138511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239107+0000","last_change":"2024-03-20T07:58:26.119480+0000","last_active":"2024-03-20T07:58:27.239107+0000","last_peered":"2024-03-20T07:58:27.239107+0000","last_clean":"2024-03-20T07:58:27.239107+0000","last_became_active":"2024-03-20T07:58:26.119047+0000","last_became_peered":"2024-03-20T07:58:26.119047+0000","last_unstale":"2024-03-20T07:58:27.239107+0000","last_undegraded":"2024-03-20T07:58:27.239107+0000","last_fullsized":"2024-03-20T07:58:27.239107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:09:31.918661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134663+0000","last_change":"2024-03-20T07:58:26.133160+0000","last_active":"2024-03-20T07:58:26.134663+0000","last_peered":"2024-03-20T07:58:26.134663+0000","last_clean":"2024-03-20T07:58:26.134663+0000","last_became_active":"2024-03-20T07:58:26.132793+0000","last_became_peered":"2024-03-20T07:58:26.132793+0000","last_unstale":"2024-03-20T07:58:26.134663+0000","last_undegraded":"2024-03-20T07:58:26.134663+0000","last_fullsized":"2024-03-20T07:58:26.134663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":159,"acting":159,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":87,"acting":87,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201286,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":838705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462923,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1183744,"data_stored":831669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717512,"statfs":{"total":95995035648,"available":95966732288,"internally_reserved":0,"allocated":626688,"data_stored":255702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.33100000000000002,"15min":0.32300000000000001},"min":{"1min":0.26200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.35599999999999998,"15min":0.34399999999999997},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.313}]},{"osd":2,"last update":"Wed Mar 20 07:58:07 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.39300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.25},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.308,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.27800000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.37}]},{"osd":4,"last update":"Wed Mar 20 07:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738410,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1228800,"data_stored":846638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 20 07:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.26300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.32200000000000001,"15min":0.312},"min":{"1min":0.26300000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Mar 20 07:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33000000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.318},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.374}]},{"osd":4,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.34100000000000003}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.39400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411359,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717520,"statfs":{"total":95995035648,"available":95966740480,"internally_reserved":0,"allocated":618496,"data_stored":255542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.36799999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35699999999999998}]},{"osd":1,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.33900000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38100000000000001},"min":{"1min":0.33000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52000000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.311},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Mar 20 07:58:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149721,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717556,"statfs":{"total":95995035648,"available":95966777344,"internally_reserved":0,"allocated":581632,"data_stored":235385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.36499999999999999}]},{"osd":1,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.626},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Mar 20 07:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986197,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":634880,"data_stored":256250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.374}]},{"osd":1,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.503}]},{"osd":2,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.432}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.35699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717528,"statfs":{"total":95995035648,"available":95966748672,"internally_reserved":0,"allocated":618496,"data_stored":255542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44}]},{"osd":1,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.495}]},{"osd":3,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.38400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T07:58:29.410 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-20T07:58:29.736 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:29.738 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T07:58:29.745 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-20T07:58:29.236703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":249,"acting":249,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":749961216,"kb_used":222732,"kb_used_data":6536,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749738484,"statfs":{"total":767960285184,"available":767732207616,"internally_reserved":0,"allocated":6692864,"data_stored":3775433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.491697"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.146486+0000","last_change":"2024-03-20T07:58:26.145662+0000","last_active":"2024-03-20T07:58:26.146486+0000","last_peered":"2024-03-20T07:58:26.146486+0000","last_clean":"2024-03-20T07:58:26.146486+0000","last_became_active":"2024-03-20T07:58:26.145286+0000","last_became_peered":"2024-03-20T07:58:26.145286+0000","last_unstale":"2024-03-20T07:58:26.146486+0000","last_undegraded":"2024-03-20T07:58:26.146486+0000","last_fullsized":"2024-03-20T07:58:26.146486+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108335+0000","last_change":"2024-03-20T07:58:26.129069+0000","last_active":"2024-03-20T07:58:27.108335+0000","last_peered":"2024-03-20T07:58:27.108335+0000","last_clean":"2024-03-20T07:58:27.108335+0000","last_became_active":"2024-03-20T07:58:26.128811+0000","last_became_peered":"2024-03-20T07:58:26.128811+0000","last_unstale":"2024-03-20T07:58:27.108335+0000","last_undegraded":"2024-03-20T07:58:27.108335+0000","last_fullsized":"2024-03-20T07:58:27.108335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239393+0000","last_change":"2024-03-20T07:58:26.119030+0000","last_active":"2024-03-20T07:58:27.239393+0000","last_peered":"2024-03-20T07:58:27.239393+0000","last_clean":"2024-03-20T07:58:27.239393+0000","last_became_active":"2024-03-20T07:58:26.118755+0000","last_became_peered":"2024-03-20T07:58:26.118755+0000","last_unstale":"2024-03-20T07:58:27.239393+0000","last_undegraded":"2024-03-20T07:58:27.239393+0000","last_fullsized":"2024-03-20T07:58:27.239393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:51:41.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139286+0000","last_change":"2024-03-20T07:58:26.138067+0000","last_active":"2024-03-20T07:58:26.139286+0000","last_peered":"2024-03-20T07:58:26.139286+0000","last_clean":"2024-03-20T07:58:26.139286+0000","last_became_active":"2024-03-20T07:58:26.137768+0000","last_became_peered":"2024-03-20T07:58:26.137768+0000","last_unstale":"2024-03-20T07:58:26.139286+0000","last_undegraded":"2024-03-20T07:58:26.139286+0000","last_fullsized":"2024-03-20T07:58:26.139286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134630+0000","last_change":"2024-03-20T07:58:26.131923+0000","last_active":"2024-03-20T07:58:26.134630+0000","last_peered":"2024-03-20T07:58:26.134630+0000","last_clean":"2024-03-20T07:58:26.134630+0000","last_became_active":"2024-03-20T07:58:26.131585+0000","last_became_peered":"2024-03-20T07:58:26.131585+0000","last_unstale":"2024-03-20T07:58:26.134630+0000","last_undegraded":"2024-03-20T07:58:26.134630+0000","last_fullsized":"2024-03-20T07:58:26.134630+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139194+0000","last_change":"2024-03-20T07:58:26.137308+0000","last_active":"2024-03-20T07:58:26.139194+0000","last_peered":"2024-03-20T07:58:26.139194+0000","last_clean":"2024-03-20T07:58:26.139194+0000","last_became_active":"2024-03-20T07:58:26.135727+0000","last_became_peered":"2024-03-20T07:58:26.135727+0000","last_unstale":"2024-03-20T07:58:26.139194+0000","last_undegraded":"2024-03-20T07:58:26.139194+0000","last_fullsized":"2024-03-20T07:58:26.139194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134374+0000","last_change":"2024-03-20T07:58:26.132941+0000","last_active":"2024-03-20T07:58:26.134374+0000","last_peered":"2024-03-20T07:58:26.134374+0000","last_clean":"2024-03-20T07:58:26.134374+0000","last_became_active":"2024-03-20T07:58:26.132230+0000","last_became_peered":"2024-03-20T07:58:26.132230+0000","last_unstale":"2024-03-20T07:58:26.134374+0000","last_undegraded":"2024-03-20T07:58:26.134374+0000","last_fullsized":"2024-03-20T07:58:26.134374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145322+0000","last_change":"2024-03-20T07:58:26.144631+0000","last_active":"2024-03-20T07:58:26.145322+0000","last_peered":"2024-03-20T07:58:26.145322+0000","last_clean":"2024-03-20T07:58:26.145322+0000","last_became_active":"2024-03-20T07:58:26.144302+0000","last_became_peered":"2024-03-20T07:58:26.144302+0000","last_unstale":"2024-03-20T07:58:26.145322+0000","last_undegraded":"2024-03-20T07:58:26.145322+0000","last_fullsized":"2024-03-20T07:58:26.145322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139423+0000","last_change":"2024-03-20T07:58:26.138657+0000","last_active":"2024-03-20T07:58:26.139423+0000","last_peered":"2024-03-20T07:58:26.139423+0000","last_clean":"2024-03-20T07:58:26.139423+0000","last_became_active":"2024-03-20T07:58:26.138348+0000","last_became_peered":"2024-03-20T07:58:26.138348+0000","last_unstale":"2024-03-20T07:58:26.139423+0000","last_undegraded":"2024-03-20T07:58:26.139423+0000","last_fullsized":"2024-03-20T07:58:26.139423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138763+0000","last_change":"2024-03-20T07:58:26.137961+0000","last_active":"2024-03-20T07:58:26.138763+0000","last_peered":"2024-03-20T07:58:26.138763+0000","last_clean":"2024-03-20T07:58:26.138763+0000","last_became_active":"2024-03-20T07:58:26.137672+0000","last_became_peered":"2024-03-20T07:58:26.137672+0000","last_unstale":"2024-03-20T07:58:26.138763+0000","last_undegraded":"2024-03-20T07:58:26.138763+0000","last_fullsized":"2024-03-20T07:58:26.138763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.137140+0000","last_change":"2024-03-20T07:58:26.136246+0000","last_active":"2024-03-20T07:58:26.137140+0000","last_peered":"2024-03-20T07:58:26.137140+0000","last_clean":"2024-03-20T07:58:26.137140+0000","last_became_active":"2024-03-20T07:58:26.135935+0000","last_became_peered":"2024-03-20T07:58:26.135935+0000","last_unstale":"2024-03-20T07:58:26.137140+0000","last_undegraded":"2024-03-20T07:58:26.137140+0000","last_fullsized":"2024-03-20T07:58:26.137140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238796+0000","last_change":"2024-03-20T07:58:26.123008+0000","last_active":"2024-03-20T07:58:27.238796+0000","last_peered":"2024-03-20T07:58:27.238796+0000","last_clean":"2024-03-20T07:58:27.238796+0000","last_became_active":"2024-03-20T07:58:26.122270+0000","last_became_peered":"2024-03-20T07:58:26.122270+0000","last_unstale":"2024-03-20T07:58:27.238796+0000","last_undegraded":"2024-03-20T07:58:27.238796+0000","last_fullsized":"2024-03-20T07:58:27.238796+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:37.385767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107015+0000","last_change":"2024-03-20T07:58:26.127042+0000","last_active":"2024-03-20T07:58:27.107015+0000","last_peered":"2024-03-20T07:58:27.107015+0000","last_clean":"2024-03-20T07:58:27.107015+0000","last_became_active":"2024-03-20T07:58:26.126555+0000","last_became_peered":"2024-03-20T07:58:26.126555+0000","last_unstale":"2024-03-20T07:58:27.107015+0000","last_undegraded":"2024-03-20T07:58:27.107015+0000","last_fullsized":"2024-03-20T07:58:27.107015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.132048+0000","last_change":"2024-03-20T07:58:26.129086+0000","last_active":"2024-03-20T07:58:26.132048+0000","last_peered":"2024-03-20T07:58:26.132048+0000","last_clean":"2024-03-20T07:58:26.132048+0000","last_became_active":"2024-03-20T07:58:26.128111+0000","last_became_peered":"2024-03-20T07:58:26.128111+0000","last_unstale":"2024-03-20T07:58:26.132048+0000","last_undegraded":"2024-03-20T07:58:26.132048+0000","last_fullsized":"2024-03-20T07:58:26.132048+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.131910+0000","last_change":"2024-03-20T07:58:26.128812+0000","last_active":"2024-03-20T07:58:26.131910+0000","last_peered":"2024-03-20T07:58:26.131910+0000","last_clean":"2024-03-20T07:58:26.131910+0000","last_became_active":"2024-03-20T07:58:26.128501+0000","last_became_peered":"2024-03-20T07:58:26.128501+0000","last_unstale":"2024-03-20T07:58:26.131910+0000","last_undegraded":"2024-03-20T07:58:26.131910+0000","last_fullsized":"2024-03-20T07:58:26.131910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238912+0000","last_change":"2024-03-20T07:58:26.128211+0000","last_active":"2024-03-20T07:58:27.238912+0000","last_peered":"2024-03-20T07:58:27.238912+0000","last_clean":"2024-03-20T07:58:27.238912+0000","last_became_active":"2024-03-20T07:58:26.127943+0000","last_became_peered":"2024-03-20T07:58:26.127943+0000","last_unstale":"2024-03-20T07:58:27.238912+0000","last_undegraded":"2024-03-20T07:58:27.238912+0000","last_fullsized":"2024-03-20T07:58:27.238912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:19.979568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134420+0000","last_change":"2024-03-20T07:58:26.133112+0000","last_active":"2024-03-20T07:58:26.134420+0000","last_peered":"2024-03-20T07:58:26.134420+0000","last_clean":"2024-03-20T07:58:26.134420+0000","last_became_active":"2024-03-20T07:58:26.132794+0000","last_became_peered":"2024-03-20T07:58:26.132794+0000","last_unstale":"2024-03-20T07:58:26.134420+0000","last_undegraded":"2024-03-20T07:58:26.134420+0000","last_fullsized":"2024-03-20T07:58:26.134420+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"62'4","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239419+0000","last_change":"2024-03-20T07:58:25.774846+0000","last_active":"2024-03-20T07:58:27.239419+0000","last_peered":"2024-03-20T07:58:27.239419+0000","last_clean":"2024-03-20T07:58:27.239419+0000","last_became_active":"2024-03-20T07:58:25.774570+0000","last_became_peered":"2024-03-20T07:58:25.774570+0000","last_unstale":"2024-03-20T07:58:27.239419+0000","last_undegraded":"2024-03-20T07:58:27.239419+0000","last_fullsized":"2024-03-20T07:58:27.239419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:00.513690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239316+0000","last_change":"2024-03-20T07:58:26.118404+0000","last_active":"2024-03-20T07:58:27.239316+0000","last_peered":"2024-03-20T07:58:27.239316+0000","last_clean":"2024-03-20T07:58:27.239316+0000","last_became_active":"2024-03-20T07:58:26.118107+0000","last_became_peered":"2024-03-20T07:58:26.118107+0000","last_unstale":"2024-03-20T07:58:27.239316+0000","last_undegraded":"2024-03-20T07:58:27.239316+0000","last_fullsized":"2024-03-20T07:58:27.239316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:51.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145432+0000","last_change":"2024-03-20T07:58:25.778043+0000","last_active":"2024-03-20T07:58:26.145432+0000","last_peered":"2024-03-20T07:58:26.145432+0000","last_clean":"2024-03-20T07:58:26.145432+0000","last_became_active":"2024-03-20T07:58:25.777710+0000","last_became_peered":"2024-03-20T07:58:25.777710+0000","last_unstale":"2024-03-20T07:58:26.145432+0000","last_undegraded":"2024-03-20T07:58:26.145432+0000","last_fullsized":"2024-03-20T07:58:26.145432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.123506+0000","last_change":"2024-03-20T07:58:26.122440+0000","last_active":"2024-03-20T07:58:26.123506+0000","last_peered":"2024-03-20T07:58:26.123506+0000","last_clean":"2024-03-20T07:58:26.123506+0000","last_became_active":"2024-03-20T07:58:26.122136+0000","last_became_peered":"2024-03-20T07:58:26.122136+0000","last_unstale":"2024-03-20T07:58:26.123506+0000","last_undegraded":"2024-03-20T07:58:26.123506+0000","last_fullsized":"2024-03-20T07:58:26.123506+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138961+0000","last_change":"2024-03-20T07:58:25.776107+0000","last_active":"2024-03-20T07:58:26.138961+0000","last_peered":"2024-03-20T07:58:26.138961+0000","last_clean":"2024-03-20T07:58:26.138961+0000","last_became_active":"2024-03-20T07:58:25.775756+0000","last_became_peered":"2024-03-20T07:58:25.775756+0000","last_unstale":"2024-03-20T07:58:26.138961+0000","last_undegraded":"2024-03-20T07:58:26.138961+0000","last_fullsized":"2024-03-20T07:58:26.138961+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138778+0000","last_change":"2024-03-20T07:58:26.137236+0000","last_active":"2024-03-20T07:58:26.138778+0000","last_peered":"2024-03-20T07:58:26.138778+0000","last_clean":"2024-03-20T07:58:26.138778+0000","last_became_active":"2024-03-20T07:58:26.136660+0000","last_became_peered":"2024-03-20T07:58:26.136660+0000","last_unstale":"2024-03-20T07:58:26.138778+0000","last_undegraded":"2024-03-20T07:58:26.138778+0000","last_fullsized":"2024-03-20T07:58:26.138778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:36.426353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.119043+0000","last_change":"2024-03-20T07:58:26.118793+0000","last_active":"2024-03-20T07:58:26.119043+0000","last_peered":"2024-03-20T07:58:26.119043+0000","last_clean":"2024-03-20T07:58:26.119043+0000","last_became_active":"2024-03-20T07:58:26.118430+0000","last_became_peered":"2024-03-20T07:58:26.118430+0000","last_unstale":"2024-03-20T07:58:26.119043+0000","last_undegraded":"2024-03-20T07:58:26.119043+0000","last_fullsized":"2024-03-20T07:58:26.119043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140552+0000","last_change":"2024-03-20T07:58:25.765918+0000","last_active":"2024-03-20T07:58:26.140552+0000","last_peered":"2024-03-20T07:58:26.140552+0000","last_clean":"2024-03-20T07:58:26.140552+0000","last_became_active":"2024-03-20T07:58:25.765598+0000","last_became_peered":"2024-03-20T07:58:25.765598+0000","last_unstale":"2024-03-20T07:58:26.140552+0000","last_undegraded":"2024-03-20T07:58:26.140552+0000","last_fullsized":"2024-03-20T07:58:26.140552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239315+0000","last_change":"2024-03-20T07:58:26.119313+0000","last_active":"2024-03-20T07:58:27.239315+0000","last_peered":"2024-03-20T07:58:27.239315+0000","last_clean":"2024-03-20T07:58:27.239315+0000","last_became_active":"2024-03-20T07:58:26.118645+0000","last_became_peered":"2024-03-20T07:58:26.118645+0000","last_unstale":"2024-03-20T07:58:27.239315+0000","last_undegraded":"2024-03-20T07:58:27.239315+0000","last_fullsized":"2024-03-20T07:58:27.239315+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:13.410704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111175+0000","last_change":"2024-03-20T07:58:25.769007+0000","last_active":"2024-03-20T07:58:26.111175+0000","last_peered":"2024-03-20T07:58:26.111175+0000","last_clean":"2024-03-20T07:58:26.111175+0000","last_became_active":"2024-03-20T07:58:25.768716+0000","last_became_peered":"2024-03-20T07:58:25.768716+0000","last_unstale":"2024-03-20T07:58:26.111175+0000","last_undegraded":"2024-03-20T07:58:26.111175+0000","last_fullsized":"2024-03-20T07:58:26.111175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107247+0000","last_change":"2024-03-20T07:58:26.119518+0000","last_active":"2024-03-20T07:58:27.107247+0000","last_peered":"2024-03-20T07:58:27.107247+0000","last_clean":"2024-03-20T07:58:27.107247+0000","last_became_active":"2024-03-20T07:58:26.119184+0000","last_became_peered":"2024-03-20T07:58:26.119184+0000","last_unstale":"2024-03-20T07:58:27.107247+0000","last_undegraded":"2024-03-20T07:58:27.107247+0000","last_fullsized":"2024-03-20T07:58:27.107247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.128555+0000","last_change":"2024-03-20T07:58:25.769300+0000","last_active":"2024-03-20T07:58:26.128555+0000","last_peered":"2024-03-20T07:58:26.128555+0000","last_clean":"2024-03-20T07:58:26.128555+0000","last_became_active":"2024-03-20T07:58:25.768916+0000","last_became_peered":"2024-03-20T07:58:25.768916+0000","last_unstale":"2024-03-20T07:58:26.128555+0000","last_undegraded":"2024-03-20T07:58:26.128555+0000","last_fullsized":"2024-03-20T07:58:26.128555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.133696+0000","last_change":"2024-03-20T07:58:26.130616+0000","last_active":"2024-03-20T07:58:26.133696+0000","last_peered":"2024-03-20T07:58:26.133696+0000","last_clean":"2024-03-20T07:58:26.133696+0000","last_became_active":"2024-03-20T07:58:26.130332+0000","last_became_peered":"2024-03-20T07:58:26.130332+0000","last_unstale":"2024-03-20T07:58:26.133696+0000","last_undegraded":"2024-03-20T07:58:26.133696+0000","last_fullsized":"2024-03-20T07:58:26.133696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111307+0000","last_change":"2024-03-20T07:58:25.769790+0000","last_active":"2024-03-20T07:58:26.111307+0000","last_peered":"2024-03-20T07:58:26.111307+0000","last_clean":"2024-03-20T07:58:26.111307+0000","last_became_active":"2024-03-20T07:58:25.769492+0000","last_became_peered":"2024-03-20T07:58:25.769492+0000","last_unstale":"2024-03-20T07:58:26.111307+0000","last_undegraded":"2024-03-20T07:58:26.111307+0000","last_fullsized":"2024-03-20T07:58:26.111307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239245+0000","last_change":"2024-03-20T07:58:26.116344+0000","last_active":"2024-03-20T07:58:27.239245+0000","last_peered":"2024-03-20T07:58:27.239245+0000","last_clean":"2024-03-20T07:58:27.239245+0000","last_became_active":"2024-03-20T07:58:26.116037+0000","last_became_peered":"2024-03-20T07:58:26.116037+0000","last_unstale":"2024-03-20T07:58:27.239245+0000","last_undegraded":"2024-03-20T07:58:27.239245+0000","last_fullsized":"2024-03-20T07:58:27.239245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:16.183856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108079+0000","last_change":"2024-03-20T07:58:25.770988+0000","last_active":"2024-03-20T07:58:27.108079+0000","last_peered":"2024-03-20T07:58:27.108079+0000","last_clean":"2024-03-20T07:58:27.108079+0000","last_became_active":"2024-03-20T07:58:25.770657+0000","last_became_peered":"2024-03-20T07:58:25.770657+0000","last_unstale":"2024-03-20T07:58:27.108079+0000","last_undegraded":"2024-03-20T07:58:27.108079+0000","last_fullsized":"2024-03-20T07:58:27.108079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.131447+0000","last_change":"2024-03-20T07:58:26.128506+0000","last_active":"2024-03-20T07:58:26.131447+0000","last_peered":"2024-03-20T07:58:26.131447+0000","last_clean":"2024-03-20T07:58:26.131447+0000","last_became_active":"2024-03-20T07:58:26.128201+0000","last_became_peered":"2024-03-20T07:58:26.128201+0000","last_unstale":"2024-03-20T07:58:26.131447+0000","last_undegraded":"2024-03-20T07:58:26.131447+0000","last_fullsized":"2024-03-20T07:58:26.131447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.147717+0000","last_change":"2024-03-20T07:58:25.767830+0000","last_active":"2024-03-20T07:58:26.147717+0000","last_peered":"2024-03-20T07:58:26.147717+0000","last_clean":"2024-03-20T07:58:26.147717+0000","last_became_active":"2024-03-20T07:58:25.767533+0000","last_became_peered":"2024-03-20T07:58:25.767533+0000","last_unstale":"2024-03-20T07:58:26.147717+0000","last_undegraded":"2024-03-20T07:58:26.147717+0000","last_fullsized":"2024-03-20T07:58:26.147717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140470+0000","last_change":"2024-03-20T07:58:26.140127+0000","last_active":"2024-03-20T07:58:26.140470+0000","last_peered":"2024-03-20T07:58:26.140470+0000","last_clean":"2024-03-20T07:58:26.140470+0000","last_became_active":"2024-03-20T07:58:26.139362+0000","last_became_peered":"2024-03-20T07:58:26.139362+0000","last_unstale":"2024-03-20T07:58:26.140470+0000","last_undegraded":"2024-03-20T07:58:26.140470+0000","last_fullsized":"2024-03-20T07:58:26.140470+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239407+0000","last_change":"2024-03-20T07:58:25.771772+0000","last_active":"2024-03-20T07:58:27.239407+0000","last_peered":"2024-03-20T07:58:27.239407+0000","last_clean":"2024-03-20T07:58:27.239407+0000","last_became_active":"2024-03-20T07:58:25.771499+0000","last_became_peered":"2024-03-20T07:58:25.771499+0000","last_unstale":"2024-03-20T07:58:27.239407+0000","last_undegraded":"2024-03-20T07:58:27.239407+0000","last_fullsized":"2024-03-20T07:58:27.239407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:19:45.833311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238700+0000","last_change":"2024-03-20T07:58:26.122549+0000","last_active":"2024-03-20T07:58:27.238700+0000","last_peered":"2024-03-20T07:58:27.238700+0000","last_clean":"2024-03-20T07:58:27.238700+0000","last_became_active":"2024-03-20T07:58:26.121780+0000","last_became_peered":"2024-03-20T07:58:26.121780+0000","last_unstale":"2024-03-20T07:58:27.238700+0000","last_undegraded":"2024-03-20T07:58:27.238700+0000","last_fullsized":"2024-03-20T07:58:27.238700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:37:01.164741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.112184+0000","last_change":"2024-03-20T07:58:25.771839+0000","last_active":"2024-03-20T07:58:26.112184+0000","last_peered":"2024-03-20T07:58:26.112184+0000","last_clean":"2024-03-20T07:58:26.112184+0000","last_became_active":"2024-03-20T07:58:25.771464+0000","last_became_peered":"2024-03-20T07:58:25.771464+0000","last_unstale":"2024-03-20T07:58:26.112184+0000","last_undegraded":"2024-03-20T07:58:26.112184+0000","last_fullsized":"2024-03-20T07:58:26.112184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.129894+0000","last_change":"2024-03-20T07:58:26.129094+0000","last_active":"2024-03-20T07:58:26.129894+0000","last_peered":"2024-03-20T07:58:26.129894+0000","last_clean":"2024-03-20T07:58:26.129894+0000","last_became_active":"2024-03-20T07:58:26.128675+0000","last_became_peered":"2024-03-20T07:58:26.128675+0000","last_unstale":"2024-03-20T07:58:26.129894+0000","last_undegraded":"2024-03-20T07:58:26.129894+0000","last_fullsized":"2024-03-20T07:58:26.129894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.124730+0000","last_change":"2024-03-20T07:58:25.767586+0000","last_active":"2024-03-20T07:58:26.124730+0000","last_peered":"2024-03-20T07:58:26.124730+0000","last_clean":"2024-03-20T07:58:26.124730+0000","last_became_active":"2024-03-20T07:58:25.767268+0000","last_became_peered":"2024-03-20T07:58:25.767268+0000","last_unstale":"2024-03-20T07:58:26.124730+0000","last_undegraded":"2024-03-20T07:58:26.124730+0000","last_fullsized":"2024-03-20T07:58:26.124730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.144101+0000","last_change":"2024-03-20T07:58:26.139925+0000","last_active":"2024-03-20T07:58:26.144101+0000","last_peered":"2024-03-20T07:58:26.144101+0000","last_clean":"2024-03-20T07:58:26.144101+0000","last_became_active":"2024-03-20T07:58:26.138286+0000","last_became_peered":"2024-03-20T07:58:26.138286+0000","last_unstale":"2024-03-20T07:58:26.144101+0000","last_undegraded":"2024-03-20T07:58:26.144101+0000","last_fullsized":"2024-03-20T07:58:26.144101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238203+0000","last_change":"2024-03-20T07:58:25.757217+0000","last_active":"2024-03-20T07:58:27.238203+0000","last_peered":"2024-03-20T07:58:27.238203+0000","last_clean":"2024-03-20T07:58:27.238203+0000","last_became_active":"2024-03-20T07:58:25.756879+0000","last_became_peered":"2024-03-20T07:58:25.756879+0000","last_unstale":"2024-03-20T07:58:27.238203+0000","last_undegraded":"2024-03-20T07:58:27.238203+0000","last_fullsized":"2024-03-20T07:58:27.238203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:28.214822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134394+0000","last_change":"2024-03-20T07:58:26.133140+0000","last_active":"2024-03-20T07:58:26.134394+0000","last_peered":"2024-03-20T07:58:26.134394+0000","last_clean":"2024-03-20T07:58:26.134394+0000","last_became_active":"2024-03-20T07:58:26.132630+0000","last_became_peered":"2024-03-20T07:58:26.132630+0000","last_unstale":"2024-03-20T07:58:26.134394+0000","last_undegraded":"2024-03-20T07:58:26.134394+0000","last_fullsized":"2024-03-20T07:58:26.134394+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.135400+0000","last_change":"2024-03-20T07:58:25.771322+0000","last_active":"2024-03-20T07:58:26.135400+0000","last_peered":"2024-03-20T07:58:26.135400+0000","last_clean":"2024-03-20T07:58:26.135400+0000","last_became_active":"2024-03-20T07:58:25.770984+0000","last_became_peered":"2024-03-20T07:58:25.770984+0000","last_unstale":"2024-03-20T07:58:26.135400+0000","last_undegraded":"2024-03-20T07:58:26.135400+0000","last_fullsized":"2024-03-20T07:58:26.135400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145263+0000","last_change":"2024-03-20T07:58:26.144254+0000","last_active":"2024-03-20T07:58:26.145263+0000","last_peered":"2024-03-20T07:58:26.145263+0000","last_clean":"2024-03-20T07:58:26.145263+0000","last_became_active":"2024-03-20T07:58:26.143683+0000","last_became_peered":"2024-03-20T07:58:26.143683+0000","last_unstale":"2024-03-20T07:58:26.145263+0000","last_undegraded":"2024-03-20T07:58:26.145263+0000","last_fullsized":"2024-03-20T07:58:26.145263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107428+0000","last_change":"2024-03-20T07:58:25.773911+0000","last_active":"2024-03-20T07:58:27.107428+0000","last_peered":"2024-03-20T07:58:27.107428+0000","last_clean":"2024-03-20T07:58:27.107428+0000","last_became_active":"2024-03-20T07:58:25.773526+0000","last_became_peered":"2024-03-20T07:58:25.773526+0000","last_unstale":"2024-03-20T07:58:27.107428+0000","last_undegraded":"2024-03-20T07:58:27.107428+0000","last_fullsized":"2024-03-20T07:58:27.107428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.123504+0000","last_change":"2024-03-20T07:58:26.122909+0000","last_active":"2024-03-20T07:58:26.123504+0000","last_peered":"2024-03-20T07:58:26.123504+0000","last_clean":"2024-03-20T07:58:26.123504+0000","last_became_active":"2024-03-20T07:58:26.121608+0000","last_became_peered":"2024-03-20T07:58:26.121608+0000","last_unstale":"2024-03-20T07:58:26.123504+0000","last_undegraded":"2024-03-20T07:58:26.123504+0000","last_fullsized":"2024-03-20T07:58:26.123504+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111928+0000","last_change":"2024-03-20T07:58:25.770569+0000","last_active":"2024-03-20T07:58:26.111928+0000","last_peered":"2024-03-20T07:58:26.111928+0000","last_clean":"2024-03-20T07:58:26.111928+0000","last_became_active":"2024-03-20T07:58:25.770298+0000","last_became_peered":"2024-03-20T07:58:25.770298+0000","last_unstale":"2024-03-20T07:58:26.111928+0000","last_undegraded":"2024-03-20T07:58:26.111928+0000","last_fullsized":"2024-03-20T07:58:26.111928+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.137205+0000","last_change":"2024-03-20T07:58:26.136200+0000","last_active":"2024-03-20T07:58:26.137205+0000","last_peered":"2024-03-20T07:58:26.137205+0000","last_clean":"2024-03-20T07:58:26.137205+0000","last_became_active":"2024-03-20T07:58:26.135909+0000","last_became_peered":"2024-03-20T07:58:26.135909+0000","last_unstale":"2024-03-20T07:58:26.137205+0000","last_undegraded":"2024-03-20T07:58:26.137205+0000","last_fullsized":"2024-03-20T07:58:26.137205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107104+0000","last_change":"2024-03-20T07:58:26.130079+0000","last_active":"2024-03-20T07:58:27.107104+0000","last_peered":"2024-03-20T07:58:27.107104+0000","last_clean":"2024-03-20T07:58:27.107104+0000","last_became_active":"2024-03-20T07:58:26.129798+0000","last_became_peered":"2024-03-20T07:58:26.129798+0000","last_unstale":"2024-03-20T07:58:27.107104+0000","last_undegraded":"2024-03-20T07:58:27.107104+0000","last_fullsized":"2024-03-20T07:58:27.107104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238173+0000","last_change":"2024-03-20T07:58:25.764736+0000","last_active":"2024-03-20T07:58:27.238173+0000","last_peered":"2024-03-20T07:58:27.238173+0000","last_clean":"2024-03-20T07:58:27.238173+0000","last_became_active":"2024-03-20T07:58:25.764390+0000","last_became_peered":"2024-03-20T07:58:25.764390+0000","last_unstale":"2024-03-20T07:58:27.238173+0000","last_undegraded":"2024-03-20T07:58:27.238173+0000","last_fullsized":"2024-03-20T07:58:27.238173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:27:58.881448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.110613+0000","last_change":"2024-03-20T07:58:25.766643+0000","last_active":"2024-03-20T07:58:26.110613+0000","last_peered":"2024-03-20T07:58:26.110613+0000","last_clean":"2024-03-20T07:58:26.110613+0000","last_became_active":"2024-03-20T07:58:25.766341+0000","last_became_peered":"2024-03-20T07:58:25.766341+0000","last_unstale":"2024-03-20T07:58:26.110613+0000","last_undegraded":"2024-03-20T07:58:26.110613+0000","last_fullsized":"2024-03-20T07:58:26.110613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"22'117","reported_seq":218,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-20T07:58:25.769715+0000","last_change":"2024-03-20T07:58:09.964395+0000","last_active":"2024-03-20T07:58:25.769715+0000","last_peered":"2024-03-20T07:58:25.769715+0000","last_clean":"2024-03-20T07:58:25.769715+0000","last_became_active":"2024-03-20T07:58:09.963927+0000","last_became_peered":"2024-03-20T07:58:09.963927+0000","last_unstale":"2024-03-20T07:58:25.769715+0000","last_undegraded":"2024-03-20T07:58:25.769715+0000","last_fullsized":"2024-03-20T07:58:25.769715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238574+0000","last_change":"2024-03-20T07:58:26.125436+0000","last_active":"2024-03-20T07:58:27.238574+0000","last_peered":"2024-03-20T07:58:27.238574+0000","last_clean":"2024-03-20T07:58:27.238574+0000","last_became_active":"2024-03-20T07:58:26.125161+0000","last_became_peered":"2024-03-20T07:58:26.125161+0000","last_unstale":"2024-03-20T07:58:27.238574+0000","last_undegraded":"2024-03-20T07:58:27.238574+0000","last_fullsized":"2024-03-20T07:58:27.238574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:53:55.085832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.118188+0000","last_change":"2024-03-20T07:58:25.779129+0000","last_active":"2024-03-20T07:58:26.118188+0000","last_peered":"2024-03-20T07:58:26.118188+0000","last_clean":"2024-03-20T07:58:26.118188+0000","last_became_active":"2024-03-20T07:58:25.778258+0000","last_became_peered":"2024-03-20T07:58:25.778258+0000","last_unstale":"2024-03-20T07:58:26.118188+0000","last_undegraded":"2024-03-20T07:58:26.118188+0000","last_fullsized":"2024-03-20T07:58:26.118188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.130223+0000","last_change":"2024-03-20T07:58:25.777973+0000","last_active":"2024-03-20T07:58:26.130223+0000","last_peered":"2024-03-20T07:58:26.130223+0000","last_clean":"2024-03-20T07:58:26.130223+0000","last_became_active":"2024-03-20T07:58:25.777666+0000","last_became_peered":"2024-03-20T07:58:25.777666+0000","last_unstale":"2024-03-20T07:58:26.130223+0000","last_undegraded":"2024-03-20T07:58:26.130223+0000","last_fullsized":"2024-03-20T07:58:26.130223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107508+0000","last_change":"2024-03-20T07:58:26.126692+0000","last_active":"2024-03-20T07:58:27.107508+0000","last_peered":"2024-03-20T07:58:27.107508+0000","last_clean":"2024-03-20T07:58:27.107508+0000","last_became_active":"2024-03-20T07:58:26.126397+0000","last_became_peered":"2024-03-20T07:58:26.126397+0000","last_unstale":"2024-03-20T07:58:27.107508+0000","last_undegraded":"2024-03-20T07:58:27.107508+0000","last_fullsized":"2024-03-20T07:58:27.107508+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108422+0000","last_change":"2024-03-20T07:58:25.769337+0000","last_active":"2024-03-20T07:58:27.108422+0000","last_peered":"2024-03-20T07:58:27.108422+0000","last_clean":"2024-03-20T07:58:27.108422+0000","last_became_active":"2024-03-20T07:58:25.769010+0000","last_became_peered":"2024-03-20T07:58:25.769010+0000","last_unstale":"2024-03-20T07:58:27.108422+0000","last_undegraded":"2024-03-20T07:58:27.108422+0000","last_fullsized":"2024-03-20T07:58:27.108422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.141871+0000","last_change":"2024-03-20T07:58:25.771097+0000","last_active":"2024-03-20T07:58:26.141871+0000","last_peered":"2024-03-20T07:58:26.141871+0000","last_clean":"2024-03-20T07:58:26.141871+0000","last_became_active":"2024-03-20T07:58:25.770737+0000","last_became_peered":"2024-03-20T07:58:25.770737+0000","last_unstale":"2024-03-20T07:58:26.141871+0000","last_undegraded":"2024-03-20T07:58:26.141871+0000","last_fullsized":"2024-03-20T07:58:26.141871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239163+0000","last_change":"2024-03-20T07:58:26.126230+0000","last_active":"2024-03-20T07:58:27.239163+0000","last_peered":"2024-03-20T07:58:27.239163+0000","last_clean":"2024-03-20T07:58:27.239163+0000","last_became_active":"2024-03-20T07:58:26.125980+0000","last_became_peered":"2024-03-20T07:58:26.125980+0000","last_unstale":"2024-03-20T07:58:27.239163+0000","last_undegraded":"2024-03-20T07:58:27.239163+0000","last_fullsized":"2024-03-20T07:58:27.239163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:18:03.971589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.117985+0000","last_change":"2024-03-20T07:58:25.777961+0000","last_active":"2024-03-20T07:58:26.117985+0000","last_peered":"2024-03-20T07:58:26.117985+0000","last_clean":"2024-03-20T07:58:26.117985+0000","last_became_active":"2024-03-20T07:58:25.777649+0000","last_became_peered":"2024-03-20T07:58:25.777649+0000","last_unstale":"2024-03-20T07:58:26.117985+0000","last_undegraded":"2024-03-20T07:58:26.117985+0000","last_fullsized":"2024-03-20T07:58:26.117985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.124307+0000","last_change":"2024-03-20T07:58:26.120305+0000","last_active":"2024-03-20T07:58:26.124307+0000","last_peered":"2024-03-20T07:58:26.124307+0000","last_clean":"2024-03-20T07:58:26.124307+0000","last_became_active":"2024-03-20T07:58:26.119966+0000","last_became_peered":"2024-03-20T07:58:26.119966+0000","last_unstale":"2024-03-20T07:58:26.124307+0000","last_undegraded":"2024-03-20T07:58:26.124307+0000","last_fullsized":"2024-03-20T07:58:26.124307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107099+0000","last_change":"2024-03-20T07:58:25.765548+0000","last_active":"2024-03-20T07:58:27.107099+0000","last_peered":"2024-03-20T07:58:27.107099+0000","last_clean":"2024-03-20T07:58:27.107099+0000","last_became_active":"2024-03-20T07:58:25.765273+0000","last_became_peered":"2024-03-20T07:58:25.765273+0000","last_unstale":"2024-03-20T07:58:27.107099+0000","last_undegraded":"2024-03-20T07:58:27.107099+0000","last_fullsized":"2024-03-20T07:58:27.107099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107547+0000","last_change":"2024-03-20T07:58:25.773811+0000","last_active":"2024-03-20T07:58:27.107547+0000","last_peered":"2024-03-20T07:58:27.107547+0000","last_clean":"2024-03-20T07:58:27.107547+0000","last_became_active":"2024-03-20T07:58:25.773520+0000","last_became_peered":"2024-03-20T07:58:25.773520+0000","last_unstale":"2024-03-20T07:58:27.107547+0000","last_undegraded":"2024-03-20T07:58:27.107547+0000","last_fullsized":"2024-03-20T07:58:27.107547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138933+0000","last_change":"2024-03-20T07:58:25.776066+0000","last_active":"2024-03-20T07:58:26.138933+0000","last_peered":"2024-03-20T07:58:26.138933+0000","last_clean":"2024-03-20T07:58:26.138933+0000","last_became_active":"2024-03-20T07:58:25.775708+0000","last_became_peered":"2024-03-20T07:58:25.775708+0000","last_unstale":"2024-03-20T07:58:26.138933+0000","last_undegraded":"2024-03-20T07:58:26.138933+0000","last_fullsized":"2024-03-20T07:58:26.138933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:36:20.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134688+0000","last_change":"2024-03-20T07:58:26.134067+0000","last_active":"2024-03-20T07:58:26.134688+0000","last_peered":"2024-03-20T07:58:26.134688+0000","last_clean":"2024-03-20T07:58:26.134688+0000","last_became_active":"2024-03-20T07:58:26.133784+0000","last_became_peered":"2024-03-20T07:58:26.133784+0000","last_unstale":"2024-03-20T07:58:26.134688+0000","last_undegraded":"2024-03-20T07:58:26.134688+0000","last_fullsized":"2024-03-20T07:58:26.134688+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238423+0000","last_change":"2024-03-20T07:58:25.763015+0000","last_active":"2024-03-20T07:58:27.238423+0000","last_peered":"2024-03-20T07:58:27.238423+0000","last_clean":"2024-03-20T07:58:27.238423+0000","last_became_active":"2024-03-20T07:58:25.762682+0000","last_became_peered":"2024-03-20T07:58:25.762682+0000","last_unstale":"2024-03-20T07:58:27.238423+0000","last_undegraded":"2024-03-20T07:58:27.238423+0000","last_fullsized":"2024-03-20T07:58:27.238423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:21:45.137669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.142797+0000","last_change":"2024-03-20T07:58:26.142445+0000","last_active":"2024-03-20T07:58:26.142797+0000","last_peered":"2024-03-20T07:58:26.142797+0000","last_clean":"2024-03-20T07:58:26.142797+0000","last_became_active":"2024-03-20T07:58:26.142148+0000","last_became_peered":"2024-03-20T07:58:26.142148+0000","last_unstale":"2024-03-20T07:58:26.142797+0000","last_undegraded":"2024-03-20T07:58:26.142797+0000","last_fullsized":"2024-03-20T07:58:26.142797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"62'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107988+0000","last_change":"2024-03-20T07:58:25.766427+0000","last_active":"2024-03-20T07:58:27.107988+0000","last_peered":"2024-03-20T07:58:27.107988+0000","last_clean":"2024-03-20T07:58:27.107988+0000","last_became_active":"2024-03-20T07:58:25.766111+0000","last_became_peered":"2024-03-20T07:58:25.766111+0000","last_unstale":"2024-03-20T07:58:27.107988+0000","last_undegraded":"2024-03-20T07:58:27.107988+0000","last_fullsized":"2024-03-20T07:58:27.107988+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.136811+0000","last_change":"2024-03-20T07:58:26.135339+0000","last_active":"2024-03-20T07:58:26.136811+0000","last_peered":"2024-03-20T07:58:26.136811+0000","last_clean":"2024-03-20T07:58:26.136811+0000","last_became_active":"2024-03-20T07:58:26.134903+0000","last_became_peered":"2024-03-20T07:58:26.134903+0000","last_unstale":"2024-03-20T07:58:26.136811+0000","last_undegraded":"2024-03-20T07:58:26.136811+0000","last_fullsized":"2024-03-20T07:58:26.136811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"62'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238866+0000","last_change":"2024-03-20T07:58:25.766233+0000","last_active":"2024-03-20T07:58:27.238866+0000","last_peered":"2024-03-20T07:58:27.238866+0000","last_clean":"2024-03-20T07:58:27.238866+0000","last_became_active":"2024-03-20T07:58:25.765948+0000","last_became_peered":"2024-03-20T07:58:25.765948+0000","last_unstale":"2024-03-20T07:58:27.238866+0000","last_undegraded":"2024-03-20T07:58:27.238866+0000","last_fullsized":"2024-03-20T07:58:27.238866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:34:01.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145073+0000","last_change":"2024-03-20T07:58:26.144008+0000","last_active":"2024-03-20T07:58:26.145073+0000","last_peered":"2024-03-20T07:58:26.145073+0000","last_clean":"2024-03-20T07:58:26.145073+0000","last_became_active":"2024-03-20T07:58:26.143696+0000","last_became_peered":"2024-03-20T07:58:26.143696+0000","last_unstale":"2024-03-20T07:58:26.145073+0000","last_undegraded":"2024-03-20T07:58:26.145073+0000","last_fullsized":"2024-03-20T07:58:26.145073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140012+0000","last_change":"2024-03-20T07:58:26.138096+0000","last_active":"2024-03-20T07:58:26.140012+0000","last_peered":"2024-03-20T07:58:26.140012+0000","last_clean":"2024-03-20T07:58:26.140012+0000","last_became_active":"2024-03-20T07:58:26.137804+0000","last_became_peered":"2024-03-20T07:58:26.137804+0000","last_unstale":"2024-03-20T07:58:26.140012+0000","last_undegraded":"2024-03-20T07:58:26.140012+0000","last_fullsized":"2024-03-20T07:58:26.140012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238668+0000","last_change":"2024-03-20T07:58:26.126085+0000","last_active":"2024-03-20T07:58:27.238668+0000","last_peered":"2024-03-20T07:58:27.238668+0000","last_clean":"2024-03-20T07:58:27.238668+0000","last_became_active":"2024-03-20T07:58:26.125793+0000","last_became_peered":"2024-03-20T07:58:26.125793+0000","last_unstale":"2024-03-20T07:58:27.238668+0000","last_undegraded":"2024-03-20T07:58:27.238668+0000","last_fullsized":"2024-03-20T07:58:27.238668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:11:45.470755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.126540+0000","last_change":"2024-03-20T07:58:26.125004+0000","last_active":"2024-03-20T07:58:26.126540+0000","last_peered":"2024-03-20T07:58:26.126540+0000","last_clean":"2024-03-20T07:58:26.126540+0000","last_became_active":"2024-03-20T07:58:26.124689+0000","last_became_peered":"2024-03-20T07:58:26.124689+0000","last_unstale":"2024-03-20T07:58:26.126540+0000","last_undegraded":"2024-03-20T07:58:26.126540+0000","last_fullsized":"2024-03-20T07:58:26.126540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239208+0000","last_change":"2024-03-20T07:58:26.120048+0000","last_active":"2024-03-20T07:58:27.239208+0000","last_peered":"2024-03-20T07:58:27.239208+0000","last_clean":"2024-03-20T07:58:27.239208+0000","last_became_active":"2024-03-20T07:58:26.119755+0000","last_became_peered":"2024-03-20T07:58:26.119755+0000","last_unstale":"2024-03-20T07:58:27.239208+0000","last_undegraded":"2024-03-20T07:58:27.239208+0000","last_fullsized":"2024-03-20T07:58:27.239208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:39:07.689571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107012+0000","last_change":"2024-03-20T07:58:26.126113+0000","last_active":"2024-03-20T07:58:27.107012+0000","last_peered":"2024-03-20T07:58:27.107012+0000","last_clean":"2024-03-20T07:58:27.107012+0000","last_became_active":"2024-03-20T07:58:26.122635+0000","last_became_peered":"2024-03-20T07:58:26.122635+0000","last_unstale":"2024-03-20T07:58:27.107012+0000","last_undegraded":"2024-03-20T07:58:27.107012+0000","last_fullsized":"2024-03-20T07:58:27.107012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.144469+0000","last_change":"2024-03-20T07:58:26.141379+0000","last_active":"2024-03-20T07:58:26.144469+0000","last_peered":"2024-03-20T07:58:26.144469+0000","last_clean":"2024-03-20T07:58:26.144469+0000","last_became_active":"2024-03-20T07:58:26.141099+0000","last_became_peered":"2024-03-20T07:58:26.141099+0000","last_unstale":"2024-03-20T07:58:26.144469+0000","last_undegraded":"2024-03-20T07:58:26.144469+0000","last_fullsized":"2024-03-20T07:58:26.144469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138838+0000","last_change":"2024-03-20T07:58:26.137638+0000","last_active":"2024-03-20T07:58:26.138838+0000","last_peered":"2024-03-20T07:58:26.138838+0000","last_clean":"2024-03-20T07:58:26.138838+0000","last_became_active":"2024-03-20T07:58:26.137344+0000","last_became_peered":"2024-03-20T07:58:26.137344+0000","last_unstale":"2024-03-20T07:58:26.138838+0000","last_undegraded":"2024-03-20T07:58:26.138838+0000","last_fullsized":"2024-03-20T07:58:26.138838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138511+0000","last_change":"2024-03-20T07:58:26.137117+0000","last_active":"2024-03-20T07:58:26.138511+0000","last_peered":"2024-03-20T07:58:26.138511+0000","last_clean":"2024-03-20T07:58:26.138511+0000","last_became_active":"2024-03-20T07:58:26.136135+0000","last_became_peered":"2024-03-20T07:58:26.136135+0000","last_unstale":"2024-03-20T07:58:26.138511+0000","last_undegraded":"2024-03-20T07:58:26.138511+0000","last_fullsized":"2024-03-20T07:58:26.138511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239107+0000","last_change":"2024-03-20T07:58:26.119480+0000","last_active":"2024-03-20T07:58:27.239107+0000","last_peered":"2024-03-20T07:58:27.239107+0000","last_clean":"2024-03-20T07:58:27.239107+0000","last_became_active":"2024-03-20T07:58:26.119047+0000","last_became_peered":"2024-03-20T07:58:26.119047+0000","last_unstale":"2024-03-20T07:58:27.239107+0000","last_undegraded":"2024-03-20T07:58:27.239107+0000","last_fullsized":"2024-03-20T07:58:27.239107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:09:31.918661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134663+0000","last_change":"2024-03-20T07:58:26.133160+0000","last_active":"2024-03-20T07:58:26.134663+0000","last_peered":"2024-03-20T07:58:26.134663+0000","last_clean":"2024-03-20T07:58:26.134663+0000","last_became_active":"2024-03-20T07:58:26.132793+0000","last_became_peered":"2024-03-20T07:58:26.132793+0000","last_unstale":"2024-03-20T07:58:26.134663+0000","last_undegraded":"2024-03-20T07:58:26.134663+0000","last_fullsized":"2024-03-20T07:58:26.134663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":159,"acting":159,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":87,"acting":87,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201286,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":838705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462923,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1183744,"data_stored":831669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717512,"statfs":{"total":95995035648,"available":95966732288,"internally_reserved":0,"allocated":626688,"data_stored":255702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.33100000000000002,"15min":0.32300000000000001},"min":{"1min":0.26200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.35599999999999998,"15min":0.34399999999999997},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.313}]},{"osd":2,"last update":"Wed Mar 20 07:58:07 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.39300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.25},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.308,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.27800000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.37}]},{"osd":4,"last update":"Wed Mar 20 07:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738410,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1228800,"data_stored":846638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 20 07:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.26300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.32200000000000001,"15min":0.312},"min":{"1min":0.26300000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Mar 20 07:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33000000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.318},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.374}]},{"osd":4,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.34100000000000003}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.39400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411359,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717520,"statfs":{"total":95995035648,"available":95966740480,"internally_reserved":0,"allocated":618496,"data_stored":255542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.36799999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35699999999999998}]},{"osd":1,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.33900000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38100000000000001},"min":{"1min":0.33000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52000000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.311},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Mar 20 07:58:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149721,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717556,"statfs":{"total":95995035648,"available":95966777344,"internally_reserved":0,"allocated":581632,"data_stored":235385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.36499999999999999}]},{"osd":1,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.626},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Mar 20 07:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986197,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":634880,"data_stored":256250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.374}]},{"osd":1,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.503}]},{"osd":2,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.432}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.35699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717528,"statfs":{"total":95995035648,"available":95966748672,"internally_reserved":0,"allocated":618496,"data_stored":255542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44}]},{"osd":1,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.495}]},{"osd":3,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.38400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T07:58:29.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-03-20T07:58:29.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.754 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-03-20T07:58:29.754 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-03-20T07:58:29.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-03-20T07:58:29.755 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:24.742760+0000', 'last_change': '2024-03-20T07:58:24.742760+0000', 'last_active': '2024-03-20T07:58:24.742760+0000', 'last_peered': '2024-03-20T07:58:24.742760+0000', 'last_clean': '2024-03-20T07:58:24.742760+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:24.742760+0000', 'last_undegraded': '2024-03-20T07:58:24.742760+0000', 'last_fullsized': '2024-03-20T07:58:24.742760+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.755 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-03-20T07:58:29.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:24.742760+0000', 'last_change': '2024-03-20T07:58:24.742760+0000', 'last_active': '2024-03-20T07:58:24.742760+0000', 'last_peered': '2024-03-20T07:58:24.742760+0000', 'last_clean': '2024-03-20T07:58:24.742760+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:24.742760+0000', 'last_undegraded': '2024-03-20T07:58:24.742760+0000', 'last_fullsized': '2024-03-20T07:58:24.742760+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-03-20T07:58:29.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:24.742760+0000', 'last_change': '2024-03-20T07:58:24.742760+0000', 'last_active': '2024-03-20T07:58:24.742760+0000', 'last_peered': '2024-03-20T07:58:24.742760+0000', 'last_clean': '2024-03-20T07:58:24.742760+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:24.742760+0000', 'last_undegraded': '2024-03-20T07:58:24.742760+0000', 'last_fullsized': '2024-03-20T07:58:24.742760+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:24.742760+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.756 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-03-20T07:58:29.756 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-03-20T07:58:29.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-03-20T07:58:29.757 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.757 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-03-20T07:58:29.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-03-20T07:58:29.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.758 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-03-20T07:58:29.758 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-03-20T07:58:29.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.759 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-03-20T07:58:29.759 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-20T07:58:25.744971+0000', 'last_change': '2024-03-20T07:58:25.744971+0000', 'last_active': '2024-03-20T07:58:25.744971+0000', 'last_peered': '2024-03-20T07:58:25.744971+0000', 'last_clean': '2024-03-20T07:58:25.744971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-20T07:58:25.744971+0000', 'last_undegraded': '2024-03-20T07:58:25.744971+0000', 'last_fullsized': '2024-03-20T07:58:25.744971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'last_clean_scrub_stamp': '2024-03-20T07:58:25.744971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-20T07:58:29.760 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-20T07:58:30.130 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:30.131 INFO:teuthology.orchestra.run.smithi118.stdout:{"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":235,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":63,"num_osds":8,"num_up_osds":8,"osd_up_since":1710921485,"num_in_osds":8,"osd_in_since":1710921469,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":83},{"state_name":"unknown","count":14}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":228077568,"bytes_avail":767732207616,"bytes_total":767960285184,"unknown_pgs_ratio":0.14432989060878754,"write_bytes_sec":3191,"read_op_per_sec":0,"write_op_per_sec":9},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24343}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.118:8443/"}},"servicemap":{"epoch":11,"modified":"2024-03-20T07:58:17.234735+0000","services":{"osd":{"daemons":{"summary":"","6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-20T07:58:30.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:29 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/801042279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:30.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:29 smithi196 ceph-mon[31033]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:29 smithi196 ceph-mon[31033]: pgmap v195: 97 pgs: 83 active+clean, 14 unknown; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s wr, 9 op/s 2024-03-20T07:58:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:29 smithi196 ceph-mon[31033]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:30.142 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:29 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/801042279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:30.142 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:29 smithi118 ceph-mon[27391]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:30.142 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:29 smithi118 ceph-mon[27391]: pgmap v195: 97 pgs: 83 active+clean, 14 unknown; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s wr, 9 op/s 2024-03-20T07:58:30.142 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:29 smithi118 ceph-mon[27391]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:30.142 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:29 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/801042279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:30.142 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:29 smithi118 ceph-mon[38981]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:30.142 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:29 smithi118 ceph-mon[38981]: pgmap v195: 97 pgs: 83 active+clean, 14 unknown; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s wr, 9 op/s 2024-03-20T07:58:30.143 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:29 smithi118 ceph-mon[38981]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:30.143 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-20T07:58:30.144 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-20T07:58:30.432 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:30.434 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T07:58:30.443 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-20T07:58:29.236703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":249,"acting":249,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":749961216,"kb_used":222732,"kb_used_data":6536,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749738484,"statfs":{"total":767960285184,"available":767732207616,"internally_reserved":0,"allocated":6692864,"data_stored":3775433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.491697"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.146486+0000","last_change":"2024-03-20T07:58:26.145662+0000","last_active":"2024-03-20T07:58:26.146486+0000","last_peered":"2024-03-20T07:58:26.146486+0000","last_clean":"2024-03-20T07:58:26.146486+0000","last_became_active":"2024-03-20T07:58:26.145286+0000","last_became_peered":"2024-03-20T07:58:26.145286+0000","last_unstale":"2024-03-20T07:58:26.146486+0000","last_undegraded":"2024-03-20T07:58:26.146486+0000","last_fullsized":"2024-03-20T07:58:26.146486+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108335+0000","last_change":"2024-03-20T07:58:26.129069+0000","last_active":"2024-03-20T07:58:27.108335+0000","last_peered":"2024-03-20T07:58:27.108335+0000","last_clean":"2024-03-20T07:58:27.108335+0000","last_became_active":"2024-03-20T07:58:26.128811+0000","last_became_peered":"2024-03-20T07:58:26.128811+0000","last_unstale":"2024-03-20T07:58:27.108335+0000","last_undegraded":"2024-03-20T07:58:27.108335+0000","last_fullsized":"2024-03-20T07:58:27.108335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239393+0000","last_change":"2024-03-20T07:58:26.119030+0000","last_active":"2024-03-20T07:58:27.239393+0000","last_peered":"2024-03-20T07:58:27.239393+0000","last_clean":"2024-03-20T07:58:27.239393+0000","last_became_active":"2024-03-20T07:58:26.118755+0000","last_became_peered":"2024-03-20T07:58:26.118755+0000","last_unstale":"2024-03-20T07:58:27.239393+0000","last_undegraded":"2024-03-20T07:58:27.239393+0000","last_fullsized":"2024-03-20T07:58:27.239393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:51:41.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139286+0000","last_change":"2024-03-20T07:58:26.138067+0000","last_active":"2024-03-20T07:58:26.139286+0000","last_peered":"2024-03-20T07:58:26.139286+0000","last_clean":"2024-03-20T07:58:26.139286+0000","last_became_active":"2024-03-20T07:58:26.137768+0000","last_became_peered":"2024-03-20T07:58:26.137768+0000","last_unstale":"2024-03-20T07:58:26.139286+0000","last_undegraded":"2024-03-20T07:58:26.139286+0000","last_fullsized":"2024-03-20T07:58:26.139286+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134630+0000","last_change":"2024-03-20T07:58:26.131923+0000","last_active":"2024-03-20T07:58:26.134630+0000","last_peered":"2024-03-20T07:58:26.134630+0000","last_clean":"2024-03-20T07:58:26.134630+0000","last_became_active":"2024-03-20T07:58:26.131585+0000","last_became_peered":"2024-03-20T07:58:26.131585+0000","last_unstale":"2024-03-20T07:58:26.134630+0000","last_undegraded":"2024-03-20T07:58:26.134630+0000","last_fullsized":"2024-03-20T07:58:26.134630+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139194+0000","last_change":"2024-03-20T07:58:26.137308+0000","last_active":"2024-03-20T07:58:26.139194+0000","last_peered":"2024-03-20T07:58:26.139194+0000","last_clean":"2024-03-20T07:58:26.139194+0000","last_became_active":"2024-03-20T07:58:26.135727+0000","last_became_peered":"2024-03-20T07:58:26.135727+0000","last_unstale":"2024-03-20T07:58:26.139194+0000","last_undegraded":"2024-03-20T07:58:26.139194+0000","last_fullsized":"2024-03-20T07:58:26.139194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134374+0000","last_change":"2024-03-20T07:58:26.132941+0000","last_active":"2024-03-20T07:58:26.134374+0000","last_peered":"2024-03-20T07:58:26.134374+0000","last_clean":"2024-03-20T07:58:26.134374+0000","last_became_active":"2024-03-20T07:58:26.132230+0000","last_became_peered":"2024-03-20T07:58:26.132230+0000","last_unstale":"2024-03-20T07:58:26.134374+0000","last_undegraded":"2024-03-20T07:58:26.134374+0000","last_fullsized":"2024-03-20T07:58:26.134374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145322+0000","last_change":"2024-03-20T07:58:26.144631+0000","last_active":"2024-03-20T07:58:26.145322+0000","last_peered":"2024-03-20T07:58:26.145322+0000","last_clean":"2024-03-20T07:58:26.145322+0000","last_became_active":"2024-03-20T07:58:26.144302+0000","last_became_peered":"2024-03-20T07:58:26.144302+0000","last_unstale":"2024-03-20T07:58:26.145322+0000","last_undegraded":"2024-03-20T07:58:26.145322+0000","last_fullsized":"2024-03-20T07:58:26.145322+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.139423+0000","last_change":"2024-03-20T07:58:26.138657+0000","last_active":"2024-03-20T07:58:26.139423+0000","last_peered":"2024-03-20T07:58:26.139423+0000","last_clean":"2024-03-20T07:58:26.139423+0000","last_became_active":"2024-03-20T07:58:26.138348+0000","last_became_peered":"2024-03-20T07:58:26.138348+0000","last_unstale":"2024-03-20T07:58:26.139423+0000","last_undegraded":"2024-03-20T07:58:26.139423+0000","last_fullsized":"2024-03-20T07:58:26.139423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138763+0000","last_change":"2024-03-20T07:58:26.137961+0000","last_active":"2024-03-20T07:58:26.138763+0000","last_peered":"2024-03-20T07:58:26.138763+0000","last_clean":"2024-03-20T07:58:26.138763+0000","last_became_active":"2024-03-20T07:58:26.137672+0000","last_became_peered":"2024-03-20T07:58:26.137672+0000","last_unstale":"2024-03-20T07:58:26.138763+0000","last_undegraded":"2024-03-20T07:58:26.138763+0000","last_fullsized":"2024-03-20T07:58:26.138763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.137140+0000","last_change":"2024-03-20T07:58:26.136246+0000","last_active":"2024-03-20T07:58:26.137140+0000","last_peered":"2024-03-20T07:58:26.137140+0000","last_clean":"2024-03-20T07:58:26.137140+0000","last_became_active":"2024-03-20T07:58:26.135935+0000","last_became_peered":"2024-03-20T07:58:26.135935+0000","last_unstale":"2024-03-20T07:58:26.137140+0000","last_undegraded":"2024-03-20T07:58:26.137140+0000","last_fullsized":"2024-03-20T07:58:26.137140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238796+0000","last_change":"2024-03-20T07:58:26.123008+0000","last_active":"2024-03-20T07:58:27.238796+0000","last_peered":"2024-03-20T07:58:27.238796+0000","last_clean":"2024-03-20T07:58:27.238796+0000","last_became_active":"2024-03-20T07:58:26.122270+0000","last_became_peered":"2024-03-20T07:58:26.122270+0000","last_unstale":"2024-03-20T07:58:27.238796+0000","last_undegraded":"2024-03-20T07:58:27.238796+0000","last_fullsized":"2024-03-20T07:58:27.238796+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:37.385767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107015+0000","last_change":"2024-03-20T07:58:26.127042+0000","last_active":"2024-03-20T07:58:27.107015+0000","last_peered":"2024-03-20T07:58:27.107015+0000","last_clean":"2024-03-20T07:58:27.107015+0000","last_became_active":"2024-03-20T07:58:26.126555+0000","last_became_peered":"2024-03-20T07:58:26.126555+0000","last_unstale":"2024-03-20T07:58:27.107015+0000","last_undegraded":"2024-03-20T07:58:27.107015+0000","last_fullsized":"2024-03-20T07:58:27.107015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.132048+0000","last_change":"2024-03-20T07:58:26.129086+0000","last_active":"2024-03-20T07:58:26.132048+0000","last_peered":"2024-03-20T07:58:26.132048+0000","last_clean":"2024-03-20T07:58:26.132048+0000","last_became_active":"2024-03-20T07:58:26.128111+0000","last_became_peered":"2024-03-20T07:58:26.128111+0000","last_unstale":"2024-03-20T07:58:26.132048+0000","last_undegraded":"2024-03-20T07:58:26.132048+0000","last_fullsized":"2024-03-20T07:58:26.132048+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.131910+0000","last_change":"2024-03-20T07:58:26.128812+0000","last_active":"2024-03-20T07:58:26.131910+0000","last_peered":"2024-03-20T07:58:26.131910+0000","last_clean":"2024-03-20T07:58:26.131910+0000","last_became_active":"2024-03-20T07:58:26.128501+0000","last_became_peered":"2024-03-20T07:58:26.128501+0000","last_unstale":"2024-03-20T07:58:26.131910+0000","last_undegraded":"2024-03-20T07:58:26.131910+0000","last_fullsized":"2024-03-20T07:58:26.131910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238912+0000","last_change":"2024-03-20T07:58:26.128211+0000","last_active":"2024-03-20T07:58:27.238912+0000","last_peered":"2024-03-20T07:58:27.238912+0000","last_clean":"2024-03-20T07:58:27.238912+0000","last_became_active":"2024-03-20T07:58:26.127943+0000","last_became_peered":"2024-03-20T07:58:26.127943+0000","last_unstale":"2024-03-20T07:58:27.238912+0000","last_undegraded":"2024-03-20T07:58:27.238912+0000","last_fullsized":"2024-03-20T07:58:27.238912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:19.979568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134420+0000","last_change":"2024-03-20T07:58:26.133112+0000","last_active":"2024-03-20T07:58:26.134420+0000","last_peered":"2024-03-20T07:58:26.134420+0000","last_clean":"2024-03-20T07:58:26.134420+0000","last_became_active":"2024-03-20T07:58:26.132794+0000","last_became_peered":"2024-03-20T07:58:26.132794+0000","last_unstale":"2024-03-20T07:58:26.134420+0000","last_undegraded":"2024-03-20T07:58:26.134420+0000","last_fullsized":"2024-03-20T07:58:26.134420+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"62'4","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239419+0000","last_change":"2024-03-20T07:58:25.774846+0000","last_active":"2024-03-20T07:58:27.239419+0000","last_peered":"2024-03-20T07:58:27.239419+0000","last_clean":"2024-03-20T07:58:27.239419+0000","last_became_active":"2024-03-20T07:58:25.774570+0000","last_became_peered":"2024-03-20T07:58:25.774570+0000","last_unstale":"2024-03-20T07:58:27.239419+0000","last_undegraded":"2024-03-20T07:58:27.239419+0000","last_fullsized":"2024-03-20T07:58:27.239419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:00.513690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239316+0000","last_change":"2024-03-20T07:58:26.118404+0000","last_active":"2024-03-20T07:58:27.239316+0000","last_peered":"2024-03-20T07:58:27.239316+0000","last_clean":"2024-03-20T07:58:27.239316+0000","last_became_active":"2024-03-20T07:58:26.118107+0000","last_became_peered":"2024-03-20T07:58:26.118107+0000","last_unstale":"2024-03-20T07:58:27.239316+0000","last_undegraded":"2024-03-20T07:58:27.239316+0000","last_fullsized":"2024-03-20T07:58:27.239316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:51.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145432+0000","last_change":"2024-03-20T07:58:25.778043+0000","last_active":"2024-03-20T07:58:26.145432+0000","last_peered":"2024-03-20T07:58:26.145432+0000","last_clean":"2024-03-20T07:58:26.145432+0000","last_became_active":"2024-03-20T07:58:25.777710+0000","last_became_peered":"2024-03-20T07:58:25.777710+0000","last_unstale":"2024-03-20T07:58:26.145432+0000","last_undegraded":"2024-03-20T07:58:26.145432+0000","last_fullsized":"2024-03-20T07:58:26.145432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.123506+0000","last_change":"2024-03-20T07:58:26.122440+0000","last_active":"2024-03-20T07:58:26.123506+0000","last_peered":"2024-03-20T07:58:26.123506+0000","last_clean":"2024-03-20T07:58:26.123506+0000","last_became_active":"2024-03-20T07:58:26.122136+0000","last_became_peered":"2024-03-20T07:58:26.122136+0000","last_unstale":"2024-03-20T07:58:26.123506+0000","last_undegraded":"2024-03-20T07:58:26.123506+0000","last_fullsized":"2024-03-20T07:58:26.123506+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138961+0000","last_change":"2024-03-20T07:58:25.776107+0000","last_active":"2024-03-20T07:58:26.138961+0000","last_peered":"2024-03-20T07:58:26.138961+0000","last_clean":"2024-03-20T07:58:26.138961+0000","last_became_active":"2024-03-20T07:58:25.775756+0000","last_became_peered":"2024-03-20T07:58:25.775756+0000","last_unstale":"2024-03-20T07:58:26.138961+0000","last_undegraded":"2024-03-20T07:58:26.138961+0000","last_fullsized":"2024-03-20T07:58:26.138961+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138778+0000","last_change":"2024-03-20T07:58:26.137236+0000","last_active":"2024-03-20T07:58:26.138778+0000","last_peered":"2024-03-20T07:58:26.138778+0000","last_clean":"2024-03-20T07:58:26.138778+0000","last_became_active":"2024-03-20T07:58:26.136660+0000","last_became_peered":"2024-03-20T07:58:26.136660+0000","last_unstale":"2024-03-20T07:58:26.138778+0000","last_undegraded":"2024-03-20T07:58:26.138778+0000","last_fullsized":"2024-03-20T07:58:26.138778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:36.426353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.119043+0000","last_change":"2024-03-20T07:58:26.118793+0000","last_active":"2024-03-20T07:58:26.119043+0000","last_peered":"2024-03-20T07:58:26.119043+0000","last_clean":"2024-03-20T07:58:26.119043+0000","last_became_active":"2024-03-20T07:58:26.118430+0000","last_became_peered":"2024-03-20T07:58:26.118430+0000","last_unstale":"2024-03-20T07:58:26.119043+0000","last_undegraded":"2024-03-20T07:58:26.119043+0000","last_fullsized":"2024-03-20T07:58:26.119043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140552+0000","last_change":"2024-03-20T07:58:25.765918+0000","last_active":"2024-03-20T07:58:26.140552+0000","last_peered":"2024-03-20T07:58:26.140552+0000","last_clean":"2024-03-20T07:58:26.140552+0000","last_became_active":"2024-03-20T07:58:25.765598+0000","last_became_peered":"2024-03-20T07:58:25.765598+0000","last_unstale":"2024-03-20T07:58:26.140552+0000","last_undegraded":"2024-03-20T07:58:26.140552+0000","last_fullsized":"2024-03-20T07:58:26.140552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239315+0000","last_change":"2024-03-20T07:58:26.119313+0000","last_active":"2024-03-20T07:58:27.239315+0000","last_peered":"2024-03-20T07:58:27.239315+0000","last_clean":"2024-03-20T07:58:27.239315+0000","last_became_active":"2024-03-20T07:58:26.118645+0000","last_became_peered":"2024-03-20T07:58:26.118645+0000","last_unstale":"2024-03-20T07:58:27.239315+0000","last_undegraded":"2024-03-20T07:58:27.239315+0000","last_fullsized":"2024-03-20T07:58:27.239315+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:13.410704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:24.742760+0000","last_change":"2024-03-20T07:58:24.742760+0000","last_active":"2024-03-20T07:58:24.742760+0000","last_peered":"2024-03-20T07:58:24.742760+0000","last_clean":"2024-03-20T07:58:24.742760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:24.742760+0000","last_undegraded":"2024-03-20T07:58:24.742760+0000","last_fullsized":"2024-03-20T07:58:24.742760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111175+0000","last_change":"2024-03-20T07:58:25.769007+0000","last_active":"2024-03-20T07:58:26.111175+0000","last_peered":"2024-03-20T07:58:26.111175+0000","last_clean":"2024-03-20T07:58:26.111175+0000","last_became_active":"2024-03-20T07:58:25.768716+0000","last_became_peered":"2024-03-20T07:58:25.768716+0000","last_unstale":"2024-03-20T07:58:26.111175+0000","last_undegraded":"2024-03-20T07:58:26.111175+0000","last_fullsized":"2024-03-20T07:58:26.111175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107247+0000","last_change":"2024-03-20T07:58:26.119518+0000","last_active":"2024-03-20T07:58:27.107247+0000","last_peered":"2024-03-20T07:58:27.107247+0000","last_clean":"2024-03-20T07:58:27.107247+0000","last_became_active":"2024-03-20T07:58:26.119184+0000","last_became_peered":"2024-03-20T07:58:26.119184+0000","last_unstale":"2024-03-20T07:58:27.107247+0000","last_undegraded":"2024-03-20T07:58:27.107247+0000","last_fullsized":"2024-03-20T07:58:27.107247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.128555+0000","last_change":"2024-03-20T07:58:25.769300+0000","last_active":"2024-03-20T07:58:26.128555+0000","last_peered":"2024-03-20T07:58:26.128555+0000","last_clean":"2024-03-20T07:58:26.128555+0000","last_became_active":"2024-03-20T07:58:25.768916+0000","last_became_peered":"2024-03-20T07:58:25.768916+0000","last_unstale":"2024-03-20T07:58:26.128555+0000","last_undegraded":"2024-03-20T07:58:26.128555+0000","last_fullsized":"2024-03-20T07:58:26.128555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.133696+0000","last_change":"2024-03-20T07:58:26.130616+0000","last_active":"2024-03-20T07:58:26.133696+0000","last_peered":"2024-03-20T07:58:26.133696+0000","last_clean":"2024-03-20T07:58:26.133696+0000","last_became_active":"2024-03-20T07:58:26.130332+0000","last_became_peered":"2024-03-20T07:58:26.130332+0000","last_unstale":"2024-03-20T07:58:26.133696+0000","last_undegraded":"2024-03-20T07:58:26.133696+0000","last_fullsized":"2024-03-20T07:58:26.133696+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111307+0000","last_change":"2024-03-20T07:58:25.769790+0000","last_active":"2024-03-20T07:58:26.111307+0000","last_peered":"2024-03-20T07:58:26.111307+0000","last_clean":"2024-03-20T07:58:26.111307+0000","last_became_active":"2024-03-20T07:58:25.769492+0000","last_became_peered":"2024-03-20T07:58:25.769492+0000","last_unstale":"2024-03-20T07:58:26.111307+0000","last_undegraded":"2024-03-20T07:58:26.111307+0000","last_fullsized":"2024-03-20T07:58:26.111307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239245+0000","last_change":"2024-03-20T07:58:26.116344+0000","last_active":"2024-03-20T07:58:27.239245+0000","last_peered":"2024-03-20T07:58:27.239245+0000","last_clean":"2024-03-20T07:58:27.239245+0000","last_became_active":"2024-03-20T07:58:26.116037+0000","last_became_peered":"2024-03-20T07:58:26.116037+0000","last_unstale":"2024-03-20T07:58:27.239245+0000","last_undegraded":"2024-03-20T07:58:27.239245+0000","last_fullsized":"2024-03-20T07:58:27.239245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:16.183856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108079+0000","last_change":"2024-03-20T07:58:25.770988+0000","last_active":"2024-03-20T07:58:27.108079+0000","last_peered":"2024-03-20T07:58:27.108079+0000","last_clean":"2024-03-20T07:58:27.108079+0000","last_became_active":"2024-03-20T07:58:25.770657+0000","last_became_peered":"2024-03-20T07:58:25.770657+0000","last_unstale":"2024-03-20T07:58:27.108079+0000","last_undegraded":"2024-03-20T07:58:27.108079+0000","last_fullsized":"2024-03-20T07:58:27.108079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.131447+0000","last_change":"2024-03-20T07:58:26.128506+0000","last_active":"2024-03-20T07:58:26.131447+0000","last_peered":"2024-03-20T07:58:26.131447+0000","last_clean":"2024-03-20T07:58:26.131447+0000","last_became_active":"2024-03-20T07:58:26.128201+0000","last_became_peered":"2024-03-20T07:58:26.128201+0000","last_unstale":"2024-03-20T07:58:26.131447+0000","last_undegraded":"2024-03-20T07:58:26.131447+0000","last_fullsized":"2024-03-20T07:58:26.131447+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.147717+0000","last_change":"2024-03-20T07:58:25.767830+0000","last_active":"2024-03-20T07:58:26.147717+0000","last_peered":"2024-03-20T07:58:26.147717+0000","last_clean":"2024-03-20T07:58:26.147717+0000","last_became_active":"2024-03-20T07:58:25.767533+0000","last_became_peered":"2024-03-20T07:58:25.767533+0000","last_unstale":"2024-03-20T07:58:26.147717+0000","last_undegraded":"2024-03-20T07:58:26.147717+0000","last_fullsized":"2024-03-20T07:58:26.147717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140470+0000","last_change":"2024-03-20T07:58:26.140127+0000","last_active":"2024-03-20T07:58:26.140470+0000","last_peered":"2024-03-20T07:58:26.140470+0000","last_clean":"2024-03-20T07:58:26.140470+0000","last_became_active":"2024-03-20T07:58:26.139362+0000","last_became_peered":"2024-03-20T07:58:26.139362+0000","last_unstale":"2024-03-20T07:58:26.140470+0000","last_undegraded":"2024-03-20T07:58:26.140470+0000","last_fullsized":"2024-03-20T07:58:26.140470+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239407+0000","last_change":"2024-03-20T07:58:25.771772+0000","last_active":"2024-03-20T07:58:27.239407+0000","last_peered":"2024-03-20T07:58:27.239407+0000","last_clean":"2024-03-20T07:58:27.239407+0000","last_became_active":"2024-03-20T07:58:25.771499+0000","last_became_peered":"2024-03-20T07:58:25.771499+0000","last_unstale":"2024-03-20T07:58:27.239407+0000","last_undegraded":"2024-03-20T07:58:27.239407+0000","last_fullsized":"2024-03-20T07:58:27.239407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:19:45.833311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238700+0000","last_change":"2024-03-20T07:58:26.122549+0000","last_active":"2024-03-20T07:58:27.238700+0000","last_peered":"2024-03-20T07:58:27.238700+0000","last_clean":"2024-03-20T07:58:27.238700+0000","last_became_active":"2024-03-20T07:58:26.121780+0000","last_became_peered":"2024-03-20T07:58:26.121780+0000","last_unstale":"2024-03-20T07:58:27.238700+0000","last_undegraded":"2024-03-20T07:58:27.238700+0000","last_fullsized":"2024-03-20T07:58:27.238700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:37:01.164741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.112184+0000","last_change":"2024-03-20T07:58:25.771839+0000","last_active":"2024-03-20T07:58:26.112184+0000","last_peered":"2024-03-20T07:58:26.112184+0000","last_clean":"2024-03-20T07:58:26.112184+0000","last_became_active":"2024-03-20T07:58:25.771464+0000","last_became_peered":"2024-03-20T07:58:25.771464+0000","last_unstale":"2024-03-20T07:58:26.112184+0000","last_undegraded":"2024-03-20T07:58:26.112184+0000","last_fullsized":"2024-03-20T07:58:26.112184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.129894+0000","last_change":"2024-03-20T07:58:26.129094+0000","last_active":"2024-03-20T07:58:26.129894+0000","last_peered":"2024-03-20T07:58:26.129894+0000","last_clean":"2024-03-20T07:58:26.129894+0000","last_became_active":"2024-03-20T07:58:26.128675+0000","last_became_peered":"2024-03-20T07:58:26.128675+0000","last_unstale":"2024-03-20T07:58:26.129894+0000","last_undegraded":"2024-03-20T07:58:26.129894+0000","last_fullsized":"2024-03-20T07:58:26.129894+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.124730+0000","last_change":"2024-03-20T07:58:25.767586+0000","last_active":"2024-03-20T07:58:26.124730+0000","last_peered":"2024-03-20T07:58:26.124730+0000","last_clean":"2024-03-20T07:58:26.124730+0000","last_became_active":"2024-03-20T07:58:25.767268+0000","last_became_peered":"2024-03-20T07:58:25.767268+0000","last_unstale":"2024-03-20T07:58:26.124730+0000","last_undegraded":"2024-03-20T07:58:26.124730+0000","last_fullsized":"2024-03-20T07:58:26.124730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.144101+0000","last_change":"2024-03-20T07:58:26.139925+0000","last_active":"2024-03-20T07:58:26.144101+0000","last_peered":"2024-03-20T07:58:26.144101+0000","last_clean":"2024-03-20T07:58:26.144101+0000","last_became_active":"2024-03-20T07:58:26.138286+0000","last_became_peered":"2024-03-20T07:58:26.138286+0000","last_unstale":"2024-03-20T07:58:26.144101+0000","last_undegraded":"2024-03-20T07:58:26.144101+0000","last_fullsized":"2024-03-20T07:58:26.144101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238203+0000","last_change":"2024-03-20T07:58:25.757217+0000","last_active":"2024-03-20T07:58:27.238203+0000","last_peered":"2024-03-20T07:58:27.238203+0000","last_clean":"2024-03-20T07:58:27.238203+0000","last_became_active":"2024-03-20T07:58:25.756879+0000","last_became_peered":"2024-03-20T07:58:25.756879+0000","last_unstale":"2024-03-20T07:58:27.238203+0000","last_undegraded":"2024-03-20T07:58:27.238203+0000","last_fullsized":"2024-03-20T07:58:27.238203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:28.214822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134394+0000","last_change":"2024-03-20T07:58:26.133140+0000","last_active":"2024-03-20T07:58:26.134394+0000","last_peered":"2024-03-20T07:58:26.134394+0000","last_clean":"2024-03-20T07:58:26.134394+0000","last_became_active":"2024-03-20T07:58:26.132630+0000","last_became_peered":"2024-03-20T07:58:26.132630+0000","last_unstale":"2024-03-20T07:58:26.134394+0000","last_undegraded":"2024-03-20T07:58:26.134394+0000","last_fullsized":"2024-03-20T07:58:26.134394+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.135400+0000","last_change":"2024-03-20T07:58:25.771322+0000","last_active":"2024-03-20T07:58:26.135400+0000","last_peered":"2024-03-20T07:58:26.135400+0000","last_clean":"2024-03-20T07:58:26.135400+0000","last_became_active":"2024-03-20T07:58:25.770984+0000","last_became_peered":"2024-03-20T07:58:25.770984+0000","last_unstale":"2024-03-20T07:58:26.135400+0000","last_undegraded":"2024-03-20T07:58:26.135400+0000","last_fullsized":"2024-03-20T07:58:26.135400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145263+0000","last_change":"2024-03-20T07:58:26.144254+0000","last_active":"2024-03-20T07:58:26.145263+0000","last_peered":"2024-03-20T07:58:26.145263+0000","last_clean":"2024-03-20T07:58:26.145263+0000","last_became_active":"2024-03-20T07:58:26.143683+0000","last_became_peered":"2024-03-20T07:58:26.143683+0000","last_unstale":"2024-03-20T07:58:26.145263+0000","last_undegraded":"2024-03-20T07:58:26.145263+0000","last_fullsized":"2024-03-20T07:58:26.145263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107428+0000","last_change":"2024-03-20T07:58:25.773911+0000","last_active":"2024-03-20T07:58:27.107428+0000","last_peered":"2024-03-20T07:58:27.107428+0000","last_clean":"2024-03-20T07:58:27.107428+0000","last_became_active":"2024-03-20T07:58:25.773526+0000","last_became_peered":"2024-03-20T07:58:25.773526+0000","last_unstale":"2024-03-20T07:58:27.107428+0000","last_undegraded":"2024-03-20T07:58:27.107428+0000","last_fullsized":"2024-03-20T07:58:27.107428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.123504+0000","last_change":"2024-03-20T07:58:26.122909+0000","last_active":"2024-03-20T07:58:26.123504+0000","last_peered":"2024-03-20T07:58:26.123504+0000","last_clean":"2024-03-20T07:58:26.123504+0000","last_became_active":"2024-03-20T07:58:26.121608+0000","last_became_peered":"2024-03-20T07:58:26.121608+0000","last_unstale":"2024-03-20T07:58:26.123504+0000","last_undegraded":"2024-03-20T07:58:26.123504+0000","last_fullsized":"2024-03-20T07:58:26.123504+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.111928+0000","last_change":"2024-03-20T07:58:25.770569+0000","last_active":"2024-03-20T07:58:26.111928+0000","last_peered":"2024-03-20T07:58:26.111928+0000","last_clean":"2024-03-20T07:58:26.111928+0000","last_became_active":"2024-03-20T07:58:25.770298+0000","last_became_peered":"2024-03-20T07:58:25.770298+0000","last_unstale":"2024-03-20T07:58:26.111928+0000","last_undegraded":"2024-03-20T07:58:26.111928+0000","last_fullsized":"2024-03-20T07:58:26.111928+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.137205+0000","last_change":"2024-03-20T07:58:26.136200+0000","last_active":"2024-03-20T07:58:26.137205+0000","last_peered":"2024-03-20T07:58:26.137205+0000","last_clean":"2024-03-20T07:58:26.137205+0000","last_became_active":"2024-03-20T07:58:26.135909+0000","last_became_peered":"2024-03-20T07:58:26.135909+0000","last_unstale":"2024-03-20T07:58:26.137205+0000","last_undegraded":"2024-03-20T07:58:26.137205+0000","last_fullsized":"2024-03-20T07:58:26.137205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107104+0000","last_change":"2024-03-20T07:58:26.130079+0000","last_active":"2024-03-20T07:58:27.107104+0000","last_peered":"2024-03-20T07:58:27.107104+0000","last_clean":"2024-03-20T07:58:27.107104+0000","last_became_active":"2024-03-20T07:58:26.129798+0000","last_became_peered":"2024-03-20T07:58:26.129798+0000","last_unstale":"2024-03-20T07:58:27.107104+0000","last_undegraded":"2024-03-20T07:58:27.107104+0000","last_fullsized":"2024-03-20T07:58:27.107104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238173+0000","last_change":"2024-03-20T07:58:25.764736+0000","last_active":"2024-03-20T07:58:27.238173+0000","last_peered":"2024-03-20T07:58:27.238173+0000","last_clean":"2024-03-20T07:58:27.238173+0000","last_became_active":"2024-03-20T07:58:25.764390+0000","last_became_peered":"2024-03-20T07:58:25.764390+0000","last_unstale":"2024-03-20T07:58:27.238173+0000","last_undegraded":"2024-03-20T07:58:27.238173+0000","last_fullsized":"2024-03-20T07:58:27.238173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:27:58.881448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.110613+0000","last_change":"2024-03-20T07:58:25.766643+0000","last_active":"2024-03-20T07:58:26.110613+0000","last_peered":"2024-03-20T07:58:26.110613+0000","last_clean":"2024-03-20T07:58:26.110613+0000","last_became_active":"2024-03-20T07:58:25.766341+0000","last_became_peered":"2024-03-20T07:58:25.766341+0000","last_unstale":"2024-03-20T07:58:26.110613+0000","last_undegraded":"2024-03-20T07:58:26.110613+0000","last_fullsized":"2024-03-20T07:58:26.110613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"22'117","reported_seq":218,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-20T07:58:25.769715+0000","last_change":"2024-03-20T07:58:09.964395+0000","last_active":"2024-03-20T07:58:25.769715+0000","last_peered":"2024-03-20T07:58:25.769715+0000","last_clean":"2024-03-20T07:58:25.769715+0000","last_became_active":"2024-03-20T07:58:09.963927+0000","last_became_peered":"2024-03-20T07:58:09.963927+0000","last_unstale":"2024-03-20T07:58:25.769715+0000","last_undegraded":"2024-03-20T07:58:25.769715+0000","last_fullsized":"2024-03-20T07:58:25.769715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238574+0000","last_change":"2024-03-20T07:58:26.125436+0000","last_active":"2024-03-20T07:58:27.238574+0000","last_peered":"2024-03-20T07:58:27.238574+0000","last_clean":"2024-03-20T07:58:27.238574+0000","last_became_active":"2024-03-20T07:58:26.125161+0000","last_became_peered":"2024-03-20T07:58:26.125161+0000","last_unstale":"2024-03-20T07:58:27.238574+0000","last_undegraded":"2024-03-20T07:58:27.238574+0000","last_fullsized":"2024-03-20T07:58:27.238574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:53:55.085832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.118188+0000","last_change":"2024-03-20T07:58:25.779129+0000","last_active":"2024-03-20T07:58:26.118188+0000","last_peered":"2024-03-20T07:58:26.118188+0000","last_clean":"2024-03-20T07:58:26.118188+0000","last_became_active":"2024-03-20T07:58:25.778258+0000","last_became_peered":"2024-03-20T07:58:25.778258+0000","last_unstale":"2024-03-20T07:58:26.118188+0000","last_undegraded":"2024-03-20T07:58:26.118188+0000","last_fullsized":"2024-03-20T07:58:26.118188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.130223+0000","last_change":"2024-03-20T07:58:25.777973+0000","last_active":"2024-03-20T07:58:26.130223+0000","last_peered":"2024-03-20T07:58:26.130223+0000","last_clean":"2024-03-20T07:58:26.130223+0000","last_became_active":"2024-03-20T07:58:25.777666+0000","last_became_peered":"2024-03-20T07:58:25.777666+0000","last_unstale":"2024-03-20T07:58:26.130223+0000","last_undegraded":"2024-03-20T07:58:26.130223+0000","last_fullsized":"2024-03-20T07:58:26.130223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107508+0000","last_change":"2024-03-20T07:58:26.126692+0000","last_active":"2024-03-20T07:58:27.107508+0000","last_peered":"2024-03-20T07:58:27.107508+0000","last_clean":"2024-03-20T07:58:27.107508+0000","last_became_active":"2024-03-20T07:58:26.126397+0000","last_became_peered":"2024-03-20T07:58:26.126397+0000","last_unstale":"2024-03-20T07:58:27.107508+0000","last_undegraded":"2024-03-20T07:58:27.107508+0000","last_fullsized":"2024-03-20T07:58:27.107508+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108422+0000","last_change":"2024-03-20T07:58:25.769337+0000","last_active":"2024-03-20T07:58:27.108422+0000","last_peered":"2024-03-20T07:58:27.108422+0000","last_clean":"2024-03-20T07:58:27.108422+0000","last_became_active":"2024-03-20T07:58:25.769010+0000","last_became_peered":"2024-03-20T07:58:25.769010+0000","last_unstale":"2024-03-20T07:58:27.108422+0000","last_undegraded":"2024-03-20T07:58:27.108422+0000","last_fullsized":"2024-03-20T07:58:27.108422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.141871+0000","last_change":"2024-03-20T07:58:25.771097+0000","last_active":"2024-03-20T07:58:26.141871+0000","last_peered":"2024-03-20T07:58:26.141871+0000","last_clean":"2024-03-20T07:58:26.141871+0000","last_became_active":"2024-03-20T07:58:25.770737+0000","last_became_peered":"2024-03-20T07:58:25.770737+0000","last_unstale":"2024-03-20T07:58:26.141871+0000","last_undegraded":"2024-03-20T07:58:26.141871+0000","last_fullsized":"2024-03-20T07:58:26.141871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239163+0000","last_change":"2024-03-20T07:58:26.126230+0000","last_active":"2024-03-20T07:58:27.239163+0000","last_peered":"2024-03-20T07:58:27.239163+0000","last_clean":"2024-03-20T07:58:27.239163+0000","last_became_active":"2024-03-20T07:58:26.125980+0000","last_became_peered":"2024-03-20T07:58:26.125980+0000","last_unstale":"2024-03-20T07:58:27.239163+0000","last_undegraded":"2024-03-20T07:58:27.239163+0000","last_fullsized":"2024-03-20T07:58:27.239163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:18:03.971589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.117985+0000","last_change":"2024-03-20T07:58:25.777961+0000","last_active":"2024-03-20T07:58:26.117985+0000","last_peered":"2024-03-20T07:58:26.117985+0000","last_clean":"2024-03-20T07:58:26.117985+0000","last_became_active":"2024-03-20T07:58:25.777649+0000","last_became_peered":"2024-03-20T07:58:25.777649+0000","last_unstale":"2024-03-20T07:58:26.117985+0000","last_undegraded":"2024-03-20T07:58:26.117985+0000","last_fullsized":"2024-03-20T07:58:26.117985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.124307+0000","last_change":"2024-03-20T07:58:26.120305+0000","last_active":"2024-03-20T07:58:26.124307+0000","last_peered":"2024-03-20T07:58:26.124307+0000","last_clean":"2024-03-20T07:58:26.124307+0000","last_became_active":"2024-03-20T07:58:26.119966+0000","last_became_peered":"2024-03-20T07:58:26.119966+0000","last_unstale":"2024-03-20T07:58:26.124307+0000","last_undegraded":"2024-03-20T07:58:26.124307+0000","last_fullsized":"2024-03-20T07:58:26.124307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107099+0000","last_change":"2024-03-20T07:58:25.765548+0000","last_active":"2024-03-20T07:58:27.107099+0000","last_peered":"2024-03-20T07:58:27.107099+0000","last_clean":"2024-03-20T07:58:27.107099+0000","last_became_active":"2024-03-20T07:58:25.765273+0000","last_became_peered":"2024-03-20T07:58:25.765273+0000","last_unstale":"2024-03-20T07:58:27.107099+0000","last_undegraded":"2024-03-20T07:58:27.107099+0000","last_fullsized":"2024-03-20T07:58:27.107099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107547+0000","last_change":"2024-03-20T07:58:25.773811+0000","last_active":"2024-03-20T07:58:27.107547+0000","last_peered":"2024-03-20T07:58:27.107547+0000","last_clean":"2024-03-20T07:58:27.107547+0000","last_became_active":"2024-03-20T07:58:25.773520+0000","last_became_peered":"2024-03-20T07:58:25.773520+0000","last_unstale":"2024-03-20T07:58:27.107547+0000","last_undegraded":"2024-03-20T07:58:27.107547+0000","last_fullsized":"2024-03-20T07:58:27.107547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138933+0000","last_change":"2024-03-20T07:58:25.776066+0000","last_active":"2024-03-20T07:58:26.138933+0000","last_peered":"2024-03-20T07:58:26.138933+0000","last_clean":"2024-03-20T07:58:26.138933+0000","last_became_active":"2024-03-20T07:58:25.775708+0000","last_became_peered":"2024-03-20T07:58:25.775708+0000","last_unstale":"2024-03-20T07:58:26.138933+0000","last_undegraded":"2024-03-20T07:58:26.138933+0000","last_fullsized":"2024-03-20T07:58:26.138933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:36:20.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134688+0000","last_change":"2024-03-20T07:58:26.134067+0000","last_active":"2024-03-20T07:58:26.134688+0000","last_peered":"2024-03-20T07:58:26.134688+0000","last_clean":"2024-03-20T07:58:26.134688+0000","last_became_active":"2024-03-20T07:58:26.133784+0000","last_became_peered":"2024-03-20T07:58:26.133784+0000","last_unstale":"2024-03-20T07:58:26.134688+0000","last_undegraded":"2024-03-20T07:58:26.134688+0000","last_fullsized":"2024-03-20T07:58:26.134688+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238423+0000","last_change":"2024-03-20T07:58:25.763015+0000","last_active":"2024-03-20T07:58:27.238423+0000","last_peered":"2024-03-20T07:58:27.238423+0000","last_clean":"2024-03-20T07:58:27.238423+0000","last_became_active":"2024-03-20T07:58:25.762682+0000","last_became_peered":"2024-03-20T07:58:25.762682+0000","last_unstale":"2024-03-20T07:58:27.238423+0000","last_undegraded":"2024-03-20T07:58:27.238423+0000","last_fullsized":"2024-03-20T07:58:27.238423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:21:45.137669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.142797+0000","last_change":"2024-03-20T07:58:26.142445+0000","last_active":"2024-03-20T07:58:26.142797+0000","last_peered":"2024-03-20T07:58:26.142797+0000","last_clean":"2024-03-20T07:58:26.142797+0000","last_became_active":"2024-03-20T07:58:26.142148+0000","last_became_peered":"2024-03-20T07:58:26.142148+0000","last_unstale":"2024-03-20T07:58:26.142797+0000","last_undegraded":"2024-03-20T07:58:26.142797+0000","last_fullsized":"2024-03-20T07:58:26.142797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"62'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107988+0000","last_change":"2024-03-20T07:58:25.766427+0000","last_active":"2024-03-20T07:58:27.107988+0000","last_peered":"2024-03-20T07:58:27.107988+0000","last_clean":"2024-03-20T07:58:27.107988+0000","last_became_active":"2024-03-20T07:58:25.766111+0000","last_became_peered":"2024-03-20T07:58:25.766111+0000","last_unstale":"2024-03-20T07:58:27.107988+0000","last_undegraded":"2024-03-20T07:58:27.107988+0000","last_fullsized":"2024-03-20T07:58:27.107988+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.136811+0000","last_change":"2024-03-20T07:58:26.135339+0000","last_active":"2024-03-20T07:58:26.136811+0000","last_peered":"2024-03-20T07:58:26.136811+0000","last_clean":"2024-03-20T07:58:26.136811+0000","last_became_active":"2024-03-20T07:58:26.134903+0000","last_became_peered":"2024-03-20T07:58:26.134903+0000","last_unstale":"2024-03-20T07:58:26.136811+0000","last_undegraded":"2024-03-20T07:58:26.136811+0000","last_fullsized":"2024-03-20T07:58:26.136811+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"62'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238866+0000","last_change":"2024-03-20T07:58:25.766233+0000","last_active":"2024-03-20T07:58:27.238866+0000","last_peered":"2024-03-20T07:58:27.238866+0000","last_clean":"2024-03-20T07:58:27.238866+0000","last_became_active":"2024-03-20T07:58:25.765948+0000","last_became_peered":"2024-03-20T07:58:25.765948+0000","last_unstale":"2024-03-20T07:58:27.238866+0000","last_undegraded":"2024-03-20T07:58:27.238866+0000","last_fullsized":"2024-03-20T07:58:27.238866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:34:01.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.145073+0000","last_change":"2024-03-20T07:58:26.144008+0000","last_active":"2024-03-20T07:58:26.145073+0000","last_peered":"2024-03-20T07:58:26.145073+0000","last_clean":"2024-03-20T07:58:26.145073+0000","last_became_active":"2024-03-20T07:58:26.143696+0000","last_became_peered":"2024-03-20T07:58:26.143696+0000","last_unstale":"2024-03-20T07:58:26.145073+0000","last_undegraded":"2024-03-20T07:58:26.145073+0000","last_fullsized":"2024-03-20T07:58:26.145073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.140012+0000","last_change":"2024-03-20T07:58:26.138096+0000","last_active":"2024-03-20T07:58:26.140012+0000","last_peered":"2024-03-20T07:58:26.140012+0000","last_clean":"2024-03-20T07:58:26.140012+0000","last_became_active":"2024-03-20T07:58:26.137804+0000","last_became_peered":"2024-03-20T07:58:26.137804+0000","last_unstale":"2024-03-20T07:58:26.140012+0000","last_undegraded":"2024-03-20T07:58:26.140012+0000","last_fullsized":"2024-03-20T07:58:26.140012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238668+0000","last_change":"2024-03-20T07:58:26.126085+0000","last_active":"2024-03-20T07:58:27.238668+0000","last_peered":"2024-03-20T07:58:27.238668+0000","last_clean":"2024-03-20T07:58:27.238668+0000","last_became_active":"2024-03-20T07:58:26.125793+0000","last_became_peered":"2024-03-20T07:58:26.125793+0000","last_unstale":"2024-03-20T07:58:27.238668+0000","last_undegraded":"2024-03-20T07:58:27.238668+0000","last_fullsized":"2024-03-20T07:58:27.238668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:11:45.470755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.126540+0000","last_change":"2024-03-20T07:58:26.125004+0000","last_active":"2024-03-20T07:58:26.126540+0000","last_peered":"2024-03-20T07:58:26.126540+0000","last_clean":"2024-03-20T07:58:26.126540+0000","last_became_active":"2024-03-20T07:58:26.124689+0000","last_became_peered":"2024-03-20T07:58:26.124689+0000","last_unstale":"2024-03-20T07:58:26.126540+0000","last_undegraded":"2024-03-20T07:58:26.126540+0000","last_fullsized":"2024-03-20T07:58:26.126540+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239208+0000","last_change":"2024-03-20T07:58:26.120048+0000","last_active":"2024-03-20T07:58:27.239208+0000","last_peered":"2024-03-20T07:58:27.239208+0000","last_clean":"2024-03-20T07:58:27.239208+0000","last_became_active":"2024-03-20T07:58:26.119755+0000","last_became_peered":"2024-03-20T07:58:26.119755+0000","last_unstale":"2024-03-20T07:58:27.239208+0000","last_undegraded":"2024-03-20T07:58:27.239208+0000","last_fullsized":"2024-03-20T07:58:27.239208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:39:07.689571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107012+0000","last_change":"2024-03-20T07:58:26.126113+0000","last_active":"2024-03-20T07:58:27.107012+0000","last_peered":"2024-03-20T07:58:27.107012+0000","last_clean":"2024-03-20T07:58:27.107012+0000","last_became_active":"2024-03-20T07:58:26.122635+0000","last_became_peered":"2024-03-20T07:58:26.122635+0000","last_unstale":"2024-03-20T07:58:27.107012+0000","last_undegraded":"2024-03-20T07:58:27.107012+0000","last_fullsized":"2024-03-20T07:58:27.107012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.144469+0000","last_change":"2024-03-20T07:58:26.141379+0000","last_active":"2024-03-20T07:58:26.144469+0000","last_peered":"2024-03-20T07:58:26.144469+0000","last_clean":"2024-03-20T07:58:26.144469+0000","last_became_active":"2024-03-20T07:58:26.141099+0000","last_became_peered":"2024-03-20T07:58:26.141099+0000","last_unstale":"2024-03-20T07:58:26.144469+0000","last_undegraded":"2024-03-20T07:58:26.144469+0000","last_fullsized":"2024-03-20T07:58:26.144469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138838+0000","last_change":"2024-03-20T07:58:26.137638+0000","last_active":"2024-03-20T07:58:26.138838+0000","last_peered":"2024-03-20T07:58:26.138838+0000","last_clean":"2024-03-20T07:58:26.138838+0000","last_became_active":"2024-03-20T07:58:26.137344+0000","last_became_peered":"2024-03-20T07:58:26.137344+0000","last_unstale":"2024-03-20T07:58:26.138838+0000","last_undegraded":"2024-03-20T07:58:26.138838+0000","last_fullsized":"2024-03-20T07:58:26.138838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-20T07:58:25.744971+0000","last_change":"2024-03-20T07:58:25.744971+0000","last_active":"2024-03-20T07:58:25.744971+0000","last_peered":"2024-03-20T07:58:25.744971+0000","last_clean":"2024-03-20T07:58:25.744971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-20T07:58:25.744971+0000","last_undegraded":"2024-03-20T07:58:25.744971+0000","last_fullsized":"2024-03-20T07:58:25.744971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.138511+0000","last_change":"2024-03-20T07:58:26.137117+0000","last_active":"2024-03-20T07:58:26.138511+0000","last_peered":"2024-03-20T07:58:26.138511+0000","last_clean":"2024-03-20T07:58:26.138511+0000","last_became_active":"2024-03-20T07:58:26.136135+0000","last_became_peered":"2024-03-20T07:58:26.136135+0000","last_unstale":"2024-03-20T07:58:26.138511+0000","last_undegraded":"2024-03-20T07:58:26.138511+0000","last_fullsized":"2024-03-20T07:58:26.138511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239107+0000","last_change":"2024-03-20T07:58:26.119480+0000","last_active":"2024-03-20T07:58:27.239107+0000","last_peered":"2024-03-20T07:58:27.239107+0000","last_clean":"2024-03-20T07:58:27.239107+0000","last_became_active":"2024-03-20T07:58:26.119047+0000","last_became_peered":"2024-03-20T07:58:26.119047+0000","last_unstale":"2024-03-20T07:58:27.239107+0000","last_undegraded":"2024-03-20T07:58:27.239107+0000","last_fullsized":"2024-03-20T07:58:27.239107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:09:31.918661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.134663+0000","last_change":"2024-03-20T07:58:26.133160+0000","last_active":"2024-03-20T07:58:26.134663+0000","last_peered":"2024-03-20T07:58:26.134663+0000","last_clean":"2024-03-20T07:58:26.134663+0000","last_became_active":"2024-03-20T07:58:26.132793+0000","last_became_peered":"2024-03-20T07:58:26.132793+0000","last_unstale":"2024-03-20T07:58:26.134663+0000","last_undegraded":"2024-03-20T07:58:26.134663+0000","last_fullsized":"2024-03-20T07:58:26.134663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":159,"acting":159,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":5418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":87,"acting":87,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201286,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28192,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716960,"statfs":{"total":95995035648,"available":95966167040,"internally_reserved":0,"allocated":1200128,"data_stored":838705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462923,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1183744,"data_stored":831669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717512,"statfs":{"total":95995035648,"available":95966732288,"internally_reserved":0,"allocated":626688,"data_stored":255702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.33100000000000002,"15min":0.32300000000000001},"min":{"1min":0.26200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.35599999999999998,"15min":0.34399999999999997},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.313}]},{"osd":2,"last update":"Wed Mar 20 07:58:07 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.39300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.25},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.308,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.27800000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.37}]},{"osd":4,"last update":"Wed Mar 20 07:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738410,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1228800,"data_stored":846638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 20 07:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.26300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.32200000000000001,"15min":0.312},"min":{"1min":0.26300000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Mar 20 07:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33000000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.318},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.374}]},{"osd":4,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.34100000000000003}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.39400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411359,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717520,"statfs":{"total":95995035648,"available":95966740480,"internally_reserved":0,"allocated":618496,"data_stored":255542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.36799999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35699999999999998}]},{"osd":1,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.33900000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38100000000000001},"min":{"1min":0.33000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52000000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.311},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed Mar 20 07:58:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149721,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717556,"statfs":{"total":95995035648,"available":95966777344,"internally_reserved":0,"allocated":581632,"data_stored":235385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.36499999999999999}]},{"osd":1,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39200000000000002}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.626},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Mar 20 07:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986197,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":634880,"data_stored":256250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.374}]},{"osd":1,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.503}]},{"osd":2,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.432}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.35699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717528,"statfs":{"total":95995035648,"available":95966748672,"internally_reserved":0,"allocated":618496,"data_stored":255542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.44}]},{"osd":1,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.495}]},{"osd":3,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.38400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T07:58:31.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:30 smithi196 ceph-mon[31033]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:30 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3725815946' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-20T07:58:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:30 smithi196 ceph-mon[31033]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:31.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:30 smithi118 ceph-mon[27391]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:31.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:30 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3725815946' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-20T07:58:31.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:30 smithi118 ceph-mon[27391]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:31.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:30 smithi118 ceph-mon[38981]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:31.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:30 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3725815946' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-20T07:58:31.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:30 smithi118 ceph-mon[38981]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:32.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:31 smithi196 ceph-mon[31033]: pgmap v196: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-20T07:58:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:31 smithi118 ceph-mon[27391]: pgmap v196: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-20T07:58:32.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:31 smithi118 ceph-mon[38981]: pgmap v196: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-20T07:58:33.448 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-20T07:58:33.749 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:58:33.750 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T07:58:33.760 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-03-20T07:58:33.237716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":279,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":222956,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749738260,"statfs":{"total":767960285184,"available":767731978240,"internally_reserved":0,"allocated":6922240,"data_stored":3876035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.488168"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689799+0000","last_change":"2024-03-20T07:58:26.145662+0000","last_active":"2024-03-20T07:58:26.689799+0000","last_peered":"2024-03-20T07:58:26.689799+0000","last_clean":"2024-03-20T07:58:26.689799+0000","last_became_active":"2024-03-20T07:58:26.145286+0000","last_became_peered":"2024-03-20T07:58:26.145286+0000","last_unstale":"2024-03-20T07:58:26.689799+0000","last_undegraded":"2024-03-20T07:58:26.689799+0000","last_fullsized":"2024-03-20T07:58:26.689799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108335+0000","last_change":"2024-03-20T07:58:26.129069+0000","last_active":"2024-03-20T07:58:27.108335+0000","last_peered":"2024-03-20T07:58:27.108335+0000","last_clean":"2024-03-20T07:58:27.108335+0000","last_became_active":"2024-03-20T07:58:26.128811+0000","last_became_peered":"2024-03-20T07:58:26.128811+0000","last_unstale":"2024-03-20T07:58:27.108335+0000","last_undegraded":"2024-03-20T07:58:27.108335+0000","last_fullsized":"2024-03-20T07:58:27.108335+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239393+0000","last_change":"2024-03-20T07:58:26.119030+0000","last_active":"2024-03-20T07:58:27.239393+0000","last_peered":"2024-03-20T07:58:27.239393+0000","last_clean":"2024-03-20T07:58:27.239393+0000","last_became_active":"2024-03-20T07:58:26.118755+0000","last_became_peered":"2024-03-20T07:58:26.118755+0000","last_unstale":"2024-03-20T07:58:27.239393+0000","last_undegraded":"2024-03-20T07:58:27.239393+0000","last_fullsized":"2024-03-20T07:58:27.239393+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:51:41.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367839+0000","last_change":"2024-03-20T07:58:26.138067+0000","last_active":"2024-03-20T07:58:26.367839+0000","last_peered":"2024-03-20T07:58:26.367839+0000","last_clean":"2024-03-20T07:58:26.367839+0000","last_became_active":"2024-03-20T07:58:26.137768+0000","last_became_peered":"2024-03-20T07:58:26.137768+0000","last_unstale":"2024-03-20T07:58:26.367839+0000","last_undegraded":"2024-03-20T07:58:26.367839+0000","last_fullsized":"2024-03-20T07:58:26.367839+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689677+0000","last_change":"2024-03-20T07:58:26.131923+0000","last_active":"2024-03-20T07:58:26.689677+0000","last_peered":"2024-03-20T07:58:26.689677+0000","last_clean":"2024-03-20T07:58:26.689677+0000","last_became_active":"2024-03-20T07:58:26.131585+0000","last_became_peered":"2024-03-20T07:58:26.131585+0000","last_unstale":"2024-03-20T07:58:26.689677+0000","last_undegraded":"2024-03-20T07:58:26.689677+0000","last_fullsized":"2024-03-20T07:58:26.689677+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367816+0000","last_change":"2024-03-20T07:58:26.137308+0000","last_active":"2024-03-20T07:58:26.367816+0000","last_peered":"2024-03-20T07:58:26.367816+0000","last_clean":"2024-03-20T07:58:26.367816+0000","last_became_active":"2024-03-20T07:58:26.135727+0000","last_became_peered":"2024-03-20T07:58:26.135727+0000","last_unstale":"2024-03-20T07:58:26.367816+0000","last_undegraded":"2024-03-20T07:58:26.367816+0000","last_fullsized":"2024-03-20T07:58:26.367816+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689635+0000","last_change":"2024-03-20T07:58:26.132941+0000","last_active":"2024-03-20T07:58:26.689635+0000","last_peered":"2024-03-20T07:58:26.689635+0000","last_clean":"2024-03-20T07:58:26.689635+0000","last_became_active":"2024-03-20T07:58:26.132230+0000","last_became_peered":"2024-03-20T07:58:26.132230+0000","last_unstale":"2024-03-20T07:58:26.689635+0000","last_undegraded":"2024-03-20T07:58:26.689635+0000","last_fullsized":"2024-03-20T07:58:26.689635+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.106681+0000","last_change":"2024-03-20T07:58:26.136216+0000","last_active":"2024-03-20T07:58:27.106681+0000","last_peered":"2024-03-20T07:58:27.106681+0000","last_clean":"2024-03-20T07:58:27.106681+0000","last_became_active":"2024-03-20T07:58:26.135920+0000","last_became_peered":"2024-03-20T07:58:26.135920+0000","last_unstale":"2024-03-20T07:58:27.106681+0000","last_undegraded":"2024-03-20T07:58:27.106681+0000","last_fullsized":"2024-03-20T07:58:27.106681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:37.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689788+0000","last_change":"2024-03-20T07:58:26.144631+0000","last_active":"2024-03-20T07:58:26.689788+0000","last_peered":"2024-03-20T07:58:26.689788+0000","last_clean":"2024-03-20T07:58:26.689788+0000","last_became_active":"2024-03-20T07:58:26.144302+0000","last_became_peered":"2024-03-20T07:58:26.144302+0000","last_unstale":"2024-03-20T07:58:26.689788+0000","last_undegraded":"2024-03-20T07:58:26.689788+0000","last_fullsized":"2024-03-20T07:58:26.689788+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298314+0000","last_change":"2024-03-20T07:58:26.138657+0000","last_active":"2024-03-20T07:58:26.298314+0000","last_peered":"2024-03-20T07:58:26.298314+0000","last_clean":"2024-03-20T07:58:26.298314+0000","last_became_active":"2024-03-20T07:58:26.138348+0000","last_became_peered":"2024-03-20T07:58:26.138348+0000","last_unstale":"2024-03-20T07:58:26.298314+0000","last_undegraded":"2024-03-20T07:58:26.298314+0000","last_fullsized":"2024-03-20T07:58:26.298314+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298304+0000","last_change":"2024-03-20T07:58:26.137961+0000","last_active":"2024-03-20T07:58:26.298304+0000","last_peered":"2024-03-20T07:58:26.298304+0000","last_clean":"2024-03-20T07:58:26.298304+0000","last_became_active":"2024-03-20T07:58:26.137672+0000","last_became_peered":"2024-03-20T07:58:26.137672+0000","last_unstale":"2024-03-20T07:58:26.298304+0000","last_undegraded":"2024-03-20T07:58:26.298304+0000","last_fullsized":"2024-03-20T07:58:26.298304+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.443760+0000","last_change":"2024-03-20T07:58:26.136246+0000","last_active":"2024-03-20T07:58:26.443760+0000","last_peered":"2024-03-20T07:58:26.443760+0000","last_clean":"2024-03-20T07:58:26.443760+0000","last_became_active":"2024-03-20T07:58:26.135935+0000","last_became_peered":"2024-03-20T07:58:26.135935+0000","last_unstale":"2024-03-20T07:58:26.443760+0000","last_undegraded":"2024-03-20T07:58:26.443760+0000","last_fullsized":"2024-03-20T07:58:26.443760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107622+0000","last_change":"2024-03-20T07:58:26.129765+0000","last_active":"2024-03-20T07:58:27.107622+0000","last_peered":"2024-03-20T07:58:27.107622+0000","last_clean":"2024-03-20T07:58:27.107622+0000","last_became_active":"2024-03-20T07:58:26.129209+0000","last_became_peered":"2024-03-20T07:58:26.129209+0000","last_unstale":"2024-03-20T07:58:27.107622+0000","last_undegraded":"2024-03-20T07:58:27.107622+0000","last_fullsized":"2024-03-20T07:58:27.107622+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:16:48.280737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238796+0000","last_change":"2024-03-20T07:58:26.123008+0000","last_active":"2024-03-20T07:58:27.238796+0000","last_peered":"2024-03-20T07:58:27.238796+0000","last_clean":"2024-03-20T07:58:27.238796+0000","last_became_active":"2024-03-20T07:58:26.122270+0000","last_became_peered":"2024-03-20T07:58:26.122270+0000","last_unstale":"2024-03-20T07:58:27.238796+0000","last_undegraded":"2024-03-20T07:58:27.238796+0000","last_fullsized":"2024-03-20T07:58:27.238796+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:37.385767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107015+0000","last_change":"2024-03-20T07:58:26.127042+0000","last_active":"2024-03-20T07:58:27.107015+0000","last_peered":"2024-03-20T07:58:27.107015+0000","last_clean":"2024-03-20T07:58:27.107015+0000","last_became_active":"2024-03-20T07:58:26.126555+0000","last_became_peered":"2024-03-20T07:58:26.126555+0000","last_unstale":"2024-03-20T07:58:27.107015+0000","last_undegraded":"2024-03-20T07:58:27.107015+0000","last_fullsized":"2024-03-20T07:58:27.107015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367723+0000","last_change":"2024-03-20T07:58:26.129086+0000","last_active":"2024-03-20T07:58:26.367723+0000","last_peered":"2024-03-20T07:58:26.367723+0000","last_clean":"2024-03-20T07:58:26.367723+0000","last_became_active":"2024-03-20T07:58:26.128111+0000","last_became_peered":"2024-03-20T07:58:26.128111+0000","last_unstale":"2024-03-20T07:58:26.367723+0000","last_undegraded":"2024-03-20T07:58:26.367723+0000","last_fullsized":"2024-03-20T07:58:26.367723+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367926+0000","last_change":"2024-03-20T07:58:26.128812+0000","last_active":"2024-03-20T07:58:26.367926+0000","last_peered":"2024-03-20T07:58:26.367926+0000","last_clean":"2024-03-20T07:58:26.367926+0000","last_became_active":"2024-03-20T07:58:26.128501+0000","last_became_peered":"2024-03-20T07:58:26.128501+0000","last_unstale":"2024-03-20T07:58:26.367926+0000","last_undegraded":"2024-03-20T07:58:26.367926+0000","last_fullsized":"2024-03-20T07:58:26.367926+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238912+0000","last_change":"2024-03-20T07:58:26.128211+0000","last_active":"2024-03-20T07:58:27.238912+0000","last_peered":"2024-03-20T07:58:27.238912+0000","last_clean":"2024-03-20T07:58:27.238912+0000","last_became_active":"2024-03-20T07:58:26.127943+0000","last_became_peered":"2024-03-20T07:58:26.127943+0000","last_unstale":"2024-03-20T07:58:27.238912+0000","last_undegraded":"2024-03-20T07:58:27.238912+0000","last_fullsized":"2024-03-20T07:58:27.238912+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:19.979568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689710+0000","last_change":"2024-03-20T07:58:26.133112+0000","last_active":"2024-03-20T07:58:26.689710+0000","last_peered":"2024-03-20T07:58:26.689710+0000","last_clean":"2024-03-20T07:58:26.689710+0000","last_became_active":"2024-03-20T07:58:26.132794+0000","last_became_peered":"2024-03-20T07:58:26.132794+0000","last_unstale":"2024-03-20T07:58:26.689710+0000","last_undegraded":"2024-03-20T07:58:26.689710+0000","last_fullsized":"2024-03-20T07:58:26.689710+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"62'4","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239419+0000","last_change":"2024-03-20T07:58:25.774846+0000","last_active":"2024-03-20T07:58:27.239419+0000","last_peered":"2024-03-20T07:58:27.239419+0000","last_clean":"2024-03-20T07:58:27.239419+0000","last_became_active":"2024-03-20T07:58:25.774570+0000","last_became_peered":"2024-03-20T07:58:25.774570+0000","last_unstale":"2024-03-20T07:58:27.239419+0000","last_undegraded":"2024-03-20T07:58:27.239419+0000","last_fullsized":"2024-03-20T07:58:27.239419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:00.513690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107346+0000","last_change":"2024-03-20T07:58:26.128852+0000","last_active":"2024-03-20T07:58:27.107346+0000","last_peered":"2024-03-20T07:58:27.107346+0000","last_clean":"2024-03-20T07:58:27.107346+0000","last_became_active":"2024-03-20T07:58:26.128584+0000","last_became_peered":"2024-03-20T07:58:26.128584+0000","last_unstale":"2024-03-20T07:58:27.107346+0000","last_undegraded":"2024-03-20T07:58:27.107346+0000","last_fullsized":"2024-03-20T07:58:27.107346+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:15:37.874514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"62'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107350+0000","last_change":"2024-03-20T07:58:25.774347+0000","last_active":"2024-03-20T07:58:27.107350+0000","last_peered":"2024-03-20T07:58:27.107350+0000","last_clean":"2024-03-20T07:58:27.107350+0000","last_became_active":"2024-03-20T07:58:25.773993+0000","last_became_peered":"2024-03-20T07:58:25.773993+0000","last_unstale":"2024-03-20T07:58:27.107350+0000","last_undegraded":"2024-03-20T07:58:27.107350+0000","last_fullsized":"2024-03-20T07:58:27.107350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:04:59.603081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239316+0000","last_change":"2024-03-20T07:58:26.118404+0000","last_active":"2024-03-20T07:58:27.239316+0000","last_peered":"2024-03-20T07:58:27.239316+0000","last_clean":"2024-03-20T07:58:27.239316+0000","last_became_active":"2024-03-20T07:58:26.118107+0000","last_became_peered":"2024-03-20T07:58:26.118107+0000","last_unstale":"2024-03-20T07:58:27.239316+0000","last_undegraded":"2024-03-20T07:58:27.239316+0000","last_fullsized":"2024-03-20T07:58:27.239316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:51.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689750+0000","last_change":"2024-03-20T07:58:25.778043+0000","last_active":"2024-03-20T07:58:26.689750+0000","last_peered":"2024-03-20T07:58:26.689750+0000","last_clean":"2024-03-20T07:58:26.689750+0000","last_became_active":"2024-03-20T07:58:25.777710+0000","last_became_peered":"2024-03-20T07:58:25.777710+0000","last_unstale":"2024-03-20T07:58:26.689750+0000","last_undegraded":"2024-03-20T07:58:26.689750+0000","last_fullsized":"2024-03-20T07:58:26.689750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367860+0000","last_change":"2024-03-20T07:58:26.122440+0000","last_active":"2024-03-20T07:58:26.367860+0000","last_peered":"2024-03-20T07:58:26.367860+0000","last_clean":"2024-03-20T07:58:26.367860+0000","last_became_active":"2024-03-20T07:58:26.122136+0000","last_became_peered":"2024-03-20T07:58:26.122136+0000","last_unstale":"2024-03-20T07:58:26.367860+0000","last_undegraded":"2024-03-20T07:58:26.367860+0000","last_fullsized":"2024-03-20T07:58:26.367860+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689698+0000","last_change":"2024-03-20T07:58:25.776107+0000","last_active":"2024-03-20T07:58:26.689698+0000","last_peered":"2024-03-20T07:58:26.689698+0000","last_clean":"2024-03-20T07:58:26.689698+0000","last_became_active":"2024-03-20T07:58:25.775756+0000","last_became_peered":"2024-03-20T07:58:25.775756+0000","last_unstale":"2024-03-20T07:58:26.689698+0000","last_undegraded":"2024-03-20T07:58:26.689698+0000","last_fullsized":"2024-03-20T07:58:26.689698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367883+0000","last_change":"2024-03-20T07:58:26.137236+0000","last_active":"2024-03-20T07:58:26.367883+0000","last_peered":"2024-03-20T07:58:26.367883+0000","last_clean":"2024-03-20T07:58:26.367883+0000","last_became_active":"2024-03-20T07:58:26.136660+0000","last_became_peered":"2024-03-20T07:58:26.136660+0000","last_unstale":"2024-03-20T07:58:26.367883+0000","last_undegraded":"2024-03-20T07:58:26.367883+0000","last_fullsized":"2024-03-20T07:58:26.367883+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:36.426353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"62'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107803+0000","last_change":"2024-03-20T07:58:25.769988+0000","last_active":"2024-03-20T07:58:27.107803+0000","last_peered":"2024-03-20T07:58:27.107803+0000","last_clean":"2024-03-20T07:58:27.107803+0000","last_became_active":"2024-03-20T07:58:25.769359+0000","last_became_peered":"2024-03-20T07:58:25.769359+0000","last_unstale":"2024-03-20T07:58:27.107803+0000","last_undegraded":"2024-03-20T07:58:27.107803+0000","last_fullsized":"2024-03-20T07:58:27.107803+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:26:35.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.443726+0000","last_change":"2024-03-20T07:58:26.118793+0000","last_active":"2024-03-20T07:58:26.443726+0000","last_peered":"2024-03-20T07:58:26.443726+0000","last_clean":"2024-03-20T07:58:26.443726+0000","last_became_active":"2024-03-20T07:58:26.118430+0000","last_became_peered":"2024-03-20T07:58:26.118430+0000","last_unstale":"2024-03-20T07:58:26.443726+0000","last_undegraded":"2024-03-20T07:58:26.443726+0000","last_fullsized":"2024-03-20T07:58:26.443726+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298270+0000","last_change":"2024-03-20T07:58:25.765918+0000","last_active":"2024-03-20T07:58:26.298270+0000","last_peered":"2024-03-20T07:58:26.298270+0000","last_clean":"2024-03-20T07:58:26.298270+0000","last_became_active":"2024-03-20T07:58:25.765598+0000","last_became_peered":"2024-03-20T07:58:25.765598+0000","last_unstale":"2024-03-20T07:58:26.298270+0000","last_undegraded":"2024-03-20T07:58:26.298270+0000","last_fullsized":"2024-03-20T07:58:26.298270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239315+0000","last_change":"2024-03-20T07:58:26.119313+0000","last_active":"2024-03-20T07:58:27.239315+0000","last_peered":"2024-03-20T07:58:27.239315+0000","last_clean":"2024-03-20T07:58:27.239315+0000","last_became_active":"2024-03-20T07:58:26.118645+0000","last_became_peered":"2024-03-20T07:58:26.118645+0000","last_unstale":"2024-03-20T07:58:27.239315+0000","last_undegraded":"2024-03-20T07:58:27.239315+0000","last_fullsized":"2024-03-20T07:58:27.239315+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:13.410704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107076+0000","last_change":"2024-03-20T07:58:25.771278+0000","last_active":"2024-03-20T07:58:27.107076+0000","last_peered":"2024-03-20T07:58:27.107076+0000","last_clean":"2024-03-20T07:58:27.107076+0000","last_became_active":"2024-03-20T07:58:25.770958+0000","last_became_peered":"2024-03-20T07:58:25.770958+0000","last_unstale":"2024-03-20T07:58:27.107076+0000","last_undegraded":"2024-03-20T07:58:27.107076+0000","last_fullsized":"2024-03-20T07:58:27.107076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:10:26.828071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.106779+0000","last_change":"2024-03-20T07:58:26.116986+0000","last_active":"2024-03-20T07:58:27.106779+0000","last_peered":"2024-03-20T07:58:27.106779+0000","last_clean":"2024-03-20T07:58:27.106779+0000","last_became_active":"2024-03-20T07:58:26.116571+0000","last_became_peered":"2024-03-20T07:58:26.116571+0000","last_unstale":"2024-03-20T07:58:27.106779+0000","last_undegraded":"2024-03-20T07:58:27.106779+0000","last_fullsized":"2024-03-20T07:58:27.106779+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:32:56.125697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367740+0000","last_change":"2024-03-20T07:58:25.769007+0000","last_active":"2024-03-20T07:58:26.367740+0000","last_peered":"2024-03-20T07:58:26.367740+0000","last_clean":"2024-03-20T07:58:26.367740+0000","last_became_active":"2024-03-20T07:58:25.768716+0000","last_became_peered":"2024-03-20T07:58:25.768716+0000","last_unstale":"2024-03-20T07:58:26.367740+0000","last_undegraded":"2024-03-20T07:58:26.367740+0000","last_fullsized":"2024-03-20T07:58:26.367740+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107247+0000","last_change":"2024-03-20T07:58:26.119518+0000","last_active":"2024-03-20T07:58:27.107247+0000","last_peered":"2024-03-20T07:58:27.107247+0000","last_clean":"2024-03-20T07:58:27.107247+0000","last_became_active":"2024-03-20T07:58:26.119184+0000","last_became_peered":"2024-03-20T07:58:26.119184+0000","last_unstale":"2024-03-20T07:58:27.107247+0000","last_undegraded":"2024-03-20T07:58:27.107247+0000","last_fullsized":"2024-03-20T07:58:27.107247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298345+0000","last_change":"2024-03-20T07:58:25.769300+0000","last_active":"2024-03-20T07:58:26.298345+0000","last_peered":"2024-03-20T07:58:26.298345+0000","last_clean":"2024-03-20T07:58:26.298345+0000","last_became_active":"2024-03-20T07:58:25.768916+0000","last_became_peered":"2024-03-20T07:58:25.768916+0000","last_unstale":"2024-03-20T07:58:26.298345+0000","last_undegraded":"2024-03-20T07:58:26.298345+0000","last_fullsized":"2024-03-20T07:58:26.298345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367907+0000","last_change":"2024-03-20T07:58:26.130616+0000","last_active":"2024-03-20T07:58:26.367907+0000","last_peered":"2024-03-20T07:58:26.367907+0000","last_clean":"2024-03-20T07:58:26.367907+0000","last_became_active":"2024-03-20T07:58:26.130332+0000","last_became_peered":"2024-03-20T07:58:26.130332+0000","last_unstale":"2024-03-20T07:58:26.367907+0000","last_undegraded":"2024-03-20T07:58:26.367907+0000","last_fullsized":"2024-03-20T07:58:26.367907+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367916+0000","last_change":"2024-03-20T07:58:25.769790+0000","last_active":"2024-03-20T07:58:26.367916+0000","last_peered":"2024-03-20T07:58:26.367916+0000","last_clean":"2024-03-20T07:58:26.367916+0000","last_became_active":"2024-03-20T07:58:25.769492+0000","last_became_peered":"2024-03-20T07:58:25.769492+0000","last_unstale":"2024-03-20T07:58:26.367916+0000","last_undegraded":"2024-03-20T07:58:26.367916+0000","last_fullsized":"2024-03-20T07:58:26.367916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239245+0000","last_change":"2024-03-20T07:58:26.116344+0000","last_active":"2024-03-20T07:58:27.239245+0000","last_peered":"2024-03-20T07:58:27.239245+0000","last_clean":"2024-03-20T07:58:27.239245+0000","last_became_active":"2024-03-20T07:58:26.116037+0000","last_became_peered":"2024-03-20T07:58:26.116037+0000","last_unstale":"2024-03-20T07:58:27.239245+0000","last_undegraded":"2024-03-20T07:58:27.239245+0000","last_fullsized":"2024-03-20T07:58:27.239245+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:16.183856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.108079+0000","last_change":"2024-03-20T07:58:25.770988+0000","last_active":"2024-03-20T07:58:27.108079+0000","last_peered":"2024-03-20T07:58:27.108079+0000","last_clean":"2024-03-20T07:58:27.108079+0000","last_became_active":"2024-03-20T07:58:25.770657+0000","last_became_peered":"2024-03-20T07:58:25.770657+0000","last_unstale":"2024-03-20T07:58:27.108079+0000","last_undegraded":"2024-03-20T07:58:27.108079+0000","last_fullsized":"2024-03-20T07:58:27.108079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.443747+0000","last_change":"2024-03-20T07:58:26.128506+0000","last_active":"2024-03-20T07:58:26.443747+0000","last_peered":"2024-03-20T07:58:26.443747+0000","last_clean":"2024-03-20T07:58:26.443747+0000","last_became_active":"2024-03-20T07:58:26.128201+0000","last_became_peered":"2024-03-20T07:58:26.128201+0000","last_unstale":"2024-03-20T07:58:26.443747+0000","last_undegraded":"2024-03-20T07:58:26.443747+0000","last_fullsized":"2024-03-20T07:58:26.443747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"62'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298282+0000","last_change":"2024-03-20T07:58:25.767830+0000","last_active":"2024-03-20T07:58:26.298282+0000","last_peered":"2024-03-20T07:58:26.298282+0000","last_clean":"2024-03-20T07:58:26.298282+0000","last_became_active":"2024-03-20T07:58:25.767533+0000","last_became_peered":"2024-03-20T07:58:25.767533+0000","last_unstale":"2024-03-20T07:58:26.298282+0000","last_undegraded":"2024-03-20T07:58:26.298282+0000","last_fullsized":"2024-03-20T07:58:26.298282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689720+0000","last_change":"2024-03-20T07:58:26.140127+0000","last_active":"2024-03-20T07:58:26.689720+0000","last_peered":"2024-03-20T07:58:26.689720+0000","last_clean":"2024-03-20T07:58:26.689720+0000","last_became_active":"2024-03-20T07:58:26.139362+0000","last_became_peered":"2024-03-20T07:58:26.139362+0000","last_unstale":"2024-03-20T07:58:26.689720+0000","last_undegraded":"2024-03-20T07:58:26.689720+0000","last_fullsized":"2024-03-20T07:58:26.689720+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239407+0000","last_change":"2024-03-20T07:58:25.771772+0000","last_active":"2024-03-20T07:58:27.239407+0000","last_peered":"2024-03-20T07:58:27.239407+0000","last_clean":"2024-03-20T07:58:27.239407+0000","last_became_active":"2024-03-20T07:58:25.771499+0000","last_became_peered":"2024-03-20T07:58:25.771499+0000","last_unstale":"2024-03-20T07:58:27.239407+0000","last_undegraded":"2024-03-20T07:58:27.239407+0000","last_fullsized":"2024-03-20T07:58:27.239407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:19:45.833311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238700+0000","last_change":"2024-03-20T07:58:26.122549+0000","last_active":"2024-03-20T07:58:27.238700+0000","last_peered":"2024-03-20T07:58:27.238700+0000","last_clean":"2024-03-20T07:58:27.238700+0000","last_became_active":"2024-03-20T07:58:26.121780+0000","last_became_peered":"2024-03-20T07:58:26.121780+0000","last_unstale":"2024-03-20T07:58:27.238700+0000","last_undegraded":"2024-03-20T07:58:27.238700+0000","last_fullsized":"2024-03-20T07:58:27.238700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:37:01.164741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367787+0000","last_change":"2024-03-20T07:58:25.771839+0000","last_active":"2024-03-20T07:58:26.367787+0000","last_peered":"2024-03-20T07:58:26.367787+0000","last_clean":"2024-03-20T07:58:26.367787+0000","last_became_active":"2024-03-20T07:58:25.771464+0000","last_became_peered":"2024-03-20T07:58:25.771464+0000","last_unstale":"2024-03-20T07:58:26.367787+0000","last_undegraded":"2024-03-20T07:58:26.367787+0000","last_fullsized":"2024-03-20T07:58:26.367787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689668+0000","last_change":"2024-03-20T07:58:26.129094+0000","last_active":"2024-03-20T07:58:26.689668+0000","last_peered":"2024-03-20T07:58:26.689668+0000","last_clean":"2024-03-20T07:58:26.689668+0000","last_became_active":"2024-03-20T07:58:26.128675+0000","last_became_peered":"2024-03-20T07:58:26.128675+0000","last_unstale":"2024-03-20T07:58:26.689668+0000","last_undegraded":"2024-03-20T07:58:26.689668+0000","last_fullsized":"2024-03-20T07:58:26.689668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298244+0000","last_change":"2024-03-20T07:58:25.767586+0000","last_active":"2024-03-20T07:58:26.298244+0000","last_peered":"2024-03-20T07:58:26.298244+0000","last_clean":"2024-03-20T07:58:26.298244+0000","last_became_active":"2024-03-20T07:58:25.767268+0000","last_became_peered":"2024-03-20T07:58:25.767268+0000","last_unstale":"2024-03-20T07:58:26.298244+0000","last_undegraded":"2024-03-20T07:58:26.298244+0000","last_fullsized":"2024-03-20T07:58:26.298244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298232+0000","last_change":"2024-03-20T07:58:26.139925+0000","last_active":"2024-03-20T07:58:26.298232+0000","last_peered":"2024-03-20T07:58:26.298232+0000","last_clean":"2024-03-20T07:58:26.298232+0000","last_became_active":"2024-03-20T07:58:26.138286+0000","last_became_peered":"2024-03-20T07:58:26.138286+0000","last_unstale":"2024-03-20T07:58:26.298232+0000","last_undegraded":"2024-03-20T07:58:26.298232+0000","last_fullsized":"2024-03-20T07:58:26.298232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238203+0000","last_change":"2024-03-20T07:58:25.757217+0000","last_active":"2024-03-20T07:58:27.238203+0000","last_peered":"2024-03-20T07:58:27.238203+0000","last_clean":"2024-03-20T07:58:27.238203+0000","last_became_active":"2024-03-20T07:58:25.756879+0000","last_became_peered":"2024-03-20T07:58:25.756879+0000","last_unstale":"2024-03-20T07:58:27.238203+0000","last_undegraded":"2024-03-20T07:58:27.238203+0000","last_fullsized":"2024-03-20T07:58:27.238203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:28.214822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689646+0000","last_change":"2024-03-20T07:58:26.133140+0000","last_active":"2024-03-20T07:58:26.689646+0000","last_peered":"2024-03-20T07:58:26.689646+0000","last_clean":"2024-03-20T07:58:26.689646+0000","last_became_active":"2024-03-20T07:58:26.132630+0000","last_became_peered":"2024-03-20T07:58:26.132630+0000","last_unstale":"2024-03-20T07:58:26.689646+0000","last_undegraded":"2024-03-20T07:58:26.689646+0000","last_fullsized":"2024-03-20T07:58:26.689646+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.443774+0000","last_change":"2024-03-20T07:58:25.771322+0000","last_active":"2024-03-20T07:58:26.443774+0000","last_peered":"2024-03-20T07:58:26.443774+0000","last_clean":"2024-03-20T07:58:26.443774+0000","last_became_active":"2024-03-20T07:58:25.770984+0000","last_became_peered":"2024-03-20T07:58:25.770984+0000","last_unstale":"2024-03-20T07:58:26.443774+0000","last_undegraded":"2024-03-20T07:58:26.443774+0000","last_fullsized":"2024-03-20T07:58:26.443774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689762+0000","last_change":"2024-03-20T07:58:26.144254+0000","last_active":"2024-03-20T07:58:26.689762+0000","last_peered":"2024-03-20T07:58:26.689762+0000","last_clean":"2024-03-20T07:58:26.689762+0000","last_became_active":"2024-03-20T07:58:26.143683+0000","last_became_peered":"2024-03-20T07:58:26.143683+0000","last_unstale":"2024-03-20T07:58:26.689762+0000","last_undegraded":"2024-03-20T07:58:26.689762+0000","last_fullsized":"2024-03-20T07:58:26.689762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107428+0000","last_change":"2024-03-20T07:58:25.773911+0000","last_active":"2024-03-20T07:58:27.107428+0000","last_peered":"2024-03-20T07:58:27.107428+0000","last_clean":"2024-03-20T07:58:27.107428+0000","last_became_active":"2024-03-20T07:58:25.773526+0000","last_became_peered":"2024-03-20T07:58:25.773526+0000","last_unstale":"2024-03-20T07:58:27.107428+0000","last_undegraded":"2024-03-20T07:58:27.107428+0000","last_fullsized":"2024-03-20T07:58:27.107428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367896+0000","last_change":"2024-03-20T07:58:26.122909+0000","last_active":"2024-03-20T07:58:26.367896+0000","last_peered":"2024-03-20T07:58:26.367896+0000","last_clean":"2024-03-20T07:58:26.367896+0000","last_became_active":"2024-03-20T07:58:26.121608+0000","last_became_peered":"2024-03-20T07:58:26.121608+0000","last_unstale":"2024-03-20T07:58:26.367896+0000","last_undegraded":"2024-03-20T07:58:26.367896+0000","last_fullsized":"2024-03-20T07:58:26.367896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367765+0000","last_change":"2024-03-20T07:58:25.770569+0000","last_active":"2024-03-20T07:58:26.367765+0000","last_peered":"2024-03-20T07:58:26.367765+0000","last_clean":"2024-03-20T07:58:26.367765+0000","last_became_active":"2024-03-20T07:58:25.770298+0000","last_became_peered":"2024-03-20T07:58:25.770298+0000","last_unstale":"2024-03-20T07:58:26.367765+0000","last_undegraded":"2024-03-20T07:58:26.367765+0000","last_fullsized":"2024-03-20T07:58:26.367765+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.443736+0000","last_change":"2024-03-20T07:58:26.136200+0000","last_active":"2024-03-20T07:58:26.443736+0000","last_peered":"2024-03-20T07:58:26.443736+0000","last_clean":"2024-03-20T07:58:26.443736+0000","last_became_active":"2024-03-20T07:58:26.135909+0000","last_became_peered":"2024-03-20T07:58:26.135909+0000","last_unstale":"2024-03-20T07:58:26.443736+0000","last_undegraded":"2024-03-20T07:58:26.443736+0000","last_fullsized":"2024-03-20T07:58:26.443736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107104+0000","last_change":"2024-03-20T07:58:26.130079+0000","last_active":"2024-03-20T07:58:27.107104+0000","last_peered":"2024-03-20T07:58:27.107104+0000","last_clean":"2024-03-20T07:58:27.107104+0000","last_became_active":"2024-03-20T07:58:26.129798+0000","last_became_peered":"2024-03-20T07:58:26.129798+0000","last_unstale":"2024-03-20T07:58:27.107104+0000","last_undegraded":"2024-03-20T07:58:27.107104+0000","last_fullsized":"2024-03-20T07:58:27.107104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238173+0000","last_change":"2024-03-20T07:58:25.764736+0000","last_active":"2024-03-20T07:58:27.238173+0000","last_peered":"2024-03-20T07:58:27.238173+0000","last_clean":"2024-03-20T07:58:27.238173+0000","last_became_active":"2024-03-20T07:58:25.764390+0000","last_became_peered":"2024-03-20T07:58:25.764390+0000","last_unstale":"2024-03-20T07:58:27.238173+0000","last_undegraded":"2024-03-20T07:58:27.238173+0000","last_fullsized":"2024-03-20T07:58:27.238173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:27:58.881448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107032+0000","last_change":"2024-03-20T07:58:26.136285+0000","last_active":"2024-03-20T07:58:27.107032+0000","last_peered":"2024-03-20T07:58:27.107032+0000","last_clean":"2024-03-20T07:58:27.107032+0000","last_became_active":"2024-03-20T07:58:26.135969+0000","last_became_peered":"2024-03-20T07:58:26.135969+0000","last_unstale":"2024-03-20T07:58:27.107032+0000","last_undegraded":"2024-03-20T07:58:27.107032+0000","last_fullsized":"2024-03-20T07:58:27.107032+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:16.328342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367828+0000","last_change":"2024-03-20T07:58:25.766643+0000","last_active":"2024-03-20T07:58:26.367828+0000","last_peered":"2024-03-20T07:58:26.367828+0000","last_clean":"2024-03-20T07:58:26.367828+0000","last_became_active":"2024-03-20T07:58:25.766341+0000","last_became_peered":"2024-03-20T07:58:25.766341+0000","last_unstale":"2024-03-20T07:58:26.367828+0000","last_undegraded":"2024-03-20T07:58:26.367828+0000","last_fullsized":"2024-03-20T07:58:26.367828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"22'117","reported_seq":219,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.118075+0000","last_change":"2024-03-20T07:58:09.964395+0000","last_active":"2024-03-20T07:58:26.118075+0000","last_peered":"2024-03-20T07:58:26.118075+0000","last_clean":"2024-03-20T07:58:26.118075+0000","last_became_active":"2024-03-20T07:58:09.963927+0000","last_became_peered":"2024-03-20T07:58:09.963927+0000","last_unstale":"2024-03-20T07:58:26.118075+0000","last_undegraded":"2024-03-20T07:58:26.118075+0000","last_fullsized":"2024-03-20T07:58:26.118075+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238574+0000","last_change":"2024-03-20T07:58:26.125436+0000","last_active":"2024-03-20T07:58:27.238574+0000","last_peered":"2024-03-20T07:58:27.238574+0000","last_clean":"2024-03-20T07:58:27.238574+0000","last_became_active":"2024-03-20T07:58:26.125161+0000","last_became_peered":"2024-03-20T07:58:26.125161+0000","last_unstale":"2024-03-20T07:58:27.238574+0000","last_undegraded":"2024-03-20T07:58:27.238574+0000","last_fullsized":"2024-03-20T07:58:27.238574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:53:55.085832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689625+0000","last_change":"2024-03-20T07:58:25.779129+0000","last_active":"2024-03-20T07:58:26.689625+0000","last_peered":"2024-03-20T07:58:26.689625+0000","last_clean":"2024-03-20T07:58:26.689625+0000","last_became_active":"2024-03-20T07:58:25.778258+0000","last_became_peered":"2024-03-20T07:58:25.778258+0000","last_unstale":"2024-03-20T07:58:26.689625+0000","last_undegraded":"2024-03-20T07:58:26.689625+0000","last_fullsized":"2024-03-20T07:58:26.689625+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689739+0000","last_change":"2024-03-20T07:58:25.777973+0000","last_active":"2024-03-20T07:58:26.689739+0000","last_peered":"2024-03-20T07:58:26.689739+0000","last_clean":"2024-03-20T07:58:26.689739+0000","last_became_active":"2024-03-20T07:58:25.777666+0000","last_became_peered":"2024-03-20T07:58:25.777666+0000","last_unstale":"2024-03-20T07:58:26.689739+0000","last_undegraded":"2024-03-20T07:58:26.689739+0000","last_fullsized":"2024-03-20T07:58:26.689739+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107508+0000","last_change":"2024-03-20T07:58:26.126692+0000","last_active":"2024-03-20T07:58:27.107508+0000","last_peered":"2024-03-20T07:58:27.107508+0000","last_clean":"2024-03-20T07:58:27.107508+0000","last_became_active":"2024-03-20T07:58:26.126397+0000","last_became_peered":"2024-03-20T07:58:26.126397+0000","last_unstale":"2024-03-20T07:58:27.107508+0000","last_undegraded":"2024-03-20T07:58:27.107508+0000","last_fullsized":"2024-03-20T07:58:27.107508+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.231250+0000","last_change":"2024-03-20T07:58:25.769337+0000","last_active":"2024-03-20T07:58:27.231250+0000","last_peered":"2024-03-20T07:58:27.231250+0000","last_clean":"2024-03-20T07:58:27.231250+0000","last_became_active":"2024-03-20T07:58:25.769010+0000","last_became_peered":"2024-03-20T07:58:25.769010+0000","last_unstale":"2024-03-20T07:58:27.231250+0000","last_undegraded":"2024-03-20T07:58:27.231250+0000","last_fullsized":"2024-03-20T07:58:27.231250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107841+0000","last_change":"2024-03-20T07:58:26.118863+0000","last_active":"2024-03-20T07:58:27.107841+0000","last_peered":"2024-03-20T07:58:27.107841+0000","last_clean":"2024-03-20T07:58:27.107841+0000","last_became_active":"2024-03-20T07:58:26.118565+0000","last_became_peered":"2024-03-20T07:58:26.118565+0000","last_unstale":"2024-03-20T07:58:27.107841+0000","last_undegraded":"2024-03-20T07:58:27.107841+0000","last_fullsized":"2024-03-20T07:58:27.107841+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:49:33.981106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298335+0000","last_change":"2024-03-20T07:58:25.771097+0000","last_active":"2024-03-20T07:58:26.298335+0000","last_peered":"2024-03-20T07:58:26.298335+0000","last_clean":"2024-03-20T07:58:26.298335+0000","last_became_active":"2024-03-20T07:58:25.770737+0000","last_became_peered":"2024-03-20T07:58:25.770737+0000","last_unstale":"2024-03-20T07:58:26.298335+0000","last_undegraded":"2024-03-20T07:58:26.298335+0000","last_fullsized":"2024-03-20T07:58:26.298335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239163+0000","last_change":"2024-03-20T07:58:26.126230+0000","last_active":"2024-03-20T07:58:27.239163+0000","last_peered":"2024-03-20T07:58:27.239163+0000","last_clean":"2024-03-20T07:58:27.239163+0000","last_became_active":"2024-03-20T07:58:26.125980+0000","last_became_peered":"2024-03-20T07:58:26.125980+0000","last_unstale":"2024-03-20T07:58:27.239163+0000","last_undegraded":"2024-03-20T07:58:27.239163+0000","last_fullsized":"2024-03-20T07:58:27.239163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:18:03.971589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689602+0000","last_change":"2024-03-20T07:58:25.777961+0000","last_active":"2024-03-20T07:58:26.689602+0000","last_peered":"2024-03-20T07:58:26.689602+0000","last_clean":"2024-03-20T07:58:26.689602+0000","last_became_active":"2024-03-20T07:58:25.777649+0000","last_became_peered":"2024-03-20T07:58:25.777649+0000","last_unstale":"2024-03-20T07:58:26.689602+0000","last_undegraded":"2024-03-20T07:58:26.689602+0000","last_fullsized":"2024-03-20T07:58:26.689602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.443708+0000","last_change":"2024-03-20T07:58:26.120305+0000","last_active":"2024-03-20T07:58:26.443708+0000","last_peered":"2024-03-20T07:58:26.443708+0000","last_clean":"2024-03-20T07:58:26.443708+0000","last_became_active":"2024-03-20T07:58:26.119966+0000","last_became_peered":"2024-03-20T07:58:26.119966+0000","last_unstale":"2024-03-20T07:58:26.443708+0000","last_undegraded":"2024-03-20T07:58:26.443708+0000","last_fullsized":"2024-03-20T07:58:26.443708+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107099+0000","last_change":"2024-03-20T07:58:25.765548+0000","last_active":"2024-03-20T07:58:27.107099+0000","last_peered":"2024-03-20T07:58:27.107099+0000","last_clean":"2024-03-20T07:58:27.107099+0000","last_became_active":"2024-03-20T07:58:25.765273+0000","last_became_peered":"2024-03-20T07:58:25.765273+0000","last_unstale":"2024-03-20T07:58:27.107099+0000","last_undegraded":"2024-03-20T07:58:27.107099+0000","last_fullsized":"2024-03-20T07:58:27.107099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.106781+0000","last_change":"2024-03-20T07:58:26.117669+0000","last_active":"2024-03-20T07:58:27.106781+0000","last_peered":"2024-03-20T07:58:27.106781+0000","last_clean":"2024-03-20T07:58:27.106781+0000","last_became_active":"2024-03-20T07:58:26.117376+0000","last_became_peered":"2024-03-20T07:58:26.117376+0000","last_unstale":"2024-03-20T07:58:27.106781+0000","last_undegraded":"2024-03-20T07:58:27.106781+0000","last_fullsized":"2024-03-20T07:58:27.106781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:42:02.274133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107547+0000","last_change":"2024-03-20T07:58:25.773811+0000","last_active":"2024-03-20T07:58:27.107547+0000","last_peered":"2024-03-20T07:58:27.107547+0000","last_clean":"2024-03-20T07:58:27.107547+0000","last_became_active":"2024-03-20T07:58:25.773520+0000","last_became_peered":"2024-03-20T07:58:25.773520+0000","last_unstale":"2024-03-20T07:58:27.107547+0000","last_undegraded":"2024-03-20T07:58:27.107547+0000","last_fullsized":"2024-03-20T07:58:27.107547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107715+0000","last_change":"2024-03-20T07:58:26.123160+0000","last_active":"2024-03-20T07:58:27.107715+0000","last_peered":"2024-03-20T07:58:27.107715+0000","last_clean":"2024-03-20T07:58:27.107715+0000","last_became_active":"2024-03-20T07:58:26.122680+0000","last_became_peered":"2024-03-20T07:58:26.122680+0000","last_unstale":"2024-03-20T07:58:27.107715+0000","last_undegraded":"2024-03-20T07:58:27.107715+0000","last_fullsized":"2024-03-20T07:58:27.107715+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:56.675972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689657+0000","last_change":"2024-03-20T07:58:25.776066+0000","last_active":"2024-03-20T07:58:26.689657+0000","last_peered":"2024-03-20T07:58:26.689657+0000","last_clean":"2024-03-20T07:58:26.689657+0000","last_became_active":"2024-03-20T07:58:25.775708+0000","last_became_peered":"2024-03-20T07:58:25.775708+0000","last_unstale":"2024-03-20T07:58:26.689657+0000","last_undegraded":"2024-03-20T07:58:26.689657+0000","last_fullsized":"2024-03-20T07:58:26.689657+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:36:20.600637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298258+0000","last_change":"2024-03-20T07:58:26.134067+0000","last_active":"2024-03-20T07:58:26.298258+0000","last_peered":"2024-03-20T07:58:26.298258+0000","last_clean":"2024-03-20T07:58:26.298258+0000","last_became_active":"2024-03-20T07:58:26.133784+0000","last_became_peered":"2024-03-20T07:58:26.133784+0000","last_unstale":"2024-03-20T07:58:26.298258+0000","last_undegraded":"2024-03-20T07:58:26.298258+0000","last_fullsized":"2024-03-20T07:58:26.298258+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238423+0000","last_change":"2024-03-20T07:58:25.763015+0000","last_active":"2024-03-20T07:58:27.238423+0000","last_peered":"2024-03-20T07:58:27.238423+0000","last_clean":"2024-03-20T07:58:27.238423+0000","last_became_active":"2024-03-20T07:58:25.762682+0000","last_became_peered":"2024-03-20T07:58:25.762682+0000","last_unstale":"2024-03-20T07:58:27.238423+0000","last_undegraded":"2024-03-20T07:58:27.238423+0000","last_fullsized":"2024-03-20T07:58:27.238423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:21:45.137669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689730+0000","last_change":"2024-03-20T07:58:26.142445+0000","last_active":"2024-03-20T07:58:26.689730+0000","last_peered":"2024-03-20T07:58:26.689730+0000","last_clean":"2024-03-20T07:58:26.689730+0000","last_became_active":"2024-03-20T07:58:26.142148+0000","last_became_peered":"2024-03-20T07:58:26.142148+0000","last_unstale":"2024-03-20T07:58:26.689730+0000","last_undegraded":"2024-03-20T07:58:26.689730+0000","last_fullsized":"2024-03-20T07:58:26.689730+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"62'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107988+0000","last_change":"2024-03-20T07:58:25.766427+0000","last_active":"2024-03-20T07:58:27.107988+0000","last_peered":"2024-03-20T07:58:27.107988+0000","last_clean":"2024-03-20T07:58:27.107988+0000","last_became_active":"2024-03-20T07:58:25.766111+0000","last_became_peered":"2024-03-20T07:58:25.766111+0000","last_unstale":"2024-03-20T07:58:27.107988+0000","last_undegraded":"2024-03-20T07:58:27.107988+0000","last_fullsized":"2024-03-20T07:58:27.107988+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367848+0000","last_change":"2024-03-20T07:58:26.135339+0000","last_active":"2024-03-20T07:58:26.367848+0000","last_peered":"2024-03-20T07:58:26.367848+0000","last_clean":"2024-03-20T07:58:26.367848+0000","last_became_active":"2024-03-20T07:58:26.134903+0000","last_became_peered":"2024-03-20T07:58:26.134903+0000","last_unstale":"2024-03-20T07:58:26.367848+0000","last_undegraded":"2024-03-20T07:58:26.367848+0000","last_fullsized":"2024-03-20T07:58:26.367848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"62'2","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238866+0000","last_change":"2024-03-20T07:58:25.766233+0000","last_active":"2024-03-20T07:58:27.238866+0000","last_peered":"2024-03-20T07:58:27.238866+0000","last_clean":"2024-03-20T07:58:27.238866+0000","last_became_active":"2024-03-20T07:58:25.765948+0000","last_became_peered":"2024-03-20T07:58:25.765948+0000","last_unstale":"2024-03-20T07:58:27.238866+0000","last_undegraded":"2024-03-20T07:58:27.238866+0000","last_fullsized":"2024-03-20T07:58:27.238866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:34:01.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.689772+0000","last_change":"2024-03-20T07:58:26.144008+0000","last_active":"2024-03-20T07:58:26.689772+0000","last_peered":"2024-03-20T07:58:26.689772+0000","last_clean":"2024-03-20T07:58:26.689772+0000","last_became_active":"2024-03-20T07:58:26.143696+0000","last_became_peered":"2024-03-20T07:58:26.143696+0000","last_unstale":"2024-03-20T07:58:26.689772+0000","last_undegraded":"2024-03-20T07:58:26.689772+0000","last_fullsized":"2024-03-20T07:58:26.689772+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298216+0000","last_change":"2024-03-20T07:58:26.138096+0000","last_active":"2024-03-20T07:58:26.298216+0000","last_peered":"2024-03-20T07:58:26.298216+0000","last_clean":"2024-03-20T07:58:26.298216+0000","last_became_active":"2024-03-20T07:58:26.137804+0000","last_became_peered":"2024-03-20T07:58:26.137804+0000","last_unstale":"2024-03-20T07:58:26.298216+0000","last_undegraded":"2024-03-20T07:58:26.298216+0000","last_fullsized":"2024-03-20T07:58:26.298216+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.238668+0000","last_change":"2024-03-20T07:58:26.126085+0000","last_active":"2024-03-20T07:58:27.238668+0000","last_peered":"2024-03-20T07:58:27.238668+0000","last_clean":"2024-03-20T07:58:27.238668+0000","last_became_active":"2024-03-20T07:58:26.125793+0000","last_became_peered":"2024-03-20T07:58:26.125793+0000","last_unstale":"2024-03-20T07:58:27.238668+0000","last_undegraded":"2024-03-20T07:58:27.238668+0000","last_fullsized":"2024-03-20T07:58:27.238668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:11:45.470755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367797+0000","last_change":"2024-03-20T07:58:26.125004+0000","last_active":"2024-03-20T07:58:26.367797+0000","last_peered":"2024-03-20T07:58:26.367797+0000","last_clean":"2024-03-20T07:58:26.367797+0000","last_became_active":"2024-03-20T07:58:26.124689+0000","last_became_peered":"2024-03-20T07:58:26.124689+0000","last_unstale":"2024-03-20T07:58:26.367797+0000","last_undegraded":"2024-03-20T07:58:26.367797+0000","last_fullsized":"2024-03-20T07:58:26.367797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239208+0000","last_change":"2024-03-20T07:58:26.120048+0000","last_active":"2024-03-20T07:58:27.239208+0000","last_peered":"2024-03-20T07:58:27.239208+0000","last_clean":"2024-03-20T07:58:27.239208+0000","last_became_active":"2024-03-20T07:58:26.119755+0000","last_became_peered":"2024-03-20T07:58:26.119755+0000","last_unstale":"2024-03-20T07:58:27.239208+0000","last_undegraded":"2024-03-20T07:58:27.239208+0000","last_fullsized":"2024-03-20T07:58:27.239208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:39:07.689571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107374+0000","last_change":"2024-03-20T07:58:26.120523+0000","last_active":"2024-03-20T07:58:27.107374+0000","last_peered":"2024-03-20T07:58:27.107374+0000","last_clean":"2024-03-20T07:58:27.107374+0000","last_became_active":"2024-03-20T07:58:26.120188+0000","last_became_peered":"2024-03-20T07:58:26.120188+0000","last_unstale":"2024-03-20T07:58:27.107374+0000","last_undegraded":"2024-03-20T07:58:27.107374+0000","last_fullsized":"2024-03-20T07:58:27.107374+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:07:20.052192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107012+0000","last_change":"2024-03-20T07:58:26.126113+0000","last_active":"2024-03-20T07:58:27.107012+0000","last_peered":"2024-03-20T07:58:27.107012+0000","last_clean":"2024-03-20T07:58:27.107012+0000","last_became_active":"2024-03-20T07:58:26.122635+0000","last_became_peered":"2024-03-20T07:58:26.122635+0000","last_unstale":"2024-03-20T07:58:27.107012+0000","last_undegraded":"2024-03-20T07:58:27.107012+0000","last_fullsized":"2024-03-20T07:58:27.107012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298324+0000","last_change":"2024-03-20T07:58:26.141379+0000","last_active":"2024-03-20T07:58:26.298324+0000","last_peered":"2024-03-20T07:58:26.298324+0000","last_clean":"2024-03-20T07:58:26.298324+0000","last_became_active":"2024-03-20T07:58:26.141099+0000","last_became_peered":"2024-03-20T07:58:26.141099+0000","last_unstale":"2024-03-20T07:58:26.298324+0000","last_undegraded":"2024-03-20T07:58:26.298324+0000","last_fullsized":"2024-03-20T07:58:26.298324+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.107930+0000","last_change":"2024-03-20T07:58:26.121389+0000","last_active":"2024-03-20T07:58:27.107930+0000","last_peered":"2024-03-20T07:58:27.107930+0000","last_clean":"2024-03-20T07:58:27.107930+0000","last_became_active":"2024-03-20T07:58:26.121094+0000","last_became_peered":"2024-03-20T07:58:26.121094+0000","last_unstale":"2024-03-20T07:58:27.107930+0000","last_undegraded":"2024-03-20T07:58:27.107930+0000","last_fullsized":"2024-03-20T07:58:27.107930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:08:53.389656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367752+0000","last_change":"2024-03-20T07:58:26.137638+0000","last_active":"2024-03-20T07:58:26.367752+0000","last_peered":"2024-03-20T07:58:26.367752+0000","last_clean":"2024-03-20T07:58:26.367752+0000","last_became_active":"2024-03-20T07:58:26.137344+0000","last_became_peered":"2024-03-20T07:58:26.137344+0000","last_unstale":"2024-03-20T07:58:26.367752+0000","last_undegraded":"2024-03-20T07:58:26.367752+0000","last_fullsized":"2024-03-20T07:58:26.367752+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.106949+0000","last_change":"2024-03-20T07:58:26.122341+0000","last_active":"2024-03-20T07:58:27.106949+0000","last_peered":"2024-03-20T07:58:27.106949+0000","last_clean":"2024-03-20T07:58:27.106949+0000","last_became_active":"2024-03-20T07:58:26.122003+0000","last_became_peered":"2024-03-20T07:58:26.122003+0000","last_unstale":"2024-03-20T07:58:27.106949+0000","last_undegraded":"2024-03-20T07:58:27.106949+0000","last_fullsized":"2024-03-20T07:58:27.106949+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:20.158675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.367806+0000","last_change":"2024-03-20T07:58:26.137117+0000","last_active":"2024-03-20T07:58:26.367806+0000","last_peered":"2024-03-20T07:58:26.367806+0000","last_clean":"2024-03-20T07:58:26.367806+0000","last_became_active":"2024-03-20T07:58:26.136135+0000","last_became_peered":"2024-03-20T07:58:26.136135+0000","last_unstale":"2024-03-20T07:58:26.367806+0000","last_undegraded":"2024-03-20T07:58:26.367806+0000","last_fullsized":"2024-03-20T07:58:26.367806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-20T07:58:27.239107+0000","last_change":"2024-03-20T07:58:26.119480+0000","last_active":"2024-03-20T07:58:27.239107+0000","last_peered":"2024-03-20T07:58:27.239107+0000","last_clean":"2024-03-20T07:58:27.239107+0000","last_became_active":"2024-03-20T07:58:26.119047+0000","last_became_peered":"2024-03-20T07:58:26.119047+0000","last_unstale":"2024-03-20T07:58:27.239107+0000","last_undegraded":"2024-03-20T07:58:27.239107+0000","last_fullsized":"2024-03-20T07:58:27.239107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:09:31.918661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-20T07:58:26.298292+0000","last_change":"2024-03-20T07:58:26.133160+0000","last_active":"2024-03-20T07:58:26.298292+0000","last_peered":"2024-03-20T07:58:26.298292+0000","last_clean":"2024-03-20T07:58:26.298292+0000","last_became_active":"2024-03-20T07:58:26.132793+0000","last_became_peered":"2024-03-20T07:58:26.132793+0000","last_unstale":"2024-03-20T07:58:26.298292+0000","last_undegraded":"2024-03-20T07:58:26.298292+0000","last_fullsized":"2024-03-20T07:58:26.298292+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":7086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201287,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1232896,"data_stored":853244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462924,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1232896,"data_stored":853873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717508,"statfs":{"total":95995035648,"available":95966728192,"internally_reserved":0,"allocated":638976,"data_stored":262432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.33100000000000002,"15min":0.32300000000000001},"min":{"1min":0.26200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.35599999999999998,"15min":0.34399999999999997},"min":{"1min":0.311,"5min":0.219,"15min":0.219},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.371}]},{"osd":2,"last update":"Wed Mar 20 07:58:07 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.39300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29999999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.308,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.308}]},{"osd":3,"last update":"Wed Mar 20 07:58:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42299999999999999,"15min":0.42199999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.57199999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:57:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.47999999999999998}]},{"osd":5,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]}]},{"osd":0,"up_from":8,"seq":34359738411,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28232,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716920,"statfs":{"total":95995035648,"available":95966126080,"internally_reserved":0,"allocated":1241088,"data_stored":853368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 20 07:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.26300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.32200000000000001,"15min":0.312},"min":{"1min":0.26300000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.314}]},{"osd":2,"last update":"Wed Mar 20 07:58:06 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.30099999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.40200000000000002,"15min":0.39700000000000002},"min":{"1min":0.31,"5min":0.254,"15min":0.254},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.38900000000000001}]},{"osd":3,"last update":"Wed Mar 20 07:58:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41299999999999998,"15min":0.41499999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46899999999999997,"15min":0.46999999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46400000000000002}]},{"osd":4,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.36399999999999999}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411360,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27648,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717504,"statfs":{"total":95995035648,"available":95966724096,"internally_reserved":0,"allocated":630784,"data_stored":262272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.39500000000000002},"min":{"1min":0.36799999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.40500000000000003}]},{"osd":1,"last update":"Wed Mar 20 07:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.33900000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.38600000000000001,"15min":0.38100000000000001},"min":{"1min":0.33000000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.52000000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.32600000000000001}]},{"osd":3,"last update":"Wed Mar 20 07:58:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.54500000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.45000000000000001,"15min":0.45300000000000001},"min":{"1min":0.307,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.36899999999999999}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.60599999999999998}]},{"osd":5,"last update":"Wed Mar 20 07:58:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149722,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27660,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717492,"statfs":{"total":95995035648,"available":95966711808,"internally_reserved":0,"allocated":651264,"data_stored":263940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.28100000000000003}]},{"osd":1,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.374}]},{"osd":2,"last update":"Wed Mar 20 07:57:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.317}]},{"osd":4,"last update":"Wed Mar 20 07:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.68200000000000005}]},{"osd":5,"last update":"Wed Mar 20 07:58:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986198,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27660,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717492,"statfs":{"total":95995035648,"available":95966711808,"internally_reserved":0,"allocated":647168,"data_stored":262980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.45700000000000002}]},{"osd":1,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48899999999999999}]},{"osd":2,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.505},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Mar 20 07:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.433}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724561,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27656,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717496,"statfs":{"total":95995035648,"available":95966715904,"internally_reserved":0,"allocated":647168,"data_stored":263926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.442}]},{"osd":1,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.436},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47399999999999998}]},{"osd":3,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.32000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T07:58:33.764 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-20T07:58:33.765 INFO:tasks.cephadm:Adding prometheus.a on smithi196 2024-03-20T07:58:33.765 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch apply prometheus '1;smithi196=a' 2024-03-20T07:58:33.978 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:58:34.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:34 smithi196 ceph-mon[31033]: pgmap v197: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-20T07:58:34.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:34 smithi118 ceph-mon[38981]: pgmap v197: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-20T07:58:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:34 smithi118 ceph-mon[27391]: pgmap v197: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-03-20T07:58:35.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:35 smithi196 ceph-mon[31033]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:35 smithi118 ceph-mon[27391]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:35.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:35 smithi118 ceph-mon[38981]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T07:58:35.935 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled prometheus update... 2024-03-20T07:58:36.574 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:36 smithi196 ceph-mon[31033]: pgmap v198: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-03-20T07:58:36.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:36.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:36.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:36.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:36.575 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:36 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:36.594 DEBUG:teuthology.orchestra.run.smithi196:prometheus.a> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service 2024-03-20T07:58:36.596 INFO:tasks.cephadm:Adding node-exporter.a on smithi118 2024-03-20T07:58:36.596 INFO:tasks.cephadm:Adding node-exporter.b on smithi196 2024-03-20T07:58:36.596 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch apply node-exporter '2;smithi118=a;smithi196=b' 2024-03-20T07:58:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[38981]: pgmap v198: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-03-20T07:58:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[27391]: pgmap v198: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 4 op/s 2024-03-20T07:58:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:58:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:58:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:58:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:36 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:36.863 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:58:37.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:37 smithi196 ceph-mon[31033]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:37.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:37 smithi196 ceph-mon[31033]: Saving service prometheus spec with placement smithi196=a;count:1 2024-03-20T07:58:37.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:37 smithi196 ceph-mon[31033]: Deploying daemon prometheus.a on smithi196 2024-03-20T07:58:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:37 smithi118 ceph-mon[27391]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:37 smithi118 ceph-mon[27391]: Saving service prometheus spec with placement smithi196=a;count:1 2024-03-20T07:58:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:37 smithi118 ceph-mon[27391]: Deploying daemon prometheus.a on smithi196 2024-03-20T07:58:37.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:37 smithi118 ceph-mon[38981]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:37.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:37 smithi118 ceph-mon[38981]: Saving service prometheus spec with placement smithi196=a;count:1 2024-03-20T07:58:37.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:37 smithi118 ceph-mon[38981]: Deploying daemon prometheus.a on smithi196 2024-03-20T07:58:38.560 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:38 smithi196 ceph-mon[31033]: pgmap v199: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1010 B/s wr, 2 op/s 2024-03-20T07:58:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:38 smithi118 ceph-mon[27391]: pgmap v199: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1010 B/s wr, 2 op/s 2024-03-20T07:58:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:38 smithi118 ceph-mon[38981]: pgmap v199: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1010 B/s wr, 2 op/s 2024-03-20T07:58:39.219 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled node-exporter update... 2024-03-20T07:58:40.236 DEBUG:teuthology.orchestra.run.smithi118:node-exporter.a> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.a.service 2024-03-20T07:58:40.239 DEBUG:teuthology.orchestra.run.smithi196:node-exporter.b> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.b.service 2024-03-20T07:58:40.241 INFO:tasks.cephadm:Adding alertmanager.a on smithi118 2024-03-20T07:58:40.242 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch apply alertmanager '1;smithi118=a' 2024-03-20T07:58:40.521 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:58:40.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:40 smithi196 ceph-mon[31033]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi118=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:40.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:40 smithi196 ceph-mon[31033]: Saving service node-exporter spec with placement smithi118=a;smithi196=b;count:2 2024-03-20T07:58:40.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:40 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:40.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:40 smithi196 ceph-mon[31033]: pgmap v200: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 682 B/s wr, 1 op/s 2024-03-20T07:58:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:40 smithi118 ceph-mon[27391]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi118=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:40 smithi118 ceph-mon[27391]: Saving service node-exporter spec with placement smithi118=a;smithi196=b;count:2 2024-03-20T07:58:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:40 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:40 smithi118 ceph-mon[27391]: pgmap v200: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 682 B/s wr, 1 op/s 2024-03-20T07:58:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:40 smithi118 ceph-mon[38981]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi118=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:40 smithi118 ceph-mon[38981]: Saving service node-exporter spec with placement smithi118=a;smithi196=b;count:2 2024-03-20T07:58:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:40 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:40 smithi118 ceph-mon[38981]: pgmap v200: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 682 B/s wr, 1 op/s 2024-03-20T07:58:42.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:41 smithi196 ceph-mon[31033]: pgmap v201: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T07:58:42.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:41 smithi118 ceph-mon[27391]: pgmap v201: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T07:58:42.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:41 smithi118 ceph-mon[38981]: pgmap v201: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T07:58:44.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:44 smithi196 ceph-mon[31033]: pgmap v202: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T07:58:44.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:44 smithi118 ceph-mon[27391]: pgmap v202: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T07:58:44.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:44 smithi118 ceph-mon[38981]: pgmap v202: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T07:58:45.505 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled alertmanager update... 2024-03-20T07:58:45.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:45 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-03-20T07:58:45.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:45 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [0, 2]}]: dispatch 2024-03-20T07:58:45.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:45 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1, 5, 2]}]: dispatch 2024-03-20T07:58:45.578 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:45 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [0, 1, 4, 2]}]: dispatch 2024-03-20T07:58:45.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-03-20T07:58:45.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [0, 2]}]: dispatch 2024-03-20T07:58:45.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1, 5, 2]}]: dispatch 2024-03-20T07:58:45.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:45 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [0, 1, 4, 2]}]: dispatch 2024-03-20T07:58:45.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:45 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-03-20T07:58:45.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:45 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [0, 2]}]: dispatch 2024-03-20T07:58:45.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:45 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1, 5, 2]}]: dispatch 2024-03-20T07:58:45.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:45 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [0, 1, 4, 2]}]: dispatch 2024-03-20T07:58:46.116 DEBUG:teuthology.orchestra.run.smithi118:alertmanager.a> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@alertmanager.a.service 2024-03-20T07:58:46.119 INFO:tasks.cephadm:Adding grafana.a on smithi196 2024-03-20T07:58:46.119 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph orch apply grafana '1;smithi196=a' 2024-03-20T07:58:46.371 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:46 smithi196 ceph-mon[31033]: pgmap v203: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:46.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-03-20T07:58:46.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [0, 2]}]': finished 2024-03-20T07:58:46.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1, 5, 2]}]': finished 2024-03-20T07:58:46.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [0, 1, 4, 2]}]': finished 2024-03-20T07:58:46.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:46 smithi196 ceph-mon[31033]: osdmap e64: 8 total, 8 up, 8 in 2024-03-20T07:58:46.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:46 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[27391]: pgmap v203: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-03-20T07:58:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [0, 2]}]': finished 2024-03-20T07:58:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1, 5, 2]}]': finished 2024-03-20T07:58:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [0, 1, 4, 2]}]': finished 2024-03-20T07:58:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[27391]: osdmap e64: 8 total, 8 up, 8 in 2024-03-20T07:58:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[38981]: pgmap v203: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-03-20T07:58:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [0, 2]}]': finished 2024-03-20T07:58:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 1, 5, 2]}]': finished 2024-03-20T07:58:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [0, 1, 4, 2]}]': finished 2024-03-20T07:58:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[38981]: osdmap e64: 8 total, 8 up, 8 in 2024-03-20T07:58:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:46 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:47.033 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:58:47.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:47 smithi196 ceph-mon[31033]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi118=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:47.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:47 smithi196 ceph-mon[31033]: Saving service alertmanager spec with placement smithi118=a;count:1 2024-03-20T07:58:47.404 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:47 smithi196 ceph-mon[31033]: osdmap e65: 8 total, 8 up, 8 in 2024-03-20T07:58:47.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:47 smithi118 ceph-mon[27391]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi118=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:47.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:47 smithi118 ceph-mon[27391]: Saving service alertmanager spec with placement smithi118=a;count:1 2024-03-20T07:58:47.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:47 smithi118 ceph-mon[27391]: osdmap e65: 8 total, 8 up, 8 in 2024-03-20T07:58:47.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:47 smithi118 ceph-mon[38981]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi118=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:47.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:47 smithi118 ceph-mon[38981]: Saving service alertmanager spec with placement smithi118=a;count:1 2024-03-20T07:58:47.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:47 smithi118 ceph-mon[38981]: osdmap e65: 8 total, 8 up, 8 in 2024-03-20T07:58:48.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:48 smithi196 ceph-mon[31033]: pgmap v206: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:48 smithi118 ceph-mon[27391]: pgmap v206: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:48.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:48 smithi118 ceph-mon[38981]: pgmap v206: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:58:50.572 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled grafana update... 2024-03-20T07:58:50.572 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:50 smithi196 ceph-mon[31033]: pgmap v207: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-03-20T07:58:50.572 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:50 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:50 smithi118 ceph-mon[27391]: pgmap v207: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-03-20T07:58:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:50 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:50.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:50 smithi118 ceph-mon[38981]: pgmap v207: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-03-20T07:58:50.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:50 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:51.281 DEBUG:teuthology.orchestra.run.smithi196:grafana.a> sudo journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@grafana.a.service 2024-03-20T07:58:51.283 INFO:tasks.cephadm:Setting up client nodes... 2024-03-20T07:58:51.283 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-20T07:58:51.498 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:58:51.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-mon[31033]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:51.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-mon[31033]: Saving service grafana spec with placement smithi196=a;count:1 2024-03-20T07:58:51.619 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:51 smithi118 ceph-mon[27391]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:51.619 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:51 smithi118 ceph-mon[27391]: Saving service grafana spec with placement smithi196=a;count:1 2024-03-20T07:58:51.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:51 smithi118 ceph-mon[38981]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T07:58:51.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:51 smithi118 ceph-mon[38981]: Saving service grafana spec with placement smithi196=a;count:1 2024-03-20T07:58:51.800 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 podman[60527]: 2024-03-20 07:58:51.77762658 +0000 UTC m=+0.353342049 container start 265fb70d6aa0fbb553dedc967a48b1104f4555be7ed0607eadebc87ffc21dae7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T07:58:52.141 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.801Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.801Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.801Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi196 (none))" 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.801Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.801Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.803Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.803Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.804Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.804Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.806Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.806Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.596µs 2024-03-20T07:58:52.142 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.806Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.807Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.807Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=26.078µs wal_replay_duration=208.477µs wbl_replay_duration=125ns total_replay_duration=259.319µs 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.807Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.807Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.807Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.824Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.907949ms db_storage=909ns remote_storage=1.081µs web_handler=312ns query_engine=500ns scrape=377.703µs scrape_sd=70.546µs notify=708ns notify_sd=1.252µs rules=16.257674ms tracing=6.431µs 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.824Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T07:58:51.824Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 bash[60527]: 265fb70d6aa0fbb553dedc967a48b1104f4555be7ed0607eadebc87ffc21dae7 2024-03-20T07:58:52.143 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 07:58:51 smithi196 systemd[1]: Started Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:58:52.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:52 smithi196 ceph-mon[31033]: pgmap v208: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-03-20T07:58:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:52 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:52 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:52 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:52 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-20T07:58:52.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[38981]: pgmap v208: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-03-20T07:58:52.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-20T07:58:52.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[27391]: pgmap v208: 97 pgs: 97 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-03-20T07:58:52.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' 2024-03-20T07:58:52.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:52 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-20T07:58:53.156 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ignoring --setuser ceph since I am not root 2024-03-20T07:58:53.156 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ignoring --setgroup ceph since I am not root 2024-03-20T07:58:53.388 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: ignoring --setuser ceph since I am not root 2024-03-20T07:58:53.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: ignoring --setgroup ceph since I am not root 2024-03-20T07:58:53.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-mgr[33166]: -- 172.21.15.196:0/3428678787 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561c5fad81a0 con 0x561c5faae800 2024-03-20T07:58:53.712 INFO:teuthology.orchestra.run.smithi118.stdout:[client.0] 2024-03-20T07:58:53.712 INFO:teuthology.orchestra.run.smithi118.stdout: key = AQA9l/plgSdGKhAAUhyQjNMHlqNxZHFI65O2LA== 2024-03-20T07:58:53.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:53.549+0000 7f084ad21200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T07:58:53.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:53.640+0000 7f084ad21200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T07:58:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:53.571+0000 7f8037471200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T07:58:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:53.664+0000 7f8037471200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T07:58:54.269 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-mon[31033]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-20T07:58:54.270 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-mon[31033]: mgrmap e16: y(active, since 5m), standbys: x 2024-03-20T07:58:54.270 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/4188179079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:54.270 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:53 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/4188179079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-20T07:58:54.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-mon[27391]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-20T07:58:54.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-mon[27391]: mgrmap e16: y(active, since 5m), standbys: x 2024-03-20T07:58:54.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/4188179079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:54.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/4188179079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-20T07:58:54.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-mon[38981]: from='mgr.14152 172.21.15.118:0/2108325283' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-20T07:58:54.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-mon[38981]: mgrmap e16: y(active, since 5m), standbys: x 2024-03-20T07:58:54.309 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/4188179079' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:54.309 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:53 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/4188179079' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-20T07:58:54.594 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:54.269+0000 7f084ad21200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T07:58:54.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:54.359+0000 7f084ad21200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T07:58:54.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:54.500+0000 7f084ad21200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T07:58:54.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:54.594+0000 7f084ad21200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T07:58:54.638 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:54.308+0000 7f8037471200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T07:58:54.638 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:54.400+0000 7f8037471200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T07:58:54.638 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:54.543+0000 7f8037471200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T07:58:54.638 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:54.638+0000 7f8037471200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T07:58:54.671 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T07:58:54.671 DEBUG:teuthology.orchestra.run.smithi118:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-20T07:58:54.671 DEBUG:teuthology.orchestra.run.smithi118:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-20T07:58:54.714 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-20T07:58:54.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:54.677+0000 7f084ad21200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T07:58:54.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:54.728+0000 7f8037471200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T07:58:54.927 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.b/config 2024-03-20T07:58:55.214 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:54.957+0000 7f084ad21200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T07:58:55.214 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:55.067+0000 7f084ad21200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T07:58:55.214 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:55.156+0000 7f084ad21200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T07:58:55.280 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:54.993+0000 7f8037471200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T07:58:55.280 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:55.101+0000 7f8037471200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T07:58:55.280 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:55.187+0000 7f8037471200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T07:58:55.280 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:55.281+0000 7f8037471200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T07:58:55.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:55.269+0000 7f084ad21200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T07:58:55.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:55.450+0000 7f084ad21200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T07:58:55.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:55.457+0000 7f8037471200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T07:58:56.574 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:56.435+0000 7f084ad21200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T07:58:56.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:56.406+0000 7f8037471200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T07:58:56.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:56.536+0000 7f8037471200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T07:58:56.845 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-03-20T07:58:56.846 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQBAl/plXahOMhAAxChVOYZKssuPHgWebhC6BA== 2024-03-20T07:58:56.846 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:56.574+0000 7f084ad21200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T07:58:57.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:56 smithi196 ceph-mon[31033]: from='client.? 172.21.15.196:0/1457515709' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:57.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:56 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:57.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:56 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-20T07:58:57.139 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:56.920+0000 7f084ad21200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T07:58:57.139 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:57.012+0000 7f084ad21200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T07:58:57.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-mon[27391]: from='client.? 172.21.15.196:0/1457515709' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:57.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:57.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-20T07:58:57.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-mon[38981]: from='client.? 172.21.15.196:0/1457515709' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:57.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-20T07:58:57.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-20T07:58:57.154 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:56.870+0000 7f8037471200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T07:58:57.155 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:56.959+0000 7f8037471200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T07:58:57.155 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:57.154+0000 7f8037471200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:58:57.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:57.340+0000 7f8037471200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T07:58:57.519 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:57.213+0000 7f084ad21200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:58:57.519 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:57.402+0000 7f084ad21200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T07:58:57.541 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-20T07:58:57.541 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-20T07:58:57.541 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-20T07:58:57.585 INFO:tasks.cephadm:Waiting for OSDs to come up 2024-03-20T07:58:57.586 DEBUG:teuthology.orchestra.run.smithi118:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-20T07:58:57.815 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:57.700+0000 7f084ad21200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:58:57.870 INFO:teuthology.misc.health.smithi118.stdout: 2024-03-20T07:58:57.870 INFO:teuthology.misc.health.smithi118.stdout:{"epoch":65,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:58:46.299122+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":7,"to":2}]},{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1},{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":0,"to":1},{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:58:57.885 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-20T07:58:57.885 INFO:tasks.cephadm:Creating RBD pool 2024-03-20T07:58:57.886 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- sudo ceph --cluster ceph osd pool create rbd 8 2024-03-20T07:58:57.900 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:57.627+0000 7f8037471200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T07:58:58.097 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:58:58.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:57 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/4185219044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:58.138 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:57.977+0000 7f084ad21200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T07:58:58.165 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:57 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/4185219044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:58.165 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:57.901+0000 7f8037471200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T07:58:58.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:57 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/4185219044' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:58:58.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:58.166+0000 7f8037471200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T07:58:58.638 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:58.224+0000 7f084ad21200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T07:58:59.388 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:59.031+0000 7f084ad21200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T07:58:59.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:58.986+0000 7f8037471200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T07:58:59.784 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:59.561+0000 7f8037471200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T07:58:59.784 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:59.661+0000 7f8037471200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T07:58:59.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:59.601+0000 7f084ad21200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T07:58:59.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:59.699+0000 7f084ad21200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T07:59:00.072 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: Active manager daemon y restarted 2024-03-20T07:59:00.072 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: Activating manager daemon y 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: osdmap e66: 8 total, 8 up, 8 in 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: mgrmap e17: y(active, starting, since 0.00716822s), standbys: x 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:59:00.073 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: Manager daemon y is now available 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: Standby manager daemon x restarted 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: Standby manager daemon x started 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T07:59:00.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:58:59.883+0000 7f8037471200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:58:59] ENGINE Bus STARTING 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f9aaa7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f9aaa7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f9aaa7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f9aaa7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f9aaa7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f95a9d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f95a9d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f95a9d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.075 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f95a9d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.029+0000 7f7f95a9d700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: Active manager daemon y restarted 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: Activating manager daemon y 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: osdmap e66: 8 total, 8 up, 8 in 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: mgrmap e17: y(active, starting, since 0.00716822s), standbys: x 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T07:59:00.076 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T07:59:00.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: Manager daemon y is now available 2024-03-20T07:59:00.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: Standby manager daemon x restarted 2024-03-20T07:59:00.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: Standby manager daemon x started 2024-03-20T07:59:00.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T07:59:00.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T07:59:00.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T07:59:00.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:58:59 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T07:59:00.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:58:59.920+0000 7f084ad21200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T07:59:00.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: [20/Mar/2024:07:58:59] ENGINE Bus STARTING 2024-03-20T07:59:00.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: CherryPy Checker: 2024-03-20T07:59:00.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: The Application mounted at '' has an empty config. 2024-03-20T07:59:00.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: 2024-03-20T07:59:00.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: [20/Mar/2024:07:59:00] ENGINE Serving on http://:::9283 2024-03-20T07:59:00.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[33143]: [20/Mar/2024:07:59:00] ENGINE Bus STARTED 2024-03-20T07:59:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: Active manager daemon y restarted 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: Activating manager daemon y 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: osdmap e66: 8 total, 8 up, 8 in 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: mgrmap e17: y(active, starting, since 0.00716822s), standbys: x 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T07:59:00.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: Manager daemon y is now available 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: Standby manager daemon x restarted 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: Standby manager daemon x started 2024-03-20T07:59:00.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T07:59:00.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T07:59:00.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T07:59:00.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:58:59 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/617605003' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T07:59:00.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: CherryPy Checker: 2024-03-20T07:59:00.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: The Application mounted at '' has an empty config. 2024-03-20T07:59:00.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T07:59:00.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:59:00] ENGINE Serving on http://:::9283 2024-03-20T07:59:00.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:59:00] ENGINE Bus STARTED 2024-03-20T07:59:00.983 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'rbd' created 2024-03-20T07:59:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:59:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:59:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:59:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:59:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:59:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:59:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1787691067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-03-20T07:59:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-03-20T07:59:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:00 smithi196 ceph-mon[31033]: mgrmap e18: y(active, since 1.01204s), standbys: x 2024-03-20T07:59:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1787691067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[27391]: mgrmap e18: y(active, since 1.01204s), standbys: x 2024-03-20T07:59:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1787691067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-03-20T07:59:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:00 smithi118 ceph-mon[38981]: mgrmap e18: y(active, since 1.01204s), standbys: x 2024-03-20T07:59:01.869 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:07:59:01] "GET /metrics HTTP/1.1" 200 28763 "" "Prometheus/2.43.0" 2024-03-20T07:59:01.892 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-20T07:59:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:01 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-03-20T07:59:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:01 smithi118 ceph-mon[27391]: osdmap e67: 8 total, 8 up, 8 in 2024-03-20T07:59:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:01 smithi118 ceph-mon[27391]: [20/Mar/2024:07:59:01] ENGINE Bus STARTING 2024-03-20T07:59:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:01 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-03-20T07:59:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:01 smithi118 ceph-mon[38981]: osdmap e67: 8 total, 8 up, 8 in 2024-03-20T07:59:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:01 smithi118 ceph-mon[38981]: [20/Mar/2024:07:59:01] ENGINE Bus STARTING 2024-03-20T07:59:02.337 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:01 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-03-20T07:59:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:01 smithi196 ceph-mon[31033]: osdmap e67: 8 total, 8 up, 8 in 2024-03-20T07:59:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:01 smithi196 ceph-mon[31033]: [20/Mar/2024:07:59:01] ENGINE Bus STARTING 2024-03-20T07:59:03.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: [20/Mar/2024:07:59:01] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T07:59:03.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: [20/Mar/2024:07:59:01] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T07:59:03.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: [20/Mar/2024:07:59:01] ENGINE Bus STARTED 2024-03-20T07:59:03.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: pgmap v5: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:03.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: osdmap e68: 8 total, 8 up, 8 in 2024-03-20T07:59:03.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:03.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[27391]: mgrmap e19: y(active, since 3s), standbys: x 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: [20/Mar/2024:07:59:01] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: [20/Mar/2024:07:59:01] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: [20/Mar/2024:07:59:01] ENGINE Bus STARTED 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: pgmap v5: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: osdmap e68: 8 total, 8 up, 8 in 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T07:59:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:02 smithi118 ceph-mon[38981]: mgrmap e19: y(active, since 3s), standbys: x 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: [20/Mar/2024:07:59:01] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: [20/Mar/2024:07:59:01] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: [20/Mar/2024:07:59:01] ENGINE Bus STARTED 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: pgmap v5: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: osdmap e68: 8 total, 8 up, 8 in 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T07:59:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:02 smithi196 ceph-mon[31033]: mgrmap e19: y(active, since 3s), standbys: x 2024-03-20T07:59:04.061 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: osdmap e69: 8 total, 8 up, 8 in 2024-03-20T07:59:04.061 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:04.062 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: osdmap e69: 8 total, 8 up, 8 in 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.063 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:03 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:04.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: osdmap e69: 8 total, 8 up, 8 in 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:03 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:05.378 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi196 to 2992M 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[27391]: pgmap v8: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3514701645' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[27391]: mgrmap e20: y(active, since 4s), standbys: x 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi196 to 2992M 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[38981]: pgmap v8: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3514701645' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:59:05.379 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:59:05.380 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:04 smithi118 ceph-mon[38981]: mgrmap e20: y(active, since 4s), standbys: x 2024-03-20T07:59:05.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:04 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi196 to 2992M 2024-03-20T07:59:05.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:04 smithi196 ceph-mon[31033]: pgmap v8: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:05.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:04 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:05.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:04 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:05.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:04 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3514701645' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:59:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:04 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T07:59:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:04 smithi196 ceph-mon[31033]: mgrmap e20: y(active, since 4s), standbys: x 2024-03-20T07:59:05.389 INFO:teuthology.orchestra.run.smithi118.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-20T07:59:06.114 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-20T07:59:06.114 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-20T07:59:06.115 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-20T07:59:06.347 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:06.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi118 to 2992M 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:59:06.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: osdmap e70: 8 total, 8 up, 8 in 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: pgmap v10: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:59:06.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi118 to 2992M 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:59:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-03-20T07:59:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: osdmap e70: 8 total, 8 up, 8 in 2024-03-20T07:59:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:59:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:59:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:59:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:59:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: pgmap v10: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:59:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:06 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi118 to 2992M 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: osdmap e70: 8 total, 8 up, 8 in 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:59:06.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T07:59:06.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: pgmap v10: 105 pgs: 8 unknown, 97 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:06.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:59:06.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:06 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T07:59:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:07 smithi196 ceph-mon[31033]: Deploying daemon alertmanager.a on smithi118 2024-03-20T07:59:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:07 smithi196 ceph-mon[31033]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T07:59:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:07 smithi196 ceph-mon[31033]: Cluster is now healthy 2024-03-20T07:59:07.667 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.667 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.667 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.667 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[27391]: Deploying daemon alertmanager.a on smithi118 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[27391]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[27391]: Cluster is now healthy 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[38981]: Deploying daemon alertmanager.a on smithi118 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[38981]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T07:59:07.668 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:07 smithi118 ceph-mon[38981]: Cluster is now healthy 2024-03-20T07:59:08.462 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:59:08.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:08 smithi196 ceph-mon[31033]: pgmap v11: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-20T07:59:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:08 smithi118 ceph-mon[27391]: pgmap v11: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-20T07:59:08.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:08 smithi118 ceph-mon[38981]: pgmap v11: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 34 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-20T07:59:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:09 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2750866976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-20T07:59:09.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:09 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2750866976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-20T07:59:09.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:09 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2750866976' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-20T07:59:09.717 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":20,"active_gid":14583,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6800","nonce":4166913641},{"type":"v1","addr":"172.21.15.118:6801","nonce":4166913641}]},"active_addr":"172.21.15.118:6801/4166913641","active_change":"2024-03-20T07:58:59.897210+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24454,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.118:8443/","prometheus":"http://172.21.15.118:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":66,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":2517893481}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":2309848212}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":1828201096}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":237356808}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":3840517791}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.118:0","nonce":2750248213}]}]} 2024-03-20T07:59:09.724 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-20T07:59:09.725 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-20T07:59:09.725 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-20T07:59:09.925 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:10.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:10 smithi118 ceph-mon[27391]: pgmap v12: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-20T07:59:10.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:10 smithi118 ceph-mon[38981]: pgmap v12: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-20T07:59:10.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:10 smithi196 ceph-mon[31033]: pgmap v12: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-20T07:59:11.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:07:59:11] "GET /metrics HTTP/1.1" 200 28763 "" "Prometheus/2.43.0" 2024-03-20T07:59:13.109 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:12 smithi118 ceph-mon[27391]: pgmap v13: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-20T07:59:13.110 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:12 smithi118 ceph-mon[38981]: pgmap v13: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-20T07:59:13.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:12 smithi196 ceph-mon[31033]: pgmap v13: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-20T07:59:14.397 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:59:14.397 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":70,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:59:05.381551+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-03-20T07:59:00.573805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":7,"to":2}]},{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1},{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":0,"to":1},{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/2649753508":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/3938132886":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6800/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/184826175":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/1237762586":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:59:15.286 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:14 smithi118 ceph-mon[27391]: pgmap v14: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T07:59:15.286 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:14 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/4066555098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:59:15.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:14 smithi118 ceph-mon[38981]: pgmap v14: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T07:59:15.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:14 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/4066555098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:59:15.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:14 smithi196 ceph-mon[31033]: pgmap v14: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T07:59:15.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:14 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/4066555098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:59:15.992 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-20T07:59:15.992 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-20T07:59:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:15 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:15 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:16.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:15 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:17.037 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:17.169 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:16 smithi118 ceph-mon[27391]: pgmap v15: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T07:59:17.170 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:16 smithi118 ceph-mon[38981]: pgmap v15: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T07:59:17.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:16 smithi196 ceph-mon[31033]: pgmap v15: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T07:59:18.568 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:18 smithi118 podman[72068]: 2024-03-20 07:59:18.383155285 +0000 UTC m=+0.020631712 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-20T07:59:18.568 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:18 smithi118 podman[72068]: 2024-03-20 07:59:18.569441614 +0000 UTC m=+0.206918041 container create 1b8c13084c71100e71ebea1fe24a6ead94de8f76336308e554ba7357f644d652 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T07:59:19.254 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:18 smithi118 ceph-mon[27391]: pgmap v16: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T07:59:19.255 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:18 smithi118 ceph-mon[38981]: pgmap v16: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T07:59:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:18 smithi196 ceph-mon[31033]: pgmap v16: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 podman[72068]: 2024-03-20 07:59:19.393919852 +0000 UTC m=+1.031396280 container init 1b8c13084c71100e71ebea1fe24a6ead94de8f76336308e554ba7357f644d652 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 podman[72068]: 2024-03-20 07:59:19.400136294 +0000 UTC m=+1.037612724 container start 1b8c13084c71100e71ebea1fe24a6ead94de8f76336308e554ba7357f644d652 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:19.411Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:19.411Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:19.413Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.118 port=9094 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:19.414Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:19.437Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:19.437Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-20T07:59:19.560 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:19.440Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-20T07:59:19.561 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:19.440Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-20T07:59:19.827 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 bash[72068]: 1b8c13084c71100e71ebea1fe24a6ead94de8f76336308e554ba7357f644d652 2024-03-20T07:59:19.827 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:19 smithi118 systemd[1]: Started Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:59:19.827 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T07:59:19.828 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":70,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:59:05.381551+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-03-20T07:59:00.573805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":7,"to":2}]},{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1},{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":0,"to":1},{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/2649753508":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/3938132886":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6800/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/184826175":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/1237762586":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T07:59:20.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:59:19] ENGINE Bus STOPPING 2024-03-20T07:59:20.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:59:19] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T07:59:20.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:59:19] ENGINE Bus STOPPED 2024-03-20T07:59:20.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:59:19] ENGINE Bus STARTING 2024-03-20T07:59:20.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:59:19] ENGINE Serving on http://:::9283 2024-03-20T07:59:20.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:07:59:19] ENGINE Bus STARTED 2024-03-20T07:59:20.503 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-20T07:59:20.504 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-20T07:59:20.504 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-20T07:59:20.504 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-20T07:59:20.505 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-20T07:59:20.505 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-20T07:59:20.506 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-20T07:59:20.506 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-20T07:59:20.748 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:20.748 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:20.748 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:20.748 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:20.748 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-20T07:59:20.748 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:20.748 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:20.748 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3820677067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:59:20.749 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-20T07:59:20.749 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-20T07:59:20.749 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:20.749 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: Deploying daemon grafana.a on smithi196 2024-03-20T07:59:20.749 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: pgmap v17: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:20.749 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:20.955 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:20.955 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:20.956 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:20.958 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:20.959 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:20.959 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:20.959 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:20.982 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3820677067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-20T07:59:21.095 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-20T07:59:21.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: Deploying daemon grafana.a on smithi196 2024-03-20T07:59:21.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: pgmap v17: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:21.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3820677067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: Deploying daemon grafana.a on smithi196 2024-03-20T07:59:21.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: pgmap v17: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:21.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:21.612 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:21.414Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000168955s 2024-03-20T07:59:21.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:07:59:21] "GET /metrics HTTP/1.1" 200 33647 "" "Prometheus/2.43.0" 2024-03-20T07:59:23.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:22 smithi196 ceph-mon[31033]: pgmap v18: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:22 smithi118 ceph-mon[27391]: pgmap v18: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:22 smithi118 ceph-mon[38981]: pgmap v18: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:25.274 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:24 smithi118 ceph-mon[27391]: pgmap v19: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:25.275 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:24 smithi118 ceph-mon[38981]: pgmap v19: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:25.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:24 smithi196 ceph-mon[31033]: pgmap v19: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:27.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:26 smithi118 ceph-mon[27391]: pgmap v20: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:27.298 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:26 smithi118 ceph-mon[38981]: pgmap v20: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:27.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:26 smithi196 ceph-mon[31033]: pgmap v20: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:29.213 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:28 smithi118 ceph-mon[27391]: pgmap v21: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:29.213 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:28 smithi118 ceph-mon[38981]: pgmap v21: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:29.240 INFO:teuthology.orchestra.run.smithi118.stdout:55834574895 2024-03-20T07:59:29.240 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-20T07:59:29.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:28 smithi196 ceph-mon[31033]: pgmap v21: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:29.665 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 07:59:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T07:59:29.415Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001377903s 2024-03-20T07:59:31.286 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:30 smithi118 ceph-mon[27391]: pgmap v22: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:31.286 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:30 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:31.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:30 smithi118 ceph-mon[38981]: pgmap v22: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:31.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:30 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:31.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:30 smithi196 ceph-mon[31033]: pgmap v22: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:31.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:30 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:31.918 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:07:59:31] "GET /metrics HTTP/1.1" 200 33646 "" "Prometheus/2.43.0" 2024-03-20T07:59:33.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:32 smithi118 ceph-mon[38981]: pgmap v23: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:33.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:32 smithi118 ceph-mon[27391]: pgmap v23: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:33.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:32 smithi196 ceph-mon[31033]: pgmap v23: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:33.669 INFO:teuthology.orchestra.run.smithi118.stdout:133143986210 2024-03-20T07:59:33.670 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-20T07:59:33.966 INFO:teuthology.orchestra.run.smithi118.stdout:34359738422 2024-03-20T07:59:33.966 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-20T07:59:35.075 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:34 smithi118 ceph-mon[27391]: pgmap v24: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:35.075 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:34 smithi118 ceph-mon[38981]: pgmap v24: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:35.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:34 smithi196 ceph-mon[31033]: pgmap v24: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:35.703 INFO:teuthology.orchestra.run.smithi118.stdout:111669149734 2024-03-20T07:59:35.703 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-20T07:59:35.917 INFO:teuthology.orchestra.run.smithi118.stdout:167503724573 2024-03-20T07:59:35.918 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-20T07:59:37.115 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:36 smithi118 ceph-mon[27391]: pgmap v25: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:37.115 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:36 smithi118 ceph-mon[38981]: pgmap v25: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:37.119 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:37.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:36 smithi196 ceph-mon[31033]: pgmap v25: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:37.138 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 systemd[1]: Starting Ceph grafana.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:59:37.148 INFO:teuthology.orchestra.run.smithi118.stdout:77309411372 2024-03-20T07:59:37.149 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-20T07:59:37.399 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:37.511 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 podman[62715]: 2024-03-20 07:59:37.159086284 +0000 UTC m=+0.017067763 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-03-20T07:59:37.567 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 podman[62715]: 2024-03-20 07:59:37.280167388 +0000 UTC m=+0.138148899 container create c58566782642a692ae9130a99b6d7efd87a0b30287cc7f64bd4f26bb71ba501a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, version=8.5, distribution-scope=public, maintainer=Paul Cuzner , com.redhat.component=ubi8-container, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, io.openshift.expose-services=) 2024-03-20T07:59:37.567 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 podman[62715]: 2024-03-20 07:59:37.507207443 +0000 UTC m=+0.365188925 container init c58566782642a692ae9130a99b6d7efd87a0b30287cc7f64bd4f26bb71ba501a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, architecture=x86_64, vendor=Red Hat, Inc., name=ubi8, build-date=2022-04-29T11:48:11.055897, description=Ceph Grafana Container, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, io.buildah.version=1.32.0, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container) 2024-03-20T07:59:37.567 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 podman[62715]: 2024-03-20 07:59:37.512575463 +0000 UTC m=+0.370556930 container start c58566782642a692ae9130a99b6d7efd87a0b30287cc7f64bd4f26bb71ba501a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , description=Ceph Grafana Container, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, release=239.1651231664, version=8.5, io.openshift.tags=base rhel8, architecture=x86_64, vendor=Red Hat, Inc., name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 bash[62715]: c58566782642a692ae9130a99b6d7efd87a0b30287cc7f64bd4f26bb71ba501a 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 systemd[1]: Started Ceph grafana.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668410921Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668611777Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.6686302Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668637205Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668643138Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668649137Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-03-20T07:59:37.764 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668654837Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668660448Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668666684Z level=info msg="Path Home" path=/usr/share/grafana 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668673087Z level=info msg="Path Data" path=/var/lib/grafana 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668678614Z level=info msg="Path Logs" path=/var/log/grafana 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668684474Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668690053Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=settings t=2024-03-20T07:59:37.668696519Z level=info msg="App mode production" 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=sqlstore t=2024-03-20T07:59:37.668751025Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=sqlstore t=2024-03-20T07:59:37.668768302Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.669619013Z level=info msg="Starting DB migrations" 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.669703867Z level=info msg="Executing migration" id="create migration_log table" 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.670303971Z level=info msg="Executing migration" id="create user table" 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.670765843Z level=info msg="Executing migration" id="add unique index user.login" 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.673347777Z level=info msg="Executing migration" id="add unique index user.email" 2024-03-20T07:59:37.765 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.67380106Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.676424383Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.678949272Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.681462476Z level=info msg="Executing migration" id="create user table v2" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.682277735Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.682636682Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.683072498Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.683892698Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.684271582Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.685160745Z level=info msg="Executing migration" id="Update user table charset" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.685719469Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.686163723Z level=info msg="Executing migration" id="Add missing user data" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.687089615Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.687703745Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.688526241Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-03-20T07:59:37.766 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.689479769Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-03-20T07:59:37.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.690784613Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-03-20T07:59:37.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.691262974Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-03-20T07:59:37.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.692077394Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-03-20T07:59:37.767 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.692920983Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.693316532Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.694356393Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.695033724Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.696052578Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.696738803Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.697703377Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.698471795Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.699324802Z level=info msg="Executing migration" id="create temp_user v2" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.700270344Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.701187474Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.701817566Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.702779484Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.703719577Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-03-20T07:59:37.768 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.704418475Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.704746224Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.705116988Z level=info msg="Executing migration" id="create star table" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.705606853Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.706600282Z level=info msg="Executing migration" id="create org table v1" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.707327Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.710246148Z level=info msg="Executing migration" id="create org_user table v1" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.71285664Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.715733817Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.716148588Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.718945467Z level=info msg="Executing migration" id="Update org table charset" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.72178574Z level=info msg="Executing migration" id="Update org_user table charset" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.7245273Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.724922771Z level=info msg="Executing migration" id="create dashboard table" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.725791444Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-03-20T07:59:37.769 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.726761983Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.727526687Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.727924548Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.728871967Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.72967088Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.730807715Z level=info msg="Executing migration" id="create dashboard v2" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.731455095Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.73247583Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.733226461Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.734075721Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.734459861Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.735307817Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.73581356Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.736305701Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-03-20T07:59:37.770 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.736749265Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.737167405Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.739655597Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.742401707Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.742815134Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.745522952Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.74584118Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.74638404Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.74687232Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.747365849Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.749821414Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.750233614Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.752667678Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.753106348Z level=info msg="Executing migration" id="Update dashboard title length" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.755743008Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-03-20T07:59:37.771 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.758590489Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-03-20T07:59:37.772 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.761539592Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-03-20T07:59:37.772 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.762532945Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-03-20T07:59:37.772 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.765376014Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-03-20T07:59:38.014 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.768106472Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-03-20T07:59:38.014 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.770648905Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.771162581Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.773729328Z level=info msg="Executing migration" id="Add check_sum column" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.776571431Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.779156967Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.779516043Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.782214331Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.782791062Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.784668311Z level=info msg="Executing migration" id="create data_source table" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.7875348Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.788062644Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.788478239Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.791157678Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-03-20T07:59:38.015 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.791571666Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.794105819Z level=info msg="Executing migration" id="create data_source table v2" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.794727599Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.797583026Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.798328589Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.799279404Z level=info msg="Executing migration" id="Add column with_credentials" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.799994689Z level=info msg="Executing migration" id="Add secure json data column" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.800843757Z level=info msg="Executing migration" id="Update data_source table charset" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.801643768Z level=info msg="Executing migration" id="Update initial version to 1" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.802428405Z level=info msg="Executing migration" id="Add read_only data column" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.803018636Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.803345146Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.804252453Z level=info msg="Executing migration" id="Add uid column" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.804909959Z level=info msg="Executing migration" id="Update uid value" 2024-03-20T07:59:38.016 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.80524052Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.806153091Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.806812333Z level=info msg="Executing migration" id="create api_key table" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.807226844Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.808077041Z level=info msg="Executing migration" id="add index api_key.key" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.809141274Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.809750057Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.810167231Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.811183682Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-03-20T07:59:38.017 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.811955305Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-03-20T07:59:38.018 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.812989762Z level=info msg="Executing migration" id="create api_key table v2" 2024-03-20T07:59:38.018 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.813967264Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.814789236Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.815562811Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.816575876Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.817270791Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.817637113Z level=info msg="Executing migration" id="Update api_key table charset" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.818464035Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.819043092Z level=info msg="Executing migration" id="Add service account foreign key" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.819568362Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.820497071Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.821269407Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.821809735Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.822826128Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-03-20T07:59:38.019 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.823582353Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-03-20T07:59:38.020 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.824533156Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-03-20T07:59:38.020 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.825173623Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-03-20T07:59:38.020 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.827506024Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-03-20T07:59:38.020 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.83035669Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-03-20T07:59:38.020 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.833031678Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.835840417Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.838944778Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.841756169Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.84459468Z level=info msg="Executing migration" id="create quota table v1" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.847591495Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.850368245Z level=info msg="Executing migration" id="Update quota table charset" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.853357754Z level=info msg="Executing migration" id="create plugin_setting table" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.853864683Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-03-20T07:59:38.021 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.856709864Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.859767137Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.86231483Z level=info msg="Executing migration" id="create session table" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.86280584Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.86568013Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.866056119Z level=info msg="Executing migration" id="create playlist table v2" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.866470763Z level=info msg="Executing migration" id="create playlist item table v2" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.869516573Z level=info msg="Executing migration" id="Update playlist table charset" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.872004099Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.872325072Z level=info msg="Executing migration" id="drop preferences table v2" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.872596868Z level=info msg="Executing migration" id="drop preferences table v3" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.8728694Z level=info msg="Executing migration" id="create preferences table v3" 2024-03-20T07:59:38.022 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.873269481Z level=info msg="Executing migration" id="Update preferences table charset" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.873536078Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.876204359Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.878856445Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.879888621Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.882463117Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.882787455Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.885289017Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.88772588Z level=info msg="Executing migration" id="create alert table v1" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.890304225Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.892809056Z level=info msg="Executing migration" id="add index alert state" 2024-03-20T07:59:38.023 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.895537056Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.896040698Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.896407045Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.896817173Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.897227134Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.898638432Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.899025275Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.899459825Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.899829757Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.900313038Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-03-20T07:59:38.024 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.900892055Z level=info msg="Executing migration" id="Add column is_default" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.901776926Z level=info msg="Executing migration" id="Add column frequency" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.902674079Z level=info msg="Executing migration" id="Add column send_reminder" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.903583999Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.904473402Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.905021674Z level=info msg="Executing migration" id="Update alert table charset" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.905390197Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.905749736Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.90627012Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.908553657Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.911450564Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.914362745Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.916912919Z level=info msg="Executing migration" id="Add for to alert table" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.919643619Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-03-20T07:59:38.025 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.922397971Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.925248879Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.928011411Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.930609336Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.93332323Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.936002763Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.938659167Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.941311639Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.944113958Z level=info msg="Executing migration" id="create annotation table v5" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.946469712Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.949340709Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.951774281Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.954413726Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.956943565Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-03-20T07:59:38.026 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.959688695Z level=info msg="Executing migration" id="Update annotation table charset" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.962346428Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.965080516Z level=info msg="Executing migration" id="Drop category_id index" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.967665308Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.97050775Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.973016787Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.97381686Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.974480251Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.976050613Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.976979726Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.977814787Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.97855695Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.979252716Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.979603151Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-03-20T07:59:38.027 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.980285871Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.981091001Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.981852825Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.982707884Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.983553779Z level=info msg="Executing migration" id="Add epoch_end column" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.984374546Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.98512512Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.986054615Z level=info msg="Executing migration" id="Move region to single row" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.986752407Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.98771857Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.98848535Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.989440281Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.990157488Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.991144195Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.99187042Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-03-20T07:59:38.028 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.992895612Z level=info msg="Executing migration" id="create test_data table" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.993603658Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.994447111Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.995262035Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.996086423Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.997056101Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.997792132Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.998803225Z level=info msg="Executing migration" id="create team table" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:37.999465651Z level=info msg="Executing migration" id="add index team.org_id" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.000314389Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.001198972Z level=info msg="Executing migration" id="create team member table" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.002033627Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.004978915Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.007655097Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.010455326Z level=info msg="Executing migration" id="Add column email to team table" 2024-03-20T07:59:38.029 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.013197167Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-03-20T07:59:38.030 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.015989244Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-03-20T07:59:38.030 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.018737394Z level=info msg="Executing migration" id="create dashboard acl table" 2024-03-20T07:59:38.266 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.021476307Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-03-20T07:59:38.266 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.024330673Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-03-20T07:59:38.266 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.027136415Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-03-20T07:59:38.267 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.030049268Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-03-20T07:59:38.267 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.032587995Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-03-20T07:59:38.267 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.035325753Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-03-20T07:59:38.267 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.035796567Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-03-20T07:59:38.267 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.038450997Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-03-20T07:59:38.267 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.041221333Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-03-20T07:59:38.268 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.043932192Z level=info msg="Executing migration" id="create tag table" 2024-03-20T07:59:38.268 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.044345909Z level=info msg="Executing migration" id="add index tag.key_value" 2024-03-20T07:59:38.268 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.047203107Z level=info msg="Executing migration" id="create login attempt table" 2024-03-20T07:59:38.268 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.049684583Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-03-20T07:59:38.268 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.052529184Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-03-20T07:59:38.268 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.052971879Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-03-20T07:59:38.269 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.055647573Z level=info msg="Executing migration" id="create login_attempt v2" 2024-03-20T07:59:38.269 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.058381956Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-03-20T07:59:38.269 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.061004008Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-03-20T07:59:38.269 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.061334783Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-03-20T07:59:38.273 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.064062995Z level=info msg="Executing migration" id="create user auth table" 2024-03-20T07:59:38.273 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.064464248Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-03-20T07:59:38.273 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.066739509Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-03-20T07:59:38.274 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.069104974Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-03-20T07:59:38.274 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.069885189Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-03-20T07:59:38.274 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.072468824Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-03-20T07:59:38.274 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.074979188Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-03-20T07:59:38.274 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.077756926Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-03-20T07:59:38.275 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.078236476Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-03-20T07:59:38.275 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.080925385Z level=info msg="Executing migration" id="create server_lock table" 2024-03-20T07:59:38.275 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.081897859Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-03-20T07:59:38.275 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.082522187Z level=info msg="Executing migration" id="create user auth token table" 2024-03-20T07:59:38.275 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.082968001Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-03-20T07:59:38.275 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.083780351Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-03-20T07:59:38.276 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.084743602Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-03-20T07:59:38.276 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.085434419Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-03-20T07:59:38.276 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.086258064Z level=info msg="Executing migration" id="create cache_data table" 2024-03-20T07:59:38.276 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.087105965Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-03-20T07:59:38.276 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.087968543Z level=info msg="Executing migration" id="create short_url table v1" 2024-03-20T07:59:38.276 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.088784416Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-03-20T07:59:38.276 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.08926494Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-03-20T07:59:38.277 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.090074374Z level=info msg="Executing migration" id="delete alert_definition table" 2024-03-20T07:59:38.277 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.090942049Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-03-20T07:59:38.277 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.091700823Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-03-20T07:59:38.277 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.092157167Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-03-20T07:59:38.277 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.09300572Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-03-20T07:59:38.278 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.093818509Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-03-20T07:59:38.278 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.094810612Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-03-20T07:59:38.278 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.09537252Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-03-20T07:59:38.278 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.096224252Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-03-20T07:59:38.278 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.096669272Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-03-20T07:59:38.278 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.097492809Z level=info msg="Executing migration" id="drop alert_definition table" 2024-03-20T07:59:38.279 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.098365036Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-03-20T07:59:38.279 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.098696232Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-03-20T07:59:38.279 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.099577221Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-03-20T07:59:38.279 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.100521556Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-03-20T07:59:38.279 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.101216235Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-03-20T07:59:38.279 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.10151491Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-03-20T07:59:38.280 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.102350716Z level=info msg="Executing migration" id="create alert_instance table" 2024-03-20T07:59:38.280 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.102789598Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-03-20T07:59:38.280 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.103232613Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-03-20T07:59:38.280 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.104083146Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-03-20T07:59:38.280 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.104861224Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-03-20T07:59:38.280 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.105710177Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-03-20T07:59:38.281 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.106703557Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-03-20T07:59:38.281 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.110154572Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-03-20T07:59:38.281 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.113448745Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-03-20T07:59:38.281 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.113910444Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-03-20T07:59:38.281 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.114333885Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-03-20T07:59:38.281 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.115174934Z level=info msg="Executing migration" id="create alert_rule table" 2024-03-20T07:59:38.282 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.115608584Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-03-20T07:59:38.282 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.11605895Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-03-20T07:59:38.282 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.116491349Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-03-20T07:59:38.282 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.116934549Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-03-20T07:59:38.282 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.117243121Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-03-20T07:59:38.283 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.118090367Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-03-20T07:59:38.283 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.118930188Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-03-20T07:59:38.283 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.119808297Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-03-20T07:59:38.283 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.120320541Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-03-20T07:59:38.283 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.12075407Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-03-20T07:59:38.283 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.121655969Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-03-20T07:59:38.284 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.122519031Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-03-20T07:59:38.284 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.123013653Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-03-20T07:59:38.284 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.123985826Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-03-20T07:59:38.284 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.124825246Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-03-20T07:59:38.284 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.125197079Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-03-20T07:59:38.284 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.12566305Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-03-20T07:59:38.285 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.1261635Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-03-20T07:59:38.285 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.126591558Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-03-20T07:59:38.285 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.126860825Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-03-20T07:59:38.285 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.127746897Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-03-20T07:59:38.285 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.128659226Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-03-20T07:59:38.285 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.129687136Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-03-20T07:59:38.286 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.130861652Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-03-20T07:59:38.286 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.131816Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-03-20T07:59:38.286 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.132562343Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-03-20T07:59:38.286 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.132990115Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-03-20T07:59:38.286 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.133887375Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-03-20T07:59:38.286 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.134683659Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-03-20T07:59:38.287 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.135676761Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-03-20T07:59:38.287 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.136193588Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-03-20T07:59:38.287 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.137174104Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-03-20T07:59:38.287 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.138035335Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-03-20T07:59:38.287 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.138894069Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-03-20T07:59:38.287 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.140029688Z level=info msg="Executing migration" id="create provenance_type table" 2024-03-20T07:59:38.288 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.140894966Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-03-20T07:59:38.288 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.14176572Z level=info msg="Executing migration" id="create alert_image table" 2024-03-20T07:59:38.288 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.142621165Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-03-20T07:59:38.288 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.143114562Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-03-20T07:59:38.288 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.143985135Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-03-20T07:59:38.288 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.144787826Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-03-20T07:59:38.289 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.145388338Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-03-20T07:59:38.289 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.14549529Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-03-20T07:59:38.289 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.145795148Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-03-20T07:59:38.289 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.146634227Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-03-20T07:59:38.289 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.147478338Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-03-20T07:59:38.290 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.147610586Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-03-20T07:59:38.290 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.147916702Z level=info msg="alerts found to migrate" alerts=0 2024-03-20T07:59:38.290 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.148363964Z level=info msg="Executing migration" id="create library_element table v1" 2024-03-20T07:59:38.290 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.148810936Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-03-20T07:59:38.290 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.149686928Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-03-20T07:59:38.290 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.152562879Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-03-20T07:59:38.291 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.155309956Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-03-20T07:59:38.291 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.155770931Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-03-20T07:59:38.291 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.158601536Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-03-20T07:59:38.291 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.161278961Z level=info msg="Executing migration" id="create data_keys table" 2024-03-20T07:59:38.291 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.164146513Z level=info msg="Executing migration" id="create secrets table" 2024-03-20T07:59:38.291 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.164605263Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-03-20T07:59:38.292 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.168923444Z level=info msg="Executing migration" id="add name column into data_keys" 2024-03-20T07:59:38.292 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.17191214Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-03-20T07:59:38.292 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.174735965Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-03-20T07:59:38.292 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.17902635Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-03-20T07:59:38.292 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.183185967Z level=info msg="Executing migration" id="create kv_store table v1" 2024-03-20T07:59:38.292 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.1861631Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-03-20T07:59:38.293 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.188816914Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-03-20T07:59:38.293 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.189181668Z level=info msg="Executing migration" id="create permission table" 2024-03-20T07:59:38.293 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.192042988Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-03-20T07:59:38.293 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.194583887Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-03-20T07:59:38.293 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.197603521Z level=info msg="Executing migration" id="create role table" 2024-03-20T07:59:38.293 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.198071852Z level=info msg="Executing migration" id="add column display_name" 2024-03-20T07:59:38.294 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.20061867Z level=info msg="Executing migration" id="add column group_name" 2024-03-20T07:59:38.294 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.201584141Z level=info msg="Executing migration" id="add index role.org_id" 2024-03-20T07:59:38.294 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.20341724Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-03-20T07:59:38.294 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.206026746Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-03-20T07:59:38.294 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.206458192Z level=info msg="Executing migration" id="create team role table" 2024-03-20T07:59:38.295 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.209309926Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-03-20T07:59:38.295 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.211946841Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-03-20T07:59:38.295 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.214624755Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-03-20T07:59:38.295 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.215061408Z level=info msg="Executing migration" id="create user role table" 2024-03-20T07:59:38.295 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.218057664Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-03-20T07:59:38.295 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.220716658Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-03-20T07:59:38.296 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.223306489Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-03-20T07:59:38.296 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.223754145Z level=info msg="Executing migration" id="create builtin role table" 2024-03-20T07:59:38.296 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.226556552Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-03-20T07:59:38.296 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.22922343Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-03-20T07:59:38.296 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.23196622Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-03-20T07:59:38.296 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.2330597Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-03-20T07:59:38.297 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.23355784Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-03-20T07:59:38.297 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.234326521Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-03-20T07:59:38.297 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.234742186Z level=info msg="Executing migration" id="add unique index role.uid" 2024-03-20T07:59:38.297 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.235180186Z level=info msg="Executing migration" id="create seed assignment table" 2024-03-20T07:59:38.297 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.235561201Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-03-20T07:59:38.297 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.235977964Z level=info msg="Executing migration" id="add column hidden to role table" 2024-03-20T07:59:38.298 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.236940932Z level=info msg="Executing migration" id="create query_history table v1" 2024-03-20T07:59:38.298 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.237338207Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-03-20T07:59:38.298 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.237789825Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-03-20T07:59:38.298 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.2380562Z level=info msg="Executing migration" id="teams permissions migration" 2024-03-20T07:59:38.298 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.238427671Z level=info msg="Executing migration" id="dashboard permissions" 2024-03-20T07:59:38.299 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.238837725Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-03-20T07:59:38.299 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.239258375Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-03-20T07:59:38.299 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.239536266Z level=info msg="Executing migration" id="alerting notification permissions" 2024-03-20T07:59:38.299 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.239861011Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-03-20T07:59:38.299 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.240256981Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-03-20T07:59:38.299 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.240685532Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-03-20T07:59:38.300 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.241651566Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-03-20T07:59:38.300 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.241953785Z level=info msg="Executing migration" id="create correlation table v1" 2024-03-20T07:59:38.300 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.242363716Z level=info msg="Executing migration" id="add index correlations.uid" 2024-03-20T07:59:38.300 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.242777103Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-03-20T07:59:38.300 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.243204065Z level=info msg="Executing migration" id="add correlation config column" 2024-03-20T07:59:38.300 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.244187638Z level=info msg="Executing migration" id="create entity_events table" 2024-03-20T07:59:38.301 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.244604233Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-03-20T07:59:38.301 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.245017955Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-20T07:59:38.301 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.245099592Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-20T07:59:38.301 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.245372968Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-20T07:59:38.301 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.245447173Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-20T07:59:38.301 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.24569555Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-03-20T07:59:38.302 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.246098392Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-03-20T07:59:38.302 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.246494652Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-03-20T07:59:38.302 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.246949325Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-20T07:59:38.302 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.247372401Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-03-20T07:59:38.302 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.247752374Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-20T07:59:38.303 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.2481701Z level=info msg="Executing migration" id="Drop public config table" 2024-03-20T07:59:38.303 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.248556886Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-03-20T07:59:38.303 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.248936989Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-03-20T07:59:38.303 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.249360086Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-20T07:59:38.303 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.249773575Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-03-20T07:59:38.303 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.250215377Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-03-20T07:59:38.304 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.253147065Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-03-20T07:59:38.304 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.25414766Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-03-20T07:59:38.304 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.255165435Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-03-20T07:59:38.304 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.255473958Z level=info msg="Executing migration" id="add share column" 2024-03-20T07:59:38.304 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.256476992Z level=info msg="Executing migration" id="create default alerting folders" 2024-03-20T07:59:38.304 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.256897502Z level=info msg="Executing migration" id="create file table" 2024-03-20T07:59:38.305 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.257321606Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-03-20T07:59:38.305 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.257767658Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-03-20T07:59:38.305 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.258240856Z level=info msg="Executing migration" id="create file_meta table" 2024-03-20T07:59:38.305 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.259284089Z level=info msg="Executing migration" id="file table idx: path key" 2024-03-20T07:59:38.305 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.25993216Z level=info msg="Executing migration" id="set path collation in file table" 2024-03-20T07:59:38.305 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.26019994Z level=info msg="Executing migration" id="managed permissions migration" 2024-03-20T07:59:38.306 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.260551817Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-03-20T07:59:38.306 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.260838823Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-03-20T07:59:38.306 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.261263333Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-03-20T07:59:38.306 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.262280406Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-03-20T07:59:38.306 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.263144353Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-03-20T07:59:38.307 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.264110905Z level=info msg="Executing migration" id="update group index for alert rules" 2024-03-20T07:59:38.307 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.264586793Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-03-20T07:59:38.307 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.267188896Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-03-20T07:59:38.307 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.267667867Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-03-20T07:59:38.307 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.270001484Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-03-20T07:59:38.307 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.272617049Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-03-20T07:59:38.308 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.273228825Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-03-20T07:59:38.593 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.28355117Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-03-20T07:59:38.593 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.286473498Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-03-20T07:59:38.593 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.289112753Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-03-20T07:59:38.593 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.292629552Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-03-20T07:59:38.593 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.295276367Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-03-20T07:59:38.593 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.29816604Z level=info msg="Executing migration" id="create folder table" 2024-03-20T07:59:38.593 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.300827978Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-03-20T07:59:38.593 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.301331768Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.303988851Z level=info msg="Executing migration" id="Update folder title length" 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.306750237Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=migrator t=2024-03-20T07:59:38.309318912Z level=info msg="migrations completed" performed=485 skipped=0 duration=639.640408ms 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=sqlstore t=2024-03-20T07:59:38.30974815Z level=info msg="Created default organization" 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=plugin.loader t=2024-03-20T07:59:38.366321939Z level=info msg="Plugin registered" pluginID=input 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=plugin.loader t=2024-03-20T07:59:38.393323774Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=plugin.loader t=2024-03-20T07:59:38.393340559Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=secrets t=2024-03-20T07:59:38.393372284Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=query_data t=2024-03-20T07:59:38.395005318Z level=info msg="Query Service initialization" 2024-03-20T07:59:38.594 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=live.push_http t=2024-03-20T07:59:38.397427722Z level=info msg="Live Push Gateway initialization" 2024-03-20T07:59:38.812 INFO:teuthology.orchestra.run.smithi118.stdout:201863462937 2024-03-20T07:59:38.812 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-20T07:59:38.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-mon[31033]: Deploying daemon node-exporter.a on smithi118 2024-03-20T07:59:38.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:38 smithi196 ceph-mon[31033]: pgmap v26: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:38.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[38981]: Deploying daemon node-exporter.a on smithi118 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[38981]: pgmap v26: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[27391]: Deploying daemon node-exporter.a on smithi118 2024-03-20T07:59:38.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:38 smithi118 ceph-mon[27391]: pgmap v26: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=infra.usagestats.collector t=2024-03-20T07:59:39.125922284Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=provisioning.datasources t=2024-03-20T07:59:39.181318021Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=provisioning.datasources t=2024-03-20T07:59:39.213131177Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=provisioning.alerting t=2024-03-20T07:59:39.234616966Z level=info msg="starting to provision alerting" 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=provisioning.alerting t=2024-03-20T07:59:39.234635314Z level=info msg="finished to provision alerting" 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=ngalert.state.manager t=2024-03-20T07:59:39.234778514Z level=info msg="Warming state cache for startup" 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=grafanaStorageLogger t=2024-03-20T07:59:39.234911009Z level=info msg="storage starting" 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=http.server t=2024-03-20T07:59:39.238296165Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=sqlstore.transactions t=2024-03-20T07:59:39.245151118Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=ngalert.state.manager t=2024-03-20T07:59:39.247294691Z level=info msg="State cache has been initialized" states=0 duration=12.515497ms 2024-03-20T07:59:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=ticker t=2024-03-20T07:59:39.247361813Z level=info msg=starting first_tick=2024-03-20T07:59:40Z 2024-03-20T07:59:39.390 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=ngalert.multiorg.alertmanager t=2024-03-20T07:59:39.247355726Z level=info msg="starting MultiOrg Alertmanager" 2024-03-20T07:59:39.390 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 07:59:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=sqlstore.transactions t=2024-03-20T07:59:39.293378885Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-20T07:59:39.481 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:39.483 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:39.989 INFO:teuthology.orchestra.run.smithi118.stdout:236223201299 2024-03-20T07:59:39.990 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-20T07:59:40.141 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:40 smithi118 systemd[1]: Starting Ceph node-exporter.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:59:40.890 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:41.041 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:40 smithi118 ceph-mon[27391]: pgmap v27: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:41.041 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:40 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:41.041 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:40 smithi118 ceph-mon[38981]: pgmap v27: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:41.041 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:40 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:41.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:40 smithi196 ceph-mon[31033]: pgmap v27: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:40 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:41.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:07:59:41] "GET /metrics HTTP/1.1" 200 33646 "" "Prometheus/2.43.0" 2024-03-20T07:59:41.939 INFO:teuthology.orchestra.run.smithi118.stdout:55834574898 2024-03-20T07:59:42.147 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:42.247 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:41 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1763567798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-20T07:59:42.247 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:41 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1763567798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-20T07:59:42.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:41 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1763567798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-20T07:59:43.148 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:42 smithi118 ceph-mon[27391]: pgmap v28: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:43.149 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:42 smithi118 ceph-mon[38981]: pgmap v28: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:42 smithi196 ceph-mon[31033]: pgmap v28: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:44.161 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:44 smithi196 ceph-mon[31033]: pgmap v29: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:44 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:44 smithi118 ceph-mon[27391]: pgmap v29: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:44 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:45.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:44 smithi118 ceph-mon[38981]: pgmap v29: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:45.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:44 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T07:59:47.118 INFO:teuthology.orchestra.run.smithi118.stdout:111669149737 2024-03-20T07:59:47.120 INFO:teuthology.orchestra.run.smithi118.stdout:77309411374 2024-03-20T07:59:47.313 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:46 smithi118 ceph-mon[27391]: pgmap v30: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:47.313 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:46 smithi118 ceph-mon[38981]: pgmap v30: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:47.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:46 smithi196 ceph-mon[31033]: pgmap v30: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:48.142 INFO:teuthology.orchestra.run.smithi118.stdout:133143986213 2024-03-20T07:59:48.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:47 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3771244323' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-20T07:59:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:47 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/63132889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-20T07:59:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:47 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3938697877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-20T07:59:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:47 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3771244323' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-20T07:59:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:47 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/63132889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-20T07:59:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:47 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3938697877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-20T07:59:48.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:47 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3771244323' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-20T07:59:48.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:47 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/63132889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-20T07:59:48.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:47 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3938697877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-20T07:59:49.333 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:48 smithi118 ceph-mon[27391]: pgmap v31: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:49.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:48 smithi118 ceph-mon[38981]: pgmap v31: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:48 smithi196 ceph-mon[31033]: pgmap v31: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:50.100 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:50.192 INFO:teuthology.orchestra.run.smithi118.stdout:34359738426 2024-03-20T07:59:50.414 INFO:teuthology.orchestra.run.smithi118.stdout:201863462939 2024-03-20T07:59:50.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574895 got 55834574898 for osd.1 2024-03-20T07:59:50.842 DEBUG:teuthology.parallel:result is None 2024-03-20T07:59:51.247 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:51 smithi118 ceph-mon[27391]: pgmap v32: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:51.247 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:51 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/991479929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-20T07:59:51.247 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:51 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3283094501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-20T07:59:51.248 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:51 smithi118 ceph-mon[38981]: pgmap v32: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:51.248 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:51 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/991479929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-20T07:59:51.248 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:51 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3283094501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-20T07:59:51.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:51 smithi196 ceph-mon[31033]: pgmap v32: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:51 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/991479929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-20T07:59:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:51 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3283094501' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-20T07:59:51.474 INFO:teuthology.orchestra.run.smithi118.stdout:167503724576 2024-03-20T07:59:51.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 07:59:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:07:59:51] "GET /metrics HTTP/1.1" 200 33645 "" "Prometheus/2.43.0" 2024-03-20T07:59:52.049 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724573 got 167503724576 for osd.5 2024-03-20T07:59:52.050 DEBUG:teuthology.parallel:result is None 2024-03-20T07:59:52.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:52 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3796204198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-20T07:59:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:52 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3796204198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-20T07:59:52.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:52 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3796204198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-20T07:59:52.537 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149734 got 111669149737 for osd.3 2024-03-20T07:59:52.537 DEBUG:teuthology.parallel:result is None 2024-03-20T07:59:52.889 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738422 got 34359738426 for osd.0 2024-03-20T07:59:52.889 DEBUG:teuthology.parallel:result is None 2024-03-20T07:59:53.128 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:53 smithi118 ceph-mon[27391]: pgmap v33: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:53.128 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:53 smithi118 ceph-mon[38981]: pgmap v33: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:53.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:53 smithi196 ceph-mon[31033]: pgmap v33: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:53.543 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411372 got 77309411374 for osd.2 2024-03-20T07:59:53.544 DEBUG:teuthology.parallel:result is None 2024-03-20T07:59:53.981 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462937 got 201863462939 for osd.6 2024-03-20T07:59:53.981 DEBUG:teuthology.parallel:result is None 2024-03-20T07:59:54.210 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:54 smithi118 bash[75610]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-03-20T07:59:55.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:55 smithi118 ceph-mon[27391]: pgmap v34: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:55.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:55 smithi118 ceph-mon[38981]: pgmap v34: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:55.173 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:55 smithi118 bash[75610]: Getting image source signatures 2024-03-20T07:59:55.173 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:55 smithi118 bash[75610]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-03-20T07:59:55.173 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:55 smithi118 bash[75610]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-03-20T07:59:55.173 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:55 smithi118 bash[75610]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-03-20T07:59:55.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986210 got 133143986213 for osd.4 2024-03-20T07:59:55.355 DEBUG:teuthology.parallel:result is None 2024-03-20T07:59:55.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:55 smithi196 ceph-mon[31033]: pgmap v34: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:56.198 INFO:teuthology.orchestra.run.smithi118.stdout:236223201304 2024-03-20T07:59:57.340 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:57 smithi118 ceph-mon[27391]: pgmap v35: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:57.341 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:57 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1902123085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-20T07:59:57.341 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:57 smithi118 ceph-mon[38981]: pgmap v35: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:57.341 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:57 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1902123085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-20T07:59:57.341 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:57 smithi118 bash[75610]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-03-20T07:59:57.341 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:57 smithi118 bash[75610]: Writing manifest to image destination 2024-03-20T07:59:57.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:57 smithi196 ceph-mon[31033]: pgmap v35: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:57.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:57 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1902123085' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-20T07:59:57.703 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201299 got 236223201304 for osd.7 2024-03-20T07:59:57.703 DEBUG:teuthology.parallel:result is None 2024-03-20T07:59:57.703 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-20T07:59:57.703 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T07:59:57.955 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:57 smithi118 podman[75610]: 2024-03-20 07:59:57.708453611 +0000 UTC m=+3.721297380 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-20T07:59:57.956 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:57 smithi118 podman[75610]: 2024-03-20 07:59:57.823145185 +0000 UTC m=+3.835988953 container create e22b97acea5c3f0f6a0f4da6f5e91ad57a2f7d2169ff8d2dec8cbd55293c008b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T07:59:58.247 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T07:59:58.371 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 podman[75610]: 2024-03-20 07:59:58.084380231 +0000 UTC m=+4.097224006 container init e22b97acea5c3f0f6a0f4da6f5e91ad57a2f7d2169ff8d2dec8cbd55293c008b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T07:59:58.371 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 podman[75610]: 2024-03-20 07:59:58.090305755 +0000 UTC m=+4.103149524 container start e22b97acea5c3f0f6a0f4da6f5e91ad57a2f7d2169ff8d2dec8cbd55293c008b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T07:59:58.371 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.094Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-20T07:59:58.371 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.094Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-20T07:59:58.371 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.094Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-20T07:59:58.371 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.094Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-20T07:59:58.371 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.094Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-20T07:59:58.371 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.094Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-20T07:59:58.372 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-20T07:59:58.372 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=arp 2024-03-20T07:59:58.372 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-20T07:59:58.372 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-20T07:59:58.372 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-20T07:59:58.372 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-20T07:59:58.372 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-20T07:59:58.372 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=edac 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-20T07:59:58.373 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=os 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-20T07:59:58.374 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=stat 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=time 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=uname 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[75815]: ts=2024-03-20T07:59:58.095Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 bash[75610]: e22b97acea5c3f0f6a0f4da6f5e91ad57a2f7d2169ff8d2dec8cbd55293c008b 2024-03-20T07:59:58.375 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 07:59:58 smithi118 systemd[1]: Started Ceph node-exporter.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T07:59:59.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:59 smithi196 ceph-mon[31033]: pgmap v36: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:59.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:59 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:59 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 07:59:59 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.326 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 07:59:59 smithi196 systemd[1]: Starting Ceph node-exporter.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T07:59:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:59 smithi118 ceph-mon[27391]: pgmap v36: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:59 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:59 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 07:59:59 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:59 smithi118 ceph-mon[38981]: pgmap v36: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T07:59:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:59 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:59 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 07:59:59 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T07:59:59.639 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 07:59:59 smithi196 bash[63024]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-03-20T08:00:00.069 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:00 smithi118 ceph-mon[27391]: Deploying daemon node-exporter.b on smithi196 2024-03-20T08:00:00.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:00 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:00.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:00 smithi118 ceph-mon[27391]: overall HEALTH_OK 2024-03-20T08:00:00.070 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:00 smithi118 ceph-mon[38981]: Deploying daemon node-exporter.b on smithi196 2024-03-20T08:00:00.070 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:00 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:00.070 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:00 smithi118 ceph-mon[38981]: overall HEALTH_OK 2024-03-20T08:00:00.283 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:00.285 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:00:00.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:00 smithi196 ceph-mon[31033]: Deploying daemon node-exporter.b on smithi196 2024-03-20T08:00:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:00 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:00 smithi196 ceph-mon[31033]: overall HEALTH_OK 2024-03-20T08:00:00.389 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:00 smithi196 bash[63024]: Getting image source signatures 2024-03-20T08:00:00.389 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:00 smithi196 bash[63024]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-03-20T08:00:00.389 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:00 smithi196 bash[63024]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-03-20T08:00:00.389 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:00 smithi196 bash[63024]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-03-20T08:00:00.982 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-20T07:59:59.922821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593514,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":227,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":315,"acting":315,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":315,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":223608,"kb_used_data":7432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749737608,"statfs":{"total":767960285184,"available":767731310592,"internally_reserved":0,"allocated":7610368,"data_stored":4273899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002512"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844320+0000","last_change":"2024-03-20T07:58:26.145662+0000","last_active":"2024-03-20T07:59:04.844320+0000","last_peered":"2024-03-20T07:59:04.844320+0000","last_clean":"2024-03-20T07:59:04.844320+0000","last_became_active":"2024-03-20T07:58:26.145286+0000","last_became_peered":"2024-03-20T07:58:26.145286+0000","last_unstale":"2024-03-20T07:59:04.844320+0000","last_undegraded":"2024-03-20T07:59:04.844320+0000","last_fullsized":"2024-03-20T07:59:04.844320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984138+0000","last_change":"2024-03-20T07:58:26.129069+0000","last_active":"2024-03-20T07:59:02.984138+0000","last_peered":"2024-03-20T07:59:02.984138+0000","last_clean":"2024-03-20T07:59:02.984138+0000","last_became_active":"2024-03-20T07:58:26.128811+0000","last_became_peered":"2024-03-20T07:58:26.128811+0000","last_unstale":"2024-03-20T07:59:02.984138+0000","last_undegraded":"2024-03-20T07:59:02.984138+0000","last_fullsized":"2024-03-20T07:59:02.984138+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250498+0000","last_change":"2024-03-20T07:58:26.119030+0000","last_active":"2024-03-20T07:59:04.250498+0000","last_peered":"2024-03-20T07:59:04.250498+0000","last_clean":"2024-03-20T07:59:04.250498+0000","last_became_active":"2024-03-20T07:58:26.118755+0000","last_became_peered":"2024-03-20T07:58:26.118755+0000","last_unstale":"2024-03-20T07:59:04.250498+0000","last_undegraded":"2024-03-20T07:59:04.250498+0000","last_fullsized":"2024-03-20T07:59:04.250498+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:51:41.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986054+0000","last_change":"2024-03-20T07:58:26.138067+0000","last_active":"2024-03-20T07:59:02.986054+0000","last_peered":"2024-03-20T07:59:02.986054+0000","last_clean":"2024-03-20T07:59:02.986054+0000","last_became_active":"2024-03-20T07:58:26.137768+0000","last_became_peered":"2024-03-20T07:58:26.137768+0000","last_unstale":"2024-03-20T07:59:02.986054+0000","last_undegraded":"2024-03-20T07:59:02.986054+0000","last_fullsized":"2024-03-20T07:59:02.986054+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843727+0000","last_change":"2024-03-20T07:58:26.131923+0000","last_active":"2024-03-20T07:59:04.843727+0000","last_peered":"2024-03-20T07:59:04.843727+0000","last_clean":"2024-03-20T07:59:04.843727+0000","last_became_active":"2024-03-20T07:58:26.131585+0000","last_became_peered":"2024-03-20T07:58:26.131585+0000","last_unstale":"2024-03-20T07:59:04.843727+0000","last_undegraded":"2024-03-20T07:59:04.843727+0000","last_fullsized":"2024-03-20T07:59:04.843727+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986391+0000","last_change":"2024-03-20T07:58:26.137308+0000","last_active":"2024-03-20T07:59:02.986391+0000","last_peered":"2024-03-20T07:59:02.986391+0000","last_clean":"2024-03-20T07:59:02.986391+0000","last_became_active":"2024-03-20T07:58:26.135727+0000","last_became_peered":"2024-03-20T07:58:26.135727+0000","last_unstale":"2024-03-20T07:59:02.986391+0000","last_undegraded":"2024-03-20T07:59:02.986391+0000","last_fullsized":"2024-03-20T07:59:02.986391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843474+0000","last_change":"2024-03-20T07:58:26.132941+0000","last_active":"2024-03-20T07:59:04.843474+0000","last_peered":"2024-03-20T07:59:04.843474+0000","last_clean":"2024-03-20T07:59:04.843474+0000","last_became_active":"2024-03-20T07:58:26.132230+0000","last_became_peered":"2024-03-20T07:58:26.132230+0000","last_unstale":"2024-03-20T07:59:04.843474+0000","last_undegraded":"2024-03-20T07:59:04.843474+0000","last_fullsized":"2024-03-20T07:59:04.843474+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268423+0000","last_change":"2024-03-20T07:58:26.136216+0000","last_active":"2024-03-20T07:59:05.268423+0000","last_peered":"2024-03-20T07:59:05.268423+0000","last_clean":"2024-03-20T07:59:05.268423+0000","last_became_active":"2024-03-20T07:58:26.135920+0000","last_became_peered":"2024-03-20T07:58:26.135920+0000","last_unstale":"2024-03-20T07:59:05.268423+0000","last_undegraded":"2024-03-20T07:59:05.268423+0000","last_fullsized":"2024-03-20T07:59:05.268423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:37.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844260+0000","last_change":"2024-03-20T07:58:26.144631+0000","last_active":"2024-03-20T07:59:04.844260+0000","last_peered":"2024-03-20T07:59:04.844260+0000","last_clean":"2024-03-20T07:59:04.844260+0000","last_became_active":"2024-03-20T07:58:26.144302+0000","last_became_peered":"2024-03-20T07:58:26.144302+0000","last_unstale":"2024-03-20T07:59:04.844260+0000","last_undegraded":"2024-03-20T07:59:04.844260+0000","last_fullsized":"2024-03-20T07:59:04.844260+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156363+0000","last_change":"2024-03-20T07:58:26.138657+0000","last_active":"2024-03-20T07:59:03.156363+0000","last_peered":"2024-03-20T07:59:03.156363+0000","last_clean":"2024-03-20T07:59:03.156363+0000","last_became_active":"2024-03-20T07:58:26.138348+0000","last_became_peered":"2024-03-20T07:58:26.138348+0000","last_unstale":"2024-03-20T07:59:03.156363+0000","last_undegraded":"2024-03-20T07:59:03.156363+0000","last_fullsized":"2024-03-20T07:59:03.156363+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156427+0000","last_change":"2024-03-20T07:58:26.137961+0000","last_active":"2024-03-20T07:59:03.156427+0000","last_peered":"2024-03-20T07:59:03.156427+0000","last_clean":"2024-03-20T07:59:03.156427+0000","last_became_active":"2024-03-20T07:58:26.137672+0000","last_became_peered":"2024-03-20T07:58:26.137672+0000","last_unstale":"2024-03-20T07:59:03.156427+0000","last_undegraded":"2024-03-20T07:59:03.156427+0000","last_fullsized":"2024-03-20T07:59:03.156427+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269255+0000","last_change":"2024-03-20T07:58:26.136246+0000","last_active":"2024-03-20T07:59:05.269255+0000","last_peered":"2024-03-20T07:59:05.269255+0000","last_clean":"2024-03-20T07:59:05.269255+0000","last_became_active":"2024-03-20T07:58:26.135935+0000","last_became_peered":"2024-03-20T07:58:26.135935+0000","last_unstale":"2024-03-20T07:59:05.269255+0000","last_undegraded":"2024-03-20T07:59:05.269255+0000","last_fullsized":"2024-03-20T07:59:05.269255+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268785+0000","last_change":"2024-03-20T07:58:26.129765+0000","last_active":"2024-03-20T07:59:05.268785+0000","last_peered":"2024-03-20T07:59:05.268785+0000","last_clean":"2024-03-20T07:59:05.268785+0000","last_became_active":"2024-03-20T07:58:26.129209+0000","last_became_peered":"2024-03-20T07:58:26.129209+0000","last_unstale":"2024-03-20T07:59:05.268785+0000","last_undegraded":"2024-03-20T07:59:05.268785+0000","last_fullsized":"2024-03-20T07:59:05.268785+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:16:48.280737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250172+0000","last_change":"2024-03-20T07:58:26.123008+0000","last_active":"2024-03-20T07:59:04.250172+0000","last_peered":"2024-03-20T07:59:04.250172+0000","last_clean":"2024-03-20T07:59:04.250172+0000","last_became_active":"2024-03-20T07:58:26.122270+0000","last_became_peered":"2024-03-20T07:58:26.122270+0000","last_unstale":"2024-03-20T07:59:04.250172+0000","last_undegraded":"2024-03-20T07:59:04.250172+0000","last_fullsized":"2024-03-20T07:59:04.250172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:37.385767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983133+0000","last_change":"2024-03-20T07:58:26.127042+0000","last_active":"2024-03-20T07:59:02.983133+0000","last_peered":"2024-03-20T07:59:02.983133+0000","last_clean":"2024-03-20T07:59:02.983133+0000","last_became_active":"2024-03-20T07:58:26.126555+0000","last_became_peered":"2024-03-20T07:58:26.126555+0000","last_unstale":"2024-03-20T07:59:02.983133+0000","last_undegraded":"2024-03-20T07:59:02.983133+0000","last_fullsized":"2024-03-20T07:59:02.983133+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985736+0000","last_change":"2024-03-20T07:58:26.129086+0000","last_active":"2024-03-20T07:59:02.985736+0000","last_peered":"2024-03-20T07:59:02.985736+0000","last_clean":"2024-03-20T07:59:02.985736+0000","last_became_active":"2024-03-20T07:58:26.128111+0000","last_became_peered":"2024-03-20T07:58:26.128111+0000","last_unstale":"2024-03-20T07:59:02.985736+0000","last_undegraded":"2024-03-20T07:59:02.985736+0000","last_fullsized":"2024-03-20T07:59:02.985736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986251+0000","last_change":"2024-03-20T07:58:26.128812+0000","last_active":"2024-03-20T07:59:02.986251+0000","last_peered":"2024-03-20T07:59:02.986251+0000","last_clean":"2024-03-20T07:59:02.986251+0000","last_became_active":"2024-03-20T07:58:26.128501+0000","last_became_peered":"2024-03-20T07:58:26.128501+0000","last_unstale":"2024-03-20T07:59:02.986251+0000","last_undegraded":"2024-03-20T07:59:02.986251+0000","last_fullsized":"2024-03-20T07:59:02.986251+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984581+0000","last_change":"2024-03-20T07:58:26.128211+0000","last_active":"2024-03-20T07:59:02.984581+0000","last_peered":"2024-03-20T07:59:02.984581+0000","last_clean":"2024-03-20T07:59:02.984581+0000","last_became_active":"2024-03-20T07:58:26.127943+0000","last_became_peered":"2024-03-20T07:58:26.127943+0000","last_unstale":"2024-03-20T07:59:02.984581+0000","last_undegraded":"2024-03-20T07:59:02.984581+0000","last_fullsized":"2024-03-20T07:59:02.984581+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:19.979568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843899+0000","last_change":"2024-03-20T07:58:26.133112+0000","last_active":"2024-03-20T07:59:04.843899+0000","last_peered":"2024-03-20T07:59:04.843899+0000","last_clean":"2024-03-20T07:59:04.843899+0000","last_became_active":"2024-03-20T07:58:26.132794+0000","last_became_peered":"2024-03-20T07:58:26.132794+0000","last_unstale":"2024-03-20T07:59:04.843899+0000","last_undegraded":"2024-03-20T07:59:04.843899+0000","last_fullsized":"2024-03-20T07:59:04.843899+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"62'4","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250889+0000","last_change":"2024-03-20T07:58:25.774846+0000","last_active":"2024-03-20T07:59:04.250889+0000","last_peered":"2024-03-20T07:59:04.250889+0000","last_clean":"2024-03-20T07:59:04.250889+0000","last_became_active":"2024-03-20T07:58:25.774570+0000","last_became_peered":"2024-03-20T07:58:25.774570+0000","last_unstale":"2024-03-20T07:59:04.250889+0000","last_undegraded":"2024-03-20T07:59:04.250889+0000","last_fullsized":"2024-03-20T07:59:04.250889+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:00.513690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268967+0000","last_change":"2024-03-20T07:58:26.128852+0000","last_active":"2024-03-20T07:59:05.268967+0000","last_peered":"2024-03-20T07:59:05.268967+0000","last_clean":"2024-03-20T07:59:05.268967+0000","last_became_active":"2024-03-20T07:58:26.128584+0000","last_became_peered":"2024-03-20T07:58:26.128584+0000","last_unstale":"2024-03-20T07:59:05.268967+0000","last_undegraded":"2024-03-20T07:59:05.268967+0000","last_fullsized":"2024-03-20T07:59:05.268967+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:15:37.874514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269007+0000","last_change":"2024-03-20T07:58:25.774347+0000","last_active":"2024-03-20T07:59:05.269007+0000","last_peered":"2024-03-20T07:59:05.269007+0000","last_clean":"2024-03-20T07:59:05.269007+0000","last_became_active":"2024-03-20T07:58:25.773993+0000","last_became_peered":"2024-03-20T07:58:25.773993+0000","last_unstale":"2024-03-20T07:59:05.269007+0000","last_undegraded":"2024-03-20T07:59:05.269007+0000","last_fullsized":"2024-03-20T07:59:05.269007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:04:59.603081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250754+0000","last_change":"2024-03-20T07:58:26.118404+0000","last_active":"2024-03-20T07:59:04.250754+0000","last_peered":"2024-03-20T07:59:04.250754+0000","last_clean":"2024-03-20T07:59:04.250754+0000","last_became_active":"2024-03-20T07:58:26.118107+0000","last_became_peered":"2024-03-20T07:58:26.118107+0000","last_unstale":"2024-03-20T07:59:04.250754+0000","last_undegraded":"2024-03-20T07:59:04.250754+0000","last_fullsized":"2024-03-20T07:59:04.250754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:51.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844135+0000","last_change":"2024-03-20T07:58:25.778043+0000","last_active":"2024-03-20T07:59:04.844135+0000","last_peered":"2024-03-20T07:59:04.844135+0000","last_clean":"2024-03-20T07:59:04.844135+0000","last_became_active":"2024-03-20T07:58:25.777710+0000","last_became_peered":"2024-03-20T07:58:25.777710+0000","last_unstale":"2024-03-20T07:59:04.844135+0000","last_undegraded":"2024-03-20T07:59:04.844135+0000","last_fullsized":"2024-03-20T07:59:04.844135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986225+0000","last_change":"2024-03-20T07:58:26.122440+0000","last_active":"2024-03-20T07:59:02.986225+0000","last_peered":"2024-03-20T07:59:02.986225+0000","last_clean":"2024-03-20T07:59:02.986225+0000","last_became_active":"2024-03-20T07:58:26.122136+0000","last_became_peered":"2024-03-20T07:58:26.122136+0000","last_unstale":"2024-03-20T07:59:02.986225+0000","last_undegraded":"2024-03-20T07:59:02.986225+0000","last_fullsized":"2024-03-20T07:59:02.986225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843971+0000","last_change":"2024-03-20T07:58:25.776107+0000","last_active":"2024-03-20T07:59:04.843971+0000","last_peered":"2024-03-20T07:59:04.843971+0000","last_clean":"2024-03-20T07:59:04.843971+0000","last_became_active":"2024-03-20T07:58:25.775756+0000","last_became_peered":"2024-03-20T07:58:25.775756+0000","last_unstale":"2024-03-20T07:59:04.843971+0000","last_undegraded":"2024-03-20T07:59:04.843971+0000","last_fullsized":"2024-03-20T07:59:04.843971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983617+0000","last_change":"2024-03-20T07:58:46.699381+0000","last_active":"2024-03-20T07:59:02.983617+0000","last_peered":"2024-03-20T07:59:02.983617+0000","last_clean":"2024-03-20T07:59:02.983617+0000","last_became_active":"2024-03-20T07:58:46.698994+0000","last_became_peered":"2024-03-20T07:58:46.698994+0000","last_unstale":"2024-03-20T07:59:02.983617+0000","last_undegraded":"2024-03-20T07:59:02.983617+0000","last_fullsized":"2024-03-20T07:59:02.983617+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:31:09.474011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268838+0000","last_change":"2024-03-20T07:58:25.769988+0000","last_active":"2024-03-20T07:59:05.268838+0000","last_peered":"2024-03-20T07:59:05.268838+0000","last_clean":"2024-03-20T07:59:05.268838+0000","last_became_active":"2024-03-20T07:58:25.769359+0000","last_became_peered":"2024-03-20T07:58:25.769359+0000","last_unstale":"2024-03-20T07:59:05.268838+0000","last_undegraded":"2024-03-20T07:59:05.268838+0000","last_fullsized":"2024-03-20T07:59:05.268838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:26:35.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268931+0000","last_change":"2024-03-20T07:58:26.118793+0000","last_active":"2024-03-20T07:59:05.268931+0000","last_peered":"2024-03-20T07:59:05.268931+0000","last_clean":"2024-03-20T07:59:05.268931+0000","last_became_active":"2024-03-20T07:58:26.118430+0000","last_became_peered":"2024-03-20T07:58:26.118430+0000","last_unstale":"2024-03-20T07:59:05.268931+0000","last_undegraded":"2024-03-20T07:59:05.268931+0000","last_fullsized":"2024-03-20T07:59:05.268931+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156197+0000","last_change":"2024-03-20T07:58:25.765918+0000","last_active":"2024-03-20T07:59:03.156197+0000","last_peered":"2024-03-20T07:59:03.156197+0000","last_clean":"2024-03-20T07:59:03.156197+0000","last_became_active":"2024-03-20T07:58:25.765598+0000","last_became_peered":"2024-03-20T07:58:25.765598+0000","last_unstale":"2024-03-20T07:59:03.156197+0000","last_undegraded":"2024-03-20T07:59:03.156197+0000","last_fullsized":"2024-03-20T07:59:03.156197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250172+0000","last_change":"2024-03-20T07:58:26.119313+0000","last_active":"2024-03-20T07:59:04.250172+0000","last_peered":"2024-03-20T07:59:04.250172+0000","last_clean":"2024-03-20T07:59:04.250172+0000","last_became_active":"2024-03-20T07:58:26.118645+0000","last_became_peered":"2024-03-20T07:58:26.118645+0000","last_unstale":"2024-03-20T07:59:04.250172+0000","last_undegraded":"2024-03-20T07:59:04.250172+0000","last_fullsized":"2024-03-20T07:59:04.250172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:13.410704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268640+0000","last_change":"2024-03-20T07:58:25.771278+0000","last_active":"2024-03-20T07:59:05.268640+0000","last_peered":"2024-03-20T07:59:05.268640+0000","last_clean":"2024-03-20T07:59:05.268640+0000","last_became_active":"2024-03-20T07:58:25.770958+0000","last_became_peered":"2024-03-20T07:58:25.770958+0000","last_unstale":"2024-03-20T07:59:05.268640+0000","last_undegraded":"2024-03-20T07:59:05.268640+0000","last_fullsized":"2024-03-20T07:59:05.268640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:10:26.828071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268599+0000","last_change":"2024-03-20T07:58:26.116986+0000","last_active":"2024-03-20T07:59:05.268599+0000","last_peered":"2024-03-20T07:59:05.268599+0000","last_clean":"2024-03-20T07:59:05.268599+0000","last_became_active":"2024-03-20T07:58:26.116571+0000","last_became_peered":"2024-03-20T07:58:26.116571+0000","last_unstale":"2024-03-20T07:59:05.268599+0000","last_undegraded":"2024-03-20T07:59:05.268599+0000","last_fullsized":"2024-03-20T07:59:05.268599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:32:56.125697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985865+0000","last_change":"2024-03-20T07:58:25.769007+0000","last_active":"2024-03-20T07:59:02.985865+0000","last_peered":"2024-03-20T07:59:02.985865+0000","last_clean":"2024-03-20T07:59:02.985865+0000","last_became_active":"2024-03-20T07:58:25.768716+0000","last_became_peered":"2024-03-20T07:58:25.768716+0000","last_unstale":"2024-03-20T07:59:02.985865+0000","last_undegraded":"2024-03-20T07:59:02.985865+0000","last_fullsized":"2024-03-20T07:59:02.985865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983351+0000","last_change":"2024-03-20T07:58:26.119518+0000","last_active":"2024-03-20T07:59:02.983351+0000","last_peered":"2024-03-20T07:59:02.983351+0000","last_clean":"2024-03-20T07:59:02.983351+0000","last_became_active":"2024-03-20T07:58:26.119184+0000","last_became_peered":"2024-03-20T07:58:26.119184+0000","last_unstale":"2024-03-20T07:59:02.983351+0000","last_undegraded":"2024-03-20T07:59:02.983351+0000","last_fullsized":"2024-03-20T07:59:02.983351+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156497+0000","last_change":"2024-03-20T07:58:25.769300+0000","last_active":"2024-03-20T07:59:03.156497+0000","last_peered":"2024-03-20T07:59:03.156497+0000","last_clean":"2024-03-20T07:59:03.156497+0000","last_became_active":"2024-03-20T07:58:25.768916+0000","last_became_peered":"2024-03-20T07:58:25.768916+0000","last_unstale":"2024-03-20T07:59:03.156497+0000","last_undegraded":"2024-03-20T07:59:03.156497+0000","last_fullsized":"2024-03-20T07:59:03.156497+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986288+0000","last_change":"2024-03-20T07:58:26.130616+0000","last_active":"2024-03-20T07:59:02.986288+0000","last_peered":"2024-03-20T07:59:02.986288+0000","last_clean":"2024-03-20T07:59:02.986288+0000","last_became_active":"2024-03-20T07:58:26.130332+0000","last_became_peered":"2024-03-20T07:58:26.130332+0000","last_unstale":"2024-03-20T07:59:02.986288+0000","last_undegraded":"2024-03-20T07:59:02.986288+0000","last_fullsized":"2024-03-20T07:59:02.986288+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986375+0000","last_change":"2024-03-20T07:58:25.769790+0000","last_active":"2024-03-20T07:59:02.986375+0000","last_peered":"2024-03-20T07:59:02.986375+0000","last_clean":"2024-03-20T07:59:02.986375+0000","last_became_active":"2024-03-20T07:58:25.769492+0000","last_became_peered":"2024-03-20T07:58:25.769492+0000","last_unstale":"2024-03-20T07:59:02.986375+0000","last_undegraded":"2024-03-20T07:59:02.986375+0000","last_fullsized":"2024-03-20T07:59:02.986375+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250679+0000","last_change":"2024-03-20T07:58:26.116344+0000","last_active":"2024-03-20T07:59:04.250679+0000","last_peered":"2024-03-20T07:59:04.250679+0000","last_clean":"2024-03-20T07:59:04.250679+0000","last_became_active":"2024-03-20T07:58:26.116037+0000","last_became_peered":"2024-03-20T07:58:26.116037+0000","last_unstale":"2024-03-20T07:59:04.250679+0000","last_undegraded":"2024-03-20T07:59:04.250679+0000","last_fullsized":"2024-03-20T07:59:04.250679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:16.183856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983935+0000","last_change":"2024-03-20T07:58:25.770988+0000","last_active":"2024-03-20T07:59:02.983935+0000","last_peered":"2024-03-20T07:59:02.983935+0000","last_clean":"2024-03-20T07:59:02.983935+0000","last_became_active":"2024-03-20T07:58:25.770657+0000","last_became_peered":"2024-03-20T07:58:25.770657+0000","last_unstale":"2024-03-20T07:59:02.983935+0000","last_undegraded":"2024-03-20T07:59:02.983935+0000","last_fullsized":"2024-03-20T07:59:02.983935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269168+0000","last_change":"2024-03-20T07:58:26.128506+0000","last_active":"2024-03-20T07:59:05.269168+0000","last_peered":"2024-03-20T07:59:05.269168+0000","last_clean":"2024-03-20T07:59:05.269168+0000","last_became_active":"2024-03-20T07:58:26.128201+0000","last_became_peered":"2024-03-20T07:58:26.128201+0000","last_unstale":"2024-03-20T07:59:05.269168+0000","last_undegraded":"2024-03-20T07:59:05.269168+0000","last_fullsized":"2024-03-20T07:59:05.269168+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"66'9","reported_seq":28,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156321+0000","last_change":"2024-03-20T07:58:25.767830+0000","last_active":"2024-03-20T07:59:03.156321+0000","last_peered":"2024-03-20T07:59:03.156321+0000","last_clean":"2024-03-20T07:59:03.156321+0000","last_became_active":"2024-03-20T07:58:25.767533+0000","last_became_peered":"2024-03-20T07:58:25.767533+0000","last_unstale":"2024-03-20T07:59:03.156321+0000","last_undegraded":"2024-03-20T07:59:03.156321+0000","last_fullsized":"2024-03-20T07:59:03.156321+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2348,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843983+0000","last_change":"2024-03-20T07:58:26.140127+0000","last_active":"2024-03-20T07:59:04.843983+0000","last_peered":"2024-03-20T07:59:04.843983+0000","last_clean":"2024-03-20T07:59:04.843983+0000","last_became_active":"2024-03-20T07:58:26.139362+0000","last_became_peered":"2024-03-20T07:58:26.139362+0000","last_unstale":"2024-03-20T07:59:04.843983+0000","last_undegraded":"2024-03-20T07:59:04.843983+0000","last_fullsized":"2024-03-20T07:59:04.843983+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250511+0000","last_change":"2024-03-20T07:58:25.771772+0000","last_active":"2024-03-20T07:59:04.250511+0000","last_peered":"2024-03-20T07:59:04.250511+0000","last_clean":"2024-03-20T07:59:04.250511+0000","last_became_active":"2024-03-20T07:58:25.771499+0000","last_became_peered":"2024-03-20T07:58:25.771499+0000","last_unstale":"2024-03-20T07:59:04.250511+0000","last_undegraded":"2024-03-20T07:59:04.250511+0000","last_fullsized":"2024-03-20T07:59:04.250511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:19:45.833311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250093+0000","last_change":"2024-03-20T07:58:26.122549+0000","last_active":"2024-03-20T07:59:04.250093+0000","last_peered":"2024-03-20T07:59:04.250093+0000","last_clean":"2024-03-20T07:59:04.250093+0000","last_became_active":"2024-03-20T07:58:26.121780+0000","last_became_peered":"2024-03-20T07:58:26.121780+0000","last_unstale":"2024-03-20T07:59:04.250093+0000","last_undegraded":"2024-03-20T07:59:04.250093+0000","last_fullsized":"2024-03-20T07:59:04.250093+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:37:01.164741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985855+0000","last_change":"2024-03-20T07:58:25.771839+0000","last_active":"2024-03-20T07:59:02.985855+0000","last_peered":"2024-03-20T07:59:02.985855+0000","last_clean":"2024-03-20T07:59:02.985855+0000","last_became_active":"2024-03-20T07:58:25.771464+0000","last_became_peered":"2024-03-20T07:58:25.771464+0000","last_unstale":"2024-03-20T07:59:02.985855+0000","last_undegraded":"2024-03-20T07:59:02.985855+0000","last_fullsized":"2024-03-20T07:59:02.985855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843684+0000","last_change":"2024-03-20T07:58:26.129094+0000","last_active":"2024-03-20T07:59:04.843684+0000","last_peered":"2024-03-20T07:59:04.843684+0000","last_clean":"2024-03-20T07:59:04.843684+0000","last_became_active":"2024-03-20T07:58:26.128675+0000","last_became_peered":"2024-03-20T07:58:26.128675+0000","last_unstale":"2024-03-20T07:59:04.843684+0000","last_undegraded":"2024-03-20T07:59:04.843684+0000","last_fullsized":"2024-03-20T07:59:04.843684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156125+0000","last_change":"2024-03-20T07:58:25.767586+0000","last_active":"2024-03-20T07:59:03.156125+0000","last_peered":"2024-03-20T07:59:03.156125+0000","last_clean":"2024-03-20T07:59:03.156125+0000","last_became_active":"2024-03-20T07:58:25.767268+0000","last_became_peered":"2024-03-20T07:58:25.767268+0000","last_unstale":"2024-03-20T07:59:03.156125+0000","last_undegraded":"2024-03-20T07:59:03.156125+0000","last_fullsized":"2024-03-20T07:59:03.156125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156050+0000","last_change":"2024-03-20T07:58:26.139925+0000","last_active":"2024-03-20T07:59:03.156050+0000","last_peered":"2024-03-20T07:59:03.156050+0000","last_clean":"2024-03-20T07:59:03.156050+0000","last_became_active":"2024-03-20T07:58:26.138286+0000","last_became_peered":"2024-03-20T07:58:26.138286+0000","last_unstale":"2024-03-20T07:59:03.156050+0000","last_undegraded":"2024-03-20T07:59:03.156050+0000","last_fullsized":"2024-03-20T07:59:03.156050+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":27,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983262+0000","last_change":"2024-03-20T07:58:25.757217+0000","last_active":"2024-03-20T07:59:02.983262+0000","last_peered":"2024-03-20T07:59:02.983262+0000","last_clean":"2024-03-20T07:59:02.983262+0000","last_became_active":"2024-03-20T07:58:25.756879+0000","last_became_peered":"2024-03-20T07:58:25.756879+0000","last_unstale":"2024-03-20T07:59:02.983262+0000","last_undegraded":"2024-03-20T07:59:02.983262+0000","last_fullsized":"2024-03-20T07:59:02.983262+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:28.214822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843593+0000","last_change":"2024-03-20T07:58:26.133140+0000","last_active":"2024-03-20T07:59:04.843593+0000","last_peered":"2024-03-20T07:59:04.843593+0000","last_clean":"2024-03-20T07:59:04.843593+0000","last_became_active":"2024-03-20T07:58:26.132630+0000","last_became_peered":"2024-03-20T07:58:26.132630+0000","last_unstale":"2024-03-20T07:59:04.843593+0000","last_undegraded":"2024-03-20T07:59:04.843593+0000","last_fullsized":"2024-03-20T07:59:04.843593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269456+0000","last_change":"2024-03-20T07:58:25.771322+0000","last_active":"2024-03-20T07:59:05.269456+0000","last_peered":"2024-03-20T07:59:05.269456+0000","last_clean":"2024-03-20T07:59:05.269456+0000","last_became_active":"2024-03-20T07:58:25.770984+0000","last_became_peered":"2024-03-20T07:58:25.770984+0000","last_unstale":"2024-03-20T07:59:05.269456+0000","last_undegraded":"2024-03-20T07:59:05.269456+0000","last_fullsized":"2024-03-20T07:59:05.269456+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844196+0000","last_change":"2024-03-20T07:58:26.144254+0000","last_active":"2024-03-20T07:59:04.844196+0000","last_peered":"2024-03-20T07:59:04.844196+0000","last_clean":"2024-03-20T07:59:04.844196+0000","last_became_active":"2024-03-20T07:58:26.143683+0000","last_became_peered":"2024-03-20T07:58:26.143683+0000","last_unstale":"2024-03-20T07:59:04.844196+0000","last_undegraded":"2024-03-20T07:59:04.844196+0000","last_fullsized":"2024-03-20T07:59:04.844196+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.912606+0000","last_change":"2024-03-20T07:59:01.988226+0000","last_active":"2024-03-20T07:59:05.912606+0000","last_peered":"2024-03-20T07:59:05.912606+0000","last_clean":"2024-03-20T07:59:05.912606+0000","last_became_active":"2024-03-20T07:59:01.987867+0000","last_became_peered":"2024-03-20T07:59:01.987867+0000","last_unstale":"2024-03-20T07:59:05.912606+0000","last_undegraded":"2024-03-20T07:59:05.912606+0000","last_fullsized":"2024-03-20T07:59:05.912606+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:14:23.583126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983174+0000","last_change":"2024-03-20T07:58:25.773911+0000","last_active":"2024-03-20T07:59:02.983174+0000","last_peered":"2024-03-20T07:59:02.983174+0000","last_clean":"2024-03-20T07:59:02.983174+0000","last_became_active":"2024-03-20T07:58:25.773526+0000","last_became_peered":"2024-03-20T07:58:25.773526+0000","last_unstale":"2024-03-20T07:59:02.983174+0000","last_undegraded":"2024-03-20T07:59:02.983174+0000","last_fullsized":"2024-03-20T07:59:02.983174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986145+0000","last_change":"2024-03-20T07:58:26.122909+0000","last_active":"2024-03-20T07:59:02.986145+0000","last_peered":"2024-03-20T07:59:02.986145+0000","last_clean":"2024-03-20T07:59:02.986145+0000","last_became_active":"2024-03-20T07:58:26.121608+0000","last_became_peered":"2024-03-20T07:58:26.121608+0000","last_unstale":"2024-03-20T07:59:02.986145+0000","last_undegraded":"2024-03-20T07:59:02.986145+0000","last_fullsized":"2024-03-20T07:59:02.986145+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.912657+0000","last_change":"2024-03-20T07:59:01.991385+0000","last_active":"2024-03-20T07:59:05.912657+0000","last_peered":"2024-03-20T07:59:05.912657+0000","last_clean":"2024-03-20T07:59:05.912657+0000","last_became_active":"2024-03-20T07:59:01.991077+0000","last_became_peered":"2024-03-20T07:59:01.991077+0000","last_unstale":"2024-03-20T07:59:05.912657+0000","last_undegraded":"2024-03-20T07:59:05.912657+0000","last_fullsized":"2024-03-20T07:59:05.912657+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:31:07.169554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986443+0000","last_change":"2024-03-20T07:58:25.770569+0000","last_active":"2024-03-20T07:59:02.986443+0000","last_peered":"2024-03-20T07:59:02.986443+0000","last_clean":"2024-03-20T07:59:02.986443+0000","last_became_active":"2024-03-20T07:58:25.770298+0000","last_became_peered":"2024-03-20T07:58:25.770298+0000","last_unstale":"2024-03-20T07:59:02.986443+0000","last_undegraded":"2024-03-20T07:59:02.986443+0000","last_fullsized":"2024-03-20T07:59:02.986443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269157+0000","last_change":"2024-03-20T07:58:26.136200+0000","last_active":"2024-03-20T07:59:05.269157+0000","last_peered":"2024-03-20T07:59:05.269157+0000","last_clean":"2024-03-20T07:59:05.269157+0000","last_became_active":"2024-03-20T07:58:26.135909+0000","last_became_peered":"2024-03-20T07:58:26.135909+0000","last_unstale":"2024-03-20T07:59:05.269157+0000","last_undegraded":"2024-03-20T07:59:05.269157+0000","last_fullsized":"2024-03-20T07:59:05.269157+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.912725+0000","last_change":"2024-03-20T07:59:01.988864+0000","last_active":"2024-03-20T07:59:05.912725+0000","last_peered":"2024-03-20T07:59:05.912725+0000","last_clean":"2024-03-20T07:59:05.912725+0000","last_became_active":"2024-03-20T07:59:01.988575+0000","last_became_peered":"2024-03-20T07:59:01.988575+0000","last_unstale":"2024-03-20T07:59:05.912725+0000","last_undegraded":"2024-03-20T07:59:05.912725+0000","last_fullsized":"2024-03-20T07:59:05.912725+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:19:29.334160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.982959+0000","last_change":"2024-03-20T07:58:26.130079+0000","last_active":"2024-03-20T07:59:02.982959+0000","last_peered":"2024-03-20T07:59:02.982959+0000","last_clean":"2024-03-20T07:59:02.982959+0000","last_became_active":"2024-03-20T07:58:26.129798+0000","last_became_peered":"2024-03-20T07:58:26.129798+0000","last_unstale":"2024-03-20T07:59:02.982959+0000","last_undegraded":"2024-03-20T07:59:02.982959+0000","last_fullsized":"2024-03-20T07:59:02.982959+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983444+0000","last_change":"2024-03-20T07:58:25.764736+0000","last_active":"2024-03-20T07:59:02.983444+0000","last_peered":"2024-03-20T07:59:02.983444+0000","last_clean":"2024-03-20T07:59:02.983444+0000","last_became_active":"2024-03-20T07:58:25.764390+0000","last_became_peered":"2024-03-20T07:58:25.764390+0000","last_unstale":"2024-03-20T07:59:02.983444+0000","last_undegraded":"2024-03-20T07:59:02.983444+0000","last_fullsized":"2024-03-20T07:59:02.983444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:27:58.881448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.911953+0000","last_change":"2024-03-20T07:59:01.994090+0000","last_active":"2024-03-20T07:59:05.911953+0000","last_peered":"2024-03-20T07:59:05.911953+0000","last_clean":"2024-03-20T07:59:05.911953+0000","last_became_active":"2024-03-20T07:59:01.993550+0000","last_became_peered":"2024-03-20T07:59:01.993550+0000","last_unstale":"2024-03-20T07:59:05.911953+0000","last_undegraded":"2024-03-20T07:59:05.911953+0000","last_fullsized":"2024-03-20T07:59:05.911953+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:42:05.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268817+0000","last_change":"2024-03-20T07:58:26.136285+0000","last_active":"2024-03-20T07:59:05.268817+0000","last_peered":"2024-03-20T07:59:05.268817+0000","last_clean":"2024-03-20T07:59:05.268817+0000","last_became_active":"2024-03-20T07:58:26.135969+0000","last_became_peered":"2024-03-20T07:58:26.135969+0000","last_unstale":"2024-03-20T07:59:05.268817+0000","last_undegraded":"2024-03-20T07:59:05.268817+0000","last_fullsized":"2024-03-20T07:59:05.268817+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:16.328342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985973+0000","last_change":"2024-03-20T07:58:25.766643+0000","last_active":"2024-03-20T07:59:02.985973+0000","last_peered":"2024-03-20T07:59:02.985973+0000","last_clean":"2024-03-20T07:59:02.985973+0000","last_became_active":"2024-03-20T07:58:25.766341+0000","last_became_peered":"2024-03-20T07:58:25.766341+0000","last_unstale":"2024-03-20T07:59:02.985973+0000","last_undegraded":"2024-03-20T07:59:02.985973+0000","last_fullsized":"2024-03-20T07:59:02.985973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"68'128","reported_seq":265,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843565+0000","last_change":"2024-03-20T07:58:09.964395+0000","last_active":"2024-03-20T07:59:04.843565+0000","last_peered":"2024-03-20T07:59:04.843565+0000","last_clean":"2024-03-20T07:59:04.843565+0000","last_became_active":"2024-03-20T07:58:09.963927+0000","last_became_peered":"2024-03-20T07:58:09.963927+0000","last_unstale":"2024-03-20T07:59:04.843565+0000","last_undegraded":"2024-03-20T07:59:04.843565+0000","last_fullsized":"2024-03-20T07:59:04.843565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.385424+0000","last_change":"2024-03-20T07:59:01.989392+0000","last_active":"2024-03-20T07:59:05.385424+0000","last_peered":"2024-03-20T07:59:05.385424+0000","last_clean":"2024-03-20T07:59:05.385424+0000","last_became_active":"2024-03-20T07:59:01.989129+0000","last_became_peered":"2024-03-20T07:59:01.989129+0000","last_unstale":"2024-03-20T07:59:05.385424+0000","last_undegraded":"2024-03-20T07:59:05.385424+0000","last_fullsized":"2024-03-20T07:59:05.385424+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:32.163295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250102+0000","last_change":"2024-03-20T07:58:26.125436+0000","last_active":"2024-03-20T07:59:04.250102+0000","last_peered":"2024-03-20T07:59:04.250102+0000","last_clean":"2024-03-20T07:59:04.250102+0000","last_became_active":"2024-03-20T07:58:26.125161+0000","last_became_peered":"2024-03-20T07:58:26.125161+0000","last_unstale":"2024-03-20T07:59:04.250102+0000","last_undegraded":"2024-03-20T07:59:04.250102+0000","last_fullsized":"2024-03-20T07:59:04.250102+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:53:55.085832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843400+0000","last_change":"2024-03-20T07:58:25.779129+0000","last_active":"2024-03-20T07:59:04.843400+0000","last_peered":"2024-03-20T07:59:04.843400+0000","last_clean":"2024-03-20T07:59:04.843400+0000","last_became_active":"2024-03-20T07:58:25.778258+0000","last_became_peered":"2024-03-20T07:58:25.778258+0000","last_unstale":"2024-03-20T07:59:04.843400+0000","last_undegraded":"2024-03-20T07:59:04.843400+0000","last_fullsized":"2024-03-20T07:59:04.843400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.384551+0000","last_change":"2024-03-20T07:59:01.995060+0000","last_active":"2024-03-20T07:59:05.384551+0000","last_peered":"2024-03-20T07:59:05.384551+0000","last_clean":"2024-03-20T07:59:05.384551+0000","last_became_active":"2024-03-20T07:59:01.994854+0000","last_became_peered":"2024-03-20T07:59:01.994854+0000","last_unstale":"2024-03-20T07:59:05.384551+0000","last_undegraded":"2024-03-20T07:59:05.384551+0000","last_fullsized":"2024-03-20T07:59:05.384551+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:00.138918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844108+0000","last_change":"2024-03-20T07:58:25.777973+0000","last_active":"2024-03-20T07:59:04.844108+0000","last_peered":"2024-03-20T07:59:04.844108+0000","last_clean":"2024-03-20T07:59:04.844108+0000","last_became_active":"2024-03-20T07:58:25.777666+0000","last_became_peered":"2024-03-20T07:58:25.777666+0000","last_unstale":"2024-03-20T07:59:04.844108+0000","last_undegraded":"2024-03-20T07:59:04.844108+0000","last_fullsized":"2024-03-20T07:59:04.844108+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983574+0000","last_change":"2024-03-20T07:58:26.126692+0000","last_active":"2024-03-20T07:59:02.983574+0000","last_peered":"2024-03-20T07:59:02.983574+0000","last_clean":"2024-03-20T07:59:02.983574+0000","last_became_active":"2024-03-20T07:58:26.126397+0000","last_became_peered":"2024-03-20T07:58:26.126397+0000","last_unstale":"2024-03-20T07:59:02.983574+0000","last_undegraded":"2024-03-20T07:59:02.983574+0000","last_fullsized":"2024-03-20T07:59:02.983574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.437164+0000","last_change":"2024-03-20T07:59:01.993571+0000","last_active":"2024-03-20T07:59:05.437164+0000","last_peered":"2024-03-20T07:59:05.437164+0000","last_clean":"2024-03-20T07:59:05.437164+0000","last_became_active":"2024-03-20T07:59:01.993283+0000","last_became_peered":"2024-03-20T07:59:01.993283+0000","last_unstale":"2024-03-20T07:59:05.437164+0000","last_undegraded":"2024-03-20T07:59:05.437164+0000","last_fullsized":"2024-03-20T07:59:05.437164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:02:21.502002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"63'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984275+0000","last_change":"2024-03-20T07:58:25.769337+0000","last_active":"2024-03-20T07:59:02.984275+0000","last_peered":"2024-03-20T07:59:02.984275+0000","last_clean":"2024-03-20T07:59:02.984275+0000","last_became_active":"2024-03-20T07:58:25.769010+0000","last_became_peered":"2024-03-20T07:58:25.769010+0000","last_unstale":"2024-03-20T07:59:02.984275+0000","last_undegraded":"2024-03-20T07:59:02.984275+0000","last_fullsized":"2024-03-20T07:59:02.984275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269021+0000","last_change":"2024-03-20T07:58:26.118863+0000","last_active":"2024-03-20T07:59:05.269021+0000","last_peered":"2024-03-20T07:59:05.269021+0000","last_clean":"2024-03-20T07:59:05.269021+0000","last_became_active":"2024-03-20T07:58:26.118565+0000","last_became_peered":"2024-03-20T07:58:26.118565+0000","last_unstale":"2024-03-20T07:59:05.269021+0000","last_undegraded":"2024-03-20T07:59:05.269021+0000","last_fullsized":"2024-03-20T07:59:05.269021+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:49:33.981106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.913562+0000","last_change":"2024-03-20T07:59:01.989083+0000","last_active":"2024-03-20T07:59:05.913562+0000","last_peered":"2024-03-20T07:59:05.913562+0000","last_clean":"2024-03-20T07:59:05.913562+0000","last_became_active":"2024-03-20T07:59:01.988797+0000","last_became_peered":"2024-03-20T07:59:01.988797+0000","last_unstale":"2024-03-20T07:59:05.913562+0000","last_undegraded":"2024-03-20T07:59:05.913562+0000","last_fullsized":"2024-03-20T07:59:05.913562+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:42:40.380304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156515+0000","last_change":"2024-03-20T07:58:25.771097+0000","last_active":"2024-03-20T07:59:03.156515+0000","last_peered":"2024-03-20T07:59:03.156515+0000","last_clean":"2024-03-20T07:59:03.156515+0000","last_became_active":"2024-03-20T07:58:25.770737+0000","last_became_peered":"2024-03-20T07:58:25.770737+0000","last_unstale":"2024-03-20T07:59:03.156515+0000","last_undegraded":"2024-03-20T07:59:03.156515+0000","last_fullsized":"2024-03-20T07:59:03.156515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984463+0000","last_change":"2024-03-20T07:58:46.699298+0000","last_active":"2024-03-20T07:59:02.984463+0000","last_peered":"2024-03-20T07:59:02.984463+0000","last_clean":"2024-03-20T07:59:02.984463+0000","last_became_active":"2024-03-20T07:58:46.698841+0000","last_became_peered":"2024-03-20T07:58:46.698841+0000","last_unstale":"2024-03-20T07:59:02.984463+0000","last_undegraded":"2024-03-20T07:59:02.984463+0000","last_fullsized":"2024-03-20T07:59:02.984463+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:02:06.660176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843627+0000","last_change":"2024-03-20T07:58:25.777961+0000","last_active":"2024-03-20T07:59:04.843627+0000","last_peered":"2024-03-20T07:59:04.843627+0000","last_clean":"2024-03-20T07:59:04.843627+0000","last_became_active":"2024-03-20T07:58:25.777649+0000","last_became_peered":"2024-03-20T07:58:25.777649+0000","last_unstale":"2024-03-20T07:59:04.843627+0000","last_undegraded":"2024-03-20T07:59:04.843627+0000","last_fullsized":"2024-03-20T07:59:04.843627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269114+0000","last_change":"2024-03-20T07:58:26.120305+0000","last_active":"2024-03-20T07:59:05.269114+0000","last_peered":"2024-03-20T07:59:05.269114+0000","last_clean":"2024-03-20T07:59:05.269114+0000","last_became_active":"2024-03-20T07:58:26.119966+0000","last_became_peered":"2024-03-20T07:58:26.119966+0000","last_unstale":"2024-03-20T07:59:05.269114+0000","last_undegraded":"2024-03-20T07:59:05.269114+0000","last_fullsized":"2024-03-20T07:59:05.269114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983215+0000","last_change":"2024-03-20T07:58:25.765548+0000","last_active":"2024-03-20T07:59:02.983215+0000","last_peered":"2024-03-20T07:59:02.983215+0000","last_clean":"2024-03-20T07:59:02.983215+0000","last_became_active":"2024-03-20T07:58:25.765273+0000","last_became_peered":"2024-03-20T07:58:25.765273+0000","last_unstale":"2024-03-20T07:59:02.983215+0000","last_undegraded":"2024-03-20T07:59:02.983215+0000","last_fullsized":"2024-03-20T07:59:02.983215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268509+0000","last_change":"2024-03-20T07:58:26.117669+0000","last_active":"2024-03-20T07:59:05.268509+0000","last_peered":"2024-03-20T07:59:05.268509+0000","last_clean":"2024-03-20T07:59:05.268509+0000","last_became_active":"2024-03-20T07:58:26.117376+0000","last_became_peered":"2024-03-20T07:58:26.117376+0000","last_unstale":"2024-03-20T07:59:05.268509+0000","last_undegraded":"2024-03-20T07:59:05.268509+0000","last_fullsized":"2024-03-20T07:59:05.268509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:42:02.274133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983347+0000","last_change":"2024-03-20T07:58:25.773811+0000","last_active":"2024-03-20T07:59:02.983347+0000","last_peered":"2024-03-20T07:59:02.983347+0000","last_clean":"2024-03-20T07:59:02.983347+0000","last_became_active":"2024-03-20T07:58:25.773520+0000","last_became_peered":"2024-03-20T07:58:25.773520+0000","last_unstale":"2024-03-20T07:59:02.983347+0000","last_undegraded":"2024-03-20T07:59:02.983347+0000","last_fullsized":"2024-03-20T07:59:02.983347+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268703+0000","last_change":"2024-03-20T07:58:26.123160+0000","last_active":"2024-03-20T07:59:05.268703+0000","last_peered":"2024-03-20T07:59:05.268703+0000","last_clean":"2024-03-20T07:59:05.268703+0000","last_became_active":"2024-03-20T07:58:26.122680+0000","last_became_peered":"2024-03-20T07:58:26.122680+0000","last_unstale":"2024-03-20T07:59:05.268703+0000","last_undegraded":"2024-03-20T07:59:05.268703+0000","last_fullsized":"2024-03-20T07:59:05.268703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:56.675972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"62'2","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983403+0000","last_change":"2024-03-20T07:58:46.309368+0000","last_active":"2024-03-20T07:59:02.983403+0000","last_peered":"2024-03-20T07:59:02.983403+0000","last_clean":"2024-03-20T07:59:02.983403+0000","last_became_active":"2024-03-20T07:58:46.306388+0000","last_became_peered":"2024-03-20T07:58:46.306388+0000","last_unstale":"2024-03-20T07:59:02.983403+0000","last_undegraded":"2024-03-20T07:59:02.983403+0000","last_fullsized":"2024-03-20T07:59:02.983403+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:42:19.060358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156129+0000","last_change":"2024-03-20T07:58:26.134067+0000","last_active":"2024-03-20T07:59:03.156129+0000","last_peered":"2024-03-20T07:59:03.156129+0000","last_clean":"2024-03-20T07:59:03.156129+0000","last_became_active":"2024-03-20T07:58:26.133784+0000","last_became_peered":"2024-03-20T07:58:26.133784+0000","last_unstale":"2024-03-20T07:59:03.156129+0000","last_undegraded":"2024-03-20T07:59:03.156129+0000","last_fullsized":"2024-03-20T07:59:03.156129+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983922+0000","last_change":"2024-03-20T07:58:25.763015+0000","last_active":"2024-03-20T07:59:02.983922+0000","last_peered":"2024-03-20T07:59:02.983922+0000","last_clean":"2024-03-20T07:59:02.983922+0000","last_became_active":"2024-03-20T07:58:25.762682+0000","last_became_peered":"2024-03-20T07:58:25.762682+0000","last_unstale":"2024-03-20T07:59:02.983922+0000","last_undegraded":"2024-03-20T07:59:02.983922+0000","last_fullsized":"2024-03-20T07:59:02.983922+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:21:45.137669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844015+0000","last_change":"2024-03-20T07:58:26.142445+0000","last_active":"2024-03-20T07:59:04.844015+0000","last_peered":"2024-03-20T07:59:04.844015+0000","last_clean":"2024-03-20T07:59:04.844015+0000","last_became_active":"2024-03-20T07:58:26.142148+0000","last_became_peered":"2024-03-20T07:58:26.142148+0000","last_unstale":"2024-03-20T07:59:04.844015+0000","last_undegraded":"2024-03-20T07:59:04.844015+0000","last_fullsized":"2024-03-20T07:59:04.844015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983793+0000","last_change":"2024-03-20T07:58:25.766427+0000","last_active":"2024-03-20T07:59:02.983793+0000","last_peered":"2024-03-20T07:59:02.983793+0000","last_clean":"2024-03-20T07:59:02.983793+0000","last_became_active":"2024-03-20T07:58:25.766111+0000","last_became_peered":"2024-03-20T07:58:25.766111+0000","last_unstale":"2024-03-20T07:59:02.983793+0000","last_undegraded":"2024-03-20T07:59:02.983793+0000","last_fullsized":"2024-03-20T07:59:02.983793+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986095+0000","last_change":"2024-03-20T07:58:26.135339+0000","last_active":"2024-03-20T07:59:02.986095+0000","last_peered":"2024-03-20T07:59:02.986095+0000","last_clean":"2024-03-20T07:59:02.986095+0000","last_became_active":"2024-03-20T07:58:26.134903+0000","last_became_peered":"2024-03-20T07:58:26.134903+0000","last_unstale":"2024-03-20T07:59:02.986095+0000","last_undegraded":"2024-03-20T07:59:02.986095+0000","last_fullsized":"2024-03-20T07:59:02.986095+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984547+0000","last_change":"2024-03-20T07:58:25.766233+0000","last_active":"2024-03-20T07:59:02.984547+0000","last_peered":"2024-03-20T07:59:02.984547+0000","last_clean":"2024-03-20T07:59:02.984547+0000","last_became_active":"2024-03-20T07:58:25.765948+0000","last_became_peered":"2024-03-20T07:58:25.765948+0000","last_unstale":"2024-03-20T07:59:02.984547+0000","last_undegraded":"2024-03-20T07:59:02.984547+0000","last_fullsized":"2024-03-20T07:59:02.984547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:34:01.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844255+0000","last_change":"2024-03-20T07:58:26.144008+0000","last_active":"2024-03-20T07:59:04.844255+0000","last_peered":"2024-03-20T07:59:04.844255+0000","last_clean":"2024-03-20T07:59:04.844255+0000","last_became_active":"2024-03-20T07:58:26.143696+0000","last_became_peered":"2024-03-20T07:58:26.143696+0000","last_unstale":"2024-03-20T07:59:04.844255+0000","last_undegraded":"2024-03-20T07:59:04.844255+0000","last_fullsized":"2024-03-20T07:59:04.844255+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.155990+0000","last_change":"2024-03-20T07:58:26.138096+0000","last_active":"2024-03-20T07:59:03.155990+0000","last_peered":"2024-03-20T07:59:03.155990+0000","last_clean":"2024-03-20T07:59:03.155990+0000","last_became_active":"2024-03-20T07:58:26.137804+0000","last_became_peered":"2024-03-20T07:58:26.137804+0000","last_unstale":"2024-03-20T07:59:03.155990+0000","last_undegraded":"2024-03-20T07:59:03.155990+0000","last_fullsized":"2024-03-20T07:59:03.155990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983522+0000","last_change":"2024-03-20T07:58:26.126085+0000","last_active":"2024-03-20T07:59:02.983522+0000","last_peered":"2024-03-20T07:59:02.983522+0000","last_clean":"2024-03-20T07:59:02.983522+0000","last_became_active":"2024-03-20T07:58:26.125793+0000","last_became_peered":"2024-03-20T07:58:26.125793+0000","last_unstale":"2024-03-20T07:59:02.983522+0000","last_undegraded":"2024-03-20T07:59:02.983522+0000","last_fullsized":"2024-03-20T07:59:02.983522+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:11:45.470755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985945+0000","last_change":"2024-03-20T07:58:26.125004+0000","last_active":"2024-03-20T07:59:02.985945+0000","last_peered":"2024-03-20T07:59:02.985945+0000","last_clean":"2024-03-20T07:59:02.985945+0000","last_became_active":"2024-03-20T07:58:26.124689+0000","last_became_peered":"2024-03-20T07:58:26.124689+0000","last_unstale":"2024-03-20T07:59:02.985945+0000","last_undegraded":"2024-03-20T07:59:02.985945+0000","last_fullsized":"2024-03-20T07:59:02.985945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250247+0000","last_change":"2024-03-20T07:58:26.120048+0000","last_active":"2024-03-20T07:59:04.250247+0000","last_peered":"2024-03-20T07:59:04.250247+0000","last_clean":"2024-03-20T07:59:04.250247+0000","last_became_active":"2024-03-20T07:58:26.119755+0000","last_became_peered":"2024-03-20T07:58:26.119755+0000","last_unstale":"2024-03-20T07:59:04.250247+0000","last_undegraded":"2024-03-20T07:59:04.250247+0000","last_fullsized":"2024-03-20T07:59:04.250247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:39:07.689571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268900+0000","last_change":"2024-03-20T07:58:46.699520+0000","last_active":"2024-03-20T07:59:05.268900+0000","last_peered":"2024-03-20T07:59:05.268900+0000","last_clean":"2024-03-20T07:59:05.268900+0000","last_became_active":"2024-03-20T07:58:46.699085+0000","last_became_peered":"2024-03-20T07:58:46.699085+0000","last_unstale":"2024-03-20T07:59:05.268900+0000","last_undegraded":"2024-03-20T07:59:05.268900+0000","last_fullsized":"2024-03-20T07:59:05.268900+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:27.802971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983711+0000","last_change":"2024-03-20T07:58:26.126113+0000","last_active":"2024-03-20T07:59:02.983711+0000","last_peered":"2024-03-20T07:59:02.983711+0000","last_clean":"2024-03-20T07:59:02.983711+0000","last_became_active":"2024-03-20T07:58:26.122635+0000","last_became_peered":"2024-03-20T07:58:26.122635+0000","last_unstale":"2024-03-20T07:59:02.983711+0000","last_undegraded":"2024-03-20T07:59:02.983711+0000","last_fullsized":"2024-03-20T07:59:02.983711+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156462+0000","last_change":"2024-03-20T07:58:26.141379+0000","last_active":"2024-03-20T07:59:03.156462+0000","last_peered":"2024-03-20T07:59:03.156462+0000","last_clean":"2024-03-20T07:59:03.156462+0000","last_became_active":"2024-03-20T07:58:26.141099+0000","last_became_peered":"2024-03-20T07:58:26.141099+0000","last_unstale":"2024-03-20T07:59:03.156462+0000","last_undegraded":"2024-03-20T07:59:03.156462+0000","last_fullsized":"2024-03-20T07:59:03.156462+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269071+0000","last_change":"2024-03-20T07:58:26.121389+0000","last_active":"2024-03-20T07:59:05.269071+0000","last_peered":"2024-03-20T07:59:05.269071+0000","last_clean":"2024-03-20T07:59:05.269071+0000","last_became_active":"2024-03-20T07:58:26.121094+0000","last_became_peered":"2024-03-20T07:58:26.121094+0000","last_unstale":"2024-03-20T07:59:05.269071+0000","last_undegraded":"2024-03-20T07:59:05.269071+0000","last_fullsized":"2024-03-20T07:59:05.269071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:08:53.389656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985872+0000","last_change":"2024-03-20T07:58:26.137638+0000","last_active":"2024-03-20T07:59:02.985872+0000","last_peered":"2024-03-20T07:59:02.985872+0000","last_clean":"2024-03-20T07:59:02.985872+0000","last_became_active":"2024-03-20T07:58:26.137344+0000","last_became_peered":"2024-03-20T07:58:26.137344+0000","last_unstale":"2024-03-20T07:59:02.985872+0000","last_undegraded":"2024-03-20T07:59:02.985872+0000","last_fullsized":"2024-03-20T07:59:02.985872+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268556+0000","last_change":"2024-03-20T07:58:26.122341+0000","last_active":"2024-03-20T07:59:05.268556+0000","last_peered":"2024-03-20T07:59:05.268556+0000","last_clean":"2024-03-20T07:59:05.268556+0000","last_became_active":"2024-03-20T07:58:26.122003+0000","last_became_peered":"2024-03-20T07:58:26.122003+0000","last_unstale":"2024-03-20T07:59:05.268556+0000","last_undegraded":"2024-03-20T07:59:05.268556+0000","last_fullsized":"2024-03-20T07:59:05.268556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:20.158675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986235+0000","last_change":"2024-03-20T07:58:26.137117+0000","last_active":"2024-03-20T07:59:02.986235+0000","last_peered":"2024-03-20T07:59:02.986235+0000","last_clean":"2024-03-20T07:59:02.986235+0000","last_became_active":"2024-03-20T07:58:26.136135+0000","last_became_peered":"2024-03-20T07:58:26.136135+0000","last_unstale":"2024-03-20T07:59:02.986235+0000","last_undegraded":"2024-03-20T07:59:02.986235+0000","last_fullsized":"2024-03-20T07:59:02.986235+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250340+0000","last_change":"2024-03-20T07:58:26.119480+0000","last_active":"2024-03-20T07:59:04.250340+0000","last_peered":"2024-03-20T07:59:04.250340+0000","last_clean":"2024-03-20T07:59:04.250340+0000","last_became_active":"2024-03-20T07:58:26.119047+0000","last_became_peered":"2024-03-20T07:58:26.119047+0000","last_unstale":"2024-03-20T07:59:04.250340+0000","last_undegraded":"2024-03-20T07:59:04.250340+0000","last_fullsized":"2024-03-20T07:59:04.250340+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:09:31.918661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156361+0000","last_change":"2024-03-20T07:58:26.133160+0000","last_active":"2024-03-20T07:59:03.156361+0000","last_peered":"2024-03-20T07:59:03.156361+0000","last_clean":"2024-03-20T07:59:03.156361+0000","last_became_active":"2024-03-20T07:58:26.132793+0000","last_became_peered":"2024-03-20T07:58:26.132793+0000","last_unstale":"2024-03-20T07:59:03.156361+0000","last_undegraded":"2024-03-20T07:59:03.156361+0000","last_fullsized":"2024-03-20T07:59:03.156361+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3146,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":9438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201304,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28312,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716840,"statfs":{"total":95995035648,"available":95966044160,"internally_reserved":0,"allocated":1318912,"data_stored":902683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.40100000000000002}]},{"osd":1,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.31}]},{"osd":2,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.376},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.35099999999999998}]},{"osd":4,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.501}]},{"osd":5,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48799999999999999}]},{"osd":6,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.441},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.53000000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462941,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716844,"statfs":{"total":95995035648,"available":95966048256,"internally_reserved":0,"allocated":1318912,"data_stored":904096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.33400000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.747,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68300000000000005,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51200000000000001}]},{"osd":1,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68100000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.72299999999999998}]},{"osd":2,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67100000000000004,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61799999999999999}]},{"osd":3,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71999999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65500000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.65900000000000003}]},{"osd":4,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.43099999999999999,"15min":0.432},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57099999999999995,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.33700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.68200000000000005}]},{"osd":5,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69499999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.76100000000000001}]},{"osd":7,"last update":"Wed Mar 20 07:59:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.78400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717424,"statfs":{"total":95995035648,"available":95966642176,"internally_reserved":0,"allocated":724992,"data_stored":311871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.32700000000000001},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48299999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.627,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.27700000000000002}]},{"osd":2,"last update":"Wed Mar 20 07:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.38600000000000001},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.378}]},{"osd":3,"last update":"Wed Mar 20 07:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56499999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.46600000000000003,"15min":0.48099999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34000000000000002}]},{"osd":4,"last update":"Wed Mar 20 07:58:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.437,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.754},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.47099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.55900000000000005}]},{"osd":5,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77700000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Mar 20 07:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.627,"15min":0.63500000000000001},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.746,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.745}]},{"osd":7,"last update":"Wed Mar 20 07:59:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28320,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716832,"statfs":{"total":95995035648,"available":95966035968,"internally_reserved":0,"allocated":1327104,"data_stored":902807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 20 07:59:55 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.31},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.45000000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.34000000000000002,"15min":0.318},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.505,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.36599999999999999}]},{"osd":2,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.52200000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.29699999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.55400000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33100000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:59:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40699999999999997,"15min":0.41299999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61599999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.434}]},{"osd":4,"last update":"Wed Mar 20 07:59:55 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66700000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79900000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.51400000000000001}]},{"osd":5,"last update":"Wed Mar 20 07:59:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.871,"15min":0.871},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed Mar 20 07:59:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71299999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82699999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":311711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.26300000000000001,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":0.46999999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40300000000000002,"15min":0.39400000000000002},"min":{"1min":0.28299999999999997,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.248,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.44800000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.312},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.38,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.41699999999999998,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed Mar 20 07:59:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40300000000000002,"15min":0.41399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.497,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.437,"15min":0.44900000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.46899999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42399999999999999}]},{"osd":4,"last update":"Wed Mar 20 07:58:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.64000000000000001}]},{"osd":5,"last update":"Wed Mar 20 07:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78400000000000003,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54100000000000004}]},{"osd":6,"last update":"Wed Mar 20 07:59:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.61799999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149739,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717408,"statfs":{"total":95995035648,"available":95966625792,"internally_reserved":0,"allocated":737280,"data_stored":314163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:34 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.41799999999999998,"15min":0.433},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53300000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.437,"15min":0.442},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57499999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.34999999999999998}]},{"osd":1,"last update":"Wed Mar 20 07:59:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.40600000000000003,"15min":0.42799999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46800000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.47199999999999998,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.32300000000000001}]},{"osd":2,"last update":"Wed Mar 20 07:59:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45200000000000001,"15min":0.46800000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48999999999999999,"15min":0.502},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.37}]},{"osd":4,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71999999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80300000000000005,"5min":0.876,"15min":0.876},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Mar 20 07:59:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82599999999999996,"5min":1.056,"15min":1.056},"last":0.746},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86799999999999999,"5min":1.099,"15min":1.099},"last":0.77800000000000002}]},{"osd":6,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68400000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed Mar 20 07:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58599999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986215,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27740,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717412,"statfs":{"total":95995035648,"available":95966629888,"internally_reserved":0,"allocated":733184,"data_stored":312419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.39600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67200000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.443,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.63600000000000001,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67400000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.46600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.73799999999999999,"5min":0.75,"15min":0.75},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.496}]},{"osd":5,"last update":"Wed Mar 20 07:59:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.376},"min":{"1min":0.29499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.374},"min":{"1min":0.33300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.45300000000000001}]},{"osd":6,"last update":"Wed Mar 20 07:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.45300000000000001,"15min":0.441},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.42799999999999999}]},{"osd":7,"last update":"Wed Mar 20 07:59:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.47099999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724578,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":733184,"data_stored":314149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.745,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.61299999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.875,"15min":0.875},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.45400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.625,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.50900000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38,"15min":0.38100000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.50800000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.29999999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.49099999999999999}]},{"osd":6,"last update":"Wed Mar 20 07:59:43 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.442,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.312,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59699999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59899999999999998}]},{"osd":7,"last update":"Wed Mar 20 07:59:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.623}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:00:00.988 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:00:01.195 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:00:01.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:01 smithi118 ceph-mon[27391]: pgmap v37: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:01.317 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:01 smithi118 ceph-mon[38981]: pgmap v37: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:01.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:01 smithi196 ceph-mon[31033]: pgmap v37: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:01.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:00:01] "GET /metrics HTTP/1.1" 200 33648 "" "Prometheus/2.43.0" 2024-03-20T08:00:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:02 smithi196 ceph-mon[31033]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:00:02.416 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:02 smithi118 ceph-mon[38981]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:00:02.416 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:02 smithi118 ceph-mon[27391]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:00:03.128 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:03.130 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:00:03.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:03 smithi196 ceph-mon[31033]: pgmap v38: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:03.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:03 smithi118 ceph-mon[27391]: pgmap v38: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:03.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:03 smithi118 ceph-mon[38981]: pgmap v38: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:03.900 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-03-20T08:00:01.923170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593514,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":227,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":315,"acting":315,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":315,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":223608,"kb_used_data":7432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":749737608,"statfs":{"total":767960285184,"available":767731310592,"internally_reserved":0,"allocated":7610368,"data_stored":4273899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002536"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844320+0000","last_change":"2024-03-20T07:58:26.145662+0000","last_active":"2024-03-20T07:59:04.844320+0000","last_peered":"2024-03-20T07:59:04.844320+0000","last_clean":"2024-03-20T07:59:04.844320+0000","last_became_active":"2024-03-20T07:58:26.145286+0000","last_became_peered":"2024-03-20T07:58:26.145286+0000","last_unstale":"2024-03-20T07:59:04.844320+0000","last_undegraded":"2024-03-20T07:59:04.844320+0000","last_fullsized":"2024-03-20T07:59:04.844320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984138+0000","last_change":"2024-03-20T07:58:26.129069+0000","last_active":"2024-03-20T07:59:02.984138+0000","last_peered":"2024-03-20T07:59:02.984138+0000","last_clean":"2024-03-20T07:59:02.984138+0000","last_became_active":"2024-03-20T07:58:26.128811+0000","last_became_peered":"2024-03-20T07:58:26.128811+0000","last_unstale":"2024-03-20T07:59:02.984138+0000","last_undegraded":"2024-03-20T07:59:02.984138+0000","last_fullsized":"2024-03-20T07:59:02.984138+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250498+0000","last_change":"2024-03-20T07:58:26.119030+0000","last_active":"2024-03-20T07:59:04.250498+0000","last_peered":"2024-03-20T07:59:04.250498+0000","last_clean":"2024-03-20T07:59:04.250498+0000","last_became_active":"2024-03-20T07:58:26.118755+0000","last_became_peered":"2024-03-20T07:58:26.118755+0000","last_unstale":"2024-03-20T07:59:04.250498+0000","last_undegraded":"2024-03-20T07:59:04.250498+0000","last_fullsized":"2024-03-20T07:59:04.250498+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:51:41.823570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986054+0000","last_change":"2024-03-20T07:58:26.138067+0000","last_active":"2024-03-20T07:59:02.986054+0000","last_peered":"2024-03-20T07:59:02.986054+0000","last_clean":"2024-03-20T07:59:02.986054+0000","last_became_active":"2024-03-20T07:58:26.137768+0000","last_became_peered":"2024-03-20T07:58:26.137768+0000","last_unstale":"2024-03-20T07:59:02.986054+0000","last_undegraded":"2024-03-20T07:59:02.986054+0000","last_fullsized":"2024-03-20T07:59:02.986054+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843727+0000","last_change":"2024-03-20T07:58:26.131923+0000","last_active":"2024-03-20T07:59:04.843727+0000","last_peered":"2024-03-20T07:59:04.843727+0000","last_clean":"2024-03-20T07:59:04.843727+0000","last_became_active":"2024-03-20T07:58:26.131585+0000","last_became_peered":"2024-03-20T07:58:26.131585+0000","last_unstale":"2024-03-20T07:59:04.843727+0000","last_undegraded":"2024-03-20T07:59:04.843727+0000","last_fullsized":"2024-03-20T07:59:04.843727+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986391+0000","last_change":"2024-03-20T07:58:26.137308+0000","last_active":"2024-03-20T07:59:02.986391+0000","last_peered":"2024-03-20T07:59:02.986391+0000","last_clean":"2024-03-20T07:59:02.986391+0000","last_became_active":"2024-03-20T07:58:26.135727+0000","last_became_peered":"2024-03-20T07:58:26.135727+0000","last_unstale":"2024-03-20T07:59:02.986391+0000","last_undegraded":"2024-03-20T07:59:02.986391+0000","last_fullsized":"2024-03-20T07:59:02.986391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843474+0000","last_change":"2024-03-20T07:58:26.132941+0000","last_active":"2024-03-20T07:59:04.843474+0000","last_peered":"2024-03-20T07:59:04.843474+0000","last_clean":"2024-03-20T07:59:04.843474+0000","last_became_active":"2024-03-20T07:58:26.132230+0000","last_became_peered":"2024-03-20T07:58:26.132230+0000","last_unstale":"2024-03-20T07:59:04.843474+0000","last_undegraded":"2024-03-20T07:59:04.843474+0000","last_fullsized":"2024-03-20T07:59:04.843474+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268423+0000","last_change":"2024-03-20T07:58:26.136216+0000","last_active":"2024-03-20T07:59:05.268423+0000","last_peered":"2024-03-20T07:59:05.268423+0000","last_clean":"2024-03-20T07:59:05.268423+0000","last_became_active":"2024-03-20T07:58:26.135920+0000","last_became_peered":"2024-03-20T07:58:26.135920+0000","last_unstale":"2024-03-20T07:59:05.268423+0000","last_undegraded":"2024-03-20T07:59:05.268423+0000","last_fullsized":"2024-03-20T07:59:05.268423+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:37.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844260+0000","last_change":"2024-03-20T07:58:26.144631+0000","last_active":"2024-03-20T07:59:04.844260+0000","last_peered":"2024-03-20T07:59:04.844260+0000","last_clean":"2024-03-20T07:59:04.844260+0000","last_became_active":"2024-03-20T07:58:26.144302+0000","last_became_peered":"2024-03-20T07:58:26.144302+0000","last_unstale":"2024-03-20T07:59:04.844260+0000","last_undegraded":"2024-03-20T07:59:04.844260+0000","last_fullsized":"2024-03-20T07:59:04.844260+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156363+0000","last_change":"2024-03-20T07:58:26.138657+0000","last_active":"2024-03-20T07:59:03.156363+0000","last_peered":"2024-03-20T07:59:03.156363+0000","last_clean":"2024-03-20T07:59:03.156363+0000","last_became_active":"2024-03-20T07:58:26.138348+0000","last_became_peered":"2024-03-20T07:58:26.138348+0000","last_unstale":"2024-03-20T07:59:03.156363+0000","last_undegraded":"2024-03-20T07:59:03.156363+0000","last_fullsized":"2024-03-20T07:59:03.156363+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156427+0000","last_change":"2024-03-20T07:58:26.137961+0000","last_active":"2024-03-20T07:59:03.156427+0000","last_peered":"2024-03-20T07:59:03.156427+0000","last_clean":"2024-03-20T07:59:03.156427+0000","last_became_active":"2024-03-20T07:58:26.137672+0000","last_became_peered":"2024-03-20T07:58:26.137672+0000","last_unstale":"2024-03-20T07:59:03.156427+0000","last_undegraded":"2024-03-20T07:59:03.156427+0000","last_fullsized":"2024-03-20T07:59:03.156427+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269255+0000","last_change":"2024-03-20T07:58:26.136246+0000","last_active":"2024-03-20T07:59:05.269255+0000","last_peered":"2024-03-20T07:59:05.269255+0000","last_clean":"2024-03-20T07:59:05.269255+0000","last_became_active":"2024-03-20T07:58:26.135935+0000","last_became_peered":"2024-03-20T07:58:26.135935+0000","last_unstale":"2024-03-20T07:59:05.269255+0000","last_undegraded":"2024-03-20T07:59:05.269255+0000","last_fullsized":"2024-03-20T07:59:05.269255+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268785+0000","last_change":"2024-03-20T07:58:26.129765+0000","last_active":"2024-03-20T07:59:05.268785+0000","last_peered":"2024-03-20T07:59:05.268785+0000","last_clean":"2024-03-20T07:59:05.268785+0000","last_became_active":"2024-03-20T07:58:26.129209+0000","last_became_peered":"2024-03-20T07:58:26.129209+0000","last_unstale":"2024-03-20T07:59:05.268785+0000","last_undegraded":"2024-03-20T07:59:05.268785+0000","last_fullsized":"2024-03-20T07:59:05.268785+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:16:48.280737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250172+0000","last_change":"2024-03-20T07:58:26.123008+0000","last_active":"2024-03-20T07:59:04.250172+0000","last_peered":"2024-03-20T07:59:04.250172+0000","last_clean":"2024-03-20T07:59:04.250172+0000","last_became_active":"2024-03-20T07:58:26.122270+0000","last_became_peered":"2024-03-20T07:58:26.122270+0000","last_unstale":"2024-03-20T07:59:04.250172+0000","last_undegraded":"2024-03-20T07:59:04.250172+0000","last_fullsized":"2024-03-20T07:59:04.250172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:37.385767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983133+0000","last_change":"2024-03-20T07:58:26.127042+0000","last_active":"2024-03-20T07:59:02.983133+0000","last_peered":"2024-03-20T07:59:02.983133+0000","last_clean":"2024-03-20T07:59:02.983133+0000","last_became_active":"2024-03-20T07:58:26.126555+0000","last_became_peered":"2024-03-20T07:58:26.126555+0000","last_unstale":"2024-03-20T07:59:02.983133+0000","last_undegraded":"2024-03-20T07:59:02.983133+0000","last_fullsized":"2024-03-20T07:59:02.983133+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985736+0000","last_change":"2024-03-20T07:58:26.129086+0000","last_active":"2024-03-20T07:59:02.985736+0000","last_peered":"2024-03-20T07:59:02.985736+0000","last_clean":"2024-03-20T07:59:02.985736+0000","last_became_active":"2024-03-20T07:58:26.128111+0000","last_became_peered":"2024-03-20T07:58:26.128111+0000","last_unstale":"2024-03-20T07:59:02.985736+0000","last_undegraded":"2024-03-20T07:59:02.985736+0000","last_fullsized":"2024-03-20T07:59:02.985736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986251+0000","last_change":"2024-03-20T07:58:26.128812+0000","last_active":"2024-03-20T07:59:02.986251+0000","last_peered":"2024-03-20T07:59:02.986251+0000","last_clean":"2024-03-20T07:59:02.986251+0000","last_became_active":"2024-03-20T07:58:26.128501+0000","last_became_peered":"2024-03-20T07:58:26.128501+0000","last_unstale":"2024-03-20T07:59:02.986251+0000","last_undegraded":"2024-03-20T07:59:02.986251+0000","last_fullsized":"2024-03-20T07:59:02.986251+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984581+0000","last_change":"2024-03-20T07:58:26.128211+0000","last_active":"2024-03-20T07:59:02.984581+0000","last_peered":"2024-03-20T07:59:02.984581+0000","last_clean":"2024-03-20T07:59:02.984581+0000","last_became_active":"2024-03-20T07:58:26.127943+0000","last_became_peered":"2024-03-20T07:58:26.127943+0000","last_unstale":"2024-03-20T07:59:02.984581+0000","last_undegraded":"2024-03-20T07:59:02.984581+0000","last_fullsized":"2024-03-20T07:59:02.984581+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:48:19.979568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843899+0000","last_change":"2024-03-20T07:58:26.133112+0000","last_active":"2024-03-20T07:59:04.843899+0000","last_peered":"2024-03-20T07:59:04.843899+0000","last_clean":"2024-03-20T07:59:04.843899+0000","last_became_active":"2024-03-20T07:58:26.132794+0000","last_became_peered":"2024-03-20T07:58:26.132794+0000","last_unstale":"2024-03-20T07:59:04.843899+0000","last_undegraded":"2024-03-20T07:59:04.843899+0000","last_fullsized":"2024-03-20T07:59:04.843899+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"62'4","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250889+0000","last_change":"2024-03-20T07:58:25.774846+0000","last_active":"2024-03-20T07:59:04.250889+0000","last_peered":"2024-03-20T07:59:04.250889+0000","last_clean":"2024-03-20T07:59:04.250889+0000","last_became_active":"2024-03-20T07:58:25.774570+0000","last_became_peered":"2024-03-20T07:58:25.774570+0000","last_unstale":"2024-03-20T07:59:04.250889+0000","last_undegraded":"2024-03-20T07:59:04.250889+0000","last_fullsized":"2024-03-20T07:59:04.250889+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:28:00.513690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268967+0000","last_change":"2024-03-20T07:58:26.128852+0000","last_active":"2024-03-20T07:59:05.268967+0000","last_peered":"2024-03-20T07:59:05.268967+0000","last_clean":"2024-03-20T07:59:05.268967+0000","last_became_active":"2024-03-20T07:58:26.128584+0000","last_became_peered":"2024-03-20T07:58:26.128584+0000","last_unstale":"2024-03-20T07:59:05.268967+0000","last_undegraded":"2024-03-20T07:59:05.268967+0000","last_fullsized":"2024-03-20T07:59:05.268967+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:15:37.874514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269007+0000","last_change":"2024-03-20T07:58:25.774347+0000","last_active":"2024-03-20T07:59:05.269007+0000","last_peered":"2024-03-20T07:59:05.269007+0000","last_clean":"2024-03-20T07:59:05.269007+0000","last_became_active":"2024-03-20T07:58:25.773993+0000","last_became_peered":"2024-03-20T07:58:25.773993+0000","last_unstale":"2024-03-20T07:59:05.269007+0000","last_undegraded":"2024-03-20T07:59:05.269007+0000","last_fullsized":"2024-03-20T07:59:05.269007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:04:59.603081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250754+0000","last_change":"2024-03-20T07:58:26.118404+0000","last_active":"2024-03-20T07:59:04.250754+0000","last_peered":"2024-03-20T07:59:04.250754+0000","last_clean":"2024-03-20T07:59:04.250754+0000","last_became_active":"2024-03-20T07:58:26.118107+0000","last_became_peered":"2024-03-20T07:58:26.118107+0000","last_unstale":"2024-03-20T07:59:04.250754+0000","last_undegraded":"2024-03-20T07:59:04.250754+0000","last_fullsized":"2024-03-20T07:59:04.250754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:51.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844135+0000","last_change":"2024-03-20T07:58:25.778043+0000","last_active":"2024-03-20T07:59:04.844135+0000","last_peered":"2024-03-20T07:59:04.844135+0000","last_clean":"2024-03-20T07:59:04.844135+0000","last_became_active":"2024-03-20T07:58:25.777710+0000","last_became_peered":"2024-03-20T07:58:25.777710+0000","last_unstale":"2024-03-20T07:59:04.844135+0000","last_undegraded":"2024-03-20T07:59:04.844135+0000","last_fullsized":"2024-03-20T07:59:04.844135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986225+0000","last_change":"2024-03-20T07:58:26.122440+0000","last_active":"2024-03-20T07:59:02.986225+0000","last_peered":"2024-03-20T07:59:02.986225+0000","last_clean":"2024-03-20T07:59:02.986225+0000","last_became_active":"2024-03-20T07:58:26.122136+0000","last_became_peered":"2024-03-20T07:58:26.122136+0000","last_unstale":"2024-03-20T07:59:02.986225+0000","last_undegraded":"2024-03-20T07:59:02.986225+0000","last_fullsized":"2024-03-20T07:59:02.986225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843971+0000","last_change":"2024-03-20T07:58:25.776107+0000","last_active":"2024-03-20T07:59:04.843971+0000","last_peered":"2024-03-20T07:59:04.843971+0000","last_clean":"2024-03-20T07:59:04.843971+0000","last_became_active":"2024-03-20T07:58:25.775756+0000","last_became_peered":"2024-03-20T07:58:25.775756+0000","last_unstale":"2024-03-20T07:59:04.843971+0000","last_undegraded":"2024-03-20T07:59:04.843971+0000","last_fullsized":"2024-03-20T07:59:04.843971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983617+0000","last_change":"2024-03-20T07:58:46.699381+0000","last_active":"2024-03-20T07:59:02.983617+0000","last_peered":"2024-03-20T07:59:02.983617+0000","last_clean":"2024-03-20T07:59:02.983617+0000","last_became_active":"2024-03-20T07:58:46.698994+0000","last_became_peered":"2024-03-20T07:58:46.698994+0000","last_unstale":"2024-03-20T07:59:02.983617+0000","last_undegraded":"2024-03-20T07:59:02.983617+0000","last_fullsized":"2024-03-20T07:59:02.983617+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:31:09.474011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268838+0000","last_change":"2024-03-20T07:58:25.769988+0000","last_active":"2024-03-20T07:59:05.268838+0000","last_peered":"2024-03-20T07:59:05.268838+0000","last_clean":"2024-03-20T07:59:05.268838+0000","last_became_active":"2024-03-20T07:58:25.769359+0000","last_became_peered":"2024-03-20T07:58:25.769359+0000","last_unstale":"2024-03-20T07:59:05.268838+0000","last_undegraded":"2024-03-20T07:59:05.268838+0000","last_fullsized":"2024-03-20T07:59:05.268838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:26:35.889023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268931+0000","last_change":"2024-03-20T07:58:26.118793+0000","last_active":"2024-03-20T07:59:05.268931+0000","last_peered":"2024-03-20T07:59:05.268931+0000","last_clean":"2024-03-20T07:59:05.268931+0000","last_became_active":"2024-03-20T07:58:26.118430+0000","last_became_peered":"2024-03-20T07:58:26.118430+0000","last_unstale":"2024-03-20T07:59:05.268931+0000","last_undegraded":"2024-03-20T07:59:05.268931+0000","last_fullsized":"2024-03-20T07:59:05.268931+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156197+0000","last_change":"2024-03-20T07:58:25.765918+0000","last_active":"2024-03-20T07:59:03.156197+0000","last_peered":"2024-03-20T07:59:03.156197+0000","last_clean":"2024-03-20T07:59:03.156197+0000","last_became_active":"2024-03-20T07:58:25.765598+0000","last_became_peered":"2024-03-20T07:58:25.765598+0000","last_unstale":"2024-03-20T07:59:03.156197+0000","last_undegraded":"2024-03-20T07:59:03.156197+0000","last_fullsized":"2024-03-20T07:59:03.156197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250172+0000","last_change":"2024-03-20T07:58:26.119313+0000","last_active":"2024-03-20T07:59:04.250172+0000","last_peered":"2024-03-20T07:59:04.250172+0000","last_clean":"2024-03-20T07:59:04.250172+0000","last_became_active":"2024-03-20T07:58:26.118645+0000","last_became_peered":"2024-03-20T07:58:26.118645+0000","last_unstale":"2024-03-20T07:59:04.250172+0000","last_undegraded":"2024-03-20T07:59:04.250172+0000","last_fullsized":"2024-03-20T07:59:04.250172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:13.410704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268640+0000","last_change":"2024-03-20T07:58:25.771278+0000","last_active":"2024-03-20T07:59:05.268640+0000","last_peered":"2024-03-20T07:59:05.268640+0000","last_clean":"2024-03-20T07:59:05.268640+0000","last_became_active":"2024-03-20T07:58:25.770958+0000","last_became_peered":"2024-03-20T07:58:25.770958+0000","last_unstale":"2024-03-20T07:59:05.268640+0000","last_undegraded":"2024-03-20T07:59:05.268640+0000","last_fullsized":"2024-03-20T07:59:05.268640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:10:26.828071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268599+0000","last_change":"2024-03-20T07:58:26.116986+0000","last_active":"2024-03-20T07:59:05.268599+0000","last_peered":"2024-03-20T07:59:05.268599+0000","last_clean":"2024-03-20T07:59:05.268599+0000","last_became_active":"2024-03-20T07:58:26.116571+0000","last_became_peered":"2024-03-20T07:58:26.116571+0000","last_unstale":"2024-03-20T07:59:05.268599+0000","last_undegraded":"2024-03-20T07:59:05.268599+0000","last_fullsized":"2024-03-20T07:59:05.268599+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:32:56.125697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985865+0000","last_change":"2024-03-20T07:58:25.769007+0000","last_active":"2024-03-20T07:59:02.985865+0000","last_peered":"2024-03-20T07:59:02.985865+0000","last_clean":"2024-03-20T07:59:02.985865+0000","last_became_active":"2024-03-20T07:58:25.768716+0000","last_became_peered":"2024-03-20T07:58:25.768716+0000","last_unstale":"2024-03-20T07:59:02.985865+0000","last_undegraded":"2024-03-20T07:59:02.985865+0000","last_fullsized":"2024-03-20T07:59:02.985865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983351+0000","last_change":"2024-03-20T07:58:26.119518+0000","last_active":"2024-03-20T07:59:02.983351+0000","last_peered":"2024-03-20T07:59:02.983351+0000","last_clean":"2024-03-20T07:59:02.983351+0000","last_became_active":"2024-03-20T07:58:26.119184+0000","last_became_peered":"2024-03-20T07:58:26.119184+0000","last_unstale":"2024-03-20T07:59:02.983351+0000","last_undegraded":"2024-03-20T07:59:02.983351+0000","last_fullsized":"2024-03-20T07:59:02.983351+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156497+0000","last_change":"2024-03-20T07:58:25.769300+0000","last_active":"2024-03-20T07:59:03.156497+0000","last_peered":"2024-03-20T07:59:03.156497+0000","last_clean":"2024-03-20T07:59:03.156497+0000","last_became_active":"2024-03-20T07:58:25.768916+0000","last_became_peered":"2024-03-20T07:58:25.768916+0000","last_unstale":"2024-03-20T07:59:03.156497+0000","last_undegraded":"2024-03-20T07:59:03.156497+0000","last_fullsized":"2024-03-20T07:59:03.156497+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986288+0000","last_change":"2024-03-20T07:58:26.130616+0000","last_active":"2024-03-20T07:59:02.986288+0000","last_peered":"2024-03-20T07:59:02.986288+0000","last_clean":"2024-03-20T07:59:02.986288+0000","last_became_active":"2024-03-20T07:58:26.130332+0000","last_became_peered":"2024-03-20T07:58:26.130332+0000","last_unstale":"2024-03-20T07:59:02.986288+0000","last_undegraded":"2024-03-20T07:59:02.986288+0000","last_fullsized":"2024-03-20T07:59:02.986288+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986375+0000","last_change":"2024-03-20T07:58:25.769790+0000","last_active":"2024-03-20T07:59:02.986375+0000","last_peered":"2024-03-20T07:59:02.986375+0000","last_clean":"2024-03-20T07:59:02.986375+0000","last_became_active":"2024-03-20T07:58:25.769492+0000","last_became_peered":"2024-03-20T07:58:25.769492+0000","last_unstale":"2024-03-20T07:59:02.986375+0000","last_undegraded":"2024-03-20T07:59:02.986375+0000","last_fullsized":"2024-03-20T07:59:02.986375+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250679+0000","last_change":"2024-03-20T07:58:26.116344+0000","last_active":"2024-03-20T07:59:04.250679+0000","last_peered":"2024-03-20T07:59:04.250679+0000","last_clean":"2024-03-20T07:59:04.250679+0000","last_became_active":"2024-03-20T07:58:26.116037+0000","last_became_peered":"2024-03-20T07:58:26.116037+0000","last_unstale":"2024-03-20T07:59:04.250679+0000","last_undegraded":"2024-03-20T07:59:04.250679+0000","last_fullsized":"2024-03-20T07:59:04.250679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:16.183856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983935+0000","last_change":"2024-03-20T07:58:25.770988+0000","last_active":"2024-03-20T07:59:02.983935+0000","last_peered":"2024-03-20T07:59:02.983935+0000","last_clean":"2024-03-20T07:59:02.983935+0000","last_became_active":"2024-03-20T07:58:25.770657+0000","last_became_peered":"2024-03-20T07:58:25.770657+0000","last_unstale":"2024-03-20T07:59:02.983935+0000","last_undegraded":"2024-03-20T07:59:02.983935+0000","last_fullsized":"2024-03-20T07:59:02.983935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269168+0000","last_change":"2024-03-20T07:58:26.128506+0000","last_active":"2024-03-20T07:59:05.269168+0000","last_peered":"2024-03-20T07:59:05.269168+0000","last_clean":"2024-03-20T07:59:05.269168+0000","last_became_active":"2024-03-20T07:58:26.128201+0000","last_became_peered":"2024-03-20T07:58:26.128201+0000","last_unstale":"2024-03-20T07:59:05.269168+0000","last_undegraded":"2024-03-20T07:59:05.269168+0000","last_fullsized":"2024-03-20T07:59:05.269168+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"66'9","reported_seq":28,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156321+0000","last_change":"2024-03-20T07:58:25.767830+0000","last_active":"2024-03-20T07:59:03.156321+0000","last_peered":"2024-03-20T07:59:03.156321+0000","last_clean":"2024-03-20T07:59:03.156321+0000","last_became_active":"2024-03-20T07:58:25.767533+0000","last_became_peered":"2024-03-20T07:58:25.767533+0000","last_unstale":"2024-03-20T07:59:03.156321+0000","last_undegraded":"2024-03-20T07:59:03.156321+0000","last_fullsized":"2024-03-20T07:59:03.156321+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2348,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843983+0000","last_change":"2024-03-20T07:58:26.140127+0000","last_active":"2024-03-20T07:59:04.843983+0000","last_peered":"2024-03-20T07:59:04.843983+0000","last_clean":"2024-03-20T07:59:04.843983+0000","last_became_active":"2024-03-20T07:58:26.139362+0000","last_became_peered":"2024-03-20T07:58:26.139362+0000","last_unstale":"2024-03-20T07:59:04.843983+0000","last_undegraded":"2024-03-20T07:59:04.843983+0000","last_fullsized":"2024-03-20T07:59:04.843983+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250511+0000","last_change":"2024-03-20T07:58:25.771772+0000","last_active":"2024-03-20T07:59:04.250511+0000","last_peered":"2024-03-20T07:59:04.250511+0000","last_clean":"2024-03-20T07:59:04.250511+0000","last_became_active":"2024-03-20T07:58:25.771499+0000","last_became_peered":"2024-03-20T07:58:25.771499+0000","last_unstale":"2024-03-20T07:59:04.250511+0000","last_undegraded":"2024-03-20T07:59:04.250511+0000","last_fullsized":"2024-03-20T07:59:04.250511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:19:45.833311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250093+0000","last_change":"2024-03-20T07:58:26.122549+0000","last_active":"2024-03-20T07:59:04.250093+0000","last_peered":"2024-03-20T07:59:04.250093+0000","last_clean":"2024-03-20T07:59:04.250093+0000","last_became_active":"2024-03-20T07:58:26.121780+0000","last_became_peered":"2024-03-20T07:58:26.121780+0000","last_unstale":"2024-03-20T07:59:04.250093+0000","last_undegraded":"2024-03-20T07:59:04.250093+0000","last_fullsized":"2024-03-20T07:59:04.250093+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:37:01.164741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985855+0000","last_change":"2024-03-20T07:58:25.771839+0000","last_active":"2024-03-20T07:59:02.985855+0000","last_peered":"2024-03-20T07:59:02.985855+0000","last_clean":"2024-03-20T07:59:02.985855+0000","last_became_active":"2024-03-20T07:58:25.771464+0000","last_became_peered":"2024-03-20T07:58:25.771464+0000","last_unstale":"2024-03-20T07:59:02.985855+0000","last_undegraded":"2024-03-20T07:59:02.985855+0000","last_fullsized":"2024-03-20T07:59:02.985855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843684+0000","last_change":"2024-03-20T07:58:26.129094+0000","last_active":"2024-03-20T07:59:04.843684+0000","last_peered":"2024-03-20T07:59:04.843684+0000","last_clean":"2024-03-20T07:59:04.843684+0000","last_became_active":"2024-03-20T07:58:26.128675+0000","last_became_peered":"2024-03-20T07:58:26.128675+0000","last_unstale":"2024-03-20T07:59:04.843684+0000","last_undegraded":"2024-03-20T07:59:04.843684+0000","last_fullsized":"2024-03-20T07:59:04.843684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156125+0000","last_change":"2024-03-20T07:58:25.767586+0000","last_active":"2024-03-20T07:59:03.156125+0000","last_peered":"2024-03-20T07:59:03.156125+0000","last_clean":"2024-03-20T07:59:03.156125+0000","last_became_active":"2024-03-20T07:58:25.767268+0000","last_became_peered":"2024-03-20T07:58:25.767268+0000","last_unstale":"2024-03-20T07:59:03.156125+0000","last_undegraded":"2024-03-20T07:59:03.156125+0000","last_fullsized":"2024-03-20T07:59:03.156125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156050+0000","last_change":"2024-03-20T07:58:26.139925+0000","last_active":"2024-03-20T07:59:03.156050+0000","last_peered":"2024-03-20T07:59:03.156050+0000","last_clean":"2024-03-20T07:59:03.156050+0000","last_became_active":"2024-03-20T07:58:26.138286+0000","last_became_peered":"2024-03-20T07:58:26.138286+0000","last_unstale":"2024-03-20T07:59:03.156050+0000","last_undegraded":"2024-03-20T07:59:03.156050+0000","last_fullsized":"2024-03-20T07:59:03.156050+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":27,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983262+0000","last_change":"2024-03-20T07:58:25.757217+0000","last_active":"2024-03-20T07:59:02.983262+0000","last_peered":"2024-03-20T07:59:02.983262+0000","last_clean":"2024-03-20T07:59:02.983262+0000","last_became_active":"2024-03-20T07:58:25.756879+0000","last_became_peered":"2024-03-20T07:58:25.756879+0000","last_unstale":"2024-03-20T07:59:02.983262+0000","last_undegraded":"2024-03-20T07:59:02.983262+0000","last_fullsized":"2024-03-20T07:59:02.983262+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:28.214822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843593+0000","last_change":"2024-03-20T07:58:26.133140+0000","last_active":"2024-03-20T07:59:04.843593+0000","last_peered":"2024-03-20T07:59:04.843593+0000","last_clean":"2024-03-20T07:59:04.843593+0000","last_became_active":"2024-03-20T07:58:26.132630+0000","last_became_peered":"2024-03-20T07:58:26.132630+0000","last_unstale":"2024-03-20T07:59:04.843593+0000","last_undegraded":"2024-03-20T07:59:04.843593+0000","last_fullsized":"2024-03-20T07:59:04.843593+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269456+0000","last_change":"2024-03-20T07:58:25.771322+0000","last_active":"2024-03-20T07:59:05.269456+0000","last_peered":"2024-03-20T07:59:05.269456+0000","last_clean":"2024-03-20T07:59:05.269456+0000","last_became_active":"2024-03-20T07:58:25.770984+0000","last_became_peered":"2024-03-20T07:58:25.770984+0000","last_unstale":"2024-03-20T07:59:05.269456+0000","last_undegraded":"2024-03-20T07:59:05.269456+0000","last_fullsized":"2024-03-20T07:59:05.269456+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844196+0000","last_change":"2024-03-20T07:58:26.144254+0000","last_active":"2024-03-20T07:59:04.844196+0000","last_peered":"2024-03-20T07:59:04.844196+0000","last_clean":"2024-03-20T07:59:04.844196+0000","last_became_active":"2024-03-20T07:58:26.143683+0000","last_became_peered":"2024-03-20T07:58:26.143683+0000","last_unstale":"2024-03-20T07:59:04.844196+0000","last_undegraded":"2024-03-20T07:59:04.844196+0000","last_fullsized":"2024-03-20T07:59:04.844196+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.912606+0000","last_change":"2024-03-20T07:59:01.988226+0000","last_active":"2024-03-20T07:59:05.912606+0000","last_peered":"2024-03-20T07:59:05.912606+0000","last_clean":"2024-03-20T07:59:05.912606+0000","last_became_active":"2024-03-20T07:59:01.987867+0000","last_became_peered":"2024-03-20T07:59:01.987867+0000","last_unstale":"2024-03-20T07:59:05.912606+0000","last_undegraded":"2024-03-20T07:59:05.912606+0000","last_fullsized":"2024-03-20T07:59:05.912606+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:14:23.583126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983174+0000","last_change":"2024-03-20T07:58:25.773911+0000","last_active":"2024-03-20T07:59:02.983174+0000","last_peered":"2024-03-20T07:59:02.983174+0000","last_clean":"2024-03-20T07:59:02.983174+0000","last_became_active":"2024-03-20T07:58:25.773526+0000","last_became_peered":"2024-03-20T07:58:25.773526+0000","last_unstale":"2024-03-20T07:59:02.983174+0000","last_undegraded":"2024-03-20T07:59:02.983174+0000","last_fullsized":"2024-03-20T07:59:02.983174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986145+0000","last_change":"2024-03-20T07:58:26.122909+0000","last_active":"2024-03-20T07:59:02.986145+0000","last_peered":"2024-03-20T07:59:02.986145+0000","last_clean":"2024-03-20T07:59:02.986145+0000","last_became_active":"2024-03-20T07:58:26.121608+0000","last_became_peered":"2024-03-20T07:58:26.121608+0000","last_unstale":"2024-03-20T07:59:02.986145+0000","last_undegraded":"2024-03-20T07:59:02.986145+0000","last_fullsized":"2024-03-20T07:59:02.986145+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.912657+0000","last_change":"2024-03-20T07:59:01.991385+0000","last_active":"2024-03-20T07:59:05.912657+0000","last_peered":"2024-03-20T07:59:05.912657+0000","last_clean":"2024-03-20T07:59:05.912657+0000","last_became_active":"2024-03-20T07:59:01.991077+0000","last_became_peered":"2024-03-20T07:59:01.991077+0000","last_unstale":"2024-03-20T07:59:05.912657+0000","last_undegraded":"2024-03-20T07:59:05.912657+0000","last_fullsized":"2024-03-20T07:59:05.912657+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:31:07.169554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986443+0000","last_change":"2024-03-20T07:58:25.770569+0000","last_active":"2024-03-20T07:59:02.986443+0000","last_peered":"2024-03-20T07:59:02.986443+0000","last_clean":"2024-03-20T07:59:02.986443+0000","last_became_active":"2024-03-20T07:58:25.770298+0000","last_became_peered":"2024-03-20T07:58:25.770298+0000","last_unstale":"2024-03-20T07:59:02.986443+0000","last_undegraded":"2024-03-20T07:59:02.986443+0000","last_fullsized":"2024-03-20T07:59:02.986443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269157+0000","last_change":"2024-03-20T07:58:26.136200+0000","last_active":"2024-03-20T07:59:05.269157+0000","last_peered":"2024-03-20T07:59:05.269157+0000","last_clean":"2024-03-20T07:59:05.269157+0000","last_became_active":"2024-03-20T07:58:26.135909+0000","last_became_peered":"2024-03-20T07:58:26.135909+0000","last_unstale":"2024-03-20T07:59:05.269157+0000","last_undegraded":"2024-03-20T07:59:05.269157+0000","last_fullsized":"2024-03-20T07:59:05.269157+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.912725+0000","last_change":"2024-03-20T07:59:01.988864+0000","last_active":"2024-03-20T07:59:05.912725+0000","last_peered":"2024-03-20T07:59:05.912725+0000","last_clean":"2024-03-20T07:59:05.912725+0000","last_became_active":"2024-03-20T07:59:01.988575+0000","last_became_peered":"2024-03-20T07:59:01.988575+0000","last_unstale":"2024-03-20T07:59:05.912725+0000","last_undegraded":"2024-03-20T07:59:05.912725+0000","last_fullsized":"2024-03-20T07:59:05.912725+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:19:29.334160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.982959+0000","last_change":"2024-03-20T07:58:26.130079+0000","last_active":"2024-03-20T07:59:02.982959+0000","last_peered":"2024-03-20T07:59:02.982959+0000","last_clean":"2024-03-20T07:59:02.982959+0000","last_became_active":"2024-03-20T07:58:26.129798+0000","last_became_peered":"2024-03-20T07:58:26.129798+0000","last_unstale":"2024-03-20T07:59:02.982959+0000","last_undegraded":"2024-03-20T07:59:02.982959+0000","last_fullsized":"2024-03-20T07:59:02.982959+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983444+0000","last_change":"2024-03-20T07:58:25.764736+0000","last_active":"2024-03-20T07:59:02.983444+0000","last_peered":"2024-03-20T07:59:02.983444+0000","last_clean":"2024-03-20T07:59:02.983444+0000","last_became_active":"2024-03-20T07:58:25.764390+0000","last_became_peered":"2024-03-20T07:58:25.764390+0000","last_unstale":"2024-03-20T07:59:02.983444+0000","last_undegraded":"2024-03-20T07:59:02.983444+0000","last_fullsized":"2024-03-20T07:59:02.983444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:27:58.881448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.911953+0000","last_change":"2024-03-20T07:59:01.994090+0000","last_active":"2024-03-20T07:59:05.911953+0000","last_peered":"2024-03-20T07:59:05.911953+0000","last_clean":"2024-03-20T07:59:05.911953+0000","last_became_active":"2024-03-20T07:59:01.993550+0000","last_became_peered":"2024-03-20T07:59:01.993550+0000","last_unstale":"2024-03-20T07:59:05.911953+0000","last_undegraded":"2024-03-20T07:59:05.911953+0000","last_fullsized":"2024-03-20T07:59:05.911953+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:42:05.762120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268817+0000","last_change":"2024-03-20T07:58:26.136285+0000","last_active":"2024-03-20T07:59:05.268817+0000","last_peered":"2024-03-20T07:59:05.268817+0000","last_clean":"2024-03-20T07:59:05.268817+0000","last_became_active":"2024-03-20T07:58:26.135969+0000","last_became_peered":"2024-03-20T07:58:26.135969+0000","last_unstale":"2024-03-20T07:59:05.268817+0000","last_undegraded":"2024-03-20T07:59:05.268817+0000","last_fullsized":"2024-03-20T07:59:05.268817+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:16.328342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985973+0000","last_change":"2024-03-20T07:58:25.766643+0000","last_active":"2024-03-20T07:59:02.985973+0000","last_peered":"2024-03-20T07:59:02.985973+0000","last_clean":"2024-03-20T07:59:02.985973+0000","last_became_active":"2024-03-20T07:58:25.766341+0000","last_became_peered":"2024-03-20T07:58:25.766341+0000","last_unstale":"2024-03-20T07:59:02.985973+0000","last_undegraded":"2024-03-20T07:59:02.985973+0000","last_fullsized":"2024-03-20T07:59:02.985973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"68'128","reported_seq":265,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843565+0000","last_change":"2024-03-20T07:58:09.964395+0000","last_active":"2024-03-20T07:59:04.843565+0000","last_peered":"2024-03-20T07:59:04.843565+0000","last_clean":"2024-03-20T07:59:04.843565+0000","last_became_active":"2024-03-20T07:58:09.963927+0000","last_became_peered":"2024-03-20T07:58:09.963927+0000","last_unstale":"2024-03-20T07:59:04.843565+0000","last_undegraded":"2024-03-20T07:59:04.843565+0000","last_fullsized":"2024-03-20T07:59:04.843565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.385424+0000","last_change":"2024-03-20T07:59:01.989392+0000","last_active":"2024-03-20T07:59:05.385424+0000","last_peered":"2024-03-20T07:59:05.385424+0000","last_clean":"2024-03-20T07:59:05.385424+0000","last_became_active":"2024-03-20T07:59:01.989129+0000","last_became_peered":"2024-03-20T07:59:01.989129+0000","last_unstale":"2024-03-20T07:59:05.385424+0000","last_undegraded":"2024-03-20T07:59:05.385424+0000","last_fullsized":"2024-03-20T07:59:05.385424+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:32.163295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250102+0000","last_change":"2024-03-20T07:58:26.125436+0000","last_active":"2024-03-20T07:59:04.250102+0000","last_peered":"2024-03-20T07:59:04.250102+0000","last_clean":"2024-03-20T07:59:04.250102+0000","last_became_active":"2024-03-20T07:58:26.125161+0000","last_became_peered":"2024-03-20T07:58:26.125161+0000","last_unstale":"2024-03-20T07:59:04.250102+0000","last_undegraded":"2024-03-20T07:59:04.250102+0000","last_fullsized":"2024-03-20T07:59:04.250102+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:53:55.085832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843400+0000","last_change":"2024-03-20T07:58:25.779129+0000","last_active":"2024-03-20T07:59:04.843400+0000","last_peered":"2024-03-20T07:59:04.843400+0000","last_clean":"2024-03-20T07:59:04.843400+0000","last_became_active":"2024-03-20T07:58:25.778258+0000","last_became_peered":"2024-03-20T07:58:25.778258+0000","last_unstale":"2024-03-20T07:59:04.843400+0000","last_undegraded":"2024-03-20T07:59:04.843400+0000","last_fullsized":"2024-03-20T07:59:04.843400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.384551+0000","last_change":"2024-03-20T07:59:01.995060+0000","last_active":"2024-03-20T07:59:05.384551+0000","last_peered":"2024-03-20T07:59:05.384551+0000","last_clean":"2024-03-20T07:59:05.384551+0000","last_became_active":"2024-03-20T07:59:01.994854+0000","last_became_peered":"2024-03-20T07:59:01.994854+0000","last_unstale":"2024-03-20T07:59:05.384551+0000","last_undegraded":"2024-03-20T07:59:05.384551+0000","last_fullsized":"2024-03-20T07:59:05.384551+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:00.138918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844108+0000","last_change":"2024-03-20T07:58:25.777973+0000","last_active":"2024-03-20T07:59:04.844108+0000","last_peered":"2024-03-20T07:59:04.844108+0000","last_clean":"2024-03-20T07:59:04.844108+0000","last_became_active":"2024-03-20T07:58:25.777666+0000","last_became_peered":"2024-03-20T07:58:25.777666+0000","last_unstale":"2024-03-20T07:59:04.844108+0000","last_undegraded":"2024-03-20T07:59:04.844108+0000","last_fullsized":"2024-03-20T07:59:04.844108+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983574+0000","last_change":"2024-03-20T07:58:26.126692+0000","last_active":"2024-03-20T07:59:02.983574+0000","last_peered":"2024-03-20T07:59:02.983574+0000","last_clean":"2024-03-20T07:59:02.983574+0000","last_became_active":"2024-03-20T07:58:26.126397+0000","last_became_peered":"2024-03-20T07:58:26.126397+0000","last_unstale":"2024-03-20T07:59:02.983574+0000","last_undegraded":"2024-03-20T07:59:02.983574+0000","last_fullsized":"2024-03-20T07:59:02.983574+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.437164+0000","last_change":"2024-03-20T07:59:01.993571+0000","last_active":"2024-03-20T07:59:05.437164+0000","last_peered":"2024-03-20T07:59:05.437164+0000","last_clean":"2024-03-20T07:59:05.437164+0000","last_became_active":"2024-03-20T07:59:01.993283+0000","last_became_peered":"2024-03-20T07:59:01.993283+0000","last_unstale":"2024-03-20T07:59:05.437164+0000","last_undegraded":"2024-03-20T07:59:05.437164+0000","last_fullsized":"2024-03-20T07:59:05.437164+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:02:21.502002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"63'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984275+0000","last_change":"2024-03-20T07:58:25.769337+0000","last_active":"2024-03-20T07:59:02.984275+0000","last_peered":"2024-03-20T07:59:02.984275+0000","last_clean":"2024-03-20T07:59:02.984275+0000","last_became_active":"2024-03-20T07:58:25.769010+0000","last_became_peered":"2024-03-20T07:58:25.769010+0000","last_unstale":"2024-03-20T07:59:02.984275+0000","last_undegraded":"2024-03-20T07:59:02.984275+0000","last_fullsized":"2024-03-20T07:59:02.984275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269021+0000","last_change":"2024-03-20T07:58:26.118863+0000","last_active":"2024-03-20T07:59:05.269021+0000","last_peered":"2024-03-20T07:59:05.269021+0000","last_clean":"2024-03-20T07:59:05.269021+0000","last_became_active":"2024-03-20T07:58:26.118565+0000","last_became_peered":"2024-03-20T07:58:26.118565+0000","last_unstale":"2024-03-20T07:59:05.269021+0000","last_undegraded":"2024-03-20T07:59:05.269021+0000","last_fullsized":"2024-03-20T07:59:05.269021+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:49:33.981106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.913562+0000","last_change":"2024-03-20T07:59:01.989083+0000","last_active":"2024-03-20T07:59:05.913562+0000","last_peered":"2024-03-20T07:59:05.913562+0000","last_clean":"2024-03-20T07:59:05.913562+0000","last_became_active":"2024-03-20T07:59:01.988797+0000","last_became_peered":"2024-03-20T07:59:01.988797+0000","last_unstale":"2024-03-20T07:59:05.913562+0000","last_undegraded":"2024-03-20T07:59:05.913562+0000","last_fullsized":"2024-03-20T07:59:05.913562+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:42:40.380304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156515+0000","last_change":"2024-03-20T07:58:25.771097+0000","last_active":"2024-03-20T07:59:03.156515+0000","last_peered":"2024-03-20T07:59:03.156515+0000","last_clean":"2024-03-20T07:59:03.156515+0000","last_became_active":"2024-03-20T07:58:25.770737+0000","last_became_peered":"2024-03-20T07:58:25.770737+0000","last_unstale":"2024-03-20T07:59:03.156515+0000","last_undegraded":"2024-03-20T07:59:03.156515+0000","last_fullsized":"2024-03-20T07:59:03.156515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984463+0000","last_change":"2024-03-20T07:58:46.699298+0000","last_active":"2024-03-20T07:59:02.984463+0000","last_peered":"2024-03-20T07:59:02.984463+0000","last_clean":"2024-03-20T07:59:02.984463+0000","last_became_active":"2024-03-20T07:58:46.698841+0000","last_became_peered":"2024-03-20T07:58:46.698841+0000","last_unstale":"2024-03-20T07:59:02.984463+0000","last_undegraded":"2024-03-20T07:59:02.984463+0000","last_fullsized":"2024-03-20T07:59:02.984463+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:02:06.660176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.843627+0000","last_change":"2024-03-20T07:58:25.777961+0000","last_active":"2024-03-20T07:59:04.843627+0000","last_peered":"2024-03-20T07:59:04.843627+0000","last_clean":"2024-03-20T07:59:04.843627+0000","last_became_active":"2024-03-20T07:58:25.777649+0000","last_became_peered":"2024-03-20T07:58:25.777649+0000","last_unstale":"2024-03-20T07:59:04.843627+0000","last_undegraded":"2024-03-20T07:59:04.843627+0000","last_fullsized":"2024-03-20T07:59:04.843627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269114+0000","last_change":"2024-03-20T07:58:26.120305+0000","last_active":"2024-03-20T07:59:05.269114+0000","last_peered":"2024-03-20T07:59:05.269114+0000","last_clean":"2024-03-20T07:59:05.269114+0000","last_became_active":"2024-03-20T07:58:26.119966+0000","last_became_peered":"2024-03-20T07:58:26.119966+0000","last_unstale":"2024-03-20T07:59:05.269114+0000","last_undegraded":"2024-03-20T07:59:05.269114+0000","last_fullsized":"2024-03-20T07:59:05.269114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983215+0000","last_change":"2024-03-20T07:58:25.765548+0000","last_active":"2024-03-20T07:59:02.983215+0000","last_peered":"2024-03-20T07:59:02.983215+0000","last_clean":"2024-03-20T07:59:02.983215+0000","last_became_active":"2024-03-20T07:58:25.765273+0000","last_became_peered":"2024-03-20T07:58:25.765273+0000","last_unstale":"2024-03-20T07:59:02.983215+0000","last_undegraded":"2024-03-20T07:59:02.983215+0000","last_fullsized":"2024-03-20T07:59:02.983215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268509+0000","last_change":"2024-03-20T07:58:26.117669+0000","last_active":"2024-03-20T07:59:05.268509+0000","last_peered":"2024-03-20T07:59:05.268509+0000","last_clean":"2024-03-20T07:59:05.268509+0000","last_became_active":"2024-03-20T07:58:26.117376+0000","last_became_peered":"2024-03-20T07:58:26.117376+0000","last_unstale":"2024-03-20T07:59:05.268509+0000","last_undegraded":"2024-03-20T07:59:05.268509+0000","last_fullsized":"2024-03-20T07:59:05.268509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:42:02.274133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983347+0000","last_change":"2024-03-20T07:58:25.773811+0000","last_active":"2024-03-20T07:59:02.983347+0000","last_peered":"2024-03-20T07:59:02.983347+0000","last_clean":"2024-03-20T07:59:02.983347+0000","last_became_active":"2024-03-20T07:58:25.773520+0000","last_became_peered":"2024-03-20T07:58:25.773520+0000","last_unstale":"2024-03-20T07:59:02.983347+0000","last_undegraded":"2024-03-20T07:59:02.983347+0000","last_fullsized":"2024-03-20T07:59:02.983347+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268703+0000","last_change":"2024-03-20T07:58:26.123160+0000","last_active":"2024-03-20T07:59:05.268703+0000","last_peered":"2024-03-20T07:59:05.268703+0000","last_clean":"2024-03-20T07:59:05.268703+0000","last_became_active":"2024-03-20T07:58:26.122680+0000","last_became_peered":"2024-03-20T07:58:26.122680+0000","last_unstale":"2024-03-20T07:59:05.268703+0000","last_undegraded":"2024-03-20T07:59:05.268703+0000","last_fullsized":"2024-03-20T07:59:05.268703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:09:56.675972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"62'2","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983403+0000","last_change":"2024-03-20T07:58:46.309368+0000","last_active":"2024-03-20T07:59:02.983403+0000","last_peered":"2024-03-20T07:59:02.983403+0000","last_clean":"2024-03-20T07:59:02.983403+0000","last_became_active":"2024-03-20T07:58:46.306388+0000","last_became_peered":"2024-03-20T07:58:46.306388+0000","last_unstale":"2024-03-20T07:59:02.983403+0000","last_undegraded":"2024-03-20T07:59:02.983403+0000","last_fullsized":"2024-03-20T07:59:02.983403+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:42:19.060358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156129+0000","last_change":"2024-03-20T07:58:26.134067+0000","last_active":"2024-03-20T07:59:03.156129+0000","last_peered":"2024-03-20T07:59:03.156129+0000","last_clean":"2024-03-20T07:59:03.156129+0000","last_became_active":"2024-03-20T07:58:26.133784+0000","last_became_peered":"2024-03-20T07:58:26.133784+0000","last_unstale":"2024-03-20T07:59:03.156129+0000","last_undegraded":"2024-03-20T07:59:03.156129+0000","last_fullsized":"2024-03-20T07:59:03.156129+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983922+0000","last_change":"2024-03-20T07:58:25.763015+0000","last_active":"2024-03-20T07:59:02.983922+0000","last_peered":"2024-03-20T07:59:02.983922+0000","last_clean":"2024-03-20T07:59:02.983922+0000","last_became_active":"2024-03-20T07:58:25.762682+0000","last_became_peered":"2024-03-20T07:58:25.762682+0000","last_unstale":"2024-03-20T07:59:02.983922+0000","last_undegraded":"2024-03-20T07:59:02.983922+0000","last_fullsized":"2024-03-20T07:59:02.983922+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:21:45.137669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844015+0000","last_change":"2024-03-20T07:58:26.142445+0000","last_active":"2024-03-20T07:59:04.844015+0000","last_peered":"2024-03-20T07:59:04.844015+0000","last_clean":"2024-03-20T07:59:04.844015+0000","last_became_active":"2024-03-20T07:58:26.142148+0000","last_became_peered":"2024-03-20T07:58:26.142148+0000","last_unstale":"2024-03-20T07:59:04.844015+0000","last_undegraded":"2024-03-20T07:59:04.844015+0000","last_fullsized":"2024-03-20T07:59:04.844015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"62'1","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983793+0000","last_change":"2024-03-20T07:58:25.766427+0000","last_active":"2024-03-20T07:59:02.983793+0000","last_peered":"2024-03-20T07:59:02.983793+0000","last_clean":"2024-03-20T07:59:02.983793+0000","last_became_active":"2024-03-20T07:58:25.766111+0000","last_became_peered":"2024-03-20T07:58:25.766111+0000","last_unstale":"2024-03-20T07:59:02.983793+0000","last_undegraded":"2024-03-20T07:59:02.983793+0000","last_fullsized":"2024-03-20T07:59:02.983793+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986095+0000","last_change":"2024-03-20T07:58:26.135339+0000","last_active":"2024-03-20T07:59:02.986095+0000","last_peered":"2024-03-20T07:59:02.986095+0000","last_clean":"2024-03-20T07:59:02.986095+0000","last_became_active":"2024-03-20T07:58:26.134903+0000","last_became_peered":"2024-03-20T07:58:26.134903+0000","last_unstale":"2024-03-20T07:59:02.986095+0000","last_undegraded":"2024-03-20T07:59:02.986095+0000","last_fullsized":"2024-03-20T07:59:02.986095+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"62'2","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.984547+0000","last_change":"2024-03-20T07:58:25.766233+0000","last_active":"2024-03-20T07:59:02.984547+0000","last_peered":"2024-03-20T07:59:02.984547+0000","last_clean":"2024-03-20T07:59:02.984547+0000","last_became_active":"2024-03-20T07:58:25.765948+0000","last_became_peered":"2024-03-20T07:58:25.765948+0000","last_unstale":"2024-03-20T07:59:02.984547+0000","last_undegraded":"2024-03-20T07:59:02.984547+0000","last_fullsized":"2024-03-20T07:59:02.984547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:34:01.876341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.844255+0000","last_change":"2024-03-20T07:58:26.144008+0000","last_active":"2024-03-20T07:59:04.844255+0000","last_peered":"2024-03-20T07:59:04.844255+0000","last_clean":"2024-03-20T07:59:04.844255+0000","last_became_active":"2024-03-20T07:58:26.143696+0000","last_became_peered":"2024-03-20T07:58:26.143696+0000","last_unstale":"2024-03-20T07:59:04.844255+0000","last_undegraded":"2024-03-20T07:59:04.844255+0000","last_fullsized":"2024-03-20T07:59:04.844255+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.155990+0000","last_change":"2024-03-20T07:58:26.138096+0000","last_active":"2024-03-20T07:59:03.155990+0000","last_peered":"2024-03-20T07:59:03.155990+0000","last_clean":"2024-03-20T07:59:03.155990+0000","last_became_active":"2024-03-20T07:58:26.137804+0000","last_became_peered":"2024-03-20T07:58:26.137804+0000","last_unstale":"2024-03-20T07:59:03.155990+0000","last_undegraded":"2024-03-20T07:59:03.155990+0000","last_fullsized":"2024-03-20T07:59:03.155990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983522+0000","last_change":"2024-03-20T07:58:26.126085+0000","last_active":"2024-03-20T07:59:02.983522+0000","last_peered":"2024-03-20T07:59:02.983522+0000","last_clean":"2024-03-20T07:59:02.983522+0000","last_became_active":"2024-03-20T07:58:26.125793+0000","last_became_peered":"2024-03-20T07:58:26.125793+0000","last_unstale":"2024-03-20T07:59:02.983522+0000","last_undegraded":"2024-03-20T07:59:02.983522+0000","last_fullsized":"2024-03-20T07:59:02.983522+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:11:45.470755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985945+0000","last_change":"2024-03-20T07:58:26.125004+0000","last_active":"2024-03-20T07:59:02.985945+0000","last_peered":"2024-03-20T07:59:02.985945+0000","last_clean":"2024-03-20T07:59:02.985945+0000","last_became_active":"2024-03-20T07:58:26.124689+0000","last_became_peered":"2024-03-20T07:58:26.124689+0000","last_unstale":"2024-03-20T07:59:02.985945+0000","last_undegraded":"2024-03-20T07:59:02.985945+0000","last_fullsized":"2024-03-20T07:59:02.985945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250247+0000","last_change":"2024-03-20T07:58:26.120048+0000","last_active":"2024-03-20T07:59:04.250247+0000","last_peered":"2024-03-20T07:59:04.250247+0000","last_clean":"2024-03-20T07:59:04.250247+0000","last_became_active":"2024-03-20T07:58:26.119755+0000","last_became_peered":"2024-03-20T07:58:26.119755+0000","last_unstale":"2024-03-20T07:59:04.250247+0000","last_undegraded":"2024-03-20T07:59:04.250247+0000","last_fullsized":"2024-03-20T07:59:04.250247+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:39:07.689571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":27,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268900+0000","last_change":"2024-03-20T07:58:46.699520+0000","last_active":"2024-03-20T07:59:05.268900+0000","last_peered":"2024-03-20T07:59:05.268900+0000","last_clean":"2024-03-20T07:59:05.268900+0000","last_became_active":"2024-03-20T07:58:46.699085+0000","last_became_peered":"2024-03-20T07:58:46.699085+0000","last_unstale":"2024-03-20T07:59:05.268900+0000","last_undegraded":"2024-03-20T07:59:05.268900+0000","last_fullsized":"2024-03-20T07:59:05.268900+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:27.802971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.983711+0000","last_change":"2024-03-20T07:58:26.126113+0000","last_active":"2024-03-20T07:59:02.983711+0000","last_peered":"2024-03-20T07:59:02.983711+0000","last_clean":"2024-03-20T07:59:02.983711+0000","last_became_active":"2024-03-20T07:58:26.122635+0000","last_became_peered":"2024-03-20T07:58:26.122635+0000","last_unstale":"2024-03-20T07:59:02.983711+0000","last_undegraded":"2024-03-20T07:59:02.983711+0000","last_fullsized":"2024-03-20T07:59:02.983711+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156462+0000","last_change":"2024-03-20T07:58:26.141379+0000","last_active":"2024-03-20T07:59:03.156462+0000","last_peered":"2024-03-20T07:59:03.156462+0000","last_clean":"2024-03-20T07:59:03.156462+0000","last_became_active":"2024-03-20T07:58:26.141099+0000","last_became_peered":"2024-03-20T07:58:26.141099+0000","last_unstale":"2024-03-20T07:59:03.156462+0000","last_undegraded":"2024-03-20T07:59:03.156462+0000","last_fullsized":"2024-03-20T07:59:03.156462+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.269071+0000","last_change":"2024-03-20T07:58:26.121389+0000","last_active":"2024-03-20T07:59:05.269071+0000","last_peered":"2024-03-20T07:59:05.269071+0000","last_clean":"2024-03-20T07:59:05.269071+0000","last_became_active":"2024-03-20T07:58:26.121094+0000","last_became_peered":"2024-03-20T07:58:26.121094+0000","last_unstale":"2024-03-20T07:59:05.269071+0000","last_undegraded":"2024-03-20T07:59:05.269071+0000","last_fullsized":"2024-03-20T07:59:05.269071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:08:53.389656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.985872+0000","last_change":"2024-03-20T07:58:26.137638+0000","last_active":"2024-03-20T07:59:02.985872+0000","last_peered":"2024-03-20T07:59:02.985872+0000","last_clean":"2024-03-20T07:59:02.985872+0000","last_became_active":"2024-03-20T07:58:26.137344+0000","last_became_peered":"2024-03-20T07:58:26.137344+0000","last_unstale":"2024-03-20T07:59:02.985872+0000","last_undegraded":"2024-03-20T07:59:02.985872+0000","last_fullsized":"2024-03-20T07:59:02.985872+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:05.268556+0000","last_change":"2024-03-20T07:58:26.122341+0000","last_active":"2024-03-20T07:59:05.268556+0000","last_peered":"2024-03-20T07:59:05.268556+0000","last_clean":"2024-03-20T07:59:05.268556+0000","last_became_active":"2024-03-20T07:58:26.122003+0000","last_became_peered":"2024-03-20T07:58:26.122003+0000","last_unstale":"2024-03-20T07:59:05.268556+0000","last_undegraded":"2024-03-20T07:59:05.268556+0000","last_fullsized":"2024-03-20T07:59:05.268556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:20.158675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:02.986235+0000","last_change":"2024-03-20T07:58:26.137117+0000","last_active":"2024-03-20T07:59:02.986235+0000","last_peered":"2024-03-20T07:59:02.986235+0000","last_clean":"2024-03-20T07:59:02.986235+0000","last_became_active":"2024-03-20T07:58:26.136135+0000","last_became_peered":"2024-03-20T07:58:26.136135+0000","last_unstale":"2024-03-20T07:59:02.986235+0000","last_undegraded":"2024-03-20T07:59:02.986235+0000","last_fullsized":"2024-03-20T07:59:02.986235+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:04.250340+0000","last_change":"2024-03-20T07:58:26.119480+0000","last_active":"2024-03-20T07:59:04.250340+0000","last_peered":"2024-03-20T07:59:04.250340+0000","last_clean":"2024-03-20T07:59:04.250340+0000","last_became_active":"2024-03-20T07:58:26.119047+0000","last_became_peered":"2024-03-20T07:58:26.119047+0000","last_unstale":"2024-03-20T07:59:04.250340+0000","last_undegraded":"2024-03-20T07:59:04.250340+0000","last_fullsized":"2024-03-20T07:59:04.250340+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:09:31.918661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-03-20T07:59:03.156361+0000","last_change":"2024-03-20T07:58:26.133160+0000","last_active":"2024-03-20T07:59:03.156361+0000","last_peered":"2024-03-20T07:59:03.156361+0000","last_clean":"2024-03-20T07:59:03.156361+0000","last_became_active":"2024-03-20T07:58:26.132793+0000","last_became_peered":"2024-03-20T07:58:26.132793+0000","last_unstale":"2024-03-20T07:59:03.156361+0000","last_undegraded":"2024-03-20T07:59:03.156361+0000","last_fullsized":"2024-03-20T07:59:03.156361+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3146,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":9438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201305,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28312,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716840,"statfs":{"total":95995035648,"available":95966044160,"internally_reserved":0,"allocated":1318912,"data_stored":902683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.372}]},{"osd":1,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.32000000000000001}]},{"osd":2,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.439}]},{"osd":3,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.499}]},{"osd":4,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65200000000000002}]},{"osd":5,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45400000000000001}]},{"osd":6,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.61799999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462942,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716844,"statfs":{"total":95995035648,"available":95966048256,"internally_reserved":0,"allocated":1318912,"data_stored":904096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.33400000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.747,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.5,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68300000000000005,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.68100000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51000000000000001,"15min":0.51600000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67100000000000004,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.55500000000000005,"15min":0.56499999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61699999999999999}]},{"osd":3,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71999999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65500000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.43099999999999999,"15min":0.432},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57099999999999995,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47899999999999998,"15min":0.47999999999999998},"min":{"1min":0.33700000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.69699999999999995}]},{"osd":5,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.309,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69499999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed Mar 20 07:59:07 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.751}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717424,"statfs":{"total":95995035648,"available":95966642176,"internally_reserved":0,"allocated":724992,"data_stored":311871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.32700000000000001},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48299999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.376,"15min":0.34999999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.627,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.33600000000000002}]},{"osd":2,"last update":"Wed Mar 20 07:59:11 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.38600000000000001},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.375},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.58099999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.35999999999999999}]},{"osd":3,"last update":"Wed Mar 20 07:59:33 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56499999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.46600000000000003,"15min":0.48099999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.41899999999999998}]},{"osd":4,"last update":"Wed Mar 20 07:59:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79300000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.68100000000000005,"5min":0.873,"15min":0.873},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.57099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60399999999999998,"15min":0.60199999999999998},"min":{"1min":0.49099999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77700000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.503}]},{"osd":6,"last update":"Wed Mar 20 07:59:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.627,"15min":0.63500000000000001},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.746,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.40600000000000003}]},{"osd":7,"last update":"Wed Mar 20 07:59:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.438},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.46400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28320,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716832,"statfs":{"total":95995035648,"available":95966035968,"internally_reserved":0,"allocated":1327104,"data_stored":902807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 20 07:59:55 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.32600000000000001,"15min":0.31},"min":{"1min":0.17199999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.45000000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.34000000000000002,"15min":0.318},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.505,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.31}]},{"osd":2,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.52200000000000002,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.40100000000000002,"15min":0.39700000000000002},"min":{"1min":0.29699999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.55400000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.28999999999999998}]},{"osd":3,"last update":"Wed Mar 20 07:59:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.40699999999999997,"15min":0.41299999999999998},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.46200000000000002,"15min":0.46800000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.61599999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.40899999999999997}]},{"osd":4,"last update":"Wed Mar 20 07:59:55 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66700000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79900000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.371}]},{"osd":5,"last update":"Wed Mar 20 07:59:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59899999999999998,"15min":0.60499999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.871,"15min":0.871},"last":0.39500000000000002}]},{"osd":6,"last update":"Wed Mar 20 07:59:41 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71299999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82699999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.439}]},{"osd":7,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47999999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":311711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.26300000000000001,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":0.46999999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.307},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40300000000000002,"15min":0.39400000000000002},"min":{"1min":0.28299999999999997,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47799999999999998,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.379,"15min":0.375},"min":{"1min":0.248,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.44800000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.38,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.41699999999999998,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.40400000000000003}]},{"osd":3,"last update":"Wed Mar 20 07:59:32 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.40300000000000002,"15min":0.41399999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.497,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.437,"15min":0.44900000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.46899999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.46200000000000002}]},{"osd":4,"last update":"Wed Mar 20 07:59:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.65000000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57999999999999996,"15min":0.58599999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.74299999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Wed Mar 20 07:59:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78400000000000003,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82399999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53900000000000003}]},{"osd":6,"last update":"Wed Mar 20 07:59:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.502},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52200000000000002}]},{"osd":7,"last update":"Wed Mar 20 07:59:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57599999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149740,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717408,"statfs":{"total":95995035648,"available":95966625792,"internally_reserved":0,"allocated":737280,"data_stored":314163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:34 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.41799999999999998,"15min":0.433},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53300000000000003,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.25},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.437,"15min":0.442},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.57499999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.28799999999999998}]},{"osd":1,"last update":"Wed Mar 20 07:59:34 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.40600000000000003,"15min":0.42799999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.46800000000000003,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.31},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.47199999999999998,"15min":0.48999999999999999},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Mar 20 07:59:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45200000000000001,"15min":0.46800000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48999999999999999,"15min":0.502},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.624,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.56999999999999995},"min":{"1min":0.39800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71999999999999997,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80300000000000005,"5min":0.876,"15min":0.876},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed Mar 20 07:59:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82599999999999996,"5min":1.056,"15min":1.056},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86799999999999999,"5min":1.099,"15min":1.099},"last":0.51100000000000001}]},{"osd":6,"last update":"Wed Mar 20 07:59:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68400000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.499}]},{"osd":7,"last update":"Wed Mar 20 07:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45100000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986216,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27740,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717412,"statfs":{"total":95995035648,"available":95966629888,"internally_reserved":0,"allocated":733184,"data_stored":312419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.39600000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67200000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.443,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.44}]},{"osd":1,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.63600000000000001,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.40200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67400000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.44900000000000001}]},{"osd":2,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.46600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.73799999999999999,"5min":0.75,"15min":0.75},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46800000000000003}]},{"osd":3,"last update":"Wed Mar 20 07:59:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.40699999999999997}]},{"osd":5,"last update":"Wed Mar 20 07:59:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.376},"min":{"1min":0.29499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.374},"min":{"1min":0.33300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.377}]},{"osd":6,"last update":"Wed Mar 20 07:59:46 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.45300000000000001,"15min":0.441},"min":{"1min":0.35899999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.45900000000000002},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.34100000000000003}]},{"osd":7,"last update":"Wed Mar 20 07:59:07 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39600000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724579,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":733184,"data_stored":314149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56799999999999995}]},{"osd":1,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.745,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.499},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.61299999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.875,"15min":0.875},"last":0.48199999999999998}]},{"osd":3,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.45400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.625,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68799999999999994}]},{"osd":4,"last update":"Wed Mar 20 07:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.38,"15min":0.38100000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.50800000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39500000000000002,"15min":0.39300000000000002},"min":{"1min":0.29999999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.34200000000000003}]},{"osd":6,"last update":"Wed Mar 20 07:59:43 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.31,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.31},{"interface":"front","average":{"1min":0.442,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.312,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59699999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.35499999999999998}]},{"osd":7,"last update":"Wed Mar 20 07:59:06 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.374}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:00:03.906 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-20T08:00:03.906 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-20T08:00:03.906 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-20T08:00:03.906 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-20T08:00:04.120 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:00:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:05 smithi196 ceph-mon[31033]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:00:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:05 smithi196 ceph-mon[31033]: pgmap v39: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:05.407 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:05 smithi118 ceph-mon[27391]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:00:05.407 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:05 smithi118 ceph-mon[27391]: pgmap v39: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:05.407 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:05 smithi118 ceph-mon[38981]: from='client.24674 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:00:05.407 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:05 smithi118 ceph-mon[38981]: pgmap v39: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:06.183 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:06.183 INFO:teuthology.orchestra.run.smithi118.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-20T08:00:06.826 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:06 smithi196 bash[63024]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-03-20T08:00:06.986 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-20T08:00:06.987 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T08:00:07.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-mon[31033]: pgmap v40: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:07.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2949210657' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-20T08:00:07.139 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:06 smithi196 bash[63024]: Writing manifest to image destination 2024-03-20T08:00:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:07 smithi118 ceph-mon[27391]: pgmap v40: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:07 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2949210657' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-20T08:00:07.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:07 smithi118 ceph-mon[38981]: pgmap v40: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:07.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:07 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2949210657' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-20T08:00:07.275 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:07.275 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":70,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:59:05.381551+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-03-20T07:59:00.573805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":7,"to":2}]},{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1},{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":0,"to":1},{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/2649753508":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/3938132886":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6800/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/184826175":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/1237762586":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T08:00:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-20T07:56:03.504701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-20T07:58:24.105659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-20T07:58:25.289141+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'rbd', 'create_time': '2024-03-20T07:59:00.573805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-20T08:00:07.288 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-20T08:00:07.432 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 podman[63024]: 2024-03-20 08:00:07.074944721 +0000 UTC m=+7.839609256 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-20T08:00:07.432 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 podman[63024]: 2024-03-20 08:00:07.181871717 +0000 UTC m=+7.946536239 container create fd3b2992729c450834df853343fb76eaf82d5259eaa6e522e1d73344b819274e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:00:07.432 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 podman[63024]: 2024-03-20 08:00:07.434009812 +0000 UTC m=+8.198674348 container init fd3b2992729c450834df853343fb76eaf82d5259eaa6e522e1d73344b819274e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:00:07.573 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-03-20T08:00:07.586 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-20T08:00:07.880 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 podman[63024]: 2024-03-20 08:00:07.440127194 +0000 UTC m=+8.204791717 container start fd3b2992729c450834df853343fb76eaf82d5259eaa6e522e1d73344b819274e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.444Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.444Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.444Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.445Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.445Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.445Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.445Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=arp 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-20T08:00:07.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-20T08:00:07.890 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-20T08:00:07.890 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-20T08:00:07.890 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-20T08:00:07.890 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-20T08:00:07.890 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-20T08:00:07.890 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-20T08:00:07.890 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=edac 2024-03-20T08:00:07.890 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-20T08:00:07.891 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=os 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=stat 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-20T08:00:07.892 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=time 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=uname 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[63077]: ts=2024-03-20T08:00:07.446Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 bash[63024]: fd3b2992729c450834df853343fb76eaf82d5259eaa6e522e1d73344b819274e 2024-03-20T08:00:07.893 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:00:07 smithi196 systemd[1]: Started Ceph node-exporter.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:00:07.897 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3628297128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1177949611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:00:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3474112790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3628297128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1177949611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:00:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:08 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3474112790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-20T08:00:08.226 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 64 2024-03-20T08:00:08.240 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-03-20T08:00:08.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3628297128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1177949611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T08:00:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:00:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:08 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3474112790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-20T08:00:08.533 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 8 2024-03-20T08:00:08.547 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T08:00:08.868 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:08.868 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3215808514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3215808514},{"type":"v1","addr":"172.21.15.118:6835","nonce":3215808514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-20T07:58:26.102802+0000","modified":"2024-03-20T07:58:28.113441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24343":{"gid":24343,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/5972345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":5972345},{"type":"v1","addr":"172.21.15.196:6833","nonce":5972345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-20T08:00:08.868 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 6 2024-03-20T08:00:08.881 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T08:00:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:09 smithi118 ceph-mon[27391]: pgmap v41: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:09 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/534701694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-20T08:00:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:09 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1373197285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-03-20T08:00:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:09 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/547262322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:09.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:09 smithi118 ceph-mon[38981]: pgmap v41: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:09 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/534701694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-20T08:00:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:09 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1373197285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-03-20T08:00:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:09 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/547262322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:09.195 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:09.195 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":70,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:59:05.381551+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-03-20T07:59:00.573805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":7,"to":2}]},{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1},{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":0,"to":1},{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/2649753508":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/3938132886":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6800/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/184826175":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/1237762586":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T08:00:09.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-20T07:56:03.504701+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-20T07:58:24.105659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-20T07:58:25.289141+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'rbd', 'create_time': '2024-03-20T07:59:00.573805+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-20T08:00:09.209 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-20T08:00:09.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:09 smithi196 ceph-mon[31033]: pgmap v41: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:09.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:09 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/534701694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-20T08:00:09.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:09 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1373197285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-03-20T08:00:09.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:09 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/547262322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:09.511 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 1 2024-03-20T08:00:09.525 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-20T08:00:09.818 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 32 2024-03-20T08:00:09.830 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-20T08:00:10.133 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 64 2024-03-20T08:00:10.145 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:10 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/879376080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:10.145 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:10 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/706108782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T08:00:10.145 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:10 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1993797509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-20T08:00:10.145 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:10 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:10.146 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:10 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/879376080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:10.146 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:10 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/706108782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T08:00:10.146 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:10 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1993797509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-20T08:00:10.146 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:10 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:10.147 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-03-20T08:00:10.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:10 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/879376080' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:10.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:10 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/706108782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-20T08:00:10.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:10 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1993797509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-20T08:00:10.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:10 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:10.437 INFO:teuthology.orchestra.run.smithi118.stdout:pg_num: 8 2024-03-20T08:00:10.450 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T08:00:10.772 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:10.773 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3215808514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3215808514},{"type":"v1","addr":"172.21.15.118:6835","nonce":3215808514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-20T07:58:26.102802+0000","modified":"2024-03-20T07:58:28.113441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24343":{"gid":24343,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/5972345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":5972345},{"type":"v1","addr":"172.21.15.196:6833","nonce":5972345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-20T08:00:10.773 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 6 2024-03-20T08:00:10.786 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T08:00:11.089 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:11.089 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":70,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T07:59:05.381551+0000","last_up_change":"2024-03-20T07:58:05.946971+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-03-20T07:59:00.573805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6803","nonce":4171340853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6805","nonce":4171340853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6809","nonce":4171340853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":4171340853},{"type":"v1","addr":"172.21.15.118:6807","nonce":4171340853}]},"public_addr":"172.21.15.118:6803/4171340853","cluster_addr":"172.21.15.118:6805/4171340853","heartbeat_back_addr":"172.21.15.118:6809/4171340853","heartbeat_front_addr":"172.21.15.118:6807/4171340853","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6811","nonce":4053982816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6813","nonce":4053982816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6817","nonce":4053982816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":4053982816},{"type":"v1","addr":"172.21.15.118:6815","nonce":4053982816}]},"public_addr":"172.21.15.118:6811/4053982816","cluster_addr":"172.21.15.118:6813/4053982816","heartbeat_back_addr":"172.21.15.118:6817/4053982816","heartbeat_front_addr":"172.21.15.118:6815/4053982816","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6819","nonce":1856880060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6821","nonce":1856880060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6825","nonce":1856880060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":1856880060},{"type":"v1","addr":"172.21.15.118:6823","nonce":1856880060}]},"public_addr":"172.21.15.118:6819/1856880060","cluster_addr":"172.21.15.118:6821/1856880060","heartbeat_back_addr":"172.21.15.118:6825/1856880060","heartbeat_front_addr":"172.21.15.118:6823/1856880060","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6827","nonce":3211575705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6829","nonce":3211575705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6833","nonce":3211575705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":3211575705},{"type":"v1","addr":"172.21.15.118:6831","nonce":3211575705}]},"public_addr":"172.21.15.118:6827/3211575705","cluster_addr":"172.21.15.118:6829/3211575705","heartbeat_back_addr":"172.21.15.118:6833/3211575705","heartbeat_front_addr":"172.21.15.118:6831/3211575705","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6801","nonce":2757499663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6803","nonce":2757499663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6807","nonce":2757499663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2757499663},{"type":"v1","addr":"172.21.15.196:6805","nonce":2757499663}]},"public_addr":"172.21.15.196:6801/2757499663","cluster_addr":"172.21.15.196:6803/2757499663","heartbeat_back_addr":"172.21.15.196:6807/2757499663","heartbeat_front_addr":"172.21.15.196:6805/2757499663","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6809","nonce":3669975509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6811","nonce":3669975509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6815","nonce":3669975509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":3669975509},{"type":"v1","addr":"172.21.15.196:6813","nonce":3669975509}]},"public_addr":"172.21.15.196:6809/3669975509","cluster_addr":"172.21.15.196:6811/3669975509","heartbeat_back_addr":"172.21.15.196:6815/3669975509","heartbeat_front_addr":"172.21.15.196:6813/3669975509","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6817","nonce":1568164477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6819","nonce":1568164477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6823","nonce":1568164477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":1568164477},{"type":"v1","addr":"172.21.15.196:6821","nonce":1568164477}]},"public_addr":"172.21.15.196:6817/1568164477","cluster_addr":"172.21.15.196:6819/1568164477","heartbeat_back_addr":"172.21.15.196:6823/1568164477","heartbeat_front_addr":"172.21.15.196:6821/1568164477","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6825","nonce":2088108474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6827","nonce":2088108474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6831","nonce":2088108474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2088108474},{"type":"v1","addr":"172.21.15.196:6829","nonce":2088108474}]},"public_addr":"172.21.15.196:6825/2088108474","cluster_addr":"172.21.15.196:6827/2088108474","heartbeat_back_addr":"172.21.15.196:6831/2088108474","heartbeat_front_addr":"172.21.15.196:6829/2088108474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":7,"to":2}]},{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1},{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":0,"to":1},{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.118:0/2649753508":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/3938132886":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6800/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/184826175":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/1237762586":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T08:00:11.100 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[27391]: pgmap v42: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:11.100 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2122471243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-20T08:00:11.100 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2498478817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3463723535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[38981]: pgmap v42: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2122471243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2498478817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3463723535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:11.103 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T08:00:11.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:11 smithi196 ceph-mon[31033]: pgmap v42: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:11.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:11 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2122471243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-20T08:00:11.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:11 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2498478817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]: dispatch 2024-03-20T08:00:11.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:11 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:11.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:11 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3463723535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:11.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:11 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:11.473 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:00:11.473 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14430,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.118:6835/3215808514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":3215808514},{"type":"v1","addr":"172.21.15.118:6835","nonce":3215808514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-20T07:58:26.102802+0000","modified":"2024-03-20T07:58:28.113441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24343":{"gid":24343,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6833/5972345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":5972345},{"type":"v1","addr":"172.21.15.196:6833","nonce":5972345}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-20T08:00:11.473 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 6 2024-03-20T08:00:11.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-20T07:58:26.102802+0000', 'modified': '2024-03-20T07:58:28.113441+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24343}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24343': {'gid': 24343, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6833/5972345', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 5972345}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 5972345}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-20T08:00:11.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-20T08:00:11.487 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-20T08:00:11.598 INFO:teuthology.orchestra.run.smithi118.stderr:2024-03-20T08:00:11.599+0000 7fc4f77fe640 0 client.24776 ms_handle_reset on v2:172.21.15.196:6832/5972345 2024-03-20T08:00:11.616 INFO:teuthology.orchestra.run.smithi118.stderr:2024-03-20T08:00:11.617+0000 7fc4f77fe640 0 client.14877 ms_handle_reset on v2:172.21.15.196:6832/5972345 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "cluster_fsid": "bc2566e8-e68e-11ee-95c9-87774f69a715", 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "whoami": 0, 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "id": 24343, 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "want_state": "up:active", 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "state": "up:active", 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "fs_name": "cephfs", 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "rank_uptime": 105.513171024, 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "mdsmap_epoch": 6, 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_epoch": 70, 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "osdmap_epoch_barrier": 63, 2024-03-20T08:00:11.617 INFO:teuthology.orchestra.run.smithi118.stdout: "uptime": 109.69441745899999 2024-03-20T08:00:11.618 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:00:11.631 INFO:tasks.cephadm:Setup complete, yielding 2024-03-20T08:00:11.631 INFO:teuthology.run_tasks:Running task print... 2024-03-20T08:00:11.641 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-03-20T08:00:11.641 INFO:teuthology.run_tasks:Running task print... 2024-03-20T08:00:11.660 INFO:teuthology.task.print:**** done start cephadm.shell ceph config set mgr... 2024-03-20T08:00:11.661 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-20T08:00:11.673 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi118.front.sepia.ceph.com 2024-03-20T08:00:11.673 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-20T08:00:11.873 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:00:11.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:00:11] "GET /metrics HTTP/1.1" 200 33648 "" "Prometheus/2.43.0" 2024-03-20T08:00:12.311 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3099293576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/444221498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3099293576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/444221498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:12.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:12.313 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:00:12.313 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:00:12.313 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:12 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:12 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3099293576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:00:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:12 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:12 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/444221498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:00:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:12 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:00:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:00:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:12 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:13.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:13 smithi118 ceph-mon[27391]: pgmap v43: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:13.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:13 smithi118 ceph-mon[27391]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-20T08:00:13.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:13 smithi118 ceph-mon[27391]: Reconfiguring daemon alertmanager.a on smithi118 2024-03-20T08:00:13.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:13 smithi118 ceph-mon[38981]: pgmap v43: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:13.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:13 smithi118 ceph-mon[38981]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-20T08:00:13.313 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:13 smithi118 ceph-mon[38981]: Reconfiguring daemon alertmanager.a on smithi118 2024-03-20T08:00:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:13 smithi196 ceph-mon[31033]: pgmap v43: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:13 smithi196 ceph-mon[31033]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-20T08:00:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:13 smithi196 ceph-mon[31033]: Reconfiguring daemon alertmanager.a on smithi118 2024-03-20T08:00:15.012 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:14] ENGINE Bus STOPPING 2024-03-20T08:00:15.012 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:14] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:00:15.012 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:14] ENGINE Bus STOPPED 2024-03-20T08:00:15.012 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:14] ENGINE Bus STARTING 2024-03-20T08:00:15.012 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:14] ENGINE Serving on http://:::9283 2024-03-20T08:00:15.012 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:14] ENGINE Bus STARTED 2024-03-20T08:00:15.322 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:15 smithi118 ceph-mon[27391]: pgmap v44: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:15.322 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:15 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' 2024-03-20T08:00:15.322 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:15 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:15.322 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:15 smithi118 ceph-mon[38981]: pgmap v44: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:15.322 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:15 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' 2024-03-20T08:00:15.323 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:15 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:15.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:15 smithi196 ceph-mon[31033]: pgmap v44: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:15 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' 2024-03-20T08:00:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:15 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:16.570 INFO:teuthology.run_tasks:Running task print... 2024-03-20T08:00:16.579 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-20T08:00:16.579 INFO:teuthology.run_tasks:Running task print... 2024-03-20T08:00:16.589 INFO:teuthology.task.print:**** done start telemetry reef... 2024-03-20T08:00:16.589 INFO:teuthology.run_tasks:Running task workunit... 2024-03-20T08:00:16.606 INFO:tasks.workunit:Pulling workunits from ref e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:00:16.606 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-20T08:00:16.606 DEBUG:teuthology.orchestra.run.smithi118:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-20T08:00:16.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T08:00:16.623 INFO:teuthology.orchestra.run.smithi118.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-20T08:00:16.624 DEBUG:teuthology.orchestra.run.smithi118:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-20T08:00:16.679 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-20T08:00:16.680 DEBUG:teuthology.orchestra.run.smithi118:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-20T08:00:16.736 INFO:tasks.workunit:timeout=3h 2024-03-20T08:00:16.736 INFO:tasks.workunit:cleanup=True 2024-03-20T08:00:16.737 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:00:16.792 INFO:tasks.workunit.client.0.smithi118.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-20T08:00:17.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:17 smithi118 ceph-mon[27391]: pgmap v45: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:17.185 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:17 smithi118 ceph-mon[38981]: pgmap v45: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:17.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:17 smithi196 ceph-mon[31033]: pgmap v45: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:17.499 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:17 smithi118 systemd[1]: Stopping Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:00:17.833 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[72099]: ts=2024-03-20T08:00:17.500Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-20T08:00:17.833 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:17 smithi118 podman[78263]: 2024-03-20 08:00:17.52150979 +0000 UTC m=+0.131750647 container died 1b8c13084c71100e71ebea1fe24a6ead94de8f76336308e554ba7357f644d652 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:18.172 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 podman[78263]: 2024-03-20 08:00:18.032279631 +0000 UTC m=+0.642520530 container cleanup 1b8c13084c71100e71ebea1fe24a6ead94de8f76336308e554ba7357f644d652 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:18.172 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 bash[78263]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a 2024-03-20T08:00:18.582 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 podman[78282]: 2024-03-20 08:00:18.28425258 +0000 UTC m=+0.765679308 container remove 1b8c13084c71100e71ebea1fe24a6ead94de8f76336308e554ba7357f644d652 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:18.582 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 podman[78282]: 2024-03-20 08:00:18.368212519 +0000 UTC m=+0.849639178 volume remove afe1844dbda1d025dbe46a9433559bf55e6d75af5af7e191f52dc1b704d353f0 2024-03-20T08:00:18.892 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-20T08:00:18.893 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 systemd[1]: Stopped Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:00:18.893 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 systemd[1]: Starting Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:00:18.893 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 podman[78384]: 2024-03-20 08:00:18.802431049 +0000 UTC m=+0.110691993 volume create 839383f78a10d31de2a2c2ef160af5db0a47d82eac323d9e6bf6264db05c36ac 2024-03-20T08:00:18.893 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 podman[78384]: 2024-03-20 08:00:18.709262891 +0000 UTC m=+0.017523839 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-20T08:00:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-mon[27391]: pgmap v46: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:19.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-mon[38981]: pgmap v46: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:19.172 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:18 smithi118 podman[78384]: 2024-03-20 08:00:18.894609802 +0000 UTC m=+0.202870746 container create 39fe7da5bc35889e340ba127b7098e634a1086af63d3a83be8f268b4cd71babb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:19.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:19 smithi196 ceph-mon[31033]: pgmap v46: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:19.672 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 podman[78384]: 2024-03-20 08:00:19.197668208 +0000 UTC m=+0.505929162 container init 39fe7da5bc35889e340ba127b7098e634a1086af63d3a83be8f268b4cd71babb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:19.672 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 podman[78384]: 2024-03-20 08:00:19.20329947 +0000 UTC m=+0.511560418 container start 39fe7da5bc35889e340ba127b7098e634a1086af63d3a83be8f268b4cd71babb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:19.672 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:19.215Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-20T08:00:19.672 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:19.215Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-20T08:00:19.672 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:19.216Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.118 port=9094 2024-03-20T08:00:19.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:19.217Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-20T08:00:19.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:19.240Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-20T08:00:19.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:19.240Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-20T08:00:19.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:19.242Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-20T08:00:19.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:19.242Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-20T08:00:19.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 bash[78384]: 39fe7da5bc35889e340ba127b7098e634a1086af63d3a83be8f268b4cd71babb 2024-03-20T08:00:19.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:19 smithi118 systemd[1]: Started Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:00:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:20 smithi196 ceph-mon[31033]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:00:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:20 smithi196 ceph-mon[31033]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:00:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:20 smithi196 ceph-mon[31033]: pgmap v47: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[27391]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:00:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[27391]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:00:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[27391]: pgmap v47: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[38981]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:00:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[38981]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:00:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:20 smithi118 ceph-mon[38981]: pgmap v47: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:21.646 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:21.218Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00079836s 2024-03-20T08:00:21.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:00:21] "GET /metrics HTTP/1.1" 200 33645 "" "Prometheus/2.43.0" 2024-03-20T08:00:22.829 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 systemd[1]: Stopping Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:00:23.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-mon[31033]: pgmap v48: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:23.128 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.830Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-20T08:00:23.128 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.830Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-20T08:00:23.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.830Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-20T08:00:23.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.830Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-20T08:00:23.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.830Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-20T08:00:23.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.830Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-20T08:00:23.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.830Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-20T08:00:23.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.830Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-20T08:00:23.130 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.831Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-20T08:00:23.130 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.831Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-20T08:00:23.130 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.831Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-20T08:00:23.130 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[60541]: ts=2024-03-20T08:00:22.831Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-20T08:00:23.130 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:22 smithi196 podman[64077]: 2024-03-20 08:00:22.840996371 +0000 UTC m=+0.128470361 container died 265fb70d6aa0fbb553dedc967a48b1104f4555be7ed0607eadebc87ffc21dae7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:23.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 podman[64077]: 2024-03-20 08:00:23.253049981 +0000 UTC m=+0.540523970 container cleanup 265fb70d6aa0fbb553dedc967a48b1104f4555be7ed0607eadebc87ffc21dae7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:23.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 bash[64077]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a 2024-03-20T08:00:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:22 smithi118 ceph-mon[27391]: pgmap v48: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:22 smithi118 ceph-mon[38981]: pgmap v48: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:23.875 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 podman[64096]: 2024-03-20 08:00:23.43757527 +0000 UTC m=+0.599309862 container remove 265fb70d6aa0fbb553dedc967a48b1104f4555be7ed0607eadebc87ffc21dae7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:23.876 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-20T08:00:23.876 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 systemd[1]: Stopped Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:00:23.876 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service: Consumed 1.160s CPU time. 2024-03-20T08:00:23.876 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 systemd[1]: Starting Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:00:24.128 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 podman[64197]: 2024-03-20 08:00:23.781554521 +0000 UTC m=+0.019462303 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-20T08:00:24.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:23 smithi196 podman[64197]: 2024-03-20 08:00:23.8975917 +0000 UTC m=+0.135499464 container create d713b2506adb9b0900416a40006ead7bde111e8e12cb0f461edc076a98a699b6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:24.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 podman[64197]: 2024-03-20 08:00:24.123976535 +0000 UTC m=+0.361884314 container init d713b2506adb9b0900416a40006ead7bde111e8e12cb0f461edc076a98a699b6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:24.129 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 podman[64197]: 2024-03-20 08:00:24.129779345 +0000 UTC m=+0.367687110 container start d713b2506adb9b0900416a40006ead7bde111e8e12cb0f461edc076a98a699b6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:00:24.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.153Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-20T08:00:24.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.153Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-20T08:00:24.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.153Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi196 (none))" 2024-03-20T08:00:24.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.153Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.153Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.155Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.155Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.156Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.156Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.158Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.158Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=145.476µs 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.158Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.160Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.160Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.160Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.683µs wal_replay_duration=1.663075ms wbl_replay_duration=125ns total_replay_duration=1.849881ms 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.161Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-20T08:00:24.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.161Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-20T08:00:24.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.161Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-20T08:00:24.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.177Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.32179ms db_storage=731ns remote_storage=1.1µs web_handler=413ns query_engine=618ns scrape=432.32µs scrape_sd=94.652µs notify=9.338µs notify_sd=13.025µs rules=15.53586ms tracing=5.692µs 2024-03-20T08:00:24.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.177Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-20T08:00:24.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:00:24.177Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-20T08:00:24.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 bash[64197]: d713b2506adb9b0900416a40006ead7bde111e8e12cb0f461edc076a98a699b6 2024-03-20T08:00:24.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:00:24 smithi196 systemd[1]: Started Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:00:24.599 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STOPPING 2024-03-20T08:00:24.599 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:00:24.600 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STOPPED 2024-03-20T08:00:24.600 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STARTING 2024-03-20T08:00:24.600 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Serving on http://:::9283 2024-03-20T08:00:24.600 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STARTED 2024-03-20T08:00:24.600 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STOPPING 2024-03-20T08:00:24.600 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:00:24.600 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STOPPED 2024-03-20T08:00:24.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STARTING 2024-03-20T08:00:24.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Serving on http://:::9283 2024-03-20T08:00:24.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STARTED 2024-03-20T08:00:24.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STOPPING 2024-03-20T08:00:24.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:00:24.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STOPPED 2024-03-20T08:00:24.602 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STARTING 2024-03-20T08:00:24.602 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Serving on http://:::9283 2024-03-20T08:00:24.602 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:00:24] ENGINE Bus STARTED 2024-03-20T08:00:25.241 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: pgmap v49: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:25.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:00:25.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-03-20T08:00:25.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:00:25.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-03-20T08:00:25.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.243 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:00:25.243 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-03-20T08:00:25.243 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.243 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:00:25.243 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: pgmap v49: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:25.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:00:25.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-03-20T08:00:25.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:00:25.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-03-20T08:00:25.245 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.245 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:00:25.245 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-03-20T08:00:25.245 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.245 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: pgmap v49: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-03-20T08:00:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:00:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-03-20T08:00:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:00:26.345 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:00:26.345 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-03-20T08:00:26.345 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:00:26.345 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-03-20T08:00:26.345 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:00:26.346 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-03-20T08:00:26.346 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:00:26.346 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-03-20T08:00:26.346 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:00:26.346 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-03-20T08:00:26.346 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:00:26.346 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:25 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-03-20T08:00:26.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:25 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:00:26.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:25 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi118.front.sepia.ceph.com:9093"}]: dispatch 2024-03-20T08:00:26.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:25 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:00:26.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:25 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-03-20T08:00:26.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:25 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:00:26.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:25 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-03-20T08:00:27.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:26 smithi196 ceph-mon[31033]: pgmap v50: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:27.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:26 smithi118 ceph-mon[27391]: pgmap v50: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:27.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:26 smithi118 ceph-mon[38981]: pgmap v50: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:28 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:28 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:28 smithi196 ceph-mon[31033]: pgmap v51: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:28 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:28 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:28 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:00:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:28 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:00:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:28 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[27391]: pgmap v51: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:28.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[38981]: pgmap v51: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:00:28.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:28 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:00:29.672 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:00:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:00:29.220Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003521163s 2024-03-20T08:00:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:30 smithi196 ceph-mon[31033]: pgmap v52: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:30 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:30 smithi118 ceph-mon[27391]: pgmap v52: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:30 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:30 smithi118 ceph-mon[38981]: pgmap v52: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:30 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:31.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:00:31] "GET /metrics HTTP/1.1" 200 33645 "" "Prometheus/2.43.0" 2024-03-20T08:00:32.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:31 smithi196 ceph-mon[31033]: mgrmap e21: y(active, since 91s), standbys: x 2024-03-20T08:00:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:31 smithi118 ceph-mon[27391]: mgrmap e21: y(active, since 91s), standbys: x 2024-03-20T08:00:32.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:31 smithi118 ceph-mon[38981]: mgrmap e21: y(active, since 91s), standbys: x 2024-03-20T08:00:33.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:32 smithi196 ceph-mon[31033]: pgmap v53: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:33.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:32 smithi118 ceph-mon[27391]: pgmap v53: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:33.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:32 smithi118 ceph-mon[38981]: pgmap v53: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:34 smithi196 ceph-mon[31033]: pgmap v54: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:34 smithi118 ceph-mon[27391]: pgmap v54: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:35.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:34 smithi118 ceph-mon[38981]: pgmap v54: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:37.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:36 smithi196 ceph-mon[31033]: pgmap v55: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:37.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:36 smithi118 ceph-mon[27391]: pgmap v55: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:37.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:36 smithi118 ceph-mon[38981]: pgmap v55: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:39.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:39 smithi196 ceph-mon[31033]: pgmap v56: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:39.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:39 smithi118 ceph-mon[27391]: pgmap v56: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:39.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:39 smithi118 ceph-mon[38981]: pgmap v56: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:41.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:41 smithi196 ceph-mon[31033]: pgmap v57: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:41.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:41 smithi118 ceph-mon[38981]: pgmap v57: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:41.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:41 smithi118 ceph-mon[27391]: pgmap v57: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:41.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:00:41] "GET /metrics HTTP/1.1" 200 33645 "" "Prometheus/2.43.0" 2024-03-20T08:00:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:43 smithi196 ceph-mon[31033]: pgmap v58: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:43.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:43 smithi118 ceph-mon[27391]: pgmap v58: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:43.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:43 smithi118 ceph-mon[38981]: pgmap v58: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-20T08:00:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:45 smithi196 ceph-mon[31033]: pgmap v59: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:45 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:45 smithi118 ceph-mon[27391]: pgmap v59: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:45 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:45.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:45 smithi118 ceph-mon[38981]: pgmap v59: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:45.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:45 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:00:47.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:47 smithi196 ceph-mon[31033]: pgmap v60: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:47 smithi118 ceph-mon[27391]: pgmap v60: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:47.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:47 smithi118 ceph-mon[38981]: pgmap v60: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:49 smithi196 ceph-mon[31033]: pgmap v61: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:49.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:49 smithi118 ceph-mon[27391]: pgmap v61: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:49.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:49 smithi118 ceph-mon[38981]: pgmap v61: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:51 smithi196 ceph-mon[31033]: pgmap v62: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:51.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:51 smithi118 ceph-mon[27391]: pgmap v62: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:51.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:51 smithi118 ceph-mon[38981]: pgmap v62: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:51.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:00:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:00:51] "GET /metrics HTTP/1.1" 200 33645 "" "Prometheus/2.43.0" 2024-03-20T08:00:53.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:53 smithi196 ceph-mon[31033]: pgmap v63: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:53 smithi118 ceph-mon[27391]: pgmap v63: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:53 smithi118 ceph-mon[38981]: pgmap v63: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:55.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:55 smithi196 ceph-mon[31033]: pgmap v64: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:55 smithi118 ceph-mon[27391]: pgmap v64: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:55.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:55 smithi118 ceph-mon[38981]: pgmap v64: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:57.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:57 smithi196 ceph-mon[31033]: pgmap v65: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:57.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:57 smithi118 ceph-mon[27391]: pgmap v65: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:57.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:57 smithi118 ceph-mon[38981]: pgmap v65: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:59.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:00:59 smithi196 ceph-mon[31033]: pgmap v66: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:00:59 smithi118 ceph-mon[27391]: pgmap v66: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:00:59.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:00:59 smithi118 ceph-mon[38981]: pgmap v66: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:00.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:00 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:00 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:00.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:00 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:01.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:01 smithi196 ceph-mon[31033]: pgmap v67: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:01 smithi118 ceph-mon[27391]: pgmap v67: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:01.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:01 smithi118 ceph-mon[38981]: pgmap v67: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:01.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:01:01] "GET /metrics HTTP/1.1" 200 33644 "" "Prometheus/2.43.0" 2024-03-20T08:01:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:03 smithi196 ceph-mon[31033]: pgmap v68: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:03.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:03 smithi118 ceph-mon[27391]: pgmap v68: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:03.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:03 smithi118 ceph-mon[38981]: pgmap v68: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:04.467 INFO:tasks.workunit.client.0.smithi118.stderr:Note: switching to 'e7e80a6ac9849f0f23432d0d9832f096730308ae'. 2024-03-20T08:01:04.467 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:04.468 INFO:tasks.workunit.client.0.smithi118.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-20T08:01:04.468 INFO:tasks.workunit.client.0.smithi118.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-20T08:01:04.468 INFO:tasks.workunit.client.0.smithi118.stderr:state without impacting any branches by switching back to a branch. 2024-03-20T08:01:04.468 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:04.468 INFO:tasks.workunit.client.0.smithi118.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-20T08:01:04.468 INFO:tasks.workunit.client.0.smithi118.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-20T08:01:04.469 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:04.469 INFO:tasks.workunit.client.0.smithi118.stderr: git switch -c 2024-03-20T08:01:04.469 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:04.469 INFO:tasks.workunit.client.0.smithi118.stderr:Or undo this operation with: 2024-03-20T08:01:04.469 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:04.469 INFO:tasks.workunit.client.0.smithi118.stderr: git switch - 2024-03-20T08:01:04.470 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:04.470 INFO:tasks.workunit.client.0.smithi118.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-20T08:01:04.470 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:04.470 INFO:tasks.workunit.client.0.smithi118.stderr:HEAD is now at e7e80a6ac98 Merge branch 'wip-nitzan-lazy-omap-stats-test-regex-optimize' of https://github.com/NitzanMordhai/ceph into wip-yuri5-testing-2024-03-18-1144 2024-03-20T08:01:04.473 DEBUG:teuthology.orchestra.run.smithi118:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-20T08:01:04.537 INFO:tasks.workunit.client.0.smithi118.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-20T08:01:04.539 INFO:tasks.workunit.client.0.smithi118.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-20T08:01:04.539 INFO:tasks.workunit.client.0.smithi118.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-20T08:01:05.340 INFO:tasks.workunit.client.0.smithi118.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-20T08:01:05.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:05 smithi196 ceph-mon[31033]: pgmap v69: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:05.412 INFO:tasks.workunit.client.0.smithi118.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-20T08:01:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:05 smithi118 ceph-mon[27391]: pgmap v69: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:05.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:05 smithi118 ceph-mon[38981]: pgmap v69: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:05.453 INFO:tasks.workunit.client.0.smithi118.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-20T08:01:05.455 INFO:tasks.workunit.client.0.smithi118.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-20T08:01:05.455 INFO:tasks.workunit.client.0.smithi118.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-20T08:01:05.495 INFO:tasks.workunit.client.0.smithi118.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-20T08:01:05.498 DEBUG:teuthology.orchestra.run.smithi118:> set -ex 2024-03-20T08:01:05.498 DEBUG:teuthology.orchestra.run.smithi118:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-20T08:01:05.554 INFO:tasks.workunit:Running workunits matching test_telemetry_reef.sh on client.0... 2024-03-20T08:01:05.555 INFO:tasks.workunit:Running workunit test_telemetry_reef.sh... 2024-03-20T08:01:05.556 DEBUG:teuthology.orchestra.run.smithi118:workunit test test_telemetry_reef.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e7e80a6ac9849f0f23432d0d9832f096730308ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/test_telemetry_reef.sh 2024-03-20T08:01:05.635 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph config set mgr mgr/telemetry/channel_ident true 2024-03-20T08:01:06.041 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph config set mgr mgr/telemetry/organization ceph-qa 2024-03-20T08:01:06.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:05] ENGINE Bus STOPPING 2024-03-20T08:01:06.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:01:06.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:05] ENGINE Bus STOPPED 2024-03-20T08:01:06.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:05] ENGINE Bus STARTING 2024-03-20T08:01:06.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Serving on http://:::9283 2024-03-20T08:01:06.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STARTED 2024-03-20T08:01:06.470 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph config set mgr mgr/telemetry/description 'upgrade test cluster' 2024-03-20T08:01:06.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STOPPING 2024-03-20T08:01:06.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:01:06.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STOPPED 2024-03-20T08:01:06.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STARTING 2024-03-20T08:01:06.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Serving on http://:::9283 2024-03-20T08:01:06.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STARTED 2024-03-20T08:01:06.879 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph telemetry preview 2024-03-20T08:01:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/988579722' entity='client.admin' 2024-03-20T08:01:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[27391]: pgmap v70: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1544564307' entity='client.admin' 2024-03-20T08:01:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/988579722' entity='client.admin' 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[38981]: pgmap v70: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1544564307' entity='client.admin' 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STOPPING 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STOPPED 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STARTING 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Serving on http://:::9283 2024-03-20T08:01:07.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:06] ENGINE Bus STARTED 2024-03-20T08:01:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:06 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/988579722' entity='client.admin' 2024-03-20T08:01:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:06 smithi196 ceph-mon[31033]: pgmap v70: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:06 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:06 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1544564307' entity='client.admin' 2024-03-20T08:01:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:06 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' 2024-03-20T08:01:07.413 INFO:tasks.workunit.client.0.smithi118.stdout:{ 2024-03-20T08:01:07.413 INFO:tasks.workunit.client.0.smithi118.stdout: "balancer": { 2024-03-20T08:01:07.413 INFO:tasks.workunit.client.0.smithi118.stdout: "active": true, 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "mode": "" 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "channels": [ 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "basic", 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "crash", 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "device", 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "ident" 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "channels_available": [ 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "basic", 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "ident", 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "crash", 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "device", 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: "perf" 2024-03-20T08:01:07.414 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "collections_available": [ 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_base", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "device_base", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "crash_base", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "ident_base", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_perf", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_mds_metadata", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_usage", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_usage_by_class", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_rook_v01", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_memory_metrics", 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_options_bluestore" 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "collections_opted_in": [ 2024-03-20T08:01:07.415 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_base", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "device_base", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "crash_base", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "ident_base", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_perf", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_mds_metadata", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_usage", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_usage_by_class", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_rook_v01", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_memory_metrics", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_options_bluestore" 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "config": { 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "active_changed": [ 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_allow_insecure_global_id_reclaim", 2024-03-20T08:01:07.416 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_debug", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_mon_ticket_ttl", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_service_ticket_ttl", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "bdev_debug_aio", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "client_check_pool_perm", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "client_mount_gid", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "client_mount_uid", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "client_quota", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "container_image", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "daemonize", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_asserts_on_shutdown", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mgr", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mon", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_ms", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_osd", 2024-03-20T08:01:07.417 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_paxos", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "keyring", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_block_size", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_cache_size", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_compression", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_log", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_write_buffer_size", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_file", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_journald", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_stderr", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_allow_pool_delete", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_clock_drift_allowed", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_file_level", 2024-03-20T08:01:07.418 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_file", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_journald", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_stderr", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_data_avail_warn", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_election_default_strategy", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_host", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_max_pg_per_osd", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_mgr_mkfs_grace", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_prime_pg_temp", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_reporter_subtree_level", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_pg_warn_max_object_skew", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_bytes_per_osd", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_pgs_per_osd", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-03-20T08:01:07.419 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_legacy_crush_tunables", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_too_few_osds", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_bug", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_old_message", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "no_config_file", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_default_list", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_load_list", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_crush_chooseleaf_type", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_misdirected_ops", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_op_order", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_pg_log_writeout", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_shutdown", 2024-03-20T08:01:07.420 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_cached_snaps", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_missing_on_start", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_stray_on_activate", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_deep_scrub_update_digest_min_age", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_profile", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_lim", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_res", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_wgt", 2024-03-20T08:01:07.421 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target_autotune", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue_cut_off", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_erasure_code_profile", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_pg_autoscale_mode", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recover_clone_overlap", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recovery_max_chunk", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_load_threshold", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_max_interval", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_shutdown_pgref_assert", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "public_addr", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "public_addrv", 2024-03-20T08:01:07.422 INFO:tasks.workunit.client.0.smithi118.stdout: "public_network", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd_default_features", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd_qos_exclude_ops", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "setgroup", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "setuser" 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster_changed": [ 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_allow_insecure_global_id_reclaim", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_debug", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_mon_ticket_ttl", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_service_ticket_ttl", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "bdev_debug_aio", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "container_image", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_asserts_on_shutdown", 2024-03-20T08:01:07.423 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mgr", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mon", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_ms", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_osd", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_paxos", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_file", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_journald", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_stderr", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_join_fs", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/allow_ptrace", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/container_init", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/migration_current", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/use_repo_digest", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-03-20T08:01:07.424 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/ssl_server_port", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/orchestrator/orchestrator", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/channel_ident", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/description", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/organization", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_allow_pool_delete", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_clock_drift_allowed", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_file_level", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_file", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_data_avail_warn", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_election_default_strategy", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_max_pg_per_osd", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_mgr_mkfs_grace", 2024-03-20T08:01:07.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_prime_pg_temp", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_reporter_subtree_level", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_pg_warn_max_object_skew", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_bytes_per_osd", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_pgs_per_osd", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_legacy_crush_tunables", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_too_few_osds", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_bug", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_old_message", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_default_list", 2024-03-20T08:01:07.426 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_load_list", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_misdirected_ops", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_op_order", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_pg_log_writeout", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_shutdown", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_cached_snaps", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_missing_on_start", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_stray_on_activate", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_deep_scrub_update_digest_min_age", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_profile", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target_autotune", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue_cut_off", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_erasure_code_profile", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_pg_autoscale_mode", 2024-03-20T08:01:07.427 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recover_clone_overlap", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recovery_max_chunk", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_load_threshold", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_max_interval", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_shutdown_pgref_assert", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "public_network", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_cache_enabled", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_enable_ops_log", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_enable_usage_log" 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "contact": null, 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "crashes": [], 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "created": "2024-03-20T07:52:36.029987Z", 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "crush": { 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_algs": { 2024-03-20T08:01:07.428 INFO:tasks.workunit.client.0.smithi118.stdout: "straw2": 3 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_sizes": { 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "2": 1, 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "4": 2 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_types": { 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "1": 2, 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "11": 1 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "compat_weight_set": false, 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "device_classes": [ 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: 8 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_buckets": 6, 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_devices": 8, 2024-03-20T08:01:07.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_rules": 1, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_types": 12, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_weight_sets": 0, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "tunables": { 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "allowed_bucket_algs": 54, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_local_fallback_tries": 0, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_local_tries": 0, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_total_tries": 50, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_descend_once": 1, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_stable": 1, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_vary_r": 1, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v2_rules": 0, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v3_rules": 0, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v4_buckets": 1, 2024-03-20T08:01:07.430 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v5_rules": 0, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "legacy_tunables": 0, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "minimum_required_version": "jewel", 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "optimal_tunables": 1, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "profile": "jewel", 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables": 1, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables2": 1, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables3": 1, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables5": 1, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "straw_calc_version": 1 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "description": "upgrade test cluster", 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "fs": { 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "feature_flags": { 2024-03-20T08:01:07.431 INFO:tasks.workunit.client.0.smithi118.stdout: "enable_multiple": true, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "ever_enabled_multiple": true 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "filesystems": [ 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "approx_ctime": "2024-03", 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "balancer_enabled": false, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_caps": 0, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_dns": 10, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_inos": 13, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_subtrees": 2, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "ever_allowed_features": 0, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "explicitly_allowed_features": 0, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "files": 0, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "max_mds": 1, 2024-03-20T08:01:07.432 INFO:tasks.workunit.client.0.smithi118.stdout: "num_data_pools": 1, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num_in": 1, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num_mds": 1, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": 0, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num_standby_replay": 0, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num_up": 1, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "snaps": 0, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "standby_count_wanted": 0 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num_standby_mds": 1, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "total_num_mds": 2 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "hosts": { 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num": 2, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mds": 2, 2024-03-20T08:01:07.433 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mgr": 2, 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mon": 2, 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_osd": 2 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "leaderboard": false, 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "leaderboard_description": null, 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "license": "sharing-1-0", 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "metadata": { 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 2 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.434 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 2 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 2 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 2 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 2 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.435 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 2 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 3 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:07.436 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 3 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 3 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 3 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 3 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 3 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.437 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 8 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 8 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 8 2024-03-20T08:01:07.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 8 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 8 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 8 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_objectstore": { 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": 8 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "rotational": { 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: "0": 8 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.439 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "features": { 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "optional": [], 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "persistent": [ 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "kraken", 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "luminous", 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "mimic", 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap-prune", 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "nautilus", 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "octopus", 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "pacific", 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "elector-pinging", 2024-03-20T08:01:07.440 INFO:tasks.workunit.client.0.smithi118.stdout: "quincy", 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "reef" 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "ipv4_addr_mons": 6, 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "ipv6_addr_mons": 0, 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "min_mon_release": 18, 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "v1_addr_mons": 3, 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "v2_addr_mons": 3 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "organization": "ceph-qa", 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster_network": false, 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 8, 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "require_min_compat_client": "luminous", 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: "require_osd_release": "reef" 2024-03-20T08:01:07.441 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "pools": [ 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr" 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 1, 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 1, 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 1, 2024-03-20T08:01:07.442 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 1781760, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 1781760, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 2, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 2.4430023586319294e-06, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 212, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 313344, 2024-03-20T08:01:07.443 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 590368, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 590368, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 1771104, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 179, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 2473984 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:07.444 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 32, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 32, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 2, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 98304, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 98304, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:07.445 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 22, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 1.3478666005539708e-07, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 3428, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 3428, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 10284, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 51, 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 20480 2024-03-20T08:01:07.446 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 64, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 64, 2024-03-20T08:01:07.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 3, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 0.0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:07.448 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 0, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 0, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 0, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 0, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 0 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd" 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.449 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 8, 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 8, 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 4, 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:07.450 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 0.0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 0, 2024-03-20T08:01:07.451 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 0, 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 0, 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 0 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd": { 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "mirroring_by_pool": [ 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: false 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: "num_images_by_pool": [ 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:07.452 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pools": 1 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "report_id": "f7bee648-d925-4013-8791-48f92fa7019f", 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "report_timestamp": "2024-03-20T08:01:07.405783", 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "report_version": 1, 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "rook": { 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster": { 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "external-mode": null, 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "allow-multiple-per-node": null, 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "count": null, 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "max-id": null, 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "pvc": { 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: "enabled": null 2024-03-20T08:01:07.453 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "stretch": { 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "enabled": null 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "network": { 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "provider": null 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "storage": { 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "device-set": { 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "non-portable": null, 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "portable": null, 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: "total": null 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.454 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "csi": { 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "kubernetes": { 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "node": { 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "kubernetes-total": null, 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "with-ceph-daemons": null, 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-cephfs-plugin": null, 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-nfs-plugin": null, 2024-03-20T08:01:07.455 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-rbd-plugin": null 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "usage": { 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "storage-class": { 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket": null, 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs": null, 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "nfs": null, 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd": null, 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "total": null 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.456 INFO:tasks.workunit.client.0.smithi118.stdout: "services": {}, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "usage": { 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 105, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "pools": 4, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_by_class": { 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "ssd": { 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "total_avail_bytes": 767731326976, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "total_bytes": 767960285184, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_bytes": 228958208, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_raw_bytes": 228958208, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_raw_ratio": 0.00029813809669576585 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "total_avail_bytes": 767731326976, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "total_bytes": 767960285184, 2024-03-20T08:01:07.457 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_bytes": 228958208 2024-03-20T08:01:07.458 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.458 INFO:tasks.workunit.client.0.smithi118.stdout:} 2024-03-20T08:01:07.458 INFO:tasks.workunit.client.0.smithi118.stdout:Device report is generated separately. To see it run 'ceph telemetry preview-device'. 2024-03-20T08:01:07.458 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph telemetry preview-device 2024-03-20T08:01:07.967 INFO:tasks.workunit.client.0.smithi118.stdout:{ 2024-03-20T08:01:07.967 INFO:tasks.workunit.client.0.smithi118.stdout: "01f6c8a0-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:07.967 INFO:tasks.workunit.client.0.smithi118.stdout: "INTEL_SSDPED1D480GA_01f76148-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:07.967 INFO:tasks.workunit.client.0.smithi118.stdout: "20240320-075605": { 2024-03-20T08:01:07.967 INFO:tasks.workunit.client.0.smithi118.stdout: "device": { 2024-03-20T08:01:07.967 INFO:tasks.workunit.client.0.smithi118.stdout: "info_name": "/dev/nvme0n1", 2024-03-20T08:01:07.967 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "/dev/nvme0n1", 2024-03-20T08:01:07.967 INFO:tasks.workunit.client.0.smithi118.stdout: "protocol": "NVMe", 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "nvme" 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "firmware_version": "E2010325", 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "host_id": "01f6c8a0-e690-11ee-b4ce-0cc47a8fd725", 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "json_format_version": [ 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "local_time": { 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "asctime": "Wed Mar 20 07:56:05 2024 UTC", 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "time_t": 1710921365 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "logical_block_size": 512, 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "model_name": "INTEL SSDPED1D480GA", 2024-03-20T08:01:07.968 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_controller_id": 0, 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_namespaces": [ 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "capacity": { 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "formatted_lba_size": 512, 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 1, 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "size": { 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "utilization": { 2024-03-20T08:01:07.969 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_number_of_namespaces": 1, 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_pci_vendor": { 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 32902, 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "subsystem_id": 32902 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_add_log": { 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "Device stats": { 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_fw_downgrades": { 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.970 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "crc_error_count": { 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "end_to_end_error_detection_count": { 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "erase_fail_count": { 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "host_bytes_written": { 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.971 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 6488230 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "host_ctx_wear_used": { 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "media_bytes_read": { 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "nand_bytes_written": { 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_stat_indicator": { 2024-03-20T08:01:07.972 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "pll_lock_loss_count": { 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "program_fail_count": { 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "re_alloc_sectr_cnt": { 2024-03-20T08:01:07.973 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "retry_buffer_overflow_count": { 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "soft_ecc_err_rate": { 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "thermal_throttle_status": { 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:07.974 INFO:tasks.workunit.client.0.smithi118.stdout: "cnt": 0, 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "pct": 0 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_host_reads": { 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 16 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_media_wear": { 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_timer": { 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1243878 2024-03-20T08:01:07.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "unexp_power_loss": { 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "wear_leveling": { 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "avg": 0, 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "max": 0, 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel Smart log": "nvme0n1" 2024-03-20T08:01:07.976 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_log": { 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare": 100, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare_threshold": 0, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "controller_busy_time": 2533, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "critical_warning": 0, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_read": 94253558, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_written": 425212666, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "host_reads": 1503125893, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "host_writes": 9374739071, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "media_errors": 0, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "num_err_log_entries": 0, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "percentage_used": 0, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycles": 16440, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_hours": 27966, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": 38, 2024-03-20T08:01:07.977 INFO:tasks.workunit.client.0.smithi118.stdout: "unsafe_shutdowns": 5682 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_vendor": "intel", 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycle_count": 16440, 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_time": { 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "hours": 27966 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "serial_number": "deleted", 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "smart_status": { 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme": { 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "passed": true 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.978 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl": { 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "argv": [ 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "-x", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "--json=o", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "/dev/nvme0n1" 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "build_info": "(local build)", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "exit_status": 0, 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "output": [ 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.14.0-430.el9.x86_64] (local build)", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF INFORMATION SECTION ===", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "Model Number: INTEL SSDPED1D480GA", 2024-03-20T08:01:07.979 INFO:tasks.workunit.client.0.smithi118.stdout: "Serial Number: deleted", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Version: E2010325", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller ID: 0", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Number of Namespaces: 1", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Size/Capacity: 480,103,981,056 [480 GB]", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Local Time is: Wed Mar 20 07:56:05 2024 UTC", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Updates (0x02): 1 Slot", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional Admin Commands (0x0007): Security Format Frmw_DL", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported Power States", 2024-03-20T08:01:07.980 INFO:tasks.workunit.client.0.smithi118.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 18.00W - - 0 0 0 0 0 0", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 512 0 2", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF SMART DATA SECTION ===", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "Critical Warning: 0x00", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "Temperature: 38 Celsius", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare: 100%", 2024-03-20T08:01:07.981 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare Threshold: 0%", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Percentage Used: 0%", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Read: 94,253,558 [48.2 TB]", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Written: 425,212,666 [217 TB]", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Read Commands: 1,503,125,893", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Write Commands: 9,374,739,071", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller Busy Time: 2,533", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Power Cycles: 16,440", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Power On Hours: 27,966", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Unsafe Shutdowns: 5,682", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Media and Data Integrity Errors: 0", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information Log Entries: 0", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "No Errors Logged", 2024-03-20T08:01:07.982 INFO:tasks.workunit.client.0.smithi118.stdout: "" 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: "platform_info": "x86_64-linux-5.14.0-430.el9.x86_64", 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: "svn_revision": "5022", 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: "version": [ 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": { 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: "current": 38 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: "user_capacity": { 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.983 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "01f8bd04-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "INTEL_SSDPEDMD400G4_01fa01b4-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "20240320-075607": { 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "device": { 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "info_name": "/dev/nvme0n1", 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "/dev/nvme0n1", 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "protocol": "NVMe", 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "nvme" 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "firmware_version": "8DV101H0", 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "host_id": "01f8bd04-e690-11ee-b4ce-0cc47a8fd725", 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: "json_format_version": [ 2024-03-20T08:01:07.984 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "local_time": { 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "asctime": "Wed Mar 20 07:56:07 2024 UTC", 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "time_t": 1710921367 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "logical_block_size": 512, 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_controller_id": 0, 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_namespaces": [ 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "capacity": { 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:07.985 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "formatted_lba_size": 512, 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 1, 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "size": { 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "utilization": { 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_number_of_namespaces": 1, 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_pci_vendor": { 2024-03-20T08:01:07.986 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 32902, 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "subsystem_id": 32902 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_add_log": { 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "Device stats": { 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_fw_downgrades": { 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "crc_error_count": { 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "end_to_end_error_detection_count": { 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.987 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "erase_fail_count": { 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "host_bytes_written": { 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 24021711 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "host_ctx_wear_used": { 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "media_bytes_read": { 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.988 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "nand_bytes_written": { 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 43196693 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_stat_indicator": { 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "pll_lock_loss_count": { 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "program_fail_count": { 2024-03-20T08:01:07.989 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "re_alloc_sectr_cnt": { 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "retry_buffer_overflow_count": { 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "soft_ecc_err_rate": { 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.990 INFO:tasks.workunit.client.0.smithi118.stdout: "thermal_throttle_status": { 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "cnt": 0, 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "pct": 0 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_host_reads": { 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 65535 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_media_wear": { 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 64 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_timer": { 2024-03-20T08:01:07.991 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 65535 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "unexp_power_loss": { 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "wear_leveling": { 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 91, 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "avg": 2982, 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "max": 2997, 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 2966 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.992 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel Smart log": "nvme0n1" 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_log": { 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare": 100, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare_threshold": 10, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "controller_busy_time": 7, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "critical_warning": 0, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_read": 334126040, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_written": 1574286880, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "host_reads": 5210102632, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "host_writes": 26666187474, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "media_errors": 0, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_err_log_entries": 0, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "percentage_used": 9, 2024-03-20T08:01:07.993 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycles": 54960, 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_hours": 52936, 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": 27, 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "unsafe_shutdowns": 284 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_vendor": "intel", 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycle_count": 54960, 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_time": { 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "hours": 52936 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "serial_number": "deleted", 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "smart_status": { 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme": { 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:07.994 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "passed": true 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl": { 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "argv": [ 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl", 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "-x", 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "--json=o", 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "/dev/nvme0n1" 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "build_info": "(local build)", 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "exit_status": 0, 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "output": [ 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.14.0-430.el9.x86_64] (local build)", 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-03-20T08:01:07.995 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF INFORMATION SECTION ===", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Serial Number: deleted", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Version: 8DV101H0", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller ID: 0", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Number of Namespaces: 1", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Local Time is: Wed Mar 20 07:56:07 2024 UTC", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Updates (0x02): 1 Slot", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-03-20T08:01:07.996 INFO:tasks.workunit.client.0.smithi118.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported Power States", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 512 0 2", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: " 1 - 512 8 2", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: " 2 - 512 16 2", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: " 3 - 4096 0 0", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: " 4 - 4096 8 0", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: " 5 - 4096 64 0", 2024-03-20T08:01:07.997 INFO:tasks.workunit.client.0.smithi118.stdout: " 6 - 4096 128 0", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF SMART DATA SECTION ===", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Critical Warning: 0x00", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Temperature: 27 Celsius", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare: 100%", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare Threshold: 10%", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Percentage Used: 9%", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Read: 334,126,040 [171 TB]", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Written: 1,574,286,880 [806 TB]", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Read Commands: 5,210,102,632", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Write Commands: 26,666,187,474", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller Busy Time: 7", 2024-03-20T08:01:07.998 INFO:tasks.workunit.client.0.smithi118.stdout: "Power Cycles: 54,960", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "Power On Hours: 52,936", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "Unsafe Shutdowns: 284", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "Media and Data Integrity Errors: 0", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information Log Entries: 0", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "No Errors Logged", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "" 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "platform_info": "x86_64-linux-5.14.0-430.el9.x86_64", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "svn_revision": "5022", 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: "version": [ 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:07.999 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": { 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: "current": 27 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: "user_capacity": { 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.000 INFO:tasks.workunit.client.0.smithi118.stdout:} 2024-03-20T08:01:08.001 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph telemetry preview-all 2024-03-20T08:01:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:07 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:07 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:07 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:07 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:07 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:07 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:08.483 INFO:tasks.workunit.client.0.smithi118.stdout:{ 2024-03-20T08:01:08.483 INFO:tasks.workunit.client.0.smithi118.stdout: "device_report": { 2024-03-20T08:01:08.483 INFO:tasks.workunit.client.0.smithi118.stdout: "01f6c8a0-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:08.483 INFO:tasks.workunit.client.0.smithi118.stdout: "INTEL_SSDPED1D480GA_01f76148-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "20240320-075605": { 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "device": { 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "info_name": "/dev/nvme0n1", 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "/dev/nvme0n1", 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "protocol": "NVMe", 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "nvme" 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "firmware_version": "E2010325", 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "host_id": "01f6c8a0-e690-11ee-b4ce-0cc47a8fd725", 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "json_format_version": [ 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.484 INFO:tasks.workunit.client.0.smithi118.stdout: "local_time": { 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "asctime": "Wed Mar 20 07:56:05 2024 UTC", 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "time_t": 1710921365 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "logical_block_size": 512, 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "model_name": "INTEL SSDPED1D480GA", 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_controller_id": 0, 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_namespaces": [ 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "capacity": { 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "formatted_lba_size": 512, 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 1, 2024-03-20T08:01:08.485 INFO:tasks.workunit.client.0.smithi118.stdout: "size": { 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "utilization": { 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_number_of_namespaces": 1, 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_pci_vendor": { 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 32902, 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "subsystem_id": 32902 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.486 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_add_log": { 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "Device stats": { 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_fw_downgrades": { 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "crc_error_count": { 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "end_to_end_error_detection_count": { 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "erase_fail_count": { 2024-03-20T08:01:08.487 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "host_bytes_written": { 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 6488230 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "host_ctx_wear_used": { 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "media_bytes_read": { 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "nand_bytes_written": { 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.488 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_stat_indicator": { 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "pll_lock_loss_count": { 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "program_fail_count": { 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "re_alloc_sectr_cnt": { 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.489 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "retry_buffer_overflow_count": { 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "soft_ecc_err_rate": { 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "thermal_throttle_status": { 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "cnt": 0, 2024-03-20T08:01:08.490 INFO:tasks.workunit.client.0.smithi118.stdout: "pct": 0 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_host_reads": { 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 16 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_media_wear": { 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_timer": { 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1243878 2024-03-20T08:01:08.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "unexp_power_loss": { 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "wear_leveling": { 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "avg": 0, 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "max": 0, 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.492 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel Smart log": "nvme0n1" 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_log": { 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare": 100, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare_threshold": 0, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "controller_busy_time": 2533, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "critical_warning": 0, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_read": 94253558, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_written": 425212666, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "host_reads": 1503125893, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "host_writes": 9374739071, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "media_errors": 0, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_err_log_entries": 0, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "percentage_used": 0, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycles": 16440, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_hours": 27966, 2024-03-20T08:01:08.493 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": 38, 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "unsafe_shutdowns": 5682 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_vendor": "intel", 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycle_count": 16440, 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_time": { 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "hours": 27966 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "serial_number": "deleted", 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "smart_status": { 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme": { 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "passed": true 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.494 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl": { 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "argv": [ 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "-x", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "--json=o", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "/dev/nvme0n1" 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "build_info": "(local build)", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "exit_status": 0, 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "output": [ 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.14.0-430.el9.x86_64] (local build)", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF INFORMATION SECTION ===", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "Model Number: INTEL SSDPED1D480GA", 2024-03-20T08:01:08.495 INFO:tasks.workunit.client.0.smithi118.stdout: "Serial Number: deleted", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Version: E2010325", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller ID: 0", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Number of Namespaces: 1", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Size/Capacity: 480,103,981,056 [480 GB]", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Local Time is: Wed Mar 20 07:56:05 2024 UTC", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Updates (0x02): 1 Slot", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional Admin Commands (0x0007): Security Format Frmw_DL", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported Power States", 2024-03-20T08:01:08.496 INFO:tasks.workunit.client.0.smithi118.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 18.00W - - 0 0 0 0 0 0", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 512 0 2", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF SMART DATA SECTION ===", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "Critical Warning: 0x00", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "Temperature: 38 Celsius", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare: 100%", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare Threshold: 0%", 2024-03-20T08:01:08.497 INFO:tasks.workunit.client.0.smithi118.stdout: "Percentage Used: 0%", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Read: 94,253,558 [48.2 TB]", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Written: 425,212,666 [217 TB]", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Read Commands: 1,503,125,893", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Write Commands: 9,374,739,071", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller Busy Time: 2,533", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Power Cycles: 16,440", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Power On Hours: 27,966", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Unsafe Shutdowns: 5,682", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Media and Data Integrity Errors: 0", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information Log Entries: 0", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "No Errors Logged", 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: "" 2024-03-20T08:01:08.498 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: "platform_info": "x86_64-linux-5.14.0-430.el9.x86_64", 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: "svn_revision": "5022", 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: "version": [ 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": { 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: "current": 38 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: "user_capacity": { 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:08.499 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "01f8bd04-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "INTEL_SSDPEDMD400G4_01fa01b4-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "20240320-075607": { 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "device": { 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "info_name": "/dev/nvme0n1", 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "/dev/nvme0n1", 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "protocol": "NVMe", 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "nvme" 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "firmware_version": "8DV101H0", 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "host_id": "01f8bd04-e690-11ee-b4ce-0cc47a8fd725", 2024-03-20T08:01:08.500 INFO:tasks.workunit.client.0.smithi118.stdout: "json_format_version": [ 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "local_time": { 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "asctime": "Wed Mar 20 07:56:07 2024 UTC", 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "time_t": 1710921367 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "logical_block_size": 512, 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_controller_id": 0, 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_namespaces": [ 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:08.501 INFO:tasks.workunit.client.0.smithi118.stdout: "capacity": { 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "formatted_lba_size": 512, 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 1, 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "size": { 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "utilization": { 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:08.502 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_number_of_namespaces": 1, 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_pci_vendor": { 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 32902, 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "subsystem_id": 32902 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_add_log": { 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "Device stats": { 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_fw_downgrades": { 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "crc_error_count": { 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.503 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "end_to_end_error_detection_count": { 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "erase_fail_count": { 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "host_bytes_written": { 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 24021711 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "host_ctx_wear_used": { 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.504 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "media_bytes_read": { 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "nand_bytes_written": { 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 43196693 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_stat_indicator": { 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "pll_lock_loss_count": { 2024-03-20T08:01:08.505 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "program_fail_count": { 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "re_alloc_sectr_cnt": { 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "retry_buffer_overflow_count": { 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.506 INFO:tasks.workunit.client.0.smithi118.stdout: "soft_ecc_err_rate": { 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "thermal_throttle_status": { 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "cnt": 0, 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "pct": 0 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_host_reads": { 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 65535 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.507 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_media_wear": { 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 64 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_timer": { 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 65535 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "unexp_power_loss": { 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "wear_leveling": { 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 91, 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:08.508 INFO:tasks.workunit.client.0.smithi118.stdout: "avg": 2982, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "max": 2997, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 2966 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel Smart log": "nvme0n1" 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_log": { 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare": 100, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare_threshold": 10, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "controller_busy_time": 7, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "critical_warning": 0, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_read": 334126040, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_written": 1574286880, 2024-03-20T08:01:08.509 INFO:tasks.workunit.client.0.smithi118.stdout: "host_reads": 5210102632, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "host_writes": 26666187474, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "media_errors": 0, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_err_log_entries": 0, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "percentage_used": 9, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycles": 54960, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_hours": 52936, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": 27, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "unsafe_shutdowns": 284 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_vendor": "intel", 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycle_count": 54960, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_time": { 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "hours": 52936 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.510 INFO:tasks.workunit.client.0.smithi118.stdout: "serial_number": "deleted", 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "smart_status": { 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme": { 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "passed": true 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl": { 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "argv": [ 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl", 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "-x", 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "--json=o", 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "/dev/nvme0n1" 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "build_info": "(local build)", 2024-03-20T08:01:08.511 INFO:tasks.workunit.client.0.smithi118.stdout: "exit_status": 0, 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "output": [ 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.14.0-430.el9.x86_64] (local build)", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF INFORMATION SECTION ===", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Serial Number: deleted", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Version: 8DV101H0", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller ID: 0", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Number of Namespaces: 1", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-03-20T08:01:08.512 INFO:tasks.workunit.client.0.smithi118.stdout: "Local Time is: Wed Mar 20 07:56:07 2024 UTC", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Updates (0x02): 1 Slot", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported Power States", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 512 0 2", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: " 1 - 512 8 2", 2024-03-20T08:01:08.513 INFO:tasks.workunit.client.0.smithi118.stdout: " 2 - 512 16 2", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: " 3 - 4096 0 0", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: " 4 - 4096 8 0", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: " 5 - 4096 64 0", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: " 6 - 4096 128 0", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF SMART DATA SECTION ===", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "Critical Warning: 0x00", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "Temperature: 27 Celsius", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare: 100%", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare Threshold: 10%", 2024-03-20T08:01:08.514 INFO:tasks.workunit.client.0.smithi118.stdout: "Percentage Used: 9%", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Read: 334,126,040 [171 TB]", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Written: 1,574,286,880 [806 TB]", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Read Commands: 5,210,102,632", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Write Commands: 26,666,187,474", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller Busy Time: 7", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Power Cycles: 54,960", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Power On Hours: 52,936", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Unsafe Shutdowns: 284", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Media and Data Integrity Errors: 0", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information Log Entries: 0", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "No Errors Logged", 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "" 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.515 INFO:tasks.workunit.client.0.smithi118.stdout: "platform_info": "x86_64-linux-5.14.0-430.el9.x86_64", 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: "svn_revision": "5022", 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: "version": [ 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": { 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: "current": 27 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: "user_capacity": { 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.516 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "report": { 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "balancer": { 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "active": true, 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "mode": "" 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "channels": [ 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "basic", 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "crash", 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "device", 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "ident" 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "channels_available": [ 2024-03-20T08:01:08.517 INFO:tasks.workunit.client.0.smithi118.stdout: "basic", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "ident", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "crash", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "device", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "perf" 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "collections_available": [ 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_base", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "device_base", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "crash_base", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "ident_base", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_perf", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_mds_metadata", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_usage", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_usage_by_class", 2024-03-20T08:01:08.518 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_rook_v01", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_memory_metrics", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_options_bluestore" 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "collections_opted_in": [ 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_base", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "device_base", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "crash_base", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "ident_base", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_perf", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_mds_metadata", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_usage", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_usage_by_class", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_rook_v01", 2024-03-20T08:01:08.519 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_memory_metrics", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_options_bluestore" 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "config": { 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "active_changed": [ 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_allow_insecure_global_id_reclaim", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_debug", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_mon_ticket_ttl", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_service_ticket_ttl", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "bdev_debug_aio", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "client_check_pool_perm", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "client_mount_gid", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "client_mount_uid", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "client_quota", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "container_image", 2024-03-20T08:01:08.520 INFO:tasks.workunit.client.0.smithi118.stdout: "daemonize", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_asserts_on_shutdown", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mgr", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mon", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_ms", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_osd", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_paxos", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "keyring", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_block_size", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_cache_size", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_compression", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_log", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_write_buffer_size", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_file", 2024-03-20T08:01:08.521 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_journald", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_stderr", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_allow_pool_delete", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_clock_drift_allowed", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_file_level", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_file", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_journald", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_stderr", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_data_avail_warn", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_election_default_strategy", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_host", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_max_pg_per_osd", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_mgr_mkfs_grace", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_prime_pg_temp", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_reporter_subtree_level", 2024-03-20T08:01:08.522 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_pg_warn_max_object_skew", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_bytes_per_osd", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_pgs_per_osd", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_legacy_crush_tunables", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_too_few_osds", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_bug", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_old_message", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "no_config_file", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_default_list", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_load_list", 2024-03-20T08:01:08.523 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_crush_chooseleaf_type", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_misdirected_ops", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_op_order", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_pg_log_writeout", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_shutdown", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_cached_snaps", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_missing_on_start", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_stray_on_activate", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_deep_scrub_update_digest_min_age", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_profile", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-03-20T08:01:08.524 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_lim", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_res", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_wgt", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target_autotune", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue_cut_off", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_erasure_code_profile", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_pg_autoscale_mode", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recover_clone_overlap", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recovery_max_chunk", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_load_threshold", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_max_interval", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_shutdown_pgref_assert", 2024-03-20T08:01:08.525 INFO:tasks.workunit.client.0.smithi118.stdout: "public_addr", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "public_addrv", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "public_network", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd_default_features", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd_qos_exclude_ops", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "setgroup", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "setuser" 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster_changed": [ 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_allow_insecure_global_id_reclaim", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_debug", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_mon_ticket_ttl", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_service_ticket_ttl", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "bdev_debug_aio", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "container_image", 2024-03-20T08:01:08.526 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_asserts_on_shutdown", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mgr", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mon", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_ms", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_osd", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_paxos", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_file", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_journald", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_stderr", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_join_fs", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/allow_ptrace", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/container_init", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/migration_current", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/use_repo_digest", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-03-20T08:01:08.527 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/ssl_server_port", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/orchestrator/orchestrator", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/channel_ident", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/description", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/organization", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_allow_pool_delete", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_clock_drift_allowed", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_file_level", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_file", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_data_avail_warn", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_election_default_strategy", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_max_pg_per_osd", 2024-03-20T08:01:08.528 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_mgr_mkfs_grace", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_prime_pg_temp", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_reporter_subtree_level", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_pg_warn_max_object_skew", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_bytes_per_osd", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_pgs_per_osd", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_legacy_crush_tunables", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_too_few_osds", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_bug", 2024-03-20T08:01:08.529 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_old_message", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_default_list", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_load_list", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_misdirected_ops", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_op_order", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_pg_log_writeout", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_shutdown", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_cached_snaps", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_missing_on_start", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_stray_on_activate", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_deep_scrub_update_digest_min_age", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_profile", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target_autotune", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue", 2024-03-20T08:01:08.530 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue_cut_off", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_erasure_code_profile", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_pg_autoscale_mode", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recover_clone_overlap", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recovery_max_chunk", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_load_threshold", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_max_interval", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_shutdown_pgref_assert", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "public_network", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_cache_enabled", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_enable_ops_log", 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_enable_usage_log" 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "contact": null, 2024-03-20T08:01:08.531 INFO:tasks.workunit.client.0.smithi118.stdout: "crashes": [], 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "created": "2024-03-20T07:52:36.029987Z", 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "crush": { 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_algs": { 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "straw2": 3 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_sizes": { 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "2": 1, 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "4": 2 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_types": { 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "1": 2, 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "11": 1 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "compat_weight_set": false, 2024-03-20T08:01:08.532 INFO:tasks.workunit.client.0.smithi118.stdout: "device_classes": [ 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: 8 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_buckets": 6, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_devices": 8, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_rules": 1, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_types": 12, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_weight_sets": 0, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "tunables": { 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "allowed_bucket_algs": 54, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_local_fallback_tries": 0, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_local_tries": 0, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_total_tries": 50, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_descend_once": 1, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_stable": 1, 2024-03-20T08:01:08.533 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_vary_r": 1, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v2_rules": 0, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v3_rules": 0, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v4_buckets": 1, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v5_rules": 0, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "legacy_tunables": 0, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "minimum_required_version": "jewel", 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "optimal_tunables": 1, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "profile": "jewel", 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables": 1, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables2": 1, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables3": 1, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables5": 1, 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: "straw_calc_version": 1 2024-03-20T08:01:08.534 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "description": "upgrade test cluster", 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "fs": { 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "feature_flags": { 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "enable_multiple": true, 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "ever_enabled_multiple": true 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "filesystems": [ 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "approx_ctime": "2024-03", 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "balancer_enabled": false, 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_caps": 0, 2024-03-20T08:01:08.535 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_dns": 10, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_inos": 13, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_subtrees": 2, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "ever_allowed_features": 0, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "explicitly_allowed_features": 0, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "files": 0, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "max_mds": 1, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_data_pools": 1, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_in": 1, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_mds": 1, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": 0, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_standby_replay": 0, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_up": 1, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "snaps": 0, 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: "standby_count_wanted": 0 2024-03-20T08:01:08.536 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_standby_mds": 1, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "total_num_mds": 2 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "hosts": { 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num": 2, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mds": 2, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mgr": 2, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mon": 2, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_osd": 2 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "leaderboard": false, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "leaderboard_description": null, 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "license": "sharing-1-0", 2024-03-20T08:01:08.537 INFO:tasks.workunit.client.0.smithi118.stdout: "metadata": { 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 2 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 2 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:08.538 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 2 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 2 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 2 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 2 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 3 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.539 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 3 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 3 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 3 2024-03-20T08:01:08.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 3 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 3 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 8 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.541 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 8 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 8 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 8 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 8 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.542 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 8 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_objectstore": { 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": 8 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "rotational": { 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "0": 8 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "features": { 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "optional": [], 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "persistent": [ 2024-03-20T08:01:08.543 INFO:tasks.workunit.client.0.smithi118.stdout: "kraken", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "luminous", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "mimic", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap-prune", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "nautilus", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "octopus", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "pacific", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "elector-pinging", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "quincy", 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "reef" 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "ipv4_addr_mons": 6, 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "ipv6_addr_mons": 0, 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "min_mon_release": 18, 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "v1_addr_mons": 3, 2024-03-20T08:01:08.544 INFO:tasks.workunit.client.0.smithi118.stdout: "v2_addr_mons": 3 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "organization": "ceph-qa", 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster_network": false, 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 8, 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "require_min_compat_client": "luminous", 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "require_osd_release": "reef" 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "pools": [ 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr" 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:08.545 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 1, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 1, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 1, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 1781760, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 1781760, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:08.546 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 2, 2024-03-20T08:01:08.549 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:08.549 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 2.4430023586319294e-06, 2024-03-20T08:01:08.549 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 212, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 313344, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 590368, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 590368, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 1771104, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 179, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 2473984 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.550 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 32, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 32, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 2, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 98304, 2024-03-20T08:01:08.551 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 98304, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 22, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 1.3478666005539708e-07, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 3428, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 3428, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:08.552 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 10284, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 51, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 20480 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:08.553 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 64, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 64, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 3, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 0, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 0, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 0, 2024-03-20T08:01:08.554 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 0.0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 0 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:08.555 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd" 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 8, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 8, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 4, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:08.556 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 0.0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 0, 2024-03-20T08:01:08.557 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 0, 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 0, 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 0, 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 0 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd": { 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "mirroring_by_pool": [ 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: false 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_images_by_pool": [ 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pools": 1 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "report_id": "f7bee648-d925-4013-8791-48f92fa7019f", 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "report_timestamp": "2024-03-20T08:01:08.466764", 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "report_version": 1, 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "rook": { 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster": { 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "external-mode": null, 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "allow-multiple-per-node": null, 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "count": null, 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "max-id": null, 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "pvc": { 2024-03-20T08:01:08.559 INFO:tasks.workunit.client.0.smithi118.stdout: "enabled": null 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "stretch": { 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "enabled": null 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "network": { 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "provider": null 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "storage": { 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "device-set": { 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "non-portable": null, 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "portable": null, 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: "total": null 2024-03-20T08:01:08.560 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "csi": { 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "kubernetes": { 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "node": { 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "kubernetes-total": null, 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "with-ceph-daemons": null, 2024-03-20T08:01:08.561 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-cephfs-plugin": null, 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-nfs-plugin": null, 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-rbd-plugin": null 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "usage": { 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "storage-class": { 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket": null, 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs": null, 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "nfs": null, 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd": null, 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: "total": null 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.562 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "services": {}, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "usage": { 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 105, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "pools": 4, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_by_class": { 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "ssd": { 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "total_avail_bytes": 767731326976, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "total_bytes": 767960285184, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_bytes": 228958208, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_raw_bytes": 228958208, 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_raw_ratio": 0.00029813809669576585 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:08.564 INFO:tasks.workunit.client.0.smithi118.stdout: "total_avail_bytes": 767731326976, 2024-03-20T08:01:08.564 INFO:tasks.workunit.client.0.smithi118.stdout: "total_bytes": 767960285184, 2024-03-20T08:01:08.564 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_bytes": 228958208 2024-03-20T08:01:08.564 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.564 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:08.564 INFO:tasks.workunit.client.0.smithi118.stdout:} 2024-03-20T08:01:08.564 INFO:tasks.workunit.client.0.smithi118.stderr:++ ceph telemetry collection ls 2024-03-20T08:01:08.994 INFO:tasks.workunit.client.0.smithi118.stderr:+ COLLECTIONS='New collections are available: 2024-03-20T08:01:08.994 INFO:tasks.workunit.client.0.smithi118.stderr:['\''basic_base'\'', '\''basic_mds_metadata'\'', '\''basic_pool_options_bluestore'\'', '\''basic_pool_usage'\'', '\''basic_rook_v01'\'', '\''basic_usage_by_class'\'', '\''crash_base'\'', '\''device_base'\'', '\''ident_base'\'', '\''perf_memory_metrics'\'', '\''perf_perf'\''] 2024-03-20T08:01:08.994 INFO:tasks.workunit.client.0.smithi118.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-03-20T08:01:08.994 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:08.994 INFO:tasks.workunit.client.0.smithi118.stderr:NAME STATUS DESC 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster ' 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:+ NEW_COLLECTIONS=("perf_perf" "basic_mds_metadata" "basic_pool_usage" "basic_rook_v01" "perf_memory_metrics") 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-03-20T08:01:08.995 INFO:tasks.workunit.client.0.smithi118.stderr:+ [[ New collections are available: 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:NAME STATUS DESC 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-03-20T08:01:08.996 INFO:tasks.workunit.client.0.smithi118.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *perf_perf* ]] 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:+ [[ New collections are available: 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:NAME STATUS DESC 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-03-20T08:01:08.997 INFO:tasks.workunit.client.0.smithi118.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *basic_mds_metadata* ]] 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:+ [[ New collections are available: 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:NAME STATUS DESC 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-03-20T08:01:08.998 INFO:tasks.workunit.client.0.smithi118.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *basic_pool_usage* ]] 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:+ [[ New collections are available: 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:NAME STATUS DESC 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-03-20T08:01:08.999 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *basic_rook_v01* ]] 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:+ for col in ${NEW_COLLECTIONS[@]} 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:+ [[ New collections are available: 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:['basic_base', 'basic_mds_metadata', 'basic_pool_options_bluestore', 'basic_pool_usage', 'basic_rook_v01', 'basic_usage_by_class', 'crash_base', 'device_base', 'ident_base', 'perf_memory_metrics', 'perf_perf'] 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:Run `ceph telemetry on` to opt-in to these collections. 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr: 2024-03-20T08:01:09.000 INFO:tasks.workunit.client.0.smithi118.stderr:NAME STATUS DESC 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:basic_base NOT REPORTING: NOT OPTED-IN Basic information about the cluster (capacity, number and type of daemons, version, etc.) 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:basic_mds_metadata NOT REPORTING: NOT OPTED-IN MDS metadata 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_options_bluestore NOT REPORTING: NOT OPTED-IN Per-pool bluestore config options 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:basic_pool_usage NOT REPORTING: NOT OPTED-IN Default pool application and usage statistics 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:basic_rook_v01 NOT REPORTING: NOT OPTED-IN Basic Rook deployment data 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:basic_usage_by_class NOT REPORTING: NOT OPTED-IN Default device class usage statistics 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:crash_base NOT REPORTING: NOT OPTED-IN Information about daemon crashes (daemon type and version, backtrace, etc.) 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:device_base NOT REPORTING: NOT OPTED-IN Information about device health metrics 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:ident_base NOT REPORTING: NOT OPTED-IN User-provided identifying information about the cluster 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:perf_memory_metrics NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Heap stats and mempools for mon and mds 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:perf_perf NOT REPORTING: NOT OPTED-IN, CHANNEL perf IS OFF Information about performance counters of the cluster == *perf_memory_metrics* ]] 2024-03-20T08:01:09.001 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph telemetry on --license sharing-1-0 2024-03-20T08:01:09.171 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.171 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.171 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.171 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: pgmap v71: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:09.171 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.171 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:08 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: pgmap v71: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "telemetry preview", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: pgmap v71: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: from='client.24824 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:08 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:09.506 INFO:tasks.workunit.client.0.smithi118.stdout:Telemetry is on. 2024-03-20T08:01:09.506 INFO:tasks.workunit.client.0.smithi118.stdout:Some channels are disabled, please enable with: 2024-03-20T08:01:09.506 INFO:tasks.workunit.client.0.smithi118.stdout:`ceph telemetry enable channel perf` 2024-03-20T08:01:09.517 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph telemetry enable channel perf 2024-03-20T08:01:09.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Bus STOPPING 2024-03-20T08:01:09.770 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:01:09.770 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Bus STOPPED 2024-03-20T08:01:09.770 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Bus STARTING 2024-03-20T08:01:09.770 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Serving on http://:::9283 2024-03-20T08:01:09.770 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Bus STARTED 2024-03-20T08:01:10.020 INFO:tasks.workunit.client.0.smithi118.stdout:channel_perf is enabled 2024-03-20T08:01:10.031 INFO:tasks.workunit.client.0.smithi118.stderr:++ ceph config get mgr mgr/telemetry/last_opt_revision 2024-03-20T08:01:10.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Bus STOPPING 2024-03-20T08:01:10.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Bus STOPPED 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Bus STARTING 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Serving on http://:::9283 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:01:09] ENGINE Bus STARTED 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: mgrmap e22: y(active, since 2m), standbys: x 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: mgrmap e22: y(active, since 2m), standbys: x 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:09 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "telemetry preview-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: mgrmap e22: y(active, since 2m), standbys: x 2024-03-20T08:01:10.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "telemetry collection ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:10.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:10.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:09 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:10.406 INFO:tasks.workunit.client.0.smithi118.stderr:+ LAST_OPT_REVISION=1 2024-03-20T08:01:10.406 INFO:tasks.workunit.client.0.smithi118.stderr:+ '[' 1 -ne 1 ']' 2024-03-20T08:01:10.406 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph telemetry show 2024-03-20T08:01:11.120 INFO:tasks.workunit.client.0.smithi118.stdout:{ 2024-03-20T08:01:11.120 INFO:tasks.workunit.client.0.smithi118.stdout: "balancer": { 2024-03-20T08:01:11.120 INFO:tasks.workunit.client.0.smithi118.stdout: "active": true, 2024-03-20T08:01:11.120 INFO:tasks.workunit.client.0.smithi118.stdout: "mode": "" 2024-03-20T08:01:11.120 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.120 INFO:tasks.workunit.client.0.smithi118.stdout: "channels": [ 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "basic", 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "crash", 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "device", 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "ident", 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "perf" 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "channels_available": [ 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "basic", 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "ident", 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "crash", 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "device", 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "perf" 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.121 INFO:tasks.workunit.client.0.smithi118.stdout: "collections_available": [ 2024-03-20T08:01:11.122 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_base", 2024-03-20T08:01:11.122 INFO:tasks.workunit.client.0.smithi118.stdout: "device_base", 2024-03-20T08:01:11.122 INFO:tasks.workunit.client.0.smithi118.stdout: "crash_base", 2024-03-20T08:01:11.122 INFO:tasks.workunit.client.0.smithi118.stdout: "ident_base", 2024-03-20T08:01:11.122 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_perf", 2024-03-20T08:01:11.122 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_mds_metadata", 2024-03-20T08:01:11.122 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_usage", 2024-03-20T08:01:11.122 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_usage_by_class", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_rook_v01", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_memory_metrics", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_options_bluestore" 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "collections_opted_in": [ 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_base", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "device_base", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "crash_base", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "ident_base", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_perf", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_mds_metadata", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_usage", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_usage_by_class", 2024-03-20T08:01:11.123 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_rook_v01", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_memory_metrics", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_options_bluestore" 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "config": { 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "active_changed": [ 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_allow_insecure_global_id_reclaim", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_debug", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_mon_ticket_ttl", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_service_ticket_ttl", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "bdev_debug_aio", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "client_check_pool_perm", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "client_mount_gid", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "client_mount_uid", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "client_quota", 2024-03-20T08:01:11.124 INFO:tasks.workunit.client.0.smithi118.stdout: "container_image", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "daemonize", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_asserts_on_shutdown", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mgr", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mon", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_ms", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_osd", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_paxos", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "keyring", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_block_size", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_cache_size", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_compression", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_log", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_write_buffer_size", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_file", 2024-03-20T08:01:11.125 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_journald", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_stderr", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_allow_pool_delete", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_clock_drift_allowed", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_file_level", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_file", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_journald", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_stderr", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_data_avail_warn", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_election_default_strategy", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_host", 2024-03-20T08:01:11.126 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_max_pg_per_osd", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_mgr_mkfs_grace", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_prime_pg_temp", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_reporter_subtree_level", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_pg_warn_max_object_skew", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_bytes_per_osd", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_pgs_per_osd", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-03-20T08:01:11.127 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_legacy_crush_tunables", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_too_few_osds", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_bug", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_old_message", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "no_config_file", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_default_list", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_load_list", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_crush_chooseleaf_type", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_misdirected_ops", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_op_order", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_pg_log_writeout", 2024-03-20T08:01:11.128 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_shutdown", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_cached_snaps", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_missing_on_start", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_stray_on_activate", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_deep_scrub_update_digest_min_age", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_profile", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_lim", 2024-03-20T08:01:11.129 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_res", 2024-03-20T08:01:11.130 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_wgt", 2024-03-20T08:01:11.130 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target_autotune", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue_cut_off", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_erasure_code_profile", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_pg_autoscale_mode", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recover_clone_overlap", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recovery_max_chunk", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_load_threshold", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_max_interval", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_shutdown_pgref_assert", 2024-03-20T08:01:11.131 INFO:tasks.workunit.client.0.smithi118.stdout: "public_addr", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "public_addrv", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "public_network", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd_default_features", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd_qos_exclude_ops", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "setgroup", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "setuser" 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster_changed": [ 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_allow_insecure_global_id_reclaim", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_debug", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_mon_ticket_ttl", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_service_ticket_ttl", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "bdev_debug_aio", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "container_image", 2024-03-20T08:01:11.132 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_asserts_on_shutdown", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mgr", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mon", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_ms", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_osd", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_paxos", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_file", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_journald", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_stderr", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_join_fs", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/allow_ptrace", 2024-03-20T08:01:11.133 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/container_init", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/migration_current", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/use_repo_digest", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/ssl_server_port", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/orchestrator/orchestrator", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/channel_ident", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/channel_perf", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/description", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/enabled", 2024-03-20T08:01:11.134 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/organization", 2024-03-20T08:01:11.135 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_allow_pool_delete", 2024-03-20T08:01:11.138 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_clock_drift_allowed", 2024-03-20T08:01:11.138 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_file_level", 2024-03-20T08:01:11.138 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_file", 2024-03-20T08:01:11.138 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_data_avail_warn", 2024-03-20T08:01:11.138 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_election_default_strategy", 2024-03-20T08:01:11.138 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_max_pg_per_osd", 2024-03-20T08:01:11.138 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_mgr_mkfs_grace", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_prime_pg_temp", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_reporter_subtree_level", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_pg_warn_max_object_skew", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_bytes_per_osd", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_pgs_per_osd", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_legacy_crush_tunables", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-03-20T08:01:11.139 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_too_few_osds", 2024-03-20T08:01:11.140 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_bug", 2024-03-20T08:01:11.140 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_old_message", 2024-03-20T08:01:11.140 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_default_list", 2024-03-20T08:01:11.140 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_load_list", 2024-03-20T08:01:11.140 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_misdirected_ops", 2024-03-20T08:01:11.140 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_op_order", 2024-03-20T08:01:11.140 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_pg_log_writeout", 2024-03-20T08:01:11.140 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_shutdown", 2024-03-20T08:01:11.141 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_cached_snaps", 2024-03-20T08:01:11.141 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_missing_on_start", 2024-03-20T08:01:11.141 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_stray_on_activate", 2024-03-20T08:01:11.141 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_deep_scrub_update_digest_min_age", 2024-03-20T08:01:11.141 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_profile", 2024-03-20T08:01:11.141 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target_autotune", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue_cut_off", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_erasure_code_profile", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_pg_autoscale_mode", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recover_clone_overlap", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recovery_max_chunk", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_load_threshold", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_max_interval", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_shutdown_pgref_assert", 2024-03-20T08:01:11.142 INFO:tasks.workunit.client.0.smithi118.stdout: "public_network", 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_cache_enabled", 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_enable_ops_log", 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_enable_usage_log" 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "contact": null, 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "crashes": [], 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "created": "2024-03-20T07:52:36.029987Z", 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "crush": { 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_algs": { 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: "straw2": 3 2024-03-20T08:01:11.143 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.144 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_sizes": { 2024-03-20T08:01:11.144 INFO:tasks.workunit.client.0.smithi118.stdout: "2": 1, 2024-03-20T08:01:11.144 INFO:tasks.workunit.client.0.smithi118.stdout: "4": 2 2024-03-20T08:01:11.144 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.144 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_types": { 2024-03-20T08:01:11.144 INFO:tasks.workunit.client.0.smithi118.stdout: "1": 2, 2024-03-20T08:01:11.144 INFO:tasks.workunit.client.0.smithi118.stdout: "11": 1 2024-03-20T08:01:11.144 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "compat_weight_set": false, 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "device_classes": [ 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: 8 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_buckets": 6, 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_devices": 8, 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_rules": 1, 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_types": 12, 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_weight_sets": 0, 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "tunables": { 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "allowed_bucket_algs": 54, 2024-03-20T08:01:11.145 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_local_fallback_tries": 0, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_local_tries": 0, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_total_tries": 50, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_descend_once": 1, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_stable": 1, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_vary_r": 1, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v2_rules": 0, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v3_rules": 0, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v4_buckets": 1, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v5_rules": 0, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "legacy_tunables": 0, 2024-03-20T08:01:11.146 INFO:tasks.workunit.client.0.smithi118.stdout: "minimum_required_version": "jewel", 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "optimal_tunables": 1, 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "profile": "jewel", 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables": 1, 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables2": 1, 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables3": 1, 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables5": 1, 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "straw_calc_version": 1 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "description": "upgrade test cluster", 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "fs": { 2024-03-20T08:01:11.147 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: "feature_flags": { 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: "enable_multiple": true, 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: "ever_enabled_multiple": true 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: "filesystems": [ 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: "approx_ctime": "2024-03", 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: "balancer_enabled": false, 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.148 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_caps": 0, 2024-03-20T08:01:11.149 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_dns": 10, 2024-03-20T08:01:11.149 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_inos": 13, 2024-03-20T08:01:11.149 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_subtrees": 2, 2024-03-20T08:01:11.149 INFO:tasks.workunit.client.0.smithi118.stdout: "ever_allowed_features": 0, 2024-03-20T08:01:11.149 INFO:tasks.workunit.client.0.smithi118.stdout: "explicitly_allowed_features": 0, 2024-03-20T08:01:11.149 INFO:tasks.workunit.client.0.smithi118.stdout: "files": 0, 2024-03-20T08:01:11.150 INFO:tasks.workunit.client.0.smithi118.stdout: "max_mds": 1, 2024-03-20T08:01:11.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_data_pools": 1, 2024-03-20T08:01:11.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_in": 1, 2024-03-20T08:01:11.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_mds": 1, 2024-03-20T08:01:11.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": 0, 2024-03-20T08:01:11.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_standby_replay": 0, 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_up": 1, 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "snaps": 0, 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "standby_count_wanted": 0 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_standby_mds": 1, 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "total_num_mds": 2 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_stats": { 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:11.151 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 19218432, 2024-03-20T08:01:11.152 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 348024, 2024-03-20T08:01:11.152 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:11.152 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 229376, 2024-03-20T08:01:11.152 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 0, 2024-03-20T08:01:11.152 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 307, 2024-03-20T08:01:11.152 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.152 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 970648, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 12, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 460800, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 14457072, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 19218432 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 20217856, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 401784, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 49152, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 382, 2024-03-20T08:01:11.153 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 1253312, 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 20, 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 899072, 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 14911176, 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 20267008 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 43294720, 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 1492592, 2024-03-20T08:01:11.154 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 4259840, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1742, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 12722320, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 22, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2049792, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 24277504, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 47554560 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.38b88dde2c420bbdcde70c0aa08a86a8e933770b": { 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 35586048, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 889536, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:11.155 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 761856, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 4726784, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1084, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 7996608, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 22, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1802240, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 21383296, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 40312832 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 34701312, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 1147176, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.156 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 6651904, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1154, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 8840432, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 21, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1212672, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 20748520, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 41353216 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.0": { 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 47202304, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 820120, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:11.157 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 1007616, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 32768, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1359, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4947088, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 46, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2587904, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34955992, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 47235072 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.1": { 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 45481984, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 866464, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 835584, 2024-03-20T08:01:11.158 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1285, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 5329656, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 47, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1906176, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34496104, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 46317568 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.2": { 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 43704320, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 947888, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 385024, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1223, 2024-03-20T08:01:11.159 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4633504, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 47, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2266880, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 32972464, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 44089344 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.3": { 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 44015616, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 706280, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 204800, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1196, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4833016, 2024-03-20T08:01:11.160 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 45, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1543936, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34048800, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 44220416 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.4": { 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 44810240, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 588944, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 458752, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1194, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4739448, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 48, 2024-03-20T08:01:11.161 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1812992, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34785272, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 45268992 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.5": { 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 45064192, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 812496, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 204800, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1213, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4591560, 2024-03-20T08:01:11.162 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 48, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2078720, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34697832, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 45268992 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.6": { 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 44736512, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 974016, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 8192, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 532480, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1193, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4443464, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 46, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1850368, 2024-03-20T08:01:11.163 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34576888, 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 45268992 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.7": { 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 45006848, 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 1193128, 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 1310720, 2024-03-20T08:01:11.164 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1242, 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4780376, 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 47, 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1760512, 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34389248, 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 46317568 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.165 INFO:tasks.workunit.client.0.smithi118.stdout: "hosts": { 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num": 2, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mds": 2, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mgr": 2, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mon": 2, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_osd": 2 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "io_rate": { 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_stats_delta": { 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 0, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 0, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "stamp_delta": "12.002388", 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.168 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.168 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.168 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.168 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.168 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.168 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.168 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.168 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.169 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:11.170 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 0 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "leaderboard": false, 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "leaderboard_description": null, 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "license": "sharing-1-0", 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "mempool": { 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.171 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.172 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.172 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.172 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.172 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.172 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.172 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.172 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.173 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.174 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.175 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.175 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.175 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.175 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.175 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.175 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.175 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.175 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.176 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.177 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.178 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 149384, 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 31 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.179 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 928, 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 2 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.180 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.181 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.182 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.182 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.182 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.182 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.182 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.182 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.182 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.183 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.184 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.184 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.184 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.184 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.184 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.184 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.184 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.184 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.185 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.186 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.187 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.188 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 221639, 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 65 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.189 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 51496, 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 208 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.190 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 7736, 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 121 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.191 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.192 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.193 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.194 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.195 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.196 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.197 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.198 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1164379, 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 371 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 440, 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 5 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.199 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.200 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.200 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.200 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.200 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.200 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.200 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.200 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.201 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.201 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.201 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.201 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.201 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.201 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.201 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.202 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6312, 2024-03-20T08:01:11.202 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 120 2024-03-20T08:01:11.202 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.202 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.202 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12744, 2024-03-20T08:01:11.202 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1546 2024-03-20T08:01:11.203 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.203 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.203 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 10240, 2024-03-20T08:01:11.204 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 158 2024-03-20T08:01:11.204 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.204 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.204 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.204 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.204 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.204 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.205 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.205 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.205 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.205 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.205 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.38b88dde2c420bbdcde70c0aa08a86a8e933770b": { 2024-03-20T08:01:11.205 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.205 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.206 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.206 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.206 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.206 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.206 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.206 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.207 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.207 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.207 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.207 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.207 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.207 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.207 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.208 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.208 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.208 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.208 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.208 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.208 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.209 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.209 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.209 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.209 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.209 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.209 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.210 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.211 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.211 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.211 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.211 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.211 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.211 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.211 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.212 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.212 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.212 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.212 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.212 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.213 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.213 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.213 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.213 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.213 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.213 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.214 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.214 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.214 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.214 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.214 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.214 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.215 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.215 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.215 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.215 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.215 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.215 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.216 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.216 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.216 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.216 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.216 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.216 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.216 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1330371, 2024-03-20T08:01:11.217 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 434 2024-03-20T08:01:11.217 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.217 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.217 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 176, 2024-03-20T08:01:11.217 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 2 2024-03-20T08:01:11.217 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.218 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.218 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.218 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.218 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.218 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.219 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.219 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.219 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.219 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.219 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.219 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.220 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.220 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.220 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.220 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.220 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.220 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.221 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6312, 2024-03-20T08:01:11.221 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 120 2024-03-20T08:01:11.221 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.221 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.221 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12744, 2024-03-20T08:01:11.221 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1546 2024-03-20T08:01:11.221 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.222 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.222 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5120, 2024-03-20T08:01:11.222 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 79 2024-03-20T08:01:11.222 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.222 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.222 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.223 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.223 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.223 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.223 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.223 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.223 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.224 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.226 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.226 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.226 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.226 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.226 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.226 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.226 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.226 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.227 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.228 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.229 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1090299, 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 353 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.230 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.231 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6312, 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 120 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12744, 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1546 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5120, 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 79 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.0": { 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:11.233 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5520, 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 69 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 21168, 2024-03-20T08:01:11.234 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 189 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1712, 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 45 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 606208, 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 17 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 51400, 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9559 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 138296, 2024-03-20T08:01:11.235 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 235 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 11412, 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 746 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9072, 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 189 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 31878, 2024-03-20T08:01:11.236 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 203 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 21168, 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 189 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 433520, 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 97 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.237 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2460034, 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 199 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6864, 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 78 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 521640, 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 45 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 88400, 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 174 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224104, 2024-03-20T08:01:11.239 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3444 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.240 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.1": { 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:11.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4880, 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 61 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 19824, 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 177 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1264, 2024-03-20T08:01:11.242 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 41 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16384, 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 4 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 45678, 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8647 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 129456, 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 222 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9480, 2024-03-20T08:01:11.243 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 572 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8496, 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 177 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 27946, 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 190 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 19824, 2024-03-20T08:01:11.244 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 177 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 736, 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 426003, 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 95 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12112, 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.245 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2442762, 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 185 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5016, 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 57 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 440496, 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 38 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.246 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 7280, 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 18 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224104, 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3444 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.247 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.2": { 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.248 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.249 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4240, 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 53 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 19600, 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 175 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1040, 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 39 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8192, 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 2 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.250 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 44148, 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8377 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 124696, 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 215 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8320, 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 519 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8400, 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 175 2024-03-20T08:01:11.251 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 26490, 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 183 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 19600, 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 175 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 736, 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1 2024-03-20T08:01:11.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 384192, 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 85 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12112, 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2414420, 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 176 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4400, 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 50 2024-03-20T08:01:11.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 359352, 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 31 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9360, 2024-03-20T08:01:11.254 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 26 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224616, 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3468 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.255 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.3": { 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:11.256 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 3920, 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 49 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.257 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20160, 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 180 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480, 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 34 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 44616, 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8559 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.258 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 128096, 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 220 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9348, 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 578 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8640, 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 180 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 26930, 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 188 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20160, 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 180 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2208, 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 257376, 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 54 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 26565, 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2421390, 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 178 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4664, 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 53 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 417312, 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 36 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 10400, 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 20 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224104, 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3444 2024-03-20T08:01:11.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.263 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.4": { 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.264 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 3360, 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 42 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20048, 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1376, 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 42 2024-03-20T08:01:11.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 14419, 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 45568, 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8686 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 130136, 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 223 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9648, 2024-03-20T08:01:11.266 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 578 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8592, 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 27866, 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 191 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20160, 2024-03-20T08:01:11.267 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 180 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1472, 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 2 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 921744, 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 74 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20128, 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 5 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.268 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2451826, 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 185 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4928, 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 56 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 452088, 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 39 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.269 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 7904, 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 16 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224360, 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3456 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.270 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.5": { 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.271 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2800, 2024-03-20T08:01:11.272 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 35 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20048, 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480, 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 34 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 45840, 2024-03-20T08:01:11.273 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8752 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 131496, 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 225 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9888, 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 581 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8592, 2024-03-20T08:01:11.274 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 28378, 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 193 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20048, 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2208, 2024-03-20T08:01:11.275 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 833264, 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 52 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 26565, 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2420414, 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 183 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5368, 2024-03-20T08:01:11.276 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 61 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 475272, 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 41 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.277 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 11960, 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 23 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224616, 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3468 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.278 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.6": { 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.279 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2320, 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 29 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.280 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20944, 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 256, 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 32 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 46199, 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9008 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.281 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 130136, 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 223 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 10892, 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 736 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8976, 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.282 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 29062, 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 191 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20944, 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 808329, 2024-03-20T08:01:11.283 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 46 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2424628, 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 174 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4664, 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 53 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.284 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 452088, 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 39 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 88712, 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 181 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.285 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224872, 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3480 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.286 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.7": { 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:11.287 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1760, 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 22 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20944, 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1152, 2024-03-20T08:01:11.288 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 40 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 281259, 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 48524, 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9330 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 135576, 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 231 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 11676, 2024-03-20T08:01:11.289 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 743 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8976, 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 31110, 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 199 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20944, 2024-03-20T08:01:11.290 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 145316, 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 29 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.291 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2426662, 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 200 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5896, 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 67 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 533232, 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 46 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.292 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 89336, 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 199 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224104, 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3444 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.293 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "metadata": { 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 2 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 2 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:11.295 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 2 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 2 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 2 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 2 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 3 2024-03-20T08:01:11.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 3 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 3 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 3 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.297 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 3 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 3 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 8 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:11.298 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 8 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 8 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 8 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 8 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 8 2024-03-20T08:01:11.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_objectstore": { 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": 8 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "rotational": { 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "0": 8 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "features": { 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "optional": [], 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "persistent": [ 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "kraken", 2024-03-20T08:01:11.300 INFO:tasks.workunit.client.0.smithi118.stdout: "luminous", 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "mimic", 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap-prune", 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "nautilus", 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "octopus", 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "pacific", 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "elector-pinging", 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "quincy", 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "reef" 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "ipv4_addr_mons": 6, 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "ipv6_addr_mons": 0, 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "min_mon_release": 18, 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "v1_addr_mons": 3, 2024-03-20T08:01:11.301 INFO:tasks.workunit.client.0.smithi118.stdout: "v2_addr_mons": 3 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "organization": "ceph-qa", 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster_network": false, 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 8, 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "require_min_compat_client": "luminous", 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "require_osd_release": "reef" 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_perf_histograms": [ 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "axes": [ 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:11.302 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Latency (usec)", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 100000, 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 99999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[100000, 199999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[200000, 399999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[400000, 799999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[800000, 1599999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[1600000, 3199999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[3200000, 6399999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[6400000, 12799999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[12800000, 25599999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[25600000, 51199999]", 2024-03-20T08:01:11.303 INFO:tasks.workunit.client.0.smithi118.stdout: "[51200000, 102399999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[102400000, 204799999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[204800000, 409599999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[409600000, 819199999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[819200000, 1638399999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[1638400000, 3276799999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[3276800000, 6553599999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[6553600000, 13107199999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[13107200000, 26214399999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[26214400000, 52428799999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[52428800000, 104857599999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[104857600000, 209715199999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[209715200000, 419430399999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[419430400000, 838860799999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[838860800000, 1677721599999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[1677721600000, 3355443199999]", 2024-03-20T08:01:11.304 INFO:tasks.workunit.client.0.smithi118.stdout: "[3355443200000, 6710886399999]", 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "[6710886400000, 13421772799999]", 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "[13421772800000, 26843545599999]", 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "[26843545600000, 53687091199999]", 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "[53687091200000, None]" 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Request size (bytes)", 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 512, 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 511]", 2024-03-20T08:01:11.305 INFO:tasks.workunit.client.0.smithi118.stdout: "[512, 1023]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[1024, 2047]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[2048, 4095]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[4096, 8191]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[8192, 16383]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[16384, 32767]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[32768, 65535]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[65536, 131071]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[131072, 262143]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[262144, 524287]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[524288, 1048575]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[1048576, 2097151]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[2097152, 4194303]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[4194304, 8388607]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[8388608, 16777215]", 2024-03-20T08:01:11.306 INFO:tasks.workunit.client.0.smithi118.stdout: "[16777216, 33554431]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[33554432, 67108863]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[67108864, 134217727]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[134217728, 268435455]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[268435456, 536870911]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[536870912, 1073741823]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[1073741824, 2147483647]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[2147483648, 4294967295]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[4294967296, 8589934591]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[8589934592, 17179869183]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[17179869184, 34359738367]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[34359738368, 68719476735]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[68719476736, 137438953471]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[137438953472, 274877906943]", 2024-03-20T08:01:11.307 INFO:tasks.workunit.client.0.smithi118.stdout: "[274877906944, None]" 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "osds": [ 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 0, 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.308 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.309 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 1, 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.310 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.311 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.312 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 2, 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.313 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.314 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 3, 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.315 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.316 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.317 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 4, 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.318 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.319 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 5, 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.320 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.321 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.322 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 6, 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.323 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.324 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 7, 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.325 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 69, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.326 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.327 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "axes": [ 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Latency (usec)", 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 100000, 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:11.328 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 99999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[100000, 199999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[200000, 399999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[400000, 799999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[800000, 1599999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[1600000, 3199999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[3200000, 6399999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[6400000, 12799999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[12800000, 25599999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[25600000, 51199999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[51200000, 102399999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[102400000, 204799999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[204800000, 409599999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[409600000, 819199999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[819200000, 1638399999]", 2024-03-20T08:01:11.329 INFO:tasks.workunit.client.0.smithi118.stdout: "[1638400000, 3276799999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[3276800000, 6553599999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[6553600000, 13107199999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[13107200000, 26214399999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[26214400000, 52428799999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[52428800000, 104857599999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[104857600000, 209715199999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[209715200000, 419430399999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[419430400000, 838860799999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[838860800000, 1677721599999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[1677721600000, 3355443199999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[3355443200000, 6710886399999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[6710886400000, 13421772799999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[13421772800000, 26843545599999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[26843545600000, 53687091199999]", 2024-03-20T08:01:11.330 INFO:tasks.workunit.client.0.smithi118.stdout: "[53687091200000, None]" 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Request size (bytes)", 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 512, 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 511]", 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "[512, 1023]", 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "[1024, 2047]", 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "[2048, 4095]", 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "[4096, 8191]", 2024-03-20T08:01:11.331 INFO:tasks.workunit.client.0.smithi118.stdout: "[8192, 16383]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[16384, 32767]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[32768, 65535]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[65536, 131071]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[131072, 262143]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[262144, 524287]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[524288, 1048575]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[1048576, 2097151]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[2097152, 4194303]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[4194304, 8388607]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[8388608, 16777215]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[16777216, 33554431]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[33554432, 67108863]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[67108864, 134217727]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[134217728, 268435455]", 2024-03-20T08:01:11.332 INFO:tasks.workunit.client.0.smithi118.stdout: "[268435456, 536870911]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[536870912, 1073741823]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[1073741824, 2147483647]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[2147483648, 4294967295]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[4294967296, 8589934591]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[8589934592, 17179869183]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[17179869184, 34359738367]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[34359738368, 68719476735]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[68719476736, 137438953471]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[137438953472, 274877906943]", 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "[274877906944, None]" 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: "osds": [ 2024-03-20T08:01:11.333 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 0, 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.334 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.335 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 1, 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.336 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.337 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.338 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 2, 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.339 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.340 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 3, 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.341 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.342 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.343 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 4, 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.344 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.345 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 5, 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.346 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.347 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 6, 2024-03-20T08:01:11.348 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.349 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.350 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 7, 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.351 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.352 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: "axes": [ 2024-03-20T08:01:11.353 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Latency (usec)", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 100000, 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 99999]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[100000, 199999]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[200000, 399999]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[400000, 799999]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[800000, 1599999]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[1600000, 3199999]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[3200000, 6399999]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[6400000, 12799999]", 2024-03-20T08:01:11.354 INFO:tasks.workunit.client.0.smithi118.stdout: "[12800000, 25599999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[25600000, 51199999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[51200000, 102399999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[102400000, 204799999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[204800000, 409599999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[409600000, 819199999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[819200000, 1638399999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[1638400000, 3276799999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[3276800000, 6553599999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[6553600000, 13107199999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[13107200000, 26214399999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[26214400000, 52428799999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[52428800000, 104857599999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[104857600000, 209715199999]", 2024-03-20T08:01:11.355 INFO:tasks.workunit.client.0.smithi118.stdout: "[209715200000, 419430399999]", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "[419430400000, 838860799999]", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "[838860800000, 1677721599999]", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "[1677721600000, 3355443199999]", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "[3355443200000, 6710886399999]", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "[6710886400000, 13421772799999]", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "[13421772800000, 26843545599999]", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "[26843545600000, 53687091199999]", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "[53687091200000, None]" 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Request size (bytes)", 2024-03-20T08:01:11.356 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 512, 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 511]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[512, 1023]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[1024, 2047]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[2048, 4095]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[4096, 8191]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[8192, 16383]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[16384, 32767]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[32768, 65535]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[65536, 131071]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[131072, 262143]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[262144, 524287]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[524288, 1048575]", 2024-03-20T08:01:11.357 INFO:tasks.workunit.client.0.smithi118.stdout: "[1048576, 2097151]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[2097152, 4194303]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[4194304, 8388607]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[8388608, 16777215]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[16777216, 33554431]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[33554432, 67108863]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[67108864, 134217727]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[134217728, 268435455]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[268435456, 536870911]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[536870912, 1073741823]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[1073741824, 2147483647]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[2147483648, 4294967295]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[4294967296, 8589934591]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[8589934592, 17179869183]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[17179869184, 34359738367]", 2024-03-20T08:01:11.358 INFO:tasks.workunit.client.0.smithi118.stdout: "[34359738368, 68719476735]", 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "[68719476736, 137438953471]", 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "[137438953472, 274877906943]", 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "[274877906944, None]" 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "osds": [ 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 0, 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.359 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.360 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.361 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 1, 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.362 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.363 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 2, 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.364 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.365 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.366 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 3, 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.367 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.368 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 4, 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.369 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.370 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 5, 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.371 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.372 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.373 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 6, 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.374 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.375 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 7, 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.376 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.377 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.378 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "axes": [ 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Latency (usec)", 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 100000, 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 99999]", 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "[100000, 199999]", 2024-03-20T08:01:11.379 INFO:tasks.workunit.client.0.smithi118.stdout: "[200000, 399999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[400000, 799999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[800000, 1599999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[1600000, 3199999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[3200000, 6399999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[6400000, 12799999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[12800000, 25599999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[25600000, 51199999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[51200000, 102399999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[102400000, 204799999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[204800000, 409599999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[409600000, 819199999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[819200000, 1638399999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[1638400000, 3276799999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[3276800000, 6553599999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[6553600000, 13107199999]", 2024-03-20T08:01:11.380 INFO:tasks.workunit.client.0.smithi118.stdout: "[13107200000, 26214399999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[26214400000, 52428799999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[52428800000, 104857599999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[104857600000, 209715199999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[209715200000, 419430399999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[419430400000, 838860799999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[838860800000, 1677721599999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[1677721600000, 3355443199999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[3355443200000, 6710886399999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[6710886400000, 13421772799999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[13421772800000, 26843545599999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[26843545600000, 53687091199999]", 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "[53687091200000, None]" 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:11.381 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Request size (bytes)", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 512, 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 511]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[512, 1023]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[1024, 2047]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[2048, 4095]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[4096, 8191]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[8192, 16383]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[16384, 32767]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[32768, 65535]", 2024-03-20T08:01:11.382 INFO:tasks.workunit.client.0.smithi118.stdout: "[65536, 131071]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[131072, 262143]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[262144, 524287]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[524288, 1048575]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[1048576, 2097151]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[2097152, 4194303]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[4194304, 8388607]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[8388608, 16777215]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[16777216, 33554431]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[33554432, 67108863]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[67108864, 134217727]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[134217728, 268435455]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[268435456, 536870911]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[536870912, 1073741823]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[1073741824, 2147483647]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[2147483648, 4294967295]", 2024-03-20T08:01:11.383 INFO:tasks.workunit.client.0.smithi118.stdout: "[4294967296, 8589934591]", 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "[8589934592, 17179869183]", 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "[17179869184, 34359738367]", 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "[34359738368, 68719476735]", 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "[68719476736, 137438953471]", 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "[137438953472, 274877906943]", 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "[274877906944, None]" 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "osds": [ 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 0, 2024-03-20T08:01:11.384 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.385 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.386 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 1, 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 4, 5, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 17, 3, 0, 0, 0, 0, 0, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 14, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.387 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 5, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.388 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 2, 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.389 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.390 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.391 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 3, 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.392 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.393 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 4, 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.394 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.395 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.396 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 5, 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.397 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.398 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 6, 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.399 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 5, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.400 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 7, 2024-03-20T08:01:11.401 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.402 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.403 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_counters": { 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: "caps": { 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.404 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_flush_ack": { 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_flushsnap_ack": { 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_grant": { 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_revoke": { 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_trunc": { 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.405 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_commit": { 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_fetch_complete": { 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_fetch_keys": { 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_merge": { 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.406 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_split": { 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: "exported_inodes": { 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: "forward": { 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.410 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_cap_release": { 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_caps": { 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_caps_dirty": { 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_inode_file_caps": { 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "imported_inodes": { 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "inodes": { 2024-03-20T08:01:11.411 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "inodes_expired": { 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "inodes_pinned": { 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "inodes_with_caps": { 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "load_cent": { 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "openino_dir_fetch": { 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.412 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "process_request_cap_release": { 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "reply_latency": { 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 117733 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "request": { 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "root_rbytes": { 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "root_rfiles": { 2024-03-20T08:01:11.413 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "root_rsnaps": { 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_reply": { 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "subtrees": { 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_cache": { 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_enqueue_scrub": { 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.414 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_exportdir": { 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_flush": { 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_fragmentdir": { 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_fragstats": { 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_inodestats": { 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_recovering_enqueued": { 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_recovering_prioritized": { 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_recovering_processing": { 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_strays": { 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_strays_delayed": { 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_strays_enqueuing": { 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_completed": { 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_started": { 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "strays_created": { 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "strays_enqueued": { 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "strays_migrated": { 2024-03-20T08:01:11.417 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "strays_reintegrated": { 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_log": { 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "ev": { 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "evadd": { 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "evex": { 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.418 INFO:tasks.workunit.client.0.smithi118.stdout: "evexd": { 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "evexg": { 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "evtrm": { 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "jlat": { 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11582695 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "replayed": { 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.419 INFO:tasks.workunit.client.0.smithi118.stdout: "seg": { 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "segadd": { 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "segex": { 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "segexd": { 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "segexg": { 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "segtrm": { 2024-03-20T08:01:11.420 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_mem": { 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "cap": { 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "cap+": { 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "cap-": { 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "dir": { 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12 2024-03-20T08:01:11.421 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "dir+": { 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "dir-": { 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "dn": { 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "dn+": { 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "dn-": { 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.422 INFO:tasks.workunit.client.0.smithi118.stdout: "heap": { 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 207156 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "ino": { 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "ino+": { 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "ino-": { 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "rss": { 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41064 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.423 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_server": { 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "cap_acquisition_throttle": { 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "cap_revoke_eviction": { 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_request": { 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_session": { 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_peer_request": { 2024-03-20T08:01:11.424 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "req_create_latency": { 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "req_getattr_latency": { 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 117733 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "req_getfilelock_latency": { 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "req_getvxattr_latency": { 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.425 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "req_link_latency": { 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookup_latency": { 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookuphash_latency": { 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookupino_latency": { 2024-03-20T08:01:11.426 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookupname_latency": { 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookupparent_latency": { 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookupsnap_latency": { 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.427 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lssnap_latency": { 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "req_mkdir_latency": { 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "req_mknod_latency": { 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "req_mksnap_latency": { 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.428 INFO:tasks.workunit.client.0.smithi118.stdout: "req_open_latency": { 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "req_readdir_latency": { 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "req_rename_latency": { 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "req_renamesnap_latency": { 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.429 INFO:tasks.workunit.client.0.smithi118.stdout: "req_rmdir_latency": { 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "req_rmsnap_latency": { 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "req_rmxattr_latency": { 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setattr_latency": { 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.430 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setdirlayout_latency": { 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setfilelock_latency": { 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setlayout_latency": { 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setxattr_latency": { 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.431 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "req_snapdiff_latency": { 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "req_symlink_latency": { 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "req_unlink_latency": { 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_sessions": { 2024-03-20T08:01:11.432 INFO:tasks.workunit.client.0.smithi118.stdout: "average_load": { 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "avg_session_uptime": { 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 86 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "mdthresh_evicted": { 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "session_add": { 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "session_count": { 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.433 INFO:tasks.workunit.client.0.smithi118.stdout: "session_remove": { 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "sessions_open": { 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "sessions_stale": { 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "total_load": { 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.434 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "purge_queue": { 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executed": { 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.435 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executing": { 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executing_high_water": { 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executing_ops": { 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executing_ops_high_water": { 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_item_in_journal": { 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "election_call": { 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "election_lose": { 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "election_win": { 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "num_elections": { 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.437 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": { 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 9 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "session_add": { 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "session_rm": { 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 203 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "session_trim": { 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "paxos": { 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "accept_timeout": { 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.438 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "begin": { 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1013 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_bytes": { 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4969471 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_keys": { 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3039 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_latency": { 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 149996696 2024-03-20T08:01:11.439 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "collect": { 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_bytes": { 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_keys": { 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_latency": { 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.440 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_timeout": { 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_uncommitted": { 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "commit": { 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1013 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_bytes": { 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4896530 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_keys": { 2024-03-20T08:01:11.441 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6142 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_latency": { 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 135294402 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_ack_timeout": { 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_timeout": { 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn": { 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.442 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn_latency": { 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh": { 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1014 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh_latency": { 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1014, 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2393947205 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "restart": { 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state": { 2024-03-20T08:01:11.443 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_bytes": { 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1627 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_keys": { 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "start_leader": { 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "start_peon": { 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.444 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state": { 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_bytes": { 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_keys": { 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_latency": { 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.445 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache": { 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1020054731 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42696704 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41639936 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2147483648 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.446 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1056768 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:full": { 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277021748 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 272544 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.448 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70832940 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.449 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:inc": { 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277021748 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39156 2024-03-20T08:01:11.450 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.451 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71066328 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:kv": { 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.452 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 322961408 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 184681165 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.453 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.454 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71171379 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb": { 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "compact": { 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_len": { 2024-03-20T08:01:11.455 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_merge": { 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_range": { 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "get_latency": { 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 45061, 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 174568618 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_delay_time": { 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.456 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_memtable_time": { 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_pre_and_post_time": { 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_wal_time": { 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_latency": { 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.457 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_sync_latency": { 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2188, 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 283123799 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.38b88dde2c420bbdcde70c0aa08a86a8e933770b": { 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "election_call": { 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "election_lose": { 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "election_win": { 2024-03-20T08:01:11.458 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_elections": { 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": { 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "session_add": { 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "session_rm": { 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 108 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "session_trim": { 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.459 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "paxos": { 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "accept_timeout": { 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "begin": { 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 861 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_bytes": { 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 861, 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3908052 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_keys": { 2024-03-20T08:01:11.460 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_latency": { 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 861, 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 99997793 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "collect": { 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_bytes": { 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_keys": { 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.461 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_latency": { 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_timeout": { 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_uncommitted": { 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "commit": { 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 863 2024-03-20T08:01:11.462 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_bytes": { 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_keys": { 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_latency": { 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_ack_timeout": { 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.463 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_timeout": { 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn": { 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn_latency": { 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh": { 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 862 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh_latency": { 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 862, 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1942957114 2024-03-20T08:01:11.464 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "restart": { 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 9 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state": { 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_bytes": { 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_keys": { 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.465 INFO:tasks.workunit.client.0.smithi118.stdout: "start_leader": { 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "start_peon": { 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state": { 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 862 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_bytes": { 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 862, 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7654511 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_keys": { 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 862, 2024-03-20T08:01:11.466 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6187 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_latency": { 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 862, 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 100997771 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache": { 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1020054731 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 37560320 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 32014336 2024-03-20T08:01:11.467 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2147483648 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5545984 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:full": { 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.468 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277020672 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 272974 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.469 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.470 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70833586 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:inc": { 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.471 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277020672 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41595 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.472 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.473 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71064965 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:kv": { 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 322961408 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 184680448 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.474 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.475 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71172096 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.476 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb": { 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "compact": { 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_len": { 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_merge": { 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_range": { 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "get_latency": { 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 36967, 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 97362230 2024-03-20T08:01:11.477 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_delay_time": { 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_memtable_time": { 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_pre_and_post_time": { 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_wal_time": { 2024-03-20T08:01:11.478 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_latency": { 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_sync_latency": { 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1882, 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 218751490 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:11.479 INFO:tasks.workunit.client.0.smithi118.stdout: "election_call": { 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "election_lose": { 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "election_win": { 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_elections": { 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": { 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "session_add": { 2024-03-20T08:01:11.480 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "session_rm": { 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 54 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "session_trim": { 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "paxos": { 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "accept_timeout": { 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "begin": { 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 872 2024-03-20T08:01:11.481 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_bytes": { 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 872, 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3934058 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_keys": { 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_latency": { 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 872, 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 147996580 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "collect": { 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.482 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_bytes": { 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 146 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_keys": { 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_latency": { 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2999931 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_timeout": { 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.483 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_uncommitted": { 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "commit": { 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 873 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_bytes": { 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_keys": { 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_latency": { 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.484 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_ack_timeout": { 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_timeout": { 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn": { 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn_latency": { 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh": { 2024-03-20T08:01:11.485 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 872 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh_latency": { 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 872, 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1850957235 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "restart": { 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state": { 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_bytes": { 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.486 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_keys": { 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "start_leader": { 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "start_peon": { 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state": { 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 873 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_bytes": { 2024-03-20T08:01:11.487 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 873, 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7704236 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_keys": { 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 873, 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6271 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_latency": { 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 872, 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 411990485 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache": { 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1020054731 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.488 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 38600704 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31891456 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2147483648 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6709248 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:full": { 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.489 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277020672 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 272974 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.490 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70833586 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:inc": { 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.492 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277020672 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41595 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.493 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.494 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71064965 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:kv": { 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 322961408 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.495 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 184680448 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.496 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.497 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71172096 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb": { 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "compact": { 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_len": { 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_merge": { 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.498 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_range": { 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "get_latency": { 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 37030, 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 90088410 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_delay_time": { 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_memtable_time": { 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.499 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_pre_and_post_time": { 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_wal_time": { 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_latency": { 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_sync_latency": { 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1909, 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 640777132 2024-03-20T08:01:11.500 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.0": { 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.501 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2695168 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.502 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1691648 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.503 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1552384 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:11.504 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:11.505 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:11.506 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.507 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4325376 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1327104 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:11.509 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 4, 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 170270 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.510 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 24, 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 62270 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 366, 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70407745 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 354, 2024-03-20T08:01:11.511 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16996591 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 366, 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 329151 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 366, 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70736896 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.512 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45555 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 6, 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 20853 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:11.513 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3744 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 840 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 232 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 127, 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1998302 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 337, 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 323361 2024-03-20T08:01:11.514 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 210, 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1123631 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 54317 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.515 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6564867 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 124, 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7784781 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 124, 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 521250265386 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 124, 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 136141582685 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:11.516 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 75591708725 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 80378 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 484372 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 321504765 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.517 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 160390 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 340754259 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22170005 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 902807 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.518 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 58, 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 315443 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 691545409 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 26926652 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2895425 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.519 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861614376 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43302912 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43270144 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137553817 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 32768 2024-03-20T08:01:11.520 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85470711 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.521 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.522 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 606208 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73306633 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.523 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641030334 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.524 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.525 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71999218 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.526 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 56980474 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.527 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.528 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68848182 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.529 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641030334 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2434 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.530 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8522 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 236189 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.531 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71754201 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.532 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:11.533 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 616 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 47439688 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.534 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 6, 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2828505 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 46131616 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:11.535 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.536 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.537 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:11.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 616 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 47439688 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 6, 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2828505 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:11.539 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 46131616 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1771104 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.540 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28991488 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.1": { 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:11.541 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:11.542 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2478080 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:11.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1593344 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1454080 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.544 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.545 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.546 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.547 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:11.548 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4009984 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.549 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 724992 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 177365 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:11.550 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 32, 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 79482 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:11.551 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 342, 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67561207 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 336, 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6917055 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 342, 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 337848 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 342, 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67899055 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:11.552 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39705 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6453 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.553 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 19546 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 817 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 238 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:11.554 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 135, 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2004843 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 351, 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 271377 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 216, 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1237767 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.555 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 113239 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5882558 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 122, 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7330347 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 121, 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 238160647650 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 122, 2024-03-20T08:01:11.556 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 127118562449 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 407, 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 78619356867 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 407, 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 47751 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 462679 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:11.557 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 116750324 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 171796 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 54487495 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 20599737 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.558 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 311871 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 56, 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 302498 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 198260189 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24707843 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:11.559 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2711976 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861614376 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43433984 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42557440 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.560 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137553817 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 876544 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.561 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85507548 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.562 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16384 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.563 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73859620 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641306617 2024-03-20T08:01:11.564 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.565 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71722935 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.566 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005032 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.567 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.568 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68823624 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.569 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641306617 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.570 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 221846 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.571 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71503217 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.572 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 38 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.573 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 155 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2429440 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:11.574 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 155, 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 460591849 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1063 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 155, 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 52345283 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 155, 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 175170212 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 33 2024-03-20T08:01:11.575 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4721579 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1063 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4291312 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.576 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3475180 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 22, 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 18159084 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.577 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 22, 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8782803 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 22, 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 17140728 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.578 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 100 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2429440 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 100, 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 437711186 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 100, 2024-03-20T08:01:11.579 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39271168 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 100, 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 154554304 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 590368 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.580 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28393472 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.2": { 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.581 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.582 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2179072 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:11.583 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1396736 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1257472 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.584 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.585 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.586 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.588 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3514368 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:11.589 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 716800 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 4, 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 208631 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:11.590 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16180 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.591 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 294, 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45079777 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 291, 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5480823 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 294, 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 274060 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 294, 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45353837 2024-03-20T08:01:11.592 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 66222 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.593 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4777 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 603 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:11.594 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 218 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 105, 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1001837 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 284, 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 267058 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 179, 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 620834 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.595 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 105884 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6328852 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 115, 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7580165 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:11.596 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 114, 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 251284937020 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 115, 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 125451047575 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 332, 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 75591994867 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 332, 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 40804 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.597 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 405323 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 66615035 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 121287 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22209760 2024-03-20T08:01:11.598 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16602756 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 311711 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 50, 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 220614 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 112212520 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.599 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 20252338 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2052063 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861614376 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41205760 2024-03-20T08:01:11.600 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 40730624 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137553817 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 475136 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.601 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508620 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.602 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8192 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.603 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73866740 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.604 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641314656 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.605 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.606 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71714896 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:11.607 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005747 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.608 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.609 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68822909 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.610 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641314656 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.611 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 212396 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.612 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.614 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.614 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.614 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.614 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.614 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.614 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.614 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.614 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71504628 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.615 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.616 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43961563 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:11.617 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4254833 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 37719144 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.618 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:11.619 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.620 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:11.621 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43961563 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4254833 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:11.622 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 37719144 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:11.623 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28385280 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.3": { 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.624 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1097728 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.625 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:11.626 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 806912 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 667648 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.627 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:11.628 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:11.629 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5503 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5503 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.630 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:11.631 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:11.632 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1843200 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 737280 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.633 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.634 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4750 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 150, 2024-03-20T08:01:11.635 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31275215 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 142, 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6610022 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 150, 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 183193 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 150, 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31458408 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:11.636 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 37578 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.637 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5509 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 331 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.638 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 220 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 81, 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 689093 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 140, 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 220849 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 59, 2024-03-20T08:01:11.639 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 158052 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3448519 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.640 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 53, 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3874964 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 50, 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 364806141004 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 53, 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 82325193252 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 229, 2024-03-20T08:01:11.641 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1051069 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 229, 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24281 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 237514 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 108852609 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:11.642 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65960 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 55127102 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13895819 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 314445 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:11.643 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 44, 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 196403 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 181588503 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16190214 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1442795 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.644 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861614376 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41336832 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41017344 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137553817 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.645 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 319488 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.646 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508744 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.647 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.648 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73874808 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641315582 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.650 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.651 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71713970 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.652 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005829 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.653 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.654 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68822827 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.655 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641315582 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.656 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8222 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 212617 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.657 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71495259 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.658 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 36 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:11.659 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.660 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 826294 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.661 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.662 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.663 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.664 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.665 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 826294 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.666 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28401664 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.4": { 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:11.667 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:11.668 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1187840 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.669 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 860160 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 720896 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.670 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.671 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.672 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.673 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:11.674 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:11.675 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1986560 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 733184 2024-03-20T08:01:11.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 8, 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 242713 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.677 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5202 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 163, 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43413114 2024-03-20T08:01:11.678 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 151, 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5378069 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 163, 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 160270 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 163, 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43573384 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.679 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45245 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.680 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5053 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 364 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 238 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.681 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 97, 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1057146 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 163, 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 284482 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 66, 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 318485 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:11.682 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 113090 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3567685 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 42, 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3376401 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.683 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 40, 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 174076583361 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 42, 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 59910774836 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 260, 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 503702 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 260, 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 29659 2024-03-20T08:01:11.684 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 293576 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159332190 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 147796 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:11.685 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73221492 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12492767 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 312419 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 41, 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 196260 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.686 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 249010406 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14540473 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2062376 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.687 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861620469 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42385408 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41762816 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137560985 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 622592 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.688 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508320 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.689 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8192 2024-03-20T08:01:11.690 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73867040 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:11.691 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641312403 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.692 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.693 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71717149 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:11.694 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005547 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.695 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.696 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68823109 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:11.697 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641312403 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.698 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.699 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 220872 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71498405 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.700 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.701 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.702 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16535570 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 661769 2024-03-20T08:01:11.703 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16484502 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.704 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.705 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:11.706 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.707 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16535570 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 661769 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16484502 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.708 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28405760 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.709 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.5": { 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.710 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1232896 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.711 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 921600 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:11.712 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 782336 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.713 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.714 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:11.715 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.716 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.717 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:11.718 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2093056 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 733184 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 6, 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 175599 2024-03-20T08:01:11.719 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:11.720 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5021 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 178, 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 51957555 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 166, 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7715370 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.721 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 178, 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 129777 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 178, 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 52087332 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.722 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 54355 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.723 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4809 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 332 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 245 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 91, 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 751067 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.724 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 146, 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 252293 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 55, 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 358628 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 4, 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 97849 2024-03-20T08:01:11.725 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7601436 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 39, 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2879924 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 36, 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 228721521403 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.726 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 39, 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 61505489997 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 264, 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 550360 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 264, 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 29626 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2052106 2024-03-20T08:01:11.727 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 138184191 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 81539 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 72999945 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:11.728 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13459162 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 314431 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 32, 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 143346 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 234329599 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:11.729 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15679879 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2258519 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861620469 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.730 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42385408 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42090496 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137560985 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 294912 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:11.731 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508729 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.732 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.733 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73874823 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.734 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641315470 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.735 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.736 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71714082 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.737 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005819 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.738 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.739 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68822837 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.740 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641315470 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.741 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5883 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 221230 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71489097 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.743 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:11.744 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.745 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:11.746 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.747 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.748 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.750 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28397568 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.6": { 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:11.752 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:11.753 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1212416 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:11.754 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 901120 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 761856 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:11.755 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:11.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.757 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5504 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5504 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.758 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.759 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2052096 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:11.760 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1318912 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:11.761 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5762 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 173, 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 50155890 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 162, 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13230003 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 173, 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 107893 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 173, 2024-03-20T08:01:11.763 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 50263783 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41526 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:11.764 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4040 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 346 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.765 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 252 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 91, 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 658685 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 139, 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 253902 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 48, 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 413906 2024-03-20T08:01:11.766 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5516491 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2487452 2024-03-20T08:01:11.767 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 286186428277 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57902364509 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 457537 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.768 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28096 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 217281 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 157391827 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 82856 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:11.769 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 86936491 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11926400 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 904378 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 26, 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 109000 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.770 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 262024284 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13704624 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2500277 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:11.771 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861620469 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42385408 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41852928 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137560985 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.772 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 532480 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508571 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.773 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.774 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73874981 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641314284 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.777 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71715268 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.778 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005714 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.779 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68822942 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:11.781 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641314284 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2398 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3597 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5995 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 217044 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.783 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71488362 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.784 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39 2024-03-20T08:01:11.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:11.786 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2810 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 183194403 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 15, 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6348347 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:11.787 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 101979001 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.788 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:11.789 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.790 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2810 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 183194403 2024-03-20T08:01:11.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 15, 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6348347 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 101979001 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.792 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28983296 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.7": { 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:11.793 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.794 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1171456 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:11.795 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 897024 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 757760 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.796 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:11.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5500 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:11.799 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5500 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.800 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.801 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2007040 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:11.802 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1318912 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 169944 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:11.803 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 9, 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24419 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 172, 2024-03-20T08:01:11.804 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43241157 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 169, 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13917366 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 172, 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85721 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 172, 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43326878 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.805 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 36703 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 521 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:11.806 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3672 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 611 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 281 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 119, 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2240047 2024-03-20T08:01:11.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 184, 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 257395 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 65, 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1587663 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:11.808 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 46406 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3499437 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 20, 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1769811 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 20, 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 320320280624 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:11.809 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 20, 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 32150620865 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 460087 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 30874 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192908 2024-03-20T08:01:11.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 125816192 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 87588 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 92684749 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.811 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12703064 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 902683 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 19, 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 80640 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 234931211 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.812 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14508821 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1755632 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861620469 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43433984 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:11.813 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42041344 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137560985 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1392640 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.814 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85491415 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8192 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.815 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.816 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 131072 2024-03-20T08:01:11.816 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.816 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.816 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.816 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.816 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.816 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.816 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.818 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.818 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 135168 2024-03-20T08:01:11.818 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.818 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.818 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.818 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.818 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.818 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73617705 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.819 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641185617 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.820 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.821 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71843935 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:11.822 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 56994277 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.824 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68834379 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.825 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641185617 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:11.826 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3622 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8451 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:11.827 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 228192 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71605798 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.828 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.829 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 46 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 115 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:11.830 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 115, 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 157172478 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 265081 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 117, 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 25625752 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.831 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 115, 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 110571394 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 80 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 80, 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11409892 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:11.832 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 265081 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 80, 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10167534 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 80, 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8408064 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 30 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 30, 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27269213 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:11.834 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 30, 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12349121 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 30, 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 25638026 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.835 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 118493373 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 7, 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3109097 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 76525304 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.836 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28983296 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.837 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "pools": [ 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr" 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 1, 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 1, 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 1, 2024-03-20T08:01:11.838 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 1781760, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 1781760, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 2, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 2.4430023586319294e-06, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 212, 2024-03-20T08:01:11.839 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 313344, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 590368, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 590368, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 1771104, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 179, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 2473984 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:11.840 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 32, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 32, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 2, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 98304, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:11.841 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 98304, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 22, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 1.3478666005539708e-07, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 3428, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 3428, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 10284, 2024-03-20T08:01:11.842 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 51, 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 20480 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:11.843 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 64, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 64, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 3, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 0, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 0, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 0, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:11.844 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 0.0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 0 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:11.845 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd" 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 8, 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 8, 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 4, 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:11.846 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330264330, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110100992, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 0.0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:11.847 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 0, 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 0, 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 0, 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 0, 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 0 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd": { 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: "mirroring_by_pool": [ 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: false 2024-03-20T08:01:11.848 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_images_by_pool": [ 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pools": 1 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "report_id": "f7bee648-d925-4013-8791-48f92fa7019f", 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "report_timestamp": "2024-03-20T08:01:10.995658", 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "report_version": 1, 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_stats": { 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "7.9.2" 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "rook": { 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster": { 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "external-mode": null, 2024-03-20T08:01:11.849 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "allow-multiple-per-node": null, 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "count": null, 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "max-id": null, 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "pvc": { 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "enabled": null 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "stretch": { 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "enabled": null 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "network": { 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "provider": null 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "storage": { 2024-03-20T08:01:11.850 INFO:tasks.workunit.client.0.smithi118.stdout: "device-set": { 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "non-portable": null, 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "portable": null, 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "total": null 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "csi": { 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "kubernetes": { 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "node": { 2024-03-20T08:01:11.851 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "kubernetes-total": null, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "with-ceph-daemons": null, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-cephfs-plugin": null, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-nfs-plugin": null, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-rbd-plugin": null 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "usage": { 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "storage-class": { 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket": null, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs": null, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "nfs": null, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd": null, 2024-03-20T08:01:11.852 INFO:tasks.workunit.client.0.smithi118.stdout: "total": null 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: "services": {}, 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_per_pg": [ 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:11.853 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.145286+0000", 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.145286+0000", 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.145662+0000", 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.854 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.855 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3f", 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000", 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.856 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.859 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.860 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128811+0000", 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128811+0000", 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.129069+0000", 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.861 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.862 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3e", 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.863 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000", 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.866 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:11.867 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118755+0000", 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118755+0000", 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.119030+0000", 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.869 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3d", 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.870 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:51:41.823570+0000", 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.873 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:11.874 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.137768+0000", 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.137768+0000", 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.138067+0000", 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.876 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3c", 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.877 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000", 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.880 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:11.881 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.131585+0000", 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.131585+0000", 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.131923+0000", 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.883 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3b", 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.884 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000", 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.887 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.888 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135727+0000", 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135727+0000", 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.137308+0000", 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.890 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3a", 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.891 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000", 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.894 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.895 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.132230+0000", 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.132230+0000", 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.132941+0000", 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.897 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.39", 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.898 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000", 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.901 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.902 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135920+0000", 2024-03-20T08:01:11.903 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135920+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.136216+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.904 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.38", 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.905 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:48:37.134454+0000", 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.908 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.909 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.910 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.144302+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.144302+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.144631+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.911 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.912 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.37", 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000", 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.914 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.915 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.916 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.917 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.138348+0000", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.138348+0000", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.138657+0000", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:11.918 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.919 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.36", 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000", 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.920 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.921 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.922 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.923 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.924 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.137672+0000", 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.137672+0000", 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.137961+0000", 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:11.925 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.926 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.35", 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.927 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000", 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.929 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.930 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.931 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135935+0000", 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135935+0000", 2024-03-20T08:01:11.932 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.136246+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.933 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.34", 2024-03-20T08:01:11.934 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000", 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.935 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.936 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.937 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:11.938 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.939 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.129209+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.129209+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.129765+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.940 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.941 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.33", 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:16:48.280737+0000", 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.942 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.943 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.944 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.945 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.946 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122270+0000", 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122270+0000", 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.123008+0000", 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.947 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.948 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.32", 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.949 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:40:37.385767+0000", 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.950 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.951 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.953 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.126555+0000", 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.126555+0000", 2024-03-20T08:01:11.954 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.127042+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.955 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.31", 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.956 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000", 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.957 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.958 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.960 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128111+0000", 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128111+0000", 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.129086+0000", 2024-03-20T08:01:11.961 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.962 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.30", 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.963 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000", 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.964 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.965 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.967 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128501+0000", 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128501+0000", 2024-03-20T08:01:11.968 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.128812+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.969 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2f", 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.970 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000", 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.971 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.972 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.974 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.127943+0000", 2024-03-20T08:01:11.975 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.127943+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.128211+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:11.976 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2e", 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.977 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:48:19.979568+0000", 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.978 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.979 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.981 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.982 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.132794+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.132794+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.133112+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:11.983 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.984 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2d", 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000", 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.985 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.986 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:11.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:11.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.989 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.774570+0000", 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.774570+0000", 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.774846+0000", 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:11.990 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 4, 2024-03-20T08:01:11.991 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 4, 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1f", 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:11.992 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 24, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:28:00.513690+0000", 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 582, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:11.993 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:11.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:11.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 4, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 1 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'4" 2024-03-20T08:01:11.996 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:11.997 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128584+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128584+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.128852+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:11.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:11.999 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1e", 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:15:37.874514+0000", 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.000 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.003 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.004 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.773993+0000", 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.773993+0000", 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.774347+0000", 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.006 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1e", 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:04:59.603081+0000", 2024-03-20T08:01:12.007 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.011 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118107+0000", 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118107+0000", 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.118404+0000", 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:12.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.013 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1f", 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.014 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:58:51.229913+0000", 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.017 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.018 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.019 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.019 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.019 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.019 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.019 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.019 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.019 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.019 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.777710+0000", 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.777710+0000", 2024-03-20T08:01:12.022 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.778043+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.023 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1d", 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.024 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000", 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.025 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.026 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.027 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.028 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122136+0000", 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122136+0000", 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.122440+0000", 2024-03-20T08:01:12.029 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.030 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1c", 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.031 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000", 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.033 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.035 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.775756+0000", 2024-03-20T08:01:12.036 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.775756+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.776107+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:12.037 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1c", 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.038 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000", 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.039 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.040 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.041 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.042 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:46.698994+0000", 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:46.698994+0000", 2024-03-20T08:01:12.043 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:46.699381+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 65, 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:12.044 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 64, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1d", 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.045 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 69, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 16, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:31:09.474011+0000", 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.046 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.047 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:12.049 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.050 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.769359+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.769359+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769988+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.051 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.052 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1b", 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:26:35.889023+0000", 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.053 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.054 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.056 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118430+0000", 2024-03-20T08:01:12.057 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118430+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.118793+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:12.058 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1a", 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.059 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000", 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.060 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.063 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.765598+0000", 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.765598+0000", 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.765918+0000", 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:12.064 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.065 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1a", 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.066 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000", 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.067 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.068 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.069 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.070 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118645+0000", 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118645+0000", 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.119313+0000", 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.071 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.072 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1b", 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:25:13.410704+0000", 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.073 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.074 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.075 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.076 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.077 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770958+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770958+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771278+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:12.078 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.079 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.19", 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:10:26.828071+0000", 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.080 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.081 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.082 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.083 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.084 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.116571+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.116571+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.116986+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.085 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.086 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.18", 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:32:56.125697+0000", 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.087 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.088 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.089 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.090 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.091 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.768716+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.768716+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769007+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.092 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.093 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.18", 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000", 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.094 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.095 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.096 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.097 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.119184+0000", 2024-03-20T08:01:12.098 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.119184+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.119518+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:12.099 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.19", 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.100 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000", 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.101 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.102 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.103 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.104 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.105 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.768916+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.768916+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769300+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.106 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.107 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.17", 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000", 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.108 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.109 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.110 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.111 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.112 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.130332+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.130332+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.130616+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.113 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.114 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.16", 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000", 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.115 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.116 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.117 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.118 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:12.119 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.769492+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.769492+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769790+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.120 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.121 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.16", 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000", 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.122 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.123 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.124 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.125 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.126 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.116037+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.116037+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.116344+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:12.127 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.17", 2024-03-20T08:01:12.128 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:09:16.183856+0000", 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.129 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.132 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:12.133 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770657+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770657+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.770988+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.134 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 3, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 3, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.15", 2024-03-20T08:01:12.135 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 23, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000", 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 90, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 4, 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 1 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'3" 2024-03-20T08:01:12.139 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:12.140 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128201+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128201+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.128506+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:12.141 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.14", 2024-03-20T08:01:12.142 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000", 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.146 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:12.147 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.767533+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.767533+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.767830+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:12.148 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 12, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 12, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.14", 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.149 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 32, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000", 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 2630, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.150 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 12, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 4, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 4, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.151 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.152 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 12, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 9 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "70'12" 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.153 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:12.154 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.139362+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.139362+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.140127+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:12.155 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.15", 2024-03-20T08:01:12.156 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000", 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.157 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.158 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.159 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.160 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.161 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.771499+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.771499+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771772+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.162 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.163 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.13", 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:19:45.833311+0000", 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 34, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.164 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.165 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.166 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 7 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.167 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.121780+0000", 2024-03-20T08:01:12.168 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.121780+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.122549+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.169 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.12", 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.170 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:37:01.164741+0000", 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.171 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.172 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.173 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.174 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.771464+0000", 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.771464+0000", 2024-03-20T08:01:12.175 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771839+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:12.176 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.12", 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.177 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000", 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.178 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.179 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.180 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.181 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:12.182 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128675+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128675+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.129094+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:12.183 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.13", 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.184 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000", 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.185 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.186 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.187 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.188 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.767268+0000", 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.767268+0000", 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.767586+0000", 2024-03-20T08:01:12.189 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.190 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.11", 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.191 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000", 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.192 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.193 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.194 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.195 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.138286+0000", 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.138286+0000", 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.139925+0000", 2024-03-20T08:01:12.196 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.197 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.10", 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.198 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000", 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.199 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.200 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.201 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.202 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.756879+0000", 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.756879+0000", 2024-03-20T08:01:12.203 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.757217+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 8, 2024-03-20T08:01:12.204 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 8, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.10", 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.205 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 28, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:08:28.214822+0000", 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.206 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 12, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 4, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 4, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.207 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 4, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 12, 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.208 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'8" 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.209 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.132630+0000", 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.132630+0000", 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.133140+0000", 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.210 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.211 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.11", 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.212 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000", 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.213 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.214 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.215 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.216 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770984+0000", 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770984+0000", 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771322+0000", 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.217 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.218 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.f", 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000", 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.219 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 22, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.220 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.221 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 1, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 1 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.222 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.226 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.143683+0000", 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.143683+0000", 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.144254+0000", 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.227 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.228 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.e", 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.229 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000", 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.230 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.231 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.232 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.233 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.912606+0000", 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.987867+0000", 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.987867+0000", 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.988226+0000", 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.912606+0000", 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.234 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.912606+0000", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.912606+0000", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.912606+0000", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.912606+0000", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.912606+0000", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:12.235 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.2", 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 69, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:14:23.583126+0000", 2024-03-20T08:01:12.236 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.237 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.238 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.239 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.240 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.773526+0000", 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.773526+0000", 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.773911+0000", 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.241 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.242 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.4", 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000", 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.243 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.244 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.245 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.246 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.247 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.121608+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.121608+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.122909+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:12.248 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.249 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.5", 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000", 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.250 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.251 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.252 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.253 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:12.254 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.912657+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.991077+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.991077+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.991385+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.912657+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.912657+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.912657+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.912657+0000", 2024-03-20T08:01:12.255 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.912657+0000", 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.912657+0000", 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.256 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.4", 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 69, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:31:07.169554+0000", 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.257 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.258 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.259 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.260 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.261 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770298+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770298+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.770569+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.262 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.263 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.2", 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000", 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.264 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.265 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.266 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.267 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135909+0000", 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135909+0000", 2024-03-20T08:01:12.268 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.136200+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.269 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3", 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.270 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000", 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.271 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.272 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.274 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.988575+0000", 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.988575+0000", 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.988864+0000", 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:12.275 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.276 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.7", 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 69, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.277 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:19:29.334160+0000", 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.278 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.279 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.281 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.129798+0000", 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.129798+0000", 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.130079+0000", 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.282 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.283 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.0", 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000", 2024-03-20T08:01:12.284 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.285 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.286 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.287 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:12.288 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.764390+0000", 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.764390+0000", 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.764736+0000", 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.289 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.290 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1", 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:27:58.881448+0000", 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.291 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.292 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.293 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.294 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.295 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.911953+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.993550+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.993550+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.994090+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.911953+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.911953+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.911953+0000", 2024-03-20T08:01:12.296 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.911953+0000", 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.911953+0000", 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.911953+0000", 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.297 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.5", 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 69, 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:42:05.762120+0000", 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.298 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.299 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.300 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.301 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.302 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135969+0000", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135969+0000", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.136285+0000", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:12.303 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.304 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2", 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:41:16.328342+0000", 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.305 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.306 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.307 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.308 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.309 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.766341+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.766341+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.766643+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:12.310 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.311 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.3", 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000", 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.312 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.313 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.314 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.315 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 20, 2024-03-20T08:01:12.316 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970951+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:09.963927+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:09.963927+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:09.964395+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970951+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:56:04.165332+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:56:04.165332+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 59, 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970951+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970951+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970951+0000", 2024-03-20T08:01:12.317 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:56:04.165332+0000", 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970951+0000", 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970951+0000", 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 128, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 58, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 128, 2024-03-20T08:01:12.318 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "1.0", 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 266, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000", 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 590368, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.319 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 1771104, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:12.320 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 6, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.321 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 212, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 306, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 179, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 2416 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.322 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "68'128" 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.323 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.989129+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.989129+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.989392+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.324 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.325 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.6", 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:47:32.163295+0000", 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.326 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.327 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.328 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.329 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:12.330 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.125161+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.125161+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.125436+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:12.331 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1", 2024-03-20T08:01:12.332 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:53:55.085832+0000", 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.333 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.334 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.335 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.336 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.778258+0000", 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.778258+0000", 2024-03-20T08:01:12.337 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.779129+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.338 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.0", 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.339 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000", 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.340 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.341 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.342 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.343 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384551+0000", 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.994854+0000", 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.994854+0000", 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.995060+0000", 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384551+0000", 2024-03-20T08:01:12.344 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384551+0000", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384551+0000", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384551+0000", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384551+0000", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384551+0000", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.345 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.3", 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.346 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:48:00.138918+0000", 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.347 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.348 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.349 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.350 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.777666+0000", 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.777666+0000", 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.777973+0000", 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.351 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.352 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.5", 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000", 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.353 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.354 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.355 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.356 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.357 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.126397+0000", 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.126397+0000", 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.126692+0000", 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.358 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.359 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.4", 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000", 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.360 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.361 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.362 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.363 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:12.364 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.993283+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.993283+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.993571+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:12.365 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.366 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.0", 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 13, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:02:21.502002+0000", 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.367 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.368 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.369 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.370 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.371 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.769010+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.769010+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769337+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.372 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.373 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.6", 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000", 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.374 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.375 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.376 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 1, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "63'1" 2024-03-20T08:01:12.377 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:12.378 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118565+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118565+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.118863+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:12.379 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.7", 2024-03-20T08:01:12.380 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:49:33.981106+0000", 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.381 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.382 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.383 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.384 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.913562+0000", 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.988797+0000", 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.988797+0000", 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.989083+0000", 2024-03-20T08:01:12.385 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.913562+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.913562+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.913562+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.913562+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.913562+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.913562+0000", 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.386 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.1", 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 69, 2024-03-20T08:01:12.387 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:42:40.380304+0000", 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.388 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.389 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.390 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.391 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770737+0000", 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770737+0000", 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771097+0000", 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:12.392 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.393 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.7", 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:12.394 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000", 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.395 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.396 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.397 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.398 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:46.698841+0000", 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:46.698841+0000", 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:46.699298+0000", 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.399 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 65, 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.400 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 64, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.6", 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 17, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.401 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:02:06.660176+0000", 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.402 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.403 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.404 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.405 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.777649+0000", 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.777649+0000", 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.777961+0000", 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.406 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.407 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.8", 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000", 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.408 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.409 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.410 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.411 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:12.412 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.119966+0000", 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.119966+0000", 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.120305+0000", 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:12.413 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.414 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.9", 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.415 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000", 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.416 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.417 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.418 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.419 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.765273+0000", 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.765273+0000", 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.765548+0000", 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:12.420 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.421 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.9", 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.422 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000", 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.423 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.424 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.425 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.426 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.117376+0000", 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.117376+0000", 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.117669+0000", 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.427 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.428 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.8", 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:42:02.274133+0000", 2024-03-20T08:01:12.429 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.430 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.431 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.431 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.431 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.431 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.435 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.436 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.773520+0000", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.773520+0000", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.773811+0000", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:12.437 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.438 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.a", 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000", 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.439 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.442 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.443 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122680+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122680+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.123160+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:12.444 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.445 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.b", 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:09:56.675972+0000", 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.446 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.449 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.450 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:46.306388+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:46.306388+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:46.309368+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 65, 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:12.451 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 64, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.452 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.b", 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:42:19.060358+0000", 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.453 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:12.454 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 1, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.455 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 1, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:12.456 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:12.457 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.133784+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.133784+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.134067+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.458 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.459 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.a", 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000", 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.461 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.462 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.463 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.762682+0000", 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.762682+0000", 2024-03-20T08:01:12.464 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.763015+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.465 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.c", 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.466 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:21:45.137669+0000", 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.468 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.469 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.470 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.142148+0000", 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.142148+0000", 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.142445+0000", 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.471 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.472 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.d", 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.473 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000", 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.475 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.476 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.477 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.766111+0000", 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.766111+0000", 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.766427+0000", 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.478 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.479 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.d", 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000", 2024-03-20T08:01:12.480 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 70, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.482 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 1, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 1 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.483 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.484 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.134903+0000", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.134903+0000", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.135339+0000", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:12.485 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.486 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.c", 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000", 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.487 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.489 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.490 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:12.491 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.765948+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.765948+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.766233+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.492 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.493 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.e", 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:34:01.876341+0000", 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.496 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:12.497 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.143696+0000", 2024-03-20T08:01:12.498 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.143696+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.144008+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:12.499 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.f", 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.500 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000", 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.502 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.503 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.504 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.137804+0000", 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.137804+0000", 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.138096+0000", 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.505 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.506 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.20", 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.507 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000", 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.509 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.510 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.511 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.125793+0000", 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.125793+0000", 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.126085+0000", 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.512 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.513 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.21", 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.514 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:11:45.470755+0000", 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.516 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.517 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.518 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.124689+0000", 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.124689+0000", 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.125004+0000", 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.519 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.520 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.22", 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000", 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.521 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.523 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.524 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.525 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.119755+0000", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.119755+0000", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.120048+0000", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:12.526 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.527 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.23", 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:39:07.689571+0000", 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.529 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.530 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.531 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.532 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:46.699085+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:46.699085+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:46.699520+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 65, 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.533 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 64, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.534 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.24", 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 28, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:58:27.802971+0000", 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.536 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.537 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.538 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:12.539 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122635+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122635+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.126113+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:12.540 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.25", 2024-03-20T08:01:12.541 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000", 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.543 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.544 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.545 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.141099+0000", 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.141099+0000", 2024-03-20T08:01:12.546 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.141379+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.547 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.26", 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.548 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000", 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.550 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.551 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.552 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.121094+0000", 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.121094+0000", 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.121389+0000", 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.553 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.554 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.27", 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.555 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:08:53.389656+0000", 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.557 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.558 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.559 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.137344+0000", 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.137344+0000", 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.137638+0000", 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.560 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.561 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.28", 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.562 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000", 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.563 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.564 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.565 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.566 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122003+0000", 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122003+0000", 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.122341+0000", 2024-03-20T08:01:12.567 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.568 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.29", 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.569 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:18:20.158675+0000", 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.573 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.136135+0000", 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.136135+0000", 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.137117+0000", 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:12.574 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.575 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2a", 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.576 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000", 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.579 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.580 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.119047+0000", 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.119047+0000", 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.119480+0000", 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.582 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2b", 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.583 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:09:31.918661+0000", 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.587 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.132793+0000", 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.132793+0000", 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.133160+0000", 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:12.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:12.589 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2c", 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:12.590 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000", 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_per_pool": [ 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.594 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 24, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd" 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pg": 8, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 8, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "poolid": 4, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.595 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.596 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.597 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:12.598 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 24 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 192, 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pg": 64, 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 8, 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "poolid": 3, 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.601 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:12.602 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 0, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 192 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 3, 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr" 2024-03-20T08:01:12.603 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 128, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pg": 1, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 3, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 128, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "poolid": 1, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 590368, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 1771104, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 6, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 212, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 306, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 179, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 2416 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 1781760, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 1771104, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.607 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 3 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 96, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 45, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pg": 32, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 8, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 45, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "poolid": 2, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 3428, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.608 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 66, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 22, 2024-03-20T08:01:12.609 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 22, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 14, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 1, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.610 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 51, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 20 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 98304, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 10284, 2024-03-20T08:01:12.611 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 96 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "usage": { 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 105, 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "pools": 4, 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_by_class": { 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "ssd": { 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "total_avail_bytes": 767731326976, 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "total_bytes": 767960285184, 2024-03-20T08:01:12.612 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_bytes": 228958208, 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_raw_bytes": 228958208, 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_raw_ratio": 0.00029813809669576585 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout: "total_avail_bytes": 767731326976, 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout: "total_bytes": 767960285184, 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_bytes": 228958208 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout:} 2024-03-20T08:01:12.613 INFO:tasks.workunit.client.0.smithi118.stdout:Device report is generated separately. To see it run 'ceph telemetry show-device'. 2024-03-20T08:01:12.614 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph telemetry show-device 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: pgmap v72: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/9361953' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/9361953' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.615 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: pgmap v72: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/9361953' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/9361953' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.617 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "telemetry on", "license": "sharing-1-0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: pgmap v72: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "telemetry enable channel", "channels": ["perf"], "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/9361953' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/9361953' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/telemetry/last_opt_revision"}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.620 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:10 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:11 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:12.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:12.625 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:10 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout:{ 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout: "01f6c8a0-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout: "INTEL_SSDPED1D480GA_01f76148-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout: "20240320-075605": { 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout: "device": { 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout: "info_name": "/dev/nvme0n1", 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "/dev/nvme0n1", 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout: "protocol": "NVMe", 2024-03-20T08:01:12.625 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "nvme" 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "firmware_version": "E2010325", 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "host_id": "01f6c8a0-e690-11ee-b4ce-0cc47a8fd725", 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "json_format_version": [ 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "local_time": { 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "asctime": "Wed Mar 20 07:56:05 2024 UTC", 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "time_t": 1710921365 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "logical_block_size": 512, 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "model_name": "INTEL SSDPED1D480GA", 2024-03-20T08:01:12.626 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_controller_id": 0, 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_namespaces": [ 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "capacity": { 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "formatted_lba_size": 512, 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 1, 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "size": { 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.627 INFO:tasks.workunit.client.0.smithi118.stdout: "utilization": { 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_number_of_namespaces": 1, 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_pci_vendor": { 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 32902, 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "subsystem_id": 32902 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_add_log": { 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "Device stats": { 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_fw_downgrades": { 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.628 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "crc_error_count": { 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "end_to_end_error_detection_count": { 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "erase_fail_count": { 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.629 INFO:tasks.workunit.client.0.smithi118.stdout: "host_bytes_written": { 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 6488230 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "host_ctx_wear_used": { 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "media_bytes_read": { 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "nand_bytes_written": { 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.630 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_stat_indicator": { 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "pll_lock_loss_count": { 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "program_fail_count": { 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "re_alloc_sectr_cnt": { 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.631 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "retry_buffer_overflow_count": { 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "soft_ecc_err_rate": { 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "thermal_throttle_status": { 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "cnt": 0, 2024-03-20T08:01:12.632 INFO:tasks.workunit.client.0.smithi118.stdout: "pct": 0 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_host_reads": { 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 16 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_media_wear": { 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_timer": { 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1243878 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.633 INFO:tasks.workunit.client.0.smithi118.stdout: "unexp_power_loss": { 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: "wear_leveling": { 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: "avg": 0, 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: "max": 0, 2024-03-20T08:01:12.634 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0 2024-03-20T08:01:12.638 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.638 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.638 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.638 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel Smart log": "nvme0n1" 2024-03-20T08:01:12.638 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.638 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_log": { 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare": 100, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare_threshold": 0, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "controller_busy_time": 2533, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "critical_warning": 0, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_read": 94253558, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_written": 425212666, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "host_reads": 1503125893, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "host_writes": 9374739071, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "media_errors": 0, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_err_log_entries": 0, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "percentage_used": 0, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycles": 16440, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_hours": 27966, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": 38, 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: "unsafe_shutdowns": 5682 2024-03-20T08:01:12.639 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_vendor": "intel", 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycle_count": 16440, 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_time": { 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "hours": 27966 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "serial_number": "deleted", 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "smart_status": { 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme": { 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "passed": true 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl": { 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "argv": [ 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl", 2024-03-20T08:01:12.640 INFO:tasks.workunit.client.0.smithi118.stdout: "-x", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "--json=o", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "/dev/nvme0n1" 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "build_info": "(local build)", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "exit_status": 0, 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "output": [ 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.14.0-430.el9.x86_64] (local build)", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF INFORMATION SECTION ===", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "Model Number: INTEL SSDPED1D480GA", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "Serial Number: deleted", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Version: E2010325", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-03-20T08:01:12.641 INFO:tasks.workunit.client.0.smithi118.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller ID: 0", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Number of Namespaces: 1", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Size/Capacity: 480,103,981,056 [480 GB]", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Local Time is: Wed Mar 20 07:56:05 2024 UTC", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Updates (0x02): 1 Slot", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional Admin Commands (0x0007): Security Format Frmw_DL", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported Power States", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 18.00W - - 0 0 0 0 0 0", 2024-03-20T08:01:12.642 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 512 0 2", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF SMART DATA SECTION ===", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Critical Warning: 0x00", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Temperature: 38 Celsius", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare: 100%", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare Threshold: 0%", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Percentage Used: 0%", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Read: 94,253,558 [48.2 TB]", 2024-03-20T08:01:12.643 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Written: 425,212,666 [217 TB]", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Read Commands: 1,503,125,893", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Write Commands: 9,374,739,071", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller Busy Time: 2,533", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Power Cycles: 16,440", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Power On Hours: 27,966", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Unsafe Shutdowns: 5,682", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Media and Data Integrity Errors: 0", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information Log Entries: 0", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "No Errors Logged", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "" 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "platform_info": "x86_64-linux-5.14.0-430.el9.x86_64", 2024-03-20T08:01:12.644 INFO:tasks.workunit.client.0.smithi118.stdout: "svn_revision": "5022", 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: "version": [ 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": { 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: "current": 38 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: "user_capacity": { 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.645 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "01f8bd04-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "INTEL_SSDPEDMD400G4_01fa01b4-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "20240320-075607": { 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "device": { 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "info_name": "/dev/nvme0n1", 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "/dev/nvme0n1", 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "protocol": "NVMe", 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "nvme" 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "firmware_version": "8DV101H0", 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "host_id": "01f8bd04-e690-11ee-b4ce-0cc47a8fd725", 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: "json_format_version": [ 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.646 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "local_time": { 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "asctime": "Wed Mar 20 07:56:07 2024 UTC", 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "time_t": 1710921367 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "logical_block_size": 512, 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_controller_id": 0, 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_namespaces": [ 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "capacity": { 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "formatted_lba_size": 512, 2024-03-20T08:01:12.647 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 1, 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "size": { 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "utilization": { 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_number_of_namespaces": 1, 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_pci_vendor": { 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 32902, 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: "subsystem_id": 32902 2024-03-20T08:01:12.648 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_add_log": { 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "Device stats": { 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_fw_downgrades": { 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "crc_error_count": { 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "end_to_end_error_detection_count": { 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "erase_fail_count": { 2024-03-20T08:01:12.649 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "host_bytes_written": { 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 24021711 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "host_ctx_wear_used": { 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "media_bytes_read": { 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.650 INFO:tasks.workunit.client.0.smithi118.stdout: "nand_bytes_written": { 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 43196693 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_stat_indicator": { 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "pll_lock_loss_count": { 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "program_fail_count": { 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.651 INFO:tasks.workunit.client.0.smithi118.stdout: "re_alloc_sectr_cnt": { 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "retry_buffer_overflow_count": { 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "soft_ecc_err_rate": { 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "thermal_throttle_status": { 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:12.652 INFO:tasks.workunit.client.0.smithi118.stdout: "cnt": 0, 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "pct": 0 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_host_reads": { 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 65535 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_media_wear": { 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 64 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_timer": { 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 65535 2024-03-20T08:01:12.653 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "unexp_power_loss": { 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "wear_leveling": { 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 91, 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "avg": 2982, 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "max": 2997, 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 2966 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel Smart log": "nvme0n1" 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.654 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_log": { 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare": 100, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare_threshold": 10, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "controller_busy_time": 7, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "critical_warning": 0, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_read": 334126040, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_written": 1574286880, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "host_reads": 5210102632, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "host_writes": 26666187474, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "media_errors": 0, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_err_log_entries": 0, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "percentage_used": 9, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycles": 54960, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_hours": 52936, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": 27, 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: "unsafe_shutdowns": 284 2024-03-20T08:01:12.655 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_vendor": "intel", 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycle_count": 54960, 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_time": { 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "hours": 52936 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "serial_number": "deleted", 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "smart_status": { 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme": { 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "passed": true 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl": { 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "argv": [ 2024-03-20T08:01:12.656 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "-x", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "--json=o", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "/dev/nvme0n1" 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "build_info": "(local build)", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "exit_status": 0, 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "output": [ 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.14.0-430.el9.x86_64] (local build)", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF INFORMATION SECTION ===", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-03-20T08:01:12.657 INFO:tasks.workunit.client.0.smithi118.stdout: "Serial Number: deleted", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Version: 8DV101H0", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller ID: 0", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Number of Namespaces: 1", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Local Time is: Wed Mar 20 07:56:07 2024 UTC", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Updates (0x02): 1 Slot", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.658 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported Power States", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 512 0 2", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: " 1 - 512 8 2", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: " 2 - 512 16 2", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: " 3 - 4096 0 0", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: " 4 - 4096 8 0", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: " 5 - 4096 64 0", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: " 6 - 4096 128 0", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF SMART DATA SECTION ===", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-03-20T08:01:12.659 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Critical Warning: 0x00", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Temperature: 27 Celsius", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare: 100%", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare Threshold: 10%", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Percentage Used: 9%", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Read: 334,126,040 [171 TB]", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Written: 1,574,286,880 [806 TB]", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Read Commands: 5,210,102,632", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Write Commands: 26,666,187,474", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller Busy Time: 7", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Power Cycles: 54,960", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Power On Hours: 52,936", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Unsafe Shutdowns: 284", 2024-03-20T08:01:12.660 INFO:tasks.workunit.client.0.smithi118.stdout: "Media and Data Integrity Errors: 0", 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information Log Entries: 0", 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "No Errors Logged", 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "" 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "platform_info": "x86_64-linux-5.14.0-430.el9.x86_64", 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "svn_revision": "5022", 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "version": [ 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": { 2024-03-20T08:01:12.661 INFO:tasks.workunit.client.0.smithi118.stdout: "current": 27 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout: "user_capacity": { 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.662 INFO:tasks.workunit.client.0.smithi118.stdout:} 2024-03-20T08:01:12.663 INFO:tasks.workunit.client.0.smithi118.stderr:+ ceph telemetry show-all 2024-03-20T08:01:12.663 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:01:11] "GET /metrics HTTP/1.1" 200 33644 "" "Prometheus/2.43.0" 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout:{ 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "device_report": { 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "01f6c8a0-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "INTEL_SSDPED1D480GA_01f76148-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "20240320-075605": { 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "device": { 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "info_name": "/dev/nvme0n1", 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "/dev/nvme0n1", 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "protocol": "NVMe", 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "nvme" 2024-03-20T08:01:12.670 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "firmware_version": "E2010325", 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "host_id": "01f6c8a0-e690-11ee-b4ce-0cc47a8fd725", 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "json_format_version": [ 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "local_time": { 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "asctime": "Wed Mar 20 07:56:05 2024 UTC", 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "time_t": 1710921365 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "logical_block_size": 512, 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "model_name": "INTEL SSDPED1D480GA", 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_controller_id": 0, 2024-03-20T08:01:12.671 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_namespaces": [ 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "capacity": { 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "formatted_lba_size": 512, 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 1, 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "size": { 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "utilization": { 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:12.672 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_number_of_namespaces": 1, 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_pci_vendor": { 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 32902, 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "subsystem_id": 32902 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_add_log": { 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "Device stats": { 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_fw_downgrades": { 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.673 INFO:tasks.workunit.client.0.smithi118.stdout: "crc_error_count": { 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "end_to_end_error_detection_count": { 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "erase_fail_count": { 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "host_bytes_written": { 2024-03-20T08:01:12.674 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 6488230 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "host_ctx_wear_used": { 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "media_bytes_read": { 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "nand_bytes_written": { 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.675 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_stat_indicator": { 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "pll_lock_loss_count": { 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "program_fail_count": { 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "re_alloc_sectr_cnt": { 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.676 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "retry_buffer_overflow_count": { 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "soft_ecc_err_rate": { 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "thermal_throttle_status": { 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "cnt": 0, 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: "pct": 0 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.677 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_host_reads": { 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 16 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_media_wear": { 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_timer": { 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1243878 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.678 INFO:tasks.workunit.client.0.smithi118.stdout: "unexp_power_loss": { 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "wear_leveling": { 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "avg": 0, 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "max": 0, 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel Smart log": "nvme0n1" 2024-03-20T08:01:12.679 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_log": { 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare": 100, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare_threshold": 0, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "controller_busy_time": 2533, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "critical_warning": 0, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_read": 94253558, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_written": 425212666, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "host_reads": 1503125893, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "host_writes": 9374739071, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "media_errors": 0, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "num_err_log_entries": 0, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "percentage_used": 0, 2024-03-20T08:01:12.680 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycles": 16440, 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_hours": 27966, 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": 38, 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "unsafe_shutdowns": 5682 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_vendor": "intel", 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycle_count": 16440, 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_time": { 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "hours": 27966 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "serial_number": "deleted", 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "smart_status": { 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme": { 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.681 INFO:tasks.workunit.client.0.smithi118.stdout: "passed": true 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl": { 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "argv": [ 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl", 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "-x", 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "--json=o", 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "/dev/nvme0n1" 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "build_info": "(local build)", 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "exit_status": 0, 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "output": [ 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.14.0-430.el9.x86_64] (local build)", 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-03-20T08:01:12.682 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF INFORMATION SECTION ===", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Model Number: INTEL SSDPED1D480GA", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Serial Number: deleted", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Version: E2010325", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller ID: 0", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Number of Namespaces: 1", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Size/Capacity: 480,103,981,056 [480 GB]", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Local Time is: Wed Mar 20 07:56:05 2024 UTC", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Updates (0x02): 1 Slot", 2024-03-20T08:01:12.683 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional Admin Commands (0x0007): Security Format Frmw_DL", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported Power States", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 18.00W - - 0 0 0 0 0 0", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 512 0 2", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF SMART DATA SECTION ===", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.684 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Critical Warning: 0x00", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Temperature: 38 Celsius", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare: 100%", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare Threshold: 0%", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Percentage Used: 0%", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Read: 94,253,558 [48.2 TB]", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Written: 425,212,666 [217 TB]", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Read Commands: 1,503,125,893", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Write Commands: 9,374,739,071", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller Busy Time: 2,533", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Power Cycles: 16,440", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Power On Hours: 27,966", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Unsafe Shutdowns: 5,682", 2024-03-20T08:01:12.685 INFO:tasks.workunit.client.0.smithi118.stdout: "Media and Data Integrity Errors: 0", 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information Log Entries: 0", 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "No Errors Logged", 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "" 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "platform_info": "x86_64-linux-5.14.0-430.el9.x86_64", 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "svn_revision": "5022", 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "version": [ 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": { 2024-03-20T08:01:12.686 INFO:tasks.workunit.client.0.smithi118.stdout: "current": 38 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "user_capacity": { 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 937703088, 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480103981056 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "01f8bd04-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "INTEL_SSDPEDMD400G4_01fa01b4-e690-11ee-b4ce-0cc47a8fd725": { 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "20240320-075607": { 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "device": { 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "info_name": "/dev/nvme0n1", 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "/dev/nvme0n1", 2024-03-20T08:01:12.687 INFO:tasks.workunit.client.0.smithi118.stdout: "protocol": "NVMe", 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "nvme" 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "firmware_version": "8DV101H0", 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "host_id": "01f8bd04-e690-11ee-b4ce-0cc47a8fd725", 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "json_format_version": [ 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "local_time": { 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "asctime": "Wed Mar 20 07:56:07 2024 UTC", 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "time_t": 1710921367 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "logical_block_size": 512, 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "model_name": "INTEL SSDPEDMD400G4", 2024-03-20T08:01:12.688 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_controller_id": 0, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_ieee_oui_identifier": 6083300, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_namespaces": [ 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "capacity": { 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "formatted_lba_size": 512, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 1, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "size": { 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "utilization": { 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:12.689 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_number_of_namespaces": 1, 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_pci_vendor": { 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "id": 32902, 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "subsystem_id": 32902 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_add_log": { 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "Device stats": { 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_fw_downgrades": { 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.690 INFO:tasks.workunit.client.0.smithi118.stdout: "crc_error_count": { 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "end_to_end_error_detection_count": { 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "erase_fail_count": { 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "host_bytes_written": { 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 24021711 2024-03-20T08:01:12.691 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "host_ctx_wear_used": { 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 1 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "media_bytes_read": { 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "nand_bytes_written": { 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 43196693 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_stat_indicator": { 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.692 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "pll_lock_loss_count": { 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "program_fail_count": { 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "re_alloc_sectr_cnt": { 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "retry_buffer_overflow_count": { 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.693 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "soft_ecc_err_rate": { 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "thermal_throttle_status": { 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "cnt": 0, 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "pct": 0 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_host_reads": { 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.694 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 65535 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_media_wear": { 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 64 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "timed_workload_timer": { 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 100, 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 65535 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "unexp_power_loss": { 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 0, 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": 0 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.695 INFO:tasks.workunit.client.0.smithi118.stdout: "wear_leveling": { 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "normalized": 91, 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "raw": { 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "avg": 2982, 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "max": 2997, 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 2966 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel Smart log": "nvme0n1" 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_smart_health_information_log": { 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare": 100, 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "available_spare_threshold": 10, 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "controller_busy_time": 7, 2024-03-20T08:01:12.696 INFO:tasks.workunit.client.0.smithi118.stdout: "critical_warning": 0, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_read": 334126040, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "data_units_written": 1574286880, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "host_reads": 5210102632, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "host_writes": 26666187474, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "media_errors": 0, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_err_log_entries": 0, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "percentage_used": 9, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycles": 54960, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_hours": 52936, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": 27, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "unsafe_shutdowns": 284 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme_vendor": "intel", 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "power_cycle_count": 54960, 2024-03-20T08:01:12.697 INFO:tasks.workunit.client.0.smithi118.stdout: "power_on_time": { 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "hours": 52936 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "serial_number": "deleted", 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "smart_status": { 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "nvme": { 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "passed": true 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl": { 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "argv": [ 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl", 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "-x", 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "--json=o", 2024-03-20T08:01:12.698 INFO:tasks.workunit.client.0.smithi118.stdout: "/dev/nvme0n1" 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "build_info": "(local build)", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "exit_status": 0, 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "output": [ 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.14.0-430.el9.x86_64] (local build)", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF INFORMATION SECTION ===", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "Model Number: INTEL SSDPEDMD400G4", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "Serial Number: deleted", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Version: 8DV101H0", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "PCI Vendor/Subsystem ID: 0x8086", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "IEEE OUI Identifier: 0x5cd2e4", 2024-03-20T08:01:12.699 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller ID: 0", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Number of Namespaces: 1", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Size/Capacity: 400,088,457,216 [400 GB]", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Namespace 1 Formatted LBA Size: 512", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Local Time is: Wed Mar 20 07:56:07 2024 UTC", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Firmware Updates (0x02): 1 Slot", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional Admin Commands (0x0006): Format Frmw_DL", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Optional NVM Commands (0x0006): Wr_Unc DS_Mngmt", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Maximum Data Transfer Size: 32 Pages", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported Power States", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 25.00W - - 0 0 0 0 0 0", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Supported LBA Sizes (NSID 0x1)", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: "Id Fmt Data Metadt Rel_Perf", 2024-03-20T08:01:12.700 INFO:tasks.workunit.client.0.smithi118.stdout: " 0 + 512 0 2", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: " 1 - 512 8 2", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: " 2 - 512 16 2", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: " 3 - 4096 0 0", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: " 4 - 4096 8 0", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: " 5 - 4096 64 0", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: " 6 - 4096 128 0", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "=== START OF SMART DATA SECTION ===", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART overall-health self-assessment test result: PASSED", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "SMART/Health Information (NVMe Log 0x02)", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "Critical Warning: 0x00", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "Temperature: 27 Celsius", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare: 100%", 2024-03-20T08:01:12.701 INFO:tasks.workunit.client.0.smithi118.stdout: "Available Spare Threshold: 10%", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Percentage Used: 9%", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Read: 334,126,040 [171 TB]", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Data Units Written: 1,574,286,880 [806 TB]", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Read Commands: 5,210,102,632", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Host Write Commands: 26,666,187,474", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Controller Busy Time: 7", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Power Cycles: 54,960", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Power On Hours: 52,936", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Unsafe Shutdowns: 284", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Media and Data Integrity Errors: 0", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information Log Entries: 0", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "Error Information (NVMe Log 0x01, max 64 entries)", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "No Errors Logged", 2024-03-20T08:01:12.702 INFO:tasks.workunit.client.0.smithi118.stdout: "" 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: "platform_info": "x86_64-linux-5.14.0-430.el9.x86_64", 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: "svn_revision": "5022", 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: "version": [ 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: "temperature": { 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: "current": 27 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: "user_capacity": { 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: "blocks": 781422768, 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 400088457216 2024-03-20T08:01:12.703 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "report": { 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "balancer": { 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "active": true, 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "mode": "" 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "channels": [ 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "basic", 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "crash", 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "device", 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "ident", 2024-03-20T08:01:12.704 INFO:tasks.workunit.client.0.smithi118.stdout: "perf" 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "channels_available": [ 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "basic", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "ident", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "crash", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "device", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "perf" 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "collections_available": [ 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_base", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "device_base", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "crash_base", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "ident_base", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_perf", 2024-03-20T08:01:12.705 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_mds_metadata", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_usage", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_usage_by_class", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_rook_v01", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_memory_metrics", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_options_bluestore" 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "collections_opted_in": [ 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_base", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "device_base", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "crash_base", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "ident_base", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_perf", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_mds_metadata", 2024-03-20T08:01:12.706 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_usage", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_usage_by_class", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_rook_v01", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_memory_metrics", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "basic_pool_options_bluestore" 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "config": { 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "active_changed": [ 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_allow_insecure_global_id_reclaim", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_debug", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_mon_ticket_ttl", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_service_ticket_ttl", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "bdev_debug_aio", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "client_check_pool_perm", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "client_mount_gid", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "client_mount_uid", 2024-03-20T08:01:12.707 INFO:tasks.workunit.client.0.smithi118.stdout: "client_quota", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "container_image", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "daemonize", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_asserts_on_shutdown", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mgr", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mon", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_ms", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_osd", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_paxos", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "keyring", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_block_size", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_cache_size", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_compression", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_log", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "leveldb_write_buffer_size", 2024-03-20T08:01:12.708 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_file", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_journald", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_stderr", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_allow_pool_delete", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_clock_drift_allowed", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_file_level", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_file", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_journald", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_stderr", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_data_avail_warn", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_election_default_strategy", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_host", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_max_pg_per_osd", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_mgr_mkfs_grace", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_prime_pg_temp", 2024-03-20T08:01:12.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_reporter_subtree_level", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_pg_warn_max_object_skew", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_bytes_per_osd", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_pgs_per_osd", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_legacy_crush_tunables", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_too_few_osds", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_bug", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_old_message", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "no_config_file", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_default_list", 2024-03-20T08:01:12.710 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_load_list", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_crush_chooseleaf_type", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_misdirected_ops", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_op_order", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_pg_log_writeout", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_shutdown", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_cached_snaps", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_missing_on_start", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_stray_on_activate", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_deep_scrub_update_digest_min_age", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_profile", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_lim", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_res", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_best_effort_wgt", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_lim", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_res", 2024-03-20T08:01:12.711 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_background_recovery_wgt", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_lim", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_res", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_scheduler_client_wgt", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target_autotune", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue_cut_off", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_erasure_code_profile", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_pg_autoscale_mode", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recover_clone_overlap", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recovery_max_chunk", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_load_threshold", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_max_interval", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_shutdown_pgref_assert", 2024-03-20T08:01:12.712 INFO:tasks.workunit.client.0.smithi118.stdout: "public_addr", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "public_addrv", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "public_network", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd_default_features", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd_qos_exclude_ops", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "setgroup", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "setuser" 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster_changed": [ 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_allow_insecure_global_id_reclaim", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_debug", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_mon_ticket_ttl", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "auth_service_ticket_ttl", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "bdev_debug_aio", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "container_image", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_asserts_on_shutdown", 2024-03-20T08:01:12.713 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mgr", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_mon", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_ms", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_osd", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "debug_paxos", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_file", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_journald", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "log_to_stderr", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_join_fs", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/allow_ptrace", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/container_init", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/migration_current", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/cephadm/use_repo_digest", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/ALERTMANAGER_API_HOST", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/GRAFANA_API_SSL_VERIFY", 2024-03-20T08:01:12.714 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/GRAFANA_API_URL", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/PROMETHEUS_API_HOST", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/dashboard/ssl_server_port", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/orchestrator/orchestrator", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/channel_ident", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/channel_perf", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/description", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/enabled", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr/telemetry/organization", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_allow_pool_delete", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_clock_drift_allowed", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_file_level", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_cluster_log_to_file", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_data_avail_warn", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_election_default_strategy", 2024-03-20T08:01:12.715 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_max_pg_per_osd", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_mgr_mkfs_grace", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_prime_pg_temp", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_osd_reporter_subtree_level", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_pg_warn_max_object_skew", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_bytes_per_osd", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_reweight_min_pgs_per_osd", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_crush_straw_calc_version_zero", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_legacy_crush_tunables", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_osd_down_out_interval_zero", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_pool_pg_num_not_power_of_two", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mon_warn_on_too_few_osds", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_bug", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "ms_die_on_old_message", 2024-03-20T08:01:12.716 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_default_list", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_class_load_list", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_misdirected_ops", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_op_order", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_pg_log_writeout", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_shutdown", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_cached_snaps", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_missing_on_start", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_debug_verify_stray_on_activate", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_deep_scrub_update_digest_min_age", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mclock_profile", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_memory_target_autotune", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_op_queue_cut_off", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_erasure_code_profile", 2024-03-20T08:01:12.717 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pool_default_pg_autoscale_mode", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recover_clone_overlap", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_recovery_max_chunk", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_load_threshold", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_scrub_max_interval", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_shutdown_pgref_assert", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "public_network", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_cache_enabled", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_enable_ops_log", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "rgw_enable_usage_log" 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "contact": null, 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "crashes": [], 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "created": "2024-03-20T07:52:36.029987Z", 2024-03-20T08:01:12.718 INFO:tasks.workunit.client.0.smithi118.stdout: "crush": { 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_algs": { 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "straw2": 3 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_sizes": { 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "2": 1, 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "4": 2 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket_types": { 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "1": 2, 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "11": 1 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "compat_weight_set": false, 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: "device_classes": [ 2024-03-20T08:01:12.719 INFO:tasks.workunit.client.0.smithi118.stdout: 8 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_buckets": 6, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_devices": 8, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_rules": 1, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_types": 12, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_weight_sets": 0, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "tunables": { 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "allowed_bucket_algs": 54, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_local_fallback_tries": 0, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_local_tries": 0, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "choose_total_tries": 50, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_descend_once": 1, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_stable": 1, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "chooseleaf_vary_r": 1, 2024-03-20T08:01:12.720 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v2_rules": 0, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v3_rules": 0, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v4_buckets": 1, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "has_v5_rules": 0, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "legacy_tunables": 0, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "minimum_required_version": "jewel", 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "optimal_tunables": 1, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "profile": "jewel", 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables": 1, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables2": 1, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables3": 1, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "require_feature_tunables5": 1, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "straw_calc_version": 1 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.721 INFO:tasks.workunit.client.0.smithi118.stdout: "description": "upgrade test cluster", 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "fs": { 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "feature_flags": { 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "enable_multiple": true, 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "ever_enabled_multiple": true 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "filesystems": [ 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "approx_ctime": "2024-03", 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "balancer_enabled": false, 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_caps": 0, 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_dns": 10, 2024-03-20T08:01:12.722 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_inos": 13, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "cached_subtrees": 2, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "ever_allowed_features": 0, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "explicitly_allowed_features": 0, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "files": 0, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "max_mds": 1, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "num_data_pools": 1, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "num_in": 1, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "num_mds": 1, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": 0, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "num_standby_replay": 0, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "num_up": 1, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "snaps": 0, 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: "standby_count_wanted": 0 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.723 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "num_standby_mds": 1, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "total_num_mds": 2 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_stats": { 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 19218432, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 344744, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 188416, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 0, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 309, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 946792, 2024-03-20T08:01:12.724 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 12, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 528896, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 14457072, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 19218432 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 20234240, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 399544, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 32768, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 383, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 1206352, 2024-03-20T08:01:12.725 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 20, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 964608, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 14911224, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 20267008 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 43909120, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 1453360, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 540672, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 3645440, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1745, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.726 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 12313616, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 22, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2565376, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 24283584, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 47554560 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.38b88dde2c420bbdcde70c0aa08a86a8e933770b": { 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 35856384, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 867232, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 860160, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 4456448, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1093, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 7956432, 2024-03-20T08:01:12.727 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 22, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2140160, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 21279888, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 40312832 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 34725888, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 1259928, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2752512, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 745472, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 6627328, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1156, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 7878800, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 21, 2024-03-20T08:01:12.728 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1295616, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 20793560, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 41353216 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.0": { 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 46137344, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 839384, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 1097728, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1360, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4868896, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 46, 2024-03-20T08:01:12.729 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2572032, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34973448, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 47235072 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.1": { 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 45481984, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 840960, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 835584, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1285, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 5344888, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 47, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1913344, 2024-03-20T08:01:12.730 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34499208, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 46317568 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.2": { 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 43704320, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 930336, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 385024, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1223, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4712464, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 47, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2202368, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 32975568, 2024-03-20T08:01:12.731 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 44089344 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.3": { 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 44007424, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 659736, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 212992, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1198, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4860104, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 45, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1549056, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34054944, 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 44220416 2024-03-20T08:01:12.732 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.4": { 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 44810240, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 549344, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 458752, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1194, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4838920, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 48, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1750016, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34788376, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 45268992 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.733 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.5": { 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 45047808, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 763280, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 0, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 221184, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1214, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4682760, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 48, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 2014208, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34703976, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 45268992 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.734 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.6": { 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 44744704, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 890608, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 16384, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 524288, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1195, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4577464, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 46, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1786880, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34589784, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 45268992 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.7": { 2024-03-20T08:01:12.735 INFO:tasks.workunit.client.0.smithi118.stdout: "actual_memory_used": 45031424, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "central_cache_freelist": 1180312, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "malloc_metadata": 2883584, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "page_heap_freelist": 24576, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "released_to_os": 1286144, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "spans_in_use": 1244, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "tcmalloc_page_size": 8192, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_cache_freelists": 4849032, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "thread_heaps_in_use": 47, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "transfer_cache_freelist": 1689344, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "use_by_application": 34404576, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "virtual_address_space_used": 46317568 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "hosts": { 2024-03-20T08:01:12.736 INFO:tasks.workunit.client.0.smithi118.stdout: "num": 2, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mds": 2, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mgr": 2, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_mon": 2, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "num_with_osd": 2 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "io_rate": { 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_stats_delta": { 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 0, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 0, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "stamp_delta": "12.002299", 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:12.737 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:12.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:12.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 182, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 278, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 40, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:12.740 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 0, 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 0 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "leaderboard": false, 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "leaderboard_description": null, 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "license": "sharing-1-0", 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "mempool": { 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.741 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.742 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.743 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.744 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.745 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 149384, 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 31 2024-03-20T08:01:12.746 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 928, 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 2 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.747 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.748 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.749 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.750 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.751 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.752 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.753 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 221639, 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 65 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.754 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 51496, 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 208 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.755 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 7736, 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 121 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.756 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.757 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.758 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.759 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.760 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.761 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1161259, 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 360 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.762 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 440, 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 5 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.763 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6312, 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 120 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12744, 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1546 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 10240, 2024-03-20T08:01:12.764 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 158 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.38b88dde2c420bbdcde70c0aa08a86a8e933770b": { 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.765 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.766 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.767 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.768 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.769 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1355275, 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 419 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.770 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.771 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6312, 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 120 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12744, 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1546 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5120, 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 79 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.772 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.773 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.774 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.775 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.776 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.777 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1106267, 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 355 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.778 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.779 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6312, 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 120 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12744, 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1546 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5120, 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 79 2024-03-20T08:01:12.780 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.0": { 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.781 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5520, 2024-03-20T08:01:12.782 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 69 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 21168, 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 189 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1712, 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 45 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 606208, 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 17 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 51400, 2024-03-20T08:01:12.783 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9559 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 138296, 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 235 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 11412, 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 746 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9072, 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 189 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.784 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 31878, 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 203 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 21168, 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 189 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.785 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 433520, 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 97 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2460034, 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 199 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6864, 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 78 2024-03-20T08:01:12.786 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 521640, 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 45 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 94640, 2024-03-20T08:01:12.787 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 186 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224104, 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3444 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.788 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.1": { 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:12.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4880, 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 61 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 19824, 2024-03-20T08:01:12.790 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 177 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1264, 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 41 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16384, 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 4 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 45678, 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8647 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.791 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 129456, 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 222 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9480, 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 572 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8496, 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 177 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.792 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 27946, 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 190 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 19824, 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 177 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 736, 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 426003, 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 95 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.793 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12112, 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2442762, 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 185 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5016, 2024-03-20T08:01:12.794 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 57 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 440496, 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 38 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.795 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 7280, 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 18 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224104, 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3444 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.2": { 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.797 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4240, 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 53 2024-03-20T08:01:12.798 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 19600, 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 175 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1040, 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 39 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8192, 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 2 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 44148, 2024-03-20T08:01:12.799 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8377 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 124696, 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 215 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8320, 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 519 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8400, 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 175 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.800 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 26490, 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 183 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 19600, 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 175 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 736, 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 1 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.801 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 384192, 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 85 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 12112, 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2414420, 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 176 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4400, 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 50 2024-03-20T08:01:12.802 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 359352, 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 31 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9360, 2024-03-20T08:01:12.803 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 26 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224616, 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3468 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.804 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.3": { 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:12.805 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 3920, 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 49 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.806 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20160, 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 180 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480, 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 34 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 44616, 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8559 2024-03-20T08:01:12.807 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 128096, 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 220 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9348, 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 578 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8640, 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 180 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.808 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 26930, 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 188 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20160, 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 180 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2208, 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.809 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 257376, 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 54 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 26565, 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2421390, 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 178 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4664, 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 53 2024-03-20T08:01:12.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 417312, 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 36 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 10400, 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 20 2024-03-20T08:01:12.811 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224104, 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3444 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.812 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.4": { 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.813 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 3360, 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 42 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20048, 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:12.814 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1376, 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 42 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 14419, 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 45568, 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8686 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 130136, 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 223 2024-03-20T08:01:12.815 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9648, 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 578 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8592, 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 27866, 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 191 2024-03-20T08:01:12.816 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20160, 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 180 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1472, 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 2 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 921744, 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 74 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20128, 2024-03-20T08:01:12.817 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 5 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2451826, 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 185 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4928, 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 56 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.818 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 452088, 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 39 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 7904, 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 16 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224360, 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3456 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.819 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.820 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.5": { 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:12.821 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2800, 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 35 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20048, 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 480, 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 34 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.822 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 45840, 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8752 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 131496, 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 225 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 9888, 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 581 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.823 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8592, 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 28378, 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 193 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20048, 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 179 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.824 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2208, 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 833264, 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 52 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 26565, 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2420414, 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 183 2024-03-20T08:01:12.825 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5368, 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 61 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 475272, 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 41 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.826 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 11960, 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 23 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224616, 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3468 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.827 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.6": { 2024-03-20T08:01:12.828 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.829 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2320, 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 29 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20944, 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 256, 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 32 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.830 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 46199, 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9008 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 130136, 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 223 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 10892, 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 736 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8976, 2024-03-20T08:01:12.831 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 29062, 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 191 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20944, 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.832 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 808329, 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 46 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2424628, 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 174 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.833 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 4664, 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 53 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 452088, 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 39 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.834 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 94952, 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 193 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224872, 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3480 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.835 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.7": { 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "bloom_filter": { 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.836 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 6368, 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 123 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_reader": { 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 16896, 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 8 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs_file_writer": { 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 576, 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_alloc": { 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1760, 2024-03-20T08:01:12.837 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 22 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_blob": { 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20944, 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_buffer": { 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 1152, 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 40 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.838 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_data": { 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 281259, 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_meta": { 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 48524, 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 9330 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_onode": { 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 135576, 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 231 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_cache_other": { 2024-03-20T08:01:12.842 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 11676, 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 743 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_extent": { 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 8976, 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_fsck": { 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_inline_bl": { 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 31110, 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 199 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.843 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_shared_blob": { 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 20944, 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 187 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_txc": { 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing": { 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 145316, 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 29 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore_writing_deferred": { 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.844 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_anon": { 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 2426662, 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 200 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "buffer_meta": { 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 5896, 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 67 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_co": { 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 533232, 2024-03-20T08:01:12.845 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 46 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_mapbl": { 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_pglog": { 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 96824, 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 223 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap": { 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 224104, 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 3444 2024-03-20T08:01:12.846 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap_mapping": { 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "pgmap": { 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_1": { 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "unittest_2": { 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes": 0, 2024-03-20T08:01:12.847 INFO:tasks.workunit.client.0.smithi118.stdout: "items": 0 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: "metadata": { 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 2 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 2 2024-03-20T08:01:12.848 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 2 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 2 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 2 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 2 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 2 2024-03-20T08:01:12.849 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 3 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 3 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 3 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.850 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 3 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 3 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 3 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 3 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.851 INFO:tasks.workunit.client.0.smithi118.stdout: "arch": { 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "x86_64": 8 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_version": { 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "cpu": { 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz": 8 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "distro": { 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "centos": 8 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "distro_description": { 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "CentOS Stream 8": 8 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_description": { 2024-03-20T08:01:12.852 INFO:tasks.workunit.client.0.smithi118.stdout: "#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024": 8 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: "kernel_version": { 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: "5.14.0-430.el9.x86_64": 8 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: "os": { 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: "Linux": 8 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_objectstore": { 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": 8 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: "rotational": { 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: "0": 8 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.853 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "features": { 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "optional": [], 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "persistent": [ 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "kraken", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "luminous", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "mimic", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "osdmap-prune", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "nautilus", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "octopus", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "pacific", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "elector-pinging", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "quincy", 2024-03-20T08:01:12.854 INFO:tasks.workunit.client.0.smithi118.stdout: "reef" 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "ipv4_addr_mons": 6, 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "ipv6_addr_mons": 0, 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "min_mon_release": 18, 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "v1_addr_mons": 3, 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "v2_addr_mons": 3 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "organization": "ceph-qa", 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster_network": false, 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 8, 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "require_min_compat_client": "luminous", 2024-03-20T08:01:12.855 INFO:tasks.workunit.client.0.smithi118.stdout: "require_osd_release": "reef" 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_perf_histograms": [ 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "axes": [ 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Latency (usec)", 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 100000, 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 99999]", 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "[100000, 199999]", 2024-03-20T08:01:12.856 INFO:tasks.workunit.client.0.smithi118.stdout: "[200000, 399999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[400000, 799999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[800000, 1599999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[1600000, 3199999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[3200000, 6399999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[6400000, 12799999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[12800000, 25599999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[25600000, 51199999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[51200000, 102399999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[102400000, 204799999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[204800000, 409599999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[409600000, 819199999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[819200000, 1638399999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[1638400000, 3276799999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[3276800000, 6553599999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[6553600000, 13107199999]", 2024-03-20T08:01:12.857 INFO:tasks.workunit.client.0.smithi118.stdout: "[13107200000, 26214399999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[26214400000, 52428799999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[52428800000, 104857599999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[104857600000, 209715199999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[209715200000, 419430399999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[419430400000, 838860799999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[838860800000, 1677721599999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[1677721600000, 3355443199999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[3355443200000, 6710886399999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[6710886400000, 13421772799999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[13421772800000, 26843545599999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[26843545600000, 53687091199999]", 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "[53687091200000, None]" 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:12.858 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Request size (bytes)", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 512, 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 511]", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[512, 1023]", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[1024, 2047]", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[2048, 4095]", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[4096, 8191]", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[8192, 16383]", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[16384, 32767]", 2024-03-20T08:01:12.859 INFO:tasks.workunit.client.0.smithi118.stdout: "[32768, 65535]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[65536, 131071]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[131072, 262143]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[262144, 524287]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[524288, 1048575]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[1048576, 2097151]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[2097152, 4194303]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[4194304, 8388607]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[8388608, 16777215]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[16777216, 33554431]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[33554432, 67108863]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[67108864, 134217727]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[134217728, 268435455]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[268435456, 536870911]", 2024-03-20T08:01:12.860 INFO:tasks.workunit.client.0.smithi118.stdout: "[536870912, 1073741823]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[1073741824, 2147483647]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[2147483648, 4294967295]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[4294967296, 8589934591]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[8589934592, 17179869183]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[17179869184, 34359738367]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[34359738368, 68719476735]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[68719476736, 137438953471]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[137438953472, 274877906943]", 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "[274877906944, None]" 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "osds": [ 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.861 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 0, 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.862 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.863 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 1, 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.864 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.865 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.866 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 2, 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.867 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.868 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 3, 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.869 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.870 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.871 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 4, 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.872 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.873 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 5, 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.874 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.875 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.876 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 6, 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.877 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.878 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 7, 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 97, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 5, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.879 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.880 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.881 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "axes": [ 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Latency (usec)", 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 100000, 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 99999]", 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "[100000, 199999]", 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "[200000, 399999]", 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "[400000, 799999]", 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "[800000, 1599999]", 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "[1600000, 3199999]", 2024-03-20T08:01:12.882 INFO:tasks.workunit.client.0.smithi118.stdout: "[3200000, 6399999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[6400000, 12799999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[12800000, 25599999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[25600000, 51199999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[51200000, 102399999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[102400000, 204799999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[204800000, 409599999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[409600000, 819199999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[819200000, 1638399999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[1638400000, 3276799999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[3276800000, 6553599999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[6553600000, 13107199999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[13107200000, 26214399999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[26214400000, 52428799999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[52428800000, 104857599999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[104857600000, 209715199999]", 2024-03-20T08:01:12.883 INFO:tasks.workunit.client.0.smithi118.stdout: "[209715200000, 419430399999]", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "[419430400000, 838860799999]", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "[838860800000, 1677721599999]", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "[1677721600000, 3355443199999]", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "[3355443200000, 6710886399999]", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "[6710886400000, 13421772799999]", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "[13421772800000, 26843545599999]", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "[26843545600000, 53687091199999]", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "[53687091200000, None]" 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Request size (bytes)", 2024-03-20T08:01:12.884 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 512, 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 511]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[512, 1023]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[1024, 2047]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[2048, 4095]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[4096, 8191]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[8192, 16383]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[16384, 32767]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[32768, 65535]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[65536, 131071]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[131072, 262143]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[262144, 524287]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[524288, 1048575]", 2024-03-20T08:01:12.885 INFO:tasks.workunit.client.0.smithi118.stdout: "[1048576, 2097151]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[2097152, 4194303]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[4194304, 8388607]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[8388608, 16777215]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[16777216, 33554431]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[33554432, 67108863]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[67108864, 134217727]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[134217728, 268435455]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[268435456, 536870911]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[536870912, 1073741823]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[1073741824, 2147483647]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[2147483648, 4294967295]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[4294967296, 8589934591]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[8589934592, 17179869183]", 2024-03-20T08:01:12.886 INFO:tasks.workunit.client.0.smithi118.stdout: "[17179869184, 34359738367]", 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "[34359738368, 68719476735]", 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "[68719476736, 137438953471]", 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "[137438953472, 274877906943]", 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "[274877906944, None]" 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "osds": [ 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 0, 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.887 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.888 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.889 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 1, 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.890 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.891 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 2, 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.892 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.893 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.894 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 3, 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.895 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.896 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 4, 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.897 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.898 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.899 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 5, 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.900 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.901 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 6, 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.902 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.903 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.904 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 7, 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.905 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.906 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "axes": [ 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Latency (usec)", 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 100000, 2024-03-20T08:01:12.907 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 99999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[100000, 199999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[200000, 399999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[400000, 799999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[800000, 1599999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[1600000, 3199999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[3200000, 6399999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[6400000, 12799999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[12800000, 25599999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[25600000, 51199999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[51200000, 102399999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[102400000, 204799999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[204800000, 409599999]", 2024-03-20T08:01:12.908 INFO:tasks.workunit.client.0.smithi118.stdout: "[409600000, 819199999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[819200000, 1638399999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[1638400000, 3276799999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[3276800000, 6553599999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[6553600000, 13107199999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[13107200000, 26214399999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[26214400000, 52428799999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[52428800000, 104857599999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[104857600000, 209715199999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[209715200000, 419430399999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[419430400000, 838860799999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[838860800000, 1677721599999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[1677721600000, 3355443199999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[3355443200000, 6710886399999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[6710886400000, 13421772799999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[13421772800000, 26843545599999]", 2024-03-20T08:01:12.909 INFO:tasks.workunit.client.0.smithi118.stdout: "[26843545600000, 53687091199999]", 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "[53687091200000, None]" 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Request size (bytes)", 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 512, 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 511]", 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "[512, 1023]", 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "[1024, 2047]", 2024-03-20T08:01:12.910 INFO:tasks.workunit.client.0.smithi118.stdout: "[2048, 4095]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[4096, 8191]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[8192, 16383]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[16384, 32767]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[32768, 65535]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[65536, 131071]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[131072, 262143]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[262144, 524287]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[524288, 1048575]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[1048576, 2097151]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[2097152, 4194303]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[4194304, 8388607]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[8388608, 16777215]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[16777216, 33554431]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[33554432, 67108863]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[67108864, 134217727]", 2024-03-20T08:01:12.911 INFO:tasks.workunit.client.0.smithi118.stdout: "[134217728, 268435455]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[268435456, 536870911]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[536870912, 1073741823]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[1073741824, 2147483647]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[2147483648, 4294967295]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[4294967296, 8589934591]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[8589934592, 17179869183]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[17179869184, 34359738367]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[34359738368, 68719476735]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[68719476736, 137438953471]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[137438953472, 274877906943]", 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "[274877906944, None]" 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.912 INFO:tasks.workunit.client.0.smithi118.stdout: "osds": [ 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 0, 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.913 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.914 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 1, 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.915 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.916 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 2, 2024-03-20T08:01:12.917 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.918 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.919 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 3, 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.920 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.921 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 4, 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.922 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.923 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.924 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 5, 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.925 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.926 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 6, 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.927 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.928 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.929 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 7, 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.930 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.931 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: "axes": [ 2024-03-20T08:01:12.932 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Latency (usec)", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 100000, 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 99999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[100000, 199999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[200000, 399999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[400000, 799999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[800000, 1599999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[1600000, 3199999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[3200000, 6399999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[6400000, 12799999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[12800000, 25599999]", 2024-03-20T08:01:12.933 INFO:tasks.workunit.client.0.smithi118.stdout: "[25600000, 51199999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[51200000, 102399999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[102400000, 204799999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[204800000, 409599999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[409600000, 819199999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[819200000, 1638399999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[1638400000, 3276799999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[3276800000, 6553599999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[6553600000, 13107199999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[13107200000, 26214399999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[26214400000, 52428799999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[52428800000, 104857599999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[104857600000, 209715199999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[209715200000, 419430399999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[419430400000, 838860799999]", 2024-03-20T08:01:12.934 INFO:tasks.workunit.client.0.smithi118.stdout: "[838860800000, 1677721599999]", 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "[1677721600000, 3355443199999]", 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "[3355443200000, 6710886399999]", 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "[6710886400000, 13421772799999]", 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "[13421772800000, 26843545599999]", 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "[26843545600000, 53687091199999]", 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "[53687091200000, None]" 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "buckets": 32, 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "min": 0, 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "name": "Request size (bytes)", 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "quant_size": 512, 2024-03-20T08:01:12.935 INFO:tasks.workunit.client.0.smithi118.stdout: "ranges": [ 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[None, -1]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 511]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[512, 1023]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[1024, 2047]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[2048, 4095]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[4096, 8191]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[8192, 16383]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[16384, 32767]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[32768, 65535]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[65536, 131071]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[131072, 262143]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[262144, 524287]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[524288, 1048575]", 2024-03-20T08:01:12.936 INFO:tasks.workunit.client.0.smithi118.stdout: "[1048576, 2097151]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[2097152, 4194303]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[4194304, 8388607]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[8388608, 16777215]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[16777216, 33554431]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[33554432, 67108863]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[67108864, 134217727]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[134217728, 268435455]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[268435456, 536870911]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[536870912, 1073741823]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[1073741824, 2147483647]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[2147483648, 4294967295]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[4294967296, 8589934591]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[8589934592, 17179869183]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[17179869184, 34359738367]", 2024-03-20T08:01:12.937 INFO:tasks.workunit.client.0.smithi118.stdout: "[34359738368, 68719476735]", 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "[68719476736, 137438953471]", 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "[137438953472, 274877906943]", 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "[274877906944, None]" 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "scale_type": "log2" 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "osds": [ 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 0, 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.938 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.939 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.940 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 1, 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 4, 5, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 17, 3, 0, 0, 0, 0, 0, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 14, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 5, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.941 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.942 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 2, 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.943 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.944 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.945 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 3, 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.946 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.947 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 4, 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.948 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.949 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 5, 2024-03-20T08:01:12.950 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.951 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.952 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 6, 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.953 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 5, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.954 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "osd_id": 7, 2024-03-20T08:01:12.955 INFO:tasks.workunit.client.0.smithi118.stdout: "values": [ 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.956 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.957 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]", 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: "[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]" 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: ] 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: "perf_counters": { 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: "mds.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: "mds": { 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: "caps": { 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_flush_ack": { 2024-03-20T08:01:12.958 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_flushsnap_ack": { 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_grant": { 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_revoke": { 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "ceph_cap_op_trunc": { 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_commit": { 2024-03-20T08:01:12.959 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_fetch_complete": { 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_fetch_keys": { 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_merge": { 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "dir_split": { 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "exported_inodes": { 2024-03-20T08:01:12.960 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "forward": { 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_cap_release": { 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_caps": { 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_caps_dirty": { 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_inode_file_caps": { 2024-03-20T08:01:12.961 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "imported_inodes": { 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "inodes": { 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "inodes_expired": { 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "inodes_pinned": { 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "inodes_with_caps": { 2024-03-20T08:01:12.962 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "load_cent": { 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "openino_dir_fetch": { 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "process_request_cap_release": { 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "reply_latency": { 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 117733 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.963 INFO:tasks.workunit.client.0.smithi118.stdout: "request": { 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "root_rbytes": { 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "root_rfiles": { 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "root_rsnaps": { 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_reply": { 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "subtrees": { 2024-03-20T08:01:12.964 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_cache": { 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_enqueue_scrub": { 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_exportdir": { 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_flush": { 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_fragmentdir": { 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.965 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_fragstats": { 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "ireq_inodestats": { 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_recovering_enqueued": { 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_recovering_prioritized": { 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_recovering_processing": { 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_strays": { 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_strays_delayed": { 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_strays_enqueuing": { 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_completed": { 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_started": { 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.967 INFO:tasks.workunit.client.0.smithi118.stdout: "strays_created": { 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "strays_enqueued": { 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "strays_migrated": { 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "strays_reintegrated": { 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_log": { 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "ev": { 2024-03-20T08:01:12.968 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "evadd": { 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "evex": { 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "evexd": { 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "evexg": { 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "evtrm": { 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "jlat": { 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11582695 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "replayed": { 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "seg": { 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "segadd": { 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "segex": { 2024-03-20T08:01:12.970 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "segexd": { 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "segexg": { 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "segtrm": { 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_mem": { 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "cap": { 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.971 INFO:tasks.workunit.client.0.smithi118.stdout: "cap+": { 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "cap-": { 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "dir": { 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "dir+": { 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "dir-": { 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.972 INFO:tasks.workunit.client.0.smithi118.stdout: "dn": { 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "dn+": { 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "dn-": { 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "heap": { 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 207156 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "ino": { 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.973 INFO:tasks.workunit.client.0.smithi118.stdout: "ino+": { 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "ino-": { 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "rss": { 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41064 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_server": { 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "cap_acquisition_throttle": { 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "cap_revoke_eviction": { 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.974 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_request": { 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_client_session": { 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "handle_peer_request": { 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "req_create_latency": { 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "req_getattr_latency": { 2024-03-20T08:01:12.975 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 117733 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "req_getfilelock_latency": { 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "req_getvxattr_latency": { 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "req_link_latency": { 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.976 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookup_latency": { 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookuphash_latency": { 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookupino_latency": { 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookupname_latency": { 2024-03-20T08:01:12.977 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookupparent_latency": { 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lookupsnap_latency": { 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "req_lssnap_latency": { 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.978 INFO:tasks.workunit.client.0.smithi118.stdout: "req_mkdir_latency": { 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "req_mknod_latency": { 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "req_mksnap_latency": { 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "req_open_latency": { 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.979 INFO:tasks.workunit.client.0.smithi118.stdout: "req_readdir_latency": { 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "req_rename_latency": { 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "req_renamesnap_latency": { 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "req_rmdir_latency": { 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.980 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "req_rmsnap_latency": { 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "req_rmxattr_latency": { 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setattr_latency": { 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setdirlayout_latency": { 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.981 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setfilelock_latency": { 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setlayout_latency": { 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "req_setxattr_latency": { 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "req_snapdiff_latency": { 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.982 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "req_symlink_latency": { 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "req_unlink_latency": { 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "mds_sessions": { 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "average_load": { 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.983 INFO:tasks.workunit.client.0.smithi118.stdout: "avg_session_uptime": { 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 86 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "mdthresh_evicted": { 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "session_add": { 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "session_count": { 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "session_remove": { 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.984 INFO:tasks.workunit.client.0.smithi118.stdout: "sessions_open": { 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "sessions_stale": { 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "total_load": { 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.985 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "purge_queue": { 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executed": { 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executing": { 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.986 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executing_high_water": { 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executing_ops": { 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_executing_ops_high_water": { 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "pq_item_in_journal": { 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.07b2eeb97b85886e62d94a02fd3c62a47dd17075": { 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:12.987 INFO:tasks.workunit.client.0.smithi118.stdout: "election_call": { 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "election_lose": { 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "election_win": { 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_elections": { 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": { 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 9 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.988 INFO:tasks.workunit.client.0.smithi118.stdout: "session_add": { 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "session_rm": { 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 203 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "session_trim": { 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "paxos": { 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "accept_timeout": { 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "begin": { 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1013 2024-03-20T08:01:12.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_bytes": { 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4969471 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_keys": { 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3039 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_latency": { 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 149996696 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "collect": { 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_bytes": { 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_keys": { 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_latency": { 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_timeout": { 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.991 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_uncommitted": { 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "commit": { 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1013 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_bytes": { 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4896530 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_keys": { 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6142 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_latency": { 2024-03-20T08:01:12.992 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1013, 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 135294402 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_ack_timeout": { 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_timeout": { 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn": { 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn_latency": { 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.993 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh": { 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1014 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh_latency": { 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1014, 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2393947205 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "restart": { 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state": { 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_bytes": { 2024-03-20T08:01:12.994 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1627 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_keys": { 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "start_leader": { 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "start_peon": { 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state": { 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.995 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_bytes": { 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_keys": { 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_latency": { 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache": { 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:12.996 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1020054731 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42696704 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41639936 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2147483648 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1056768 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:full": { 2024-03-20T08:01:12.997 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277021748 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 272544 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.998 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:12.999 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70832940 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:inc": { 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:13.000 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277021748 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39156 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.001 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.002 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71066328 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:kv": { 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 322961408 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.003 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 184681165 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.004 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.005 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71171379 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb": { 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "compact": { 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_len": { 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.006 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_merge": { 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_range": { 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "get_latency": { 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 45061, 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 174568618 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_delay_time": { 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_memtable_time": { 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.007 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_pre_and_post_time": { 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_wal_time": { 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_latency": { 2024-03-20T08:01:13.008 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_sync_latency": { 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2188, 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 283123799 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.38b88dde2c420bbdcde70c0aa08a86a8e933770b": { 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "election_call": { 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "election_lose": { 2024-03-20T08:01:13.009 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "election_win": { 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_elections": { 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": { 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "session_add": { 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "session_rm": { 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 108 2024-03-20T08:01:13.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "session_trim": { 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "paxos": { 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "accept_timeout": { 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "begin": { 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 861 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_bytes": { 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 861, 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3908052 2024-03-20T08:01:13.011 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_keys": { 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_latency": { 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 861, 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 99997793 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "collect": { 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_bytes": { 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.012 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_keys": { 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_latency": { 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_timeout": { 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_uncommitted": { 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "commit": { 2024-03-20T08:01:13.013 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 863 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_bytes": { 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_keys": { 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_latency": { 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_ack_timeout": { 2024-03-20T08:01:13.014 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_timeout": { 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn": { 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn_latency": { 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh": { 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 862 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.015 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh_latency": { 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 862, 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1942957114 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "restart": { 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 9 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state": { 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_bytes": { 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.016 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_keys": { 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "start_leader": { 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "start_peon": { 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state": { 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 862 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_bytes": { 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 862, 2024-03-20T08:01:13.017 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7654511 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_keys": { 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 862, 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6187 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_latency": { 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 862, 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 100997771 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache": { 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1020054731 2024-03-20T08:01:13.018 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 37560320 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 32014336 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2147483648 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5545984 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:full": { 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.019 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277020672 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 272974 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.020 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.021 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70833586 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:inc": { 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:13.022 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277020672 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41595 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.023 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.024 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71064965 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:kv": { 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 322961408 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.025 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 184680448 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.026 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.027 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71172096 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb": { 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "compact": { 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_len": { 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_merge": { 2024-03-20T08:01:13.028 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_range": { 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "get_latency": { 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 36967, 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 97362230 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_delay_time": { 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_memtable_time": { 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.029 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_pre_and_post_time": { 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_wal_time": { 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_latency": { 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_sync_latency": { 2024-03-20T08:01:13.030 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1882, 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 218751490 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "mon.84ea43e663705362e87a3b4ccae9b1e4c390d05c": { 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "election_call": { 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "election_lose": { 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "election_win": { 2024-03-20T08:01:13.031 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_elections": { 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "num_sessions": { 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "session_add": { 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "session_rm": { 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 54 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.032 INFO:tasks.workunit.client.0.smithi118.stdout: "session_trim": { 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "paxos": { 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "accept_timeout": { 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "begin": { 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 872 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_bytes": { 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 872, 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3934058 2024-03-20T08:01:13.033 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_keys": { 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "begin_latency": { 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 872, 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 147996580 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "collect": { 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_bytes": { 2024-03-20T08:01:13.034 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 146 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_keys": { 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_latency": { 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2999931 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_timeout": { 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.035 INFO:tasks.workunit.client.0.smithi118.stdout: "collect_uncommitted": { 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "commit": { 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 873 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_bytes": { 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_keys": { 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.036 INFO:tasks.workunit.client.0.smithi118.stdout: "commit_latency": { 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_ack_timeout": { 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "lease_timeout": { 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn": { 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "new_pn_latency": { 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.037 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh": { 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 872 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "refresh_latency": { 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 872, 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1850957235 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "restart": { 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state": { 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.038 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_bytes": { 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "share_state_keys": { 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "start_leader": { 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "start_peon": { 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.039 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state": { 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 873 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_bytes": { 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 873, 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7704236 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_keys": { 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 873, 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6271 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "store_state_latency": { 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 872, 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 411990485 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.040 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache": { 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1020054731 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 38600704 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31891456 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2147483648 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.041 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6709248 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:full": { 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277020672 2024-03-20T08:01:13.042 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 272974 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.046 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.047 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70833586 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:inc": { 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 348127232 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.048 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 277020672 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41595 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.049 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.050 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71064965 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "prioritycache:kv": { 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 322961408 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.051 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 184680448 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.052 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.053 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71172096 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb": { 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "compact": { 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_len": { 2024-03-20T08:01:13.054 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_queue_merge": { 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_range": { 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "get_latency": { 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 37030, 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 90088410 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_delay_time": { 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.055 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_memtable_time": { 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_pre_and_post_time": { 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_write_wal_time": { 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.056 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_latency": { 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "submit_sync_latency": { 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1909, 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 640777132 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.0": { 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.057 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.058 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2695168 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:13.059 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1691648 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1552384 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:13.060 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:13.061 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:13.062 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:13.063 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.064 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4325376 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.065 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1327104 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 4, 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 170270 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.066 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 24, 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 62270 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:13.067 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 366, 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70407745 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 354, 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16996591 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 366, 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 329151 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.068 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 366, 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 70736896 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45555 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 6, 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 20853 2024-03-20T08:01:13.069 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3744 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 840 2024-03-20T08:01:13.070 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 232 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 127, 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1998302 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 337, 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 323361 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 210, 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1123631 2024-03-20T08:01:13.071 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 54317 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6564867 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 124, 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7784781 2024-03-20T08:01:13.072 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 124, 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 521250265386 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 124, 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 136141582685 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 75591708725 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 80378 2024-03-20T08:01:13.073 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 484372 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 321504765 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 160390 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.074 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 340754259 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22170005 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 902807 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 58, 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 315443 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 691545409 2024-03-20T08:01:13.075 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 26926652 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 482, 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2895425 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861614376 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43302912 2024-03-20T08:01:13.076 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43270144 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137553817 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 32768 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.077 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85470711 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.078 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 606208 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.079 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73306633 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:13.080 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641030334 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.081 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.082 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71999218 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.083 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 56980474 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.084 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.085 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68848182 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.086 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641030334 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2434 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.087 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8522 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 236189 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.088 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71754201 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.089 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:13.090 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 616 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 47439688 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.091 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 6, 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2828505 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 46131616 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.092 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.093 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.094 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.095 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 616 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 47439688 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 6, 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2828505 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.096 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 46131616 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1771104 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:13.097 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28991488 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.1": { 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:13.098 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2478080 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:13.099 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1593344 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.100 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1454080 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.101 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:13.102 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:13.103 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:13.104 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.105 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4009984 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 724992 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:13.106 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 177365 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.107 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 32, 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 79482 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 342, 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67561207 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 336, 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6917055 2024-03-20T08:01:13.108 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 342, 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 337848 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 342, 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67899055 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39705 2024-03-20T08:01:13.109 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6453 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.110 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 19546 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 817 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 238 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 135, 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2004843 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 351, 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 271377 2024-03-20T08:01:13.111 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 216, 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1237767 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 113239 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5882558 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.112 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 122, 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7330347 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 121, 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 238160647650 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 122, 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 127118562449 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 407, 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 78619356867 2024-03-20T08:01:13.113 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 407, 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 47751 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 462679 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 116750324 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.114 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 171796 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 54487495 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 20599737 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 311871 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 56, 2024-03-20T08:01:13.115 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 302498 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 198260189 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24707843 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 408, 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2711976 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:13.116 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861614376 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43433984 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42557440 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137553817 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 876544 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.117 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85507548 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.118 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.119 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16384 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73859620 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.120 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641306617 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.121 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.122 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71722935 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:13.123 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005032 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.124 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.125 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68823624 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:13.126 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641306617 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.127 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 221846 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.128 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71503217 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.129 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 38 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.130 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 155 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2429440 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 155, 2024-03-20T08:01:13.131 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 460591849 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1063 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 155, 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 52345283 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 155, 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 175170212 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 33 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.132 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4721579 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1063 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4291312 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:13.133 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3475180 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 22, 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 18159084 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:13.134 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 22, 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8782803 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 22, 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 17140728 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.135 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 100 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2429440 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 100, 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 437711186 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 100, 2024-03-20T08:01:13.136 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39271168 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 100, 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 154554304 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 590368 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:13.137 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28393472 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.2": { 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.138 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.139 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2179072 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:13.140 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1396736 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1257472 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.141 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:13.142 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:13.143 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:13.144 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.145 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3514368 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:13.146 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 716800 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 4, 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 208631 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:13.147 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16180 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.148 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 294, 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45079777 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 291, 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5480823 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 294, 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 274060 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 294, 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45353837 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.149 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 66222 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.150 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4777 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 603 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 218 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.151 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 105, 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1001837 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 284, 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 267058 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 179, 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 620834 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:13.152 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 105884 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6328852 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 115, 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7580165 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 114, 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 251284937020 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:13.153 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 115, 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 125451047575 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 332, 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 75591994867 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 332, 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 40804 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 405323 2024-03-20T08:01:13.154 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 66615035 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 121287 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22209760 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.155 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16602756 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 311711 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 50, 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 220614 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 112212520 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 20252338 2024-03-20T08:01:13.156 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 333, 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2052063 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861614376 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41205760 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 40730624 2024-03-20T08:01:13.157 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137553817 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 475136 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.158 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508620 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.159 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8192 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.160 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73866740 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.161 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641314656 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.162 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.163 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71714896 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.164 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005747 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.165 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68822909 2024-03-20T08:01:13.166 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641314656 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.167 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.168 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 212396 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71504628 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:13.169 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:13.170 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.171 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43961563 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4254833 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 37719144 2024-03-20T08:01:13.172 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:13.173 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:13.174 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.175 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43961563 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.176 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4254833 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 10, 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 37719144 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.177 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28385280 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.3": { 2024-03-20T08:01:13.178 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.179 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1097728 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.180 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 806912 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:13.181 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 667648 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:13.182 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.183 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5503 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.184 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5503 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.185 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.186 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1843200 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:13.187 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 737280 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.188 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4750 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 150, 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31275215 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.189 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 142, 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6610022 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 150, 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 183193 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 150, 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31458408 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.190 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 37578 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.191 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5509 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 331 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 220 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 81, 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 689093 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.192 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 140, 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 220849 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 59, 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 158052 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:13.193 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3448519 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 53, 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3874964 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 50, 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 364806141004 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 53, 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 82325193252 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.194 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 229, 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1051069 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 229, 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24281 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 237514 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 108852609 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.195 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65960 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 55127102 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13895819 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 314445 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.196 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 44, 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 196403 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 181588503 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16190214 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 232, 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1442795 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.197 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861614376 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41336832 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41017344 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137553817 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.198 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 319488 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.199 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508744 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.200 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73874808 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.201 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641315582 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.202 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:13.203 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71713970 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.204 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005829 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.205 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.206 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68822827 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:13.207 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641315582 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.208 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8222 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 212617 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.209 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71495259 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.210 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 36 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:13.211 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:13.212 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 826294 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.213 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.214 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.215 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.216 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 826294 2024-03-20T08:01:13.217 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:13.218 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28401664 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.4": { 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.219 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:13.220 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1187840 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.221 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 860160 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 720896 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.222 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.223 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:13.224 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.225 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.226 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1986560 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:13.227 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 733184 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 8, 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 242713 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.228 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5202 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 163, 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43413114 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.229 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 151, 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5378069 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 163, 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 160270 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 163, 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43573384 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.230 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 45245 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.231 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5053 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 364 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 238 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 97, 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1057146 2024-03-20T08:01:13.232 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 163, 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 284482 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 66, 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 318485 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 113090 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.233 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3567685 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 42, 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3376401 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 40, 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 174076583361 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 42, 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 59910774836 2024-03-20T08:01:13.234 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 260, 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 503702 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 260, 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 29659 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 293576 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159332190 2024-03-20T08:01:13.235 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 147796 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73221492 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12492767 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 312419 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:13.236 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 41, 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 196260 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 249010406 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14540473 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 262, 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2062376 2024-03-20T08:01:13.237 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861620469 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42385408 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41762816 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137560985 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.238 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 622592 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.239 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508320 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.240 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8192 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73867040 2024-03-20T08:01:13.241 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641312403 2024-03-20T08:01:13.242 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.243 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71717149 2024-03-20T08:01:13.244 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005547 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.245 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.246 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68823109 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:13.247 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641312403 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.248 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.249 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 220872 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71498405 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.250 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.251 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.252 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16535570 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 661769 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:13.253 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16484502 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:13.254 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.255 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.256 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 22 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16535570 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.257 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 661769 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 16484502 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.258 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28405760 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.5": { 2024-03-20T08:01:13.259 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.260 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1232896 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.261 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 921600 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 782336 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:13.262 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:13.263 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:13.264 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5505 2024-03-20T08:01:13.265 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.266 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.267 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2093056 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 733184 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 6, 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 175599 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:13.268 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5021 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.269 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 178, 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 51957555 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 166, 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7715370 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 178, 2024-03-20T08:01:13.270 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 129777 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 178, 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 52087332 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 54355 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.271 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4809 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 332 2024-03-20T08:01:13.272 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 245 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 91, 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 751067 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 146, 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 252293 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 55, 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 358628 2024-03-20T08:01:13.273 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 4, 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 97849 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 7601436 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 39, 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2879924 2024-03-20T08:01:13.274 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 36, 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 228721521403 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 39, 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 61505489997 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 264, 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 550360 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 264, 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 29626 2024-03-20T08:01:13.275 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2052106 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 138184191 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 81539 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 72999945 2024-03-20T08:01:13.276 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13459162 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 314431 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 32, 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 143346 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 234329599 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.277 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15679879 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2258519 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861620469 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42385408 2024-03-20T08:01:13.278 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42090496 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137560985 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 294912 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.279 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508729 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.280 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.281 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73874823 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.282 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641315470 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.283 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.284 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71714082 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.285 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005819 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.286 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.287 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68822837 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641315470 2024-03-20T08:01:13.288 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5883 2024-03-20T08:01:13.289 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 221230 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.290 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71489097 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.291 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.292 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.293 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.294 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.295 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.296 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:13.297 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:13.298 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28397568 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.299 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.6": { 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:13.300 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1212416 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:13.301 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 901120 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 761856 2024-03-20T08:01:13.302 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.303 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.304 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5504 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:13.305 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5504 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:13.306 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.307 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2052096 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1318912 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.308 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 3, 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5762 2024-03-20T08:01:13.309 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 173, 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 50155890 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 162, 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13230003 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:13.310 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 173, 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 107893 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 173, 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 50263783 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41526 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.311 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4040 2024-03-20T08:01:13.312 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 346 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 252 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 91, 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 658685 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 139, 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 253902 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.313 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 48, 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 413906 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5516491 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.314 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2487452 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 286186428277 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 33, 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57902364509 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 457537 2024-03-20T08:01:13.315 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28096 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 217281 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 157391827 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 82856 2024-03-20T08:01:13.316 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 86936491 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11926400 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 904378 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 26, 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 109000 2024-03-20T08:01:13.317 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 262024284 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13704624 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 267, 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2500277 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:13.318 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861620469 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42385408 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 41852928 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137560985 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 532480 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.319 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85508571 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.320 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.321 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73874981 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.322 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641314284 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.323 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.324 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71715268 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.325 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 57005714 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.326 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.327 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68822942 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.328 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641314284 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2398 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3597 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.329 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5995 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 217044 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.330 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71488362 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.331 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 39 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.332 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2810 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 183194403 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:13.333 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 15, 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6348347 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 101979001 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.334 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:13.335 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.336 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2810 2024-03-20T08:01:13.337 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 183194403 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 15, 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 6348347 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 13, 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 101979001 2024-03-20T08:01:13.338 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28983296 2024-03-20T08:01:13.339 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "osd.7": { 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "bluefs": { 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_fallback": { 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_slow_size_fallback": { 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_db": { 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 65536 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_main": { 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.340 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit_wal": { 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_slow": { 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_sst": { 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12288 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_written_wal": { 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1171456 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lat": { 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.341 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "compact_lock_lat": { 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "db_total_bytes": { 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "db_used_bytes": { 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27656192 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "log_bytes": { 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 897024 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.342 INFO:tasks.workunit.client.0.smithi118.stdout: "logged_bytes": { 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 757760 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_db": { 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 31850496 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_slow": { 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "max_bytes_wal": { 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "num_files": { 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.343 INFO:tasks.workunit.client.0.smithi118.stdout: "read_bytes": { 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 206272 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "read_count": { 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 23 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes": { 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1282048 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_db": { 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1286144 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_slow": { 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.344 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_bytes_wal": { 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "read_disk_count": { 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 15 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_bytes": { 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4913 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "read_prefetch_count": { 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_bytes": { 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5500 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_buffer_count": { 2024-03-20T08:01:13.345 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_bytes": { 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5500 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_count": { 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes": { 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_db": { 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_slow": { 2024-03-20T08:01:13.346 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_bytes_wal": { 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "read_random_disk_count": { 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_total_bytes": { 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "slow_used_bytes": { 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_total_bytes": { 2024-03-20T08:01:13.347 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "wal_used_bytes": { 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "write_bytes": { 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2007040 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore": { 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "alloc_unit": { 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 4096 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": { 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1318912 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.348 INFO:tasks.workunit.client.0.smithi118.stdout: "clist_lat": { 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 169944 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_lat": { 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed": { 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_allocated": { 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "compressed_original": { 2024-03-20T08:01:13.349 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "csum_lat": { 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 9, 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 24419 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "decompress_lat": { 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_commit_lat": { 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 172, 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43241157 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.350 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_final_lat": { 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 169, 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 13917366 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_flush_lat": { 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 172, 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85721 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "kv_sync_lat": { 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 172, 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43326878 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_keys_lat": { 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.351 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_get_values_lat": { 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 36703 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_lower_bound_lat": { 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 1, 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 521 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_next_lat": { 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_seek_to_first_lat": { 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 0, 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.352 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_upper_bound_lat": { 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3672 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_hits": { 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 611 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "onode_misses": { 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 281 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "read_lat": { 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 119, 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2240047 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.353 INFO:tasks.workunit.client.0.smithi118.stdout: "read_onode_meta_lat": { 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 184, 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 257395 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "read_wait_aio_lat": { 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 65, 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1587663 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "reads_with_retries": { 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "remove_lat": { 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 2, 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 46406 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "state_aio_wait_lat": { 2024-03-20T08:01:13.354 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3499437 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_aio_wait_lat": { 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 20, 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1769811 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_cleanup_lat": { 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 20, 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 320320280624 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "state_deferred_queued_lat": { 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 20, 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 32150620865 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.355 INFO:tasks.workunit.client.0.smithi118.stdout: "state_done_lat": { 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 460087 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "state_finishing_lat": { 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 30874 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "state_io_done_lat": { 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192908 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_commiting_lat": { 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 125816192 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.356 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_done_lat": { 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 87588 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "state_kv_queued_lat": { 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 92684749 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "state_prepare_lat": { 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12703064 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": { 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 902683 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.357 INFO:tasks.workunit.client.0.smithi118.stdout: "truncate_lat": { 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 19, 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 80640 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_commit_lat": { 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 234931211 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_submit_lat": { 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 14508821 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "txc_throttle_lat": { 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 286, 2024-03-20T08:01:13.358 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1755632 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache": { 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_bytes": { 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1861620469 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "heap_bytes": { 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 43433984 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "mapped_bytes": { 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 42041344 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "target_bytes": { 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3137560985 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.359 INFO:tasks.workunit.client.0.smithi118.stdout: "unmapped_bytes": { 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 1392640 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:data": { 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 159383552 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.360 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 85491415 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8192 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 131072 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.361 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 135168 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 73617705 2024-03-20T08:01:13.362 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv": { 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 780140544 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641185617 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.363 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 2128 2024-03-20T08:01:13.364 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71843935 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:kv_onode": { 2024-03-20T08:01:13.365 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 192937984 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 67108864 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 464 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 56994277 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.366 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.367 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 68834379 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "bluestore-pricache:meta": { 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "committed_bytes": { 2024-03-20T08:01:13.368 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 713031680 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "pri0_bytes": { 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "pri10_bytes": { 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "pri11_bytes": { 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 641185617 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "pri1_bytes": { 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.369 INFO:tasks.workunit.client.0.smithi118.stdout: "pri2_bytes": { 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3622 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "pri3_bytes": { 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8451 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "pri4_bytes": { 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "pri5_bytes": { 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "pri6_bytes": { 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 228192 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.370 INFO:tasks.workunit.client.0.smithi118.stdout: "pri7_bytes": { 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "pri8_bytes": { 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "pri9_bytes": { 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "reserved_bytes": { 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 71605798 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "objecter": { 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "op_active": { 2024-03-20T08:01:13.371 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rmw": { 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "osd": { 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg": { 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 46 2024-03-20T08:01:13.372 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "numpg_removing": { 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "op": { 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 115 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_degraded": { 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "op_delayed_unreadable": { 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "op_in_bytes": { 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.373 INFO:tasks.workunit.client.0.smithi118.stdout: "op_latency": { 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 115, 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 157172478 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "op_out_bytes": { 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 265081 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "op_prepare_latency": { 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 117, 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 25625752 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "op_process_latency": { 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 115, 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 110571394 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.374 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r": { 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 80 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency": { 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 80, 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 11409892 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_latency_out_bytes_histogram": { 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_out_bytes": { 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 265081 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_prepare_latency": { 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 80, 2024-03-20T08:01:13.375 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 10167534 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "op_r_process_latency": { 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 80, 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 8408064 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw": { 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 30 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_in_bytes": { 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency": { 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 30, 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 27269213 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.376 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_in_bytes_histogram": { 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_latency_out_bytes_histogram": { 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_out_bytes": { 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_prepare_latency": { 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 30, 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 12349121 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "op_rw_process_latency": { 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 30, 2024-03-20T08:01:13.377 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 25638026 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w": { 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 5 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_in_bytes": { 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency": { 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 118493373 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_latency_in_bytes_histogram": { 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_prepare_latency": { 2024-03-20T08:01:13.378 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 7, 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 3109097 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "op_w_process_latency": { 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "count": 5, 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 76525304 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "op_wip": { 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_bytes": { 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "recovery_ops": { 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 0 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.379 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes": { 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 95995035648 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_bytes_used": { 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "value": 28983296 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "pools": [ 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr" 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:13.380 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 1, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 1, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 1, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330329866, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 1781760, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 1781760, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110117376, 2024-03-20T08:01:13.381 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 2, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 2.443002131258254e-06, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 394, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 598016, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 590368, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 590368, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 1771104, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 219, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 2473984 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.382 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 32, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 32, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 2, 2024-03-20T08:01:13.383 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330329866, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 98304, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 98304, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110117376, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 22, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 1.3478663163368765e-07, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:13.384 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 3428, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 3428, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 10284, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 51, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 20480 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:13.385 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 64, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 64, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 3, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330329866, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 0, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:13.386 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110117376, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 0.0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 0, 2024-03-20T08:01:13.387 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 0 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd" 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "cache_mode": "none", 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "erasure_code_profile": {}, 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "min_size": 2, 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "options": {}, 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_autoscale_mode": "off", 2024-03-20T08:01:13.388 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 8, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "pgp_num": 8, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "pool": 4, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "size": 3, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "stats": { 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_raw": 729330329866, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "bytes_used": 0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_bytes_used": 0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "compress_under_bytes": 0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "data_bytes_used": 0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty": 0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "max_avail": 243110117376, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "objects": 0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_bytes_used": 0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "percent_used": 0.0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_bytes": 0, 2024-03-20T08:01:13.389 INFO:tasks.workunit.client.0.smithi118.stdout: "quota_objects": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "rd": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "rd_bytes": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "stored": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_data": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_omap": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "stored_raw": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "wr": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "wr_bytes": 0 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_bytes": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "target_max_objects": 0, 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: "type": "replicated" 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.390 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd": { 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "mirroring_by_pool": [ 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: false 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "num_images_by_pool": [ 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pools": 1 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "report_id": "f7bee648-d925-4013-8791-48f92fa7019f", 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "report_timestamp": "2024-03-20T08:01:12.215469", 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "report_version": 1, 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "rocksdb_stats": { 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "7.9.2" 2024-03-20T08:01:13.391 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "rook": { 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "cluster": { 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "external-mode": null, 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "mon": { 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "allow-multiple-per-node": null, 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "count": null, 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "max-id": null, 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "pvc": { 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "enabled": null 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "stretch": { 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "enabled": null 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.392 INFO:tasks.workunit.client.0.smithi118.stdout: "network": { 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "provider": null 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "storage": { 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "device-set": { 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "non-portable": null, 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "portable": null, 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "total": null 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "csi": { 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.393 INFO:tasks.workunit.client.0.smithi118.stdout: "kubernetes": { 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "node": { 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "kubernetes-total": null, 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "with-ceph-daemons": null, 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-cephfs-plugin": null, 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-nfs-plugin": null, 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "with-csi-rbd-plugin": null 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "usage": { 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "storage-class": { 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "count": { 2024-03-20T08:01:13.394 INFO:tasks.workunit.client.0.smithi118.stdout: "bucket": null, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs": null, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: "nfs": null, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd": null, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: "total": null 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: "version": null 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: "services": {}, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_per_pg": [ 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.395 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.145286+0000", 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.145286+0000", 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.145662+0000", 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.396 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971733+0000", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.397 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3f", 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.398 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:33:33.527736+0000", 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.399 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.400 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.401 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.402 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128811+0000", 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128811+0000", 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.129069+0000", 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.403 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442684+0000", 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.404 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3e", 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:13:37.252664+0000", 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.405 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.406 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.407 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.408 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.409 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118755+0000", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118755+0000", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.119030+0000", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:13.410 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385973+0000", 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.411 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3d", 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:51:41.823570+0000", 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.412 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.413 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.414 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.415 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.416 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.137768+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.137768+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.138067+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:13.417 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384811+0000", 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.418 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3c", 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:59:00.878820+0000", 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.419 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.420 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.421 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.422 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.423 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.131585+0000", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.131585+0000", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.131923+0000", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:13.424 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971266+0000", 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.425 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3b", 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:14:50.895187+0000", 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.426 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.427 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.428 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.429 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.430 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135727+0000", 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135727+0000", 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.137308+0000", 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.431 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384791+0000", 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.432 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3a", 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:38:04.570635+0000", 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.433 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.434 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.435 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.436 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.437 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.132230+0000", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.132230+0000", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.132941+0000", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:13.438 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971041+0000", 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.439 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.39", 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:29:49.550246+0000", 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.440 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.441 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.442 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.443 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.444 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135920+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135920+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.136216+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:13.445 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437110+0000", 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.446 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.38", 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:48:37.134454+0000", 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.447 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.448 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.449 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.450 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.451 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.453 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.454 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.454 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.454 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.454 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.454 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.454 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.454 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.144302+0000", 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.144302+0000", 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.144631+0000", 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.455 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971692+0000", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.456 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.37", 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:45:28.551677+0000", 2024-03-20T08:01:13.457 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.458 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.459 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.460 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.461 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.138348+0000", 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.138348+0000", 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.138657+0000", 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.462 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595833+0000", 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.463 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.36", 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:08:43.311428+0000", 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.464 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.465 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.466 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.467 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.468 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.137672+0000", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.137672+0000", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.137961+0000", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:13.469 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595417+0000", 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.470 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.35", 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:54:39.328709+0000", 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.471 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.472 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.473 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.474 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.475 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135935+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135935+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.136246+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.476 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.188872+0000", 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.477 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.34", 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:35:32.961727+0000", 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.478 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.479 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.480 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.481 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:13.482 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.129209+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.129209+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.129765+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:13.483 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437238+0000", 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.484 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.33", 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:16:48.280737+0000", 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.485 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.486 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.487 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.488 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:13.489 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122270+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122270+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.123008+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.490 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384456+0000", 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.32", 2024-03-20T08:01:13.491 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:40:37.385767+0000", 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.492 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.493 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.494 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.495 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.126555+0000", 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.126555+0000", 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.127042+0000", 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:13.496 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442185+0000", 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.497 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.31", 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.498 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:03:41.301351+0000", 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.499 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.500 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.501 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.502 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128111+0000", 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128111+0000", 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.129086+0000", 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.503 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384469+0000", 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.504 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.30", 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:35:34.674299+0000", 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.505 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.506 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.507 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.508 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.509 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128501+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128501+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.128812+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:13.510 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385141+0000", 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.511 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2f", 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:35:51.699322+0000", 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.512 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.513 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.514 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.515 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.516 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.127943+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.127943+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.128211+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.517 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.386365+0000", 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.518 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2e", 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:48:19.979568+0000", 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.519 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.520 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.521 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.522 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.132794+0000", 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.132794+0000", 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.133112+0000", 2024-03-20T08:01:13.523 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971431+0000", 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.524 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2d", 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.525 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:31:44.493207+0000", 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.526 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.527 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.528 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.529 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.774570+0000", 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.774570+0000", 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.774846+0000", 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.530 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385804+0000", 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 4, 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.531 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 4, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1f", 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 24, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:28:00.513690+0000", 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.532 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 582, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.533 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.534 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 4, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 1 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.535 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'4" 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.536 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128584+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128584+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.128852+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.537 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437639+0000", 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.538 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1e", 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:15:37.874514+0000", 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.539 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.540 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.541 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.542 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.543 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.773993+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.773993+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.774347+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.544 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437722+0000", 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:13.545 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1e", 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:04:59.603081+0000", 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.546 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.547 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.548 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:13.549 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:13.550 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118107+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118107+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.118404+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385715+0000", 2024-03-20T08:01:13.551 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1f", 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.552 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:58:51.229913+0000", 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.553 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.554 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.555 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.556 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.777710+0000", 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.777710+0000", 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.778043+0000", 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:13.557 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970803+0000", 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:13.558 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1d", 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.559 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:35:33.084124+0000", 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.560 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.561 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:13.562 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.563 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122136+0000", 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122136+0000", 2024-03-20T08:01:13.564 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.122440+0000", 2024-03-20T08:01:13.565 INFO:tasks.workunit.client.0.smithi118.stderr:+ echo OK 2024-03-20T08:01:13.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:13.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:13.568 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384982+0000", 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.569 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1c", 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:34:29.401254+0000", 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.570 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.571 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.572 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.573 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.574 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.775756+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.775756+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.776107+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.575 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970786+0000", 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.576 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1c", 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:38:33.546280+0000", 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.577 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.578 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.579 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.580 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:46.698994+0000", 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:46.698994+0000", 2024-03-20T08:01:13.581 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:46.699381+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 65, 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:02.983617+0000", 2024-03-20T08:01:13.582 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 64, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1d", 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.583 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 69, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 16, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:31:09.474011+0000", 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.584 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.585 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.586 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.587 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.769359+0000", 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.769359+0000", 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769988+0000", 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:13.588 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437388+0000", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.589 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1b", 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.590 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:26:35.889023+0000", 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.591 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.592 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.593 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.594 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118430+0000", 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118430+0000", 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.118793+0000", 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.595 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.188748+0000", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.596 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1a", 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:51:05.805454+0000", 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.597 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.598 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.599 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.600 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.601 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.765598+0000", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.765598+0000", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.765918+0000", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:13.602 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.596148+0000", 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.603 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1a", 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:30:35.653320+0000", 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.604 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.605 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.606 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.607 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.608 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118645+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118645+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.119313+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.609 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384632+0000", 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.610 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1b", 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:25:13.410704+0000", 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.611 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.612 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.613 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.614 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770958+0000", 2024-03-20T08:01:13.615 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770958+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771278+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437456+0000", 2024-03-20T08:01:13.616 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.19", 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.617 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:10:26.828071+0000", 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.618 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.619 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.620 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.621 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.116571+0000", 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.116571+0000", 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.116986+0000", 2024-03-20T08:01:13.622 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437248+0000", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.623 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.18", 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.624 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:32:56.125697+0000", 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.625 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.626 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.627 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.628 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.768716+0000", 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.768716+0000", 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769007+0000", 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.629 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384533+0000", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.630 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.18", 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:40:23.447186+0000", 2024-03-20T08:01:13.631 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.632 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.633 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.634 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.635 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.119184+0000", 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.119184+0000", 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.119518+0000", 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.636 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.441931+0000", 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.637 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.19", 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:26:25.067260+0000", 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.638 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.639 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.640 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.641 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.642 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.768916+0000", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.768916+0000", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769300+0000", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:13.643 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595525+0000", 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.644 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.17", 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:22:35.932757+0000", 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.645 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.646 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.647 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.648 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.649 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.130332+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.130332+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.130616+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:13.650 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384969+0000", 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.651 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.16", 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:59:44.439163+0000", 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.652 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.653 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.654 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.655 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.656 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.769492+0000", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.769492+0000", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769790+0000", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:13.657 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385024+0000", 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.658 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.659 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.659 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.659 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.659 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.661 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.661 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.16", 2024-03-20T08:01:13.661 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.661 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.661 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:10:30.099079+0000", 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.662 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.663 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.664 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.665 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.116037+0000", 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.116037+0000", 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.116344+0000", 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.666 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385684+0000", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.667 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.17", 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:09:16.183856+0000", 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.668 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.669 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.670 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.671 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.672 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770657+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770657+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.770988+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.673 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442188+0000", 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 3, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 3, 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.674 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.15", 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 23, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:34:56.906803+0000", 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 90, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.675 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.676 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.677 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 4, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 1 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.678 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'3" 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.679 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128201+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128201+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.128506+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.680 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.188994+0000", 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.14", 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.681 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:11:20.984946+0000", 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.682 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.683 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.684 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.685 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.686 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.767533+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.767533+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.767830+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.687 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T08:00:30.023200+0000", 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 12, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 12, 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.688 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.14", 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 32, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:59:57.221141+0000", 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 2630, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.689 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 12, 2024-03-20T08:01:13.690 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 4, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 4, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.691 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 12, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 9 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.692 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "70'12" 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.693 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.139362+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.139362+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.140127+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:13.694 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971517+0000", 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.695 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.15", 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:14:48.943610+0000", 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.696 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.697 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.698 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.699 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.700 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.771499+0000", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.771499+0000", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771772+0000", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:13.701 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385248+0000", 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.702 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.13", 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:19:45.833311+0000", 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 34, 2024-03-20T08:01:13.703 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.704 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.705 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 7 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.706 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.707 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.121780+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.121780+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.122549+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.708 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384371+0000", 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.709 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.12", 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:37:01.164741+0000", 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.710 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.711 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.712 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.713 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.714 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.771464+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.771464+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771839+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.715 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384590+0000", 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.716 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.12", 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:53:58.942621+0000", 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.717 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.718 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.719 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.720 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.128675+0000", 2024-03-20T08:01:13.721 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.128675+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.129094+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:13.722 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971228+0000", 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.723 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.13", 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:10:55.681662+0000", 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.724 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.725 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.726 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.727 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.728 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.767268+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.767268+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.767586+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.729 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.596080+0000", 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.730 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.11", 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:46:11.177465+0000", 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.731 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.732 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.733 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.734 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.735 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.138286+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.138286+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.139925+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:13.736 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595203+0000", 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.737 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.10", 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:17:01.828875+0000", 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.738 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.739 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.740 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.741 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.742 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.756879+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.756879+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.757217+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.743 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.386022+0000", 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 8, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 8, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.744 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.10", 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 28, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:08:28.214822+0000", 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.745 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 12, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 4, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 4, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.746 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 4, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.747 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 12, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'8" 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.748 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.132630+0000", 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.132630+0000", 2024-03-20T08:01:13.749 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.133140+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971190+0000", 2024-03-20T08:01:13.750 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.11", 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.751 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:46:51.087193+0000", 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.752 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.753 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.754 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.755 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770984+0000", 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770984+0000", 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771322+0000", 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:13.756 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.189052+0000", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.757 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.f", 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:13.758 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:23:54.184586+0000", 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 22, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.759 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.760 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.761 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 1, 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 1 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.762 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.143683+0000", 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.143683+0000", 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.144254+0000", 2024-03-20T08:01:13.763 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971608+0000", 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.764 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.e", 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.765 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:33:52.892094+0000", 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.766 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.767 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.768 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.769 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442945+0000", 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.987867+0000", 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.987867+0000", 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.988226+0000", 2024-03-20T08:01:13.770 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442945+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442945+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442945+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442945+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442945+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442945+0000", 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.771 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.2", 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 13, 2024-03-20T08:01:13.772 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:14:23.583126+0000", 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.773 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.774 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.775 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.776 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.773526+0000", 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.773526+0000", 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.773911+0000", 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.777 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442240+0000", 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.778 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.4", 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:32:08.917936+0000", 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.779 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.780 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.781 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.782 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.783 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.121608+0000", 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.121608+0000", 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.122909+0000", 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.784 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384865+0000", 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.785 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.5", 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:05:37.085688+0000", 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.786 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.787 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.788 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.789 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.790 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.443114+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.991077+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.991077+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.991385+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.443114+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.443114+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.443114+0000", 2024-03-20T08:01:13.791 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.443114+0000", 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.443114+0000", 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.443114+0000", 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.792 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.4", 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 13, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:31:07.169554+0000", 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.793 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.794 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.795 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.796 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.797 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770298+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770298+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.770569+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.798 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384580+0000", 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.799 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.2", 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:21:30.545135+0000", 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.800 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.801 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.802 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.803 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:13.804 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135909+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135909+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.136200+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:13.805 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.189036+0000", 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.3", 2024-03-20T08:01:13.806 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T12:01:52.803447+0000", 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.807 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.808 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.809 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.810 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.988575+0000", 2024-03-20T08:01:13.811 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.988575+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.988864+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:13.812 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.912725+0000", 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.7", 2024-03-20T08:01:13.813 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 69, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:19:29.334160+0000", 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.814 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.815 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.816 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.817 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.129798+0000", 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.129798+0000", 2024-03-20T08:01:13.818 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.130079+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442134+0000", 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.819 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.0", 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.820 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:42:30.591840+0000", 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.821 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.822 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.823 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.824 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.764390+0000", 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.764390+0000", 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.764736+0000", 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:13.825 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384494+0000", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.826 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.1", 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.827 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:27:58.881448+0000", 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.828 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.829 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.830 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.831 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.596219+0000", 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.993550+0000", 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.993550+0000", 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.994090+0000", 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.596219+0000", 2024-03-20T08:01:13.832 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.596219+0000", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.596219+0000", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.596219+0000", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.596219+0000", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.596219+0000", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.833 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.5", 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 13, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:42:05.762120+0000", 2024-03-20T08:01:13.834 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.835 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.836 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.837 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.838 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.135969+0000", 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.135969+0000", 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.136285+0000", 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.839 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437500+0000", 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.840 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2", 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:41:16.328342+0000", 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.841 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.842 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.843 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.844 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.845 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.766341+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.766341+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.766643+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:13.846 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384647+0000", 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.847 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.3", 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:14:49.146036+0000", 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.848 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.849 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.850 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:13.851 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 20, 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.852 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T08:01:09.419984+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:09.963927+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:09.963927+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:09.964395+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T08:01:09.419984+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:56:04.165332+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:56:04.165332+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 59, 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T08:01:09.419984+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T08:01:09.419984+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T08:01:09.419984+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:56:04.165332+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T08:01:09.419984+0000", 2024-03-20T08:01:13.853 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T08:01:09.419984+0000", 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 148, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 58, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 148, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.854 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "1.0", 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 338, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:52:16.137581+0000", 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 590368, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 1771104, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.855 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.856 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 6, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 394, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 584, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.857 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 219, 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 2416 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "70'148" 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.858 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.989129+0000", 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.989129+0000", 2024-03-20T08:01:13.859 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.989392+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385424+0000", 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.860 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.6", 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.861 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 12, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:47:32.163295+0000", 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.862 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.863 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.864 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.865 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.866 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.867 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.125161+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.125161+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.125436+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.868 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385109+0000", 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.869 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.1", 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:53:55.085832+0000", 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.870 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.871 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.872 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.873 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.778258+0000", 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.778258+0000", 2024-03-20T08:01:13.874 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.779129+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970997+0000", 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.875 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.0", 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.876 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:13:03.810049+0000", 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.877 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.878 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.879 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.880 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.911214+0000", 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.994854+0000", 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.994854+0000", 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.995060+0000", 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.911214+0000", 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.881 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.911214+0000", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.911214+0000", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.911214+0000", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.911214+0000", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.911214+0000", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:13.882 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.3", 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 13, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:48:00.138918+0000", 2024-03-20T08:01:13.883 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.884 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.885 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.886 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.887 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.777666+0000", 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.777666+0000", 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.777973+0000", 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.888 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970922+0000", 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.889 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.5", 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:39:20.794255+0000", 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.890 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.891 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.892 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.893 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.894 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.126397+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.126397+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.126692+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:13.895 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442141+0000", 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.896 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.4", 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:41:47.793200+0000", 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.897 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.898 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.899 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.900 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.901 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.993283+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.993283+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.993571+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.902 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.912343+0000", 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.903 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.0", 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 13, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:02:21.502002+0000", 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.904 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.905 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.906 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.907 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.769010+0000", 2024-03-20T08:01:13.908 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.769010+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.769337+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.443286+0000", 2024-03-20T08:01:13.909 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.6", 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.910 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:25:22.276433+0000", 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.911 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.912 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 1, 2024-03-20T08:01:13.913 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "63'1" 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.914 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.118565+0000", 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.118565+0000", 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.118863+0000", 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:13.915 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437706+0000", 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.916 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.7", 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.917 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:49:33.981106+0000", 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.918 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.919 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.920 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.921 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 67, 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.189531+0000", 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:59:01.988797+0000", 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:59:01.988797+0000", 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:59:01.989083+0000", 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.189531+0000", 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.922 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 68, 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.189531+0000", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.189531+0000", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.189531+0000", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:59:00.974759+0000", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.189531+0000", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.189531+0000", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 67, 2024-03-20T08:01:13.923 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "4.1", 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 13, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.924 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:42:40.380304+0000", 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.925 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.926 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.927 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.928 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.770737+0000", 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.770737+0000", 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.771097+0000", 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:13.929 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595632+0000", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.930 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.7", 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.931 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:04:52.079932+0000", 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.932 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.933 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.934 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.935 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:46.698841+0000", 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:46.698841+0000", 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:46.699298+0000", 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 65, 2024-03-20T08:01:13.936 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.386273+0000", 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 64, 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.937 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.6", 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 17, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:02:06.660176+0000", 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.938 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.939 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.940 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.941 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.942 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.777649+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.777649+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.777961+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:13.943 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.970739+0000", 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.944 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.8", 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:14:14.279299+0000", 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.945 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.946 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.947 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.948 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 4, 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.949 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.119966+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.119966+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.120305+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.950 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.188758+0000", 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.951 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.9", 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T10:43:58.858489+0000", 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.952 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.953 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.954 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 4, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.955 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.765273+0000", 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.765273+0000", 2024-03-20T08:01:13.956 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.765548+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.441942+0000", 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.957 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.9", 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.958 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:18:05.348450+0000", 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.959 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.960 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.961 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.962 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.117376+0000", 2024-03-20T08:01:13.963 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.117376+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.117669+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437454+0000", 2024-03-20T08:01:13.964 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.8", 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.965 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:42:02.274133+0000", 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.966 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.967 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.968 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.969 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.773520+0000", 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.773520+0000", 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.773811+0000", 2024-03-20T08:01:13.970 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.441934+0000", 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.971 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.a", 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:13.972 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T19:43:26.555598+0000", 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.973 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.974 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.975 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.976 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122680+0000", 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122680+0000", 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.123160+0000", 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.977 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437286+0000", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.978 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.b", 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.979 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:09:56.675972+0000", 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.980 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.981 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.982 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.983 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:46.306388+0000", 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:46.306388+0000", 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:46.309368+0000", 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:13.984 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 65, 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384796+0000", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.985 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 64, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.b", 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:13.986 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:42:19.060358+0000", 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.987 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.988 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 1, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 1, 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.989 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.990 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.133784+0000", 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.133784+0000", 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.134067+0000", 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.991 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.596131+0000", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.992 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.a", 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:13.993 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:01:51.409625+0000", 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:13.994 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:13.995 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:13.996 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:13.997 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.762682+0000", 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.762682+0000", 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.763015+0000", 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:13.998 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.385272+0000", 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:13.999 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.c", 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 20, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:21:45.137669+0000", 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.000 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.001 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.002 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.003 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.004 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.142148+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.142148+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.142445+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.005 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971515+0000", 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.006 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.d", 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:08:52.834571+0000", 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.007 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.008 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.009 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.010 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.766111+0000", 2024-03-20T08:01:14.011 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.766111+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.766427+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442367+0000", 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.012 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 1, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 1, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.d", 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.013 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 21, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T08:36:18.901096+0000", 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 70, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.014 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.015 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 1, 2024-03-20T08:01:14.016 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 1 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: 3 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'1" 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.017 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.134903+0000", 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.134903+0000", 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.135339+0000", 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:14.018 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384855+0000", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.019 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.c", 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.020 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:22:06.681194+0000", 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.021 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.022 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.023 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:14.024 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 60, 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:25.765948+0000", 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:25.765948+0000", 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:25.766233+0000", 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:14.025 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 61, 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:24.742760+0000", 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.386278+0000", 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 2, 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 60, 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.026 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 2, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "2.e", 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 22, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:34:01.876341+0000", 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.027 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.028 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 3, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 1, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 1, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 1, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.029 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 3, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.030 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: 7 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "62'2" 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 7, 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.031 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.143696+0000", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.143696+0000", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.144008+0000", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:14.032 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.971709+0000", 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.033 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.f", 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:43:30.230310+0000", 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.034 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.035 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.036 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.037 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 7, 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.038 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.137804+0000", 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.137804+0000", 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.138096+0000", 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.039 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595342+0000", 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.040 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.20", 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:47:51.983182+0000", 2024-03-20T08:01:14.041 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.042 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.043 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.044 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: 5 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 2, 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.045 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.125793+0000", 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.125793+0000", 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.126085+0000", 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.046 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384575+0000", 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.047 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.21", 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:11:45.470755+0000", 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.048 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.049 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.050 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:14.051 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 2, 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.052 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.124689+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.124689+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.125004+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.053 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384653+0000", 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.054 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.22", 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T18:54:48.097320+0000", 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.055 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.056 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.057 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.058 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.119755+0000", 2024-03-20T08:01:14.059 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.119755+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.120048+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384704+0000", 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.060 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.23", 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.061 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T09:39:07.689571+0000", 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.062 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.063 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.064 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:14.065 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:46.699085+0000", 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:46.699085+0000", 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:46.699520+0000", 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.066 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 65, 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437574+0000", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 64, 2024-03-20T08:01:14.067 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.24", 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 28, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T16:58:27.802971+0000", 2024-03-20T08:01:14.068 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.069 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.070 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.071 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: 2, 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.073 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 1, 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122635+0000", 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122635+0000", 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.126113+0000", 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.074 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.442324+0000", 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.075 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.25", 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T15:28:33.273327+0000", 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.076 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.077 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.078 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.079 INFO:tasks.workunit.client.0.smithi118.stdout: 1, 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 1, 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.080 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.141099+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.141099+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.141379+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:14.081 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595623+0000", 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.082 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.26", 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:57:17.004349+0000", 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.083 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.084 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.085 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.086 INFO:tasks.workunit.client.0.smithi118.stdout: 4 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.087 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.121094+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.121094+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.121389+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.088 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437734+0000", 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.089 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.27", 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T14:08:53.389656+0000", 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.090 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.091 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.092 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.093 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.094 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.137344+0000", 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.137344+0000", 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.137638+0000", 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.095 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384555+0000", 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.096 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.28", 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T13:20:56.870173+0000", 2024-03-20T08:01:14.097 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.098 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.099 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.100 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: 2 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 3, 2024-03-20T08:01:14.101 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.122003+0000", 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.122003+0000", 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.122341+0000", 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.102 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.437323+0000", 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.103 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.29", 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:18:20.158675+0000", 2024-03-20T08:01:14.104 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.105 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.106 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.107 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: 3, 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 3, 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:14.108 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 5, 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.136135+0000", 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.136135+0000", 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.137117+0000", 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.109 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384697+0000", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.110 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2a", 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.111 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:58:52.302133+0000", 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.112 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.113 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.114 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: 5, 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: 6 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 5, 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:14.115 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 0, 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.119047+0000", 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.119047+0000", 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.119480+0000", 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.116 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:05.384775+0000", 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.117 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2b", 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T17:09:31.918661+0000", 2024-03-20T08:01:14.118 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.119 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.120 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.121 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: 0, 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: 7, 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: 1 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 0, 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": [ 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:14.122 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "acting_primary": 6, 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "avail_no_missing": [], 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "blocked_by": [], 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "created": 61, 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "dirty_stats_invalid": false, 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_bytes_stats_invalid": false, 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "hitset_stats_invalid": false, 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "last_active": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_active": "2024-03-20T07:58:26.132793+0000", 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "last_became_peered": "2024-03-20T07:58:26.132793+0000", 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "last_change": "2024-03-20T07:58:26.133160+0000", 2024-03-20T08:01:14.123 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_clean_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub": "0'0", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_deep_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_epoch_clean": 62, 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fresh": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_fullsized": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_peered": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub": "0'0", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_duration": 0, 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_scrub_stamp": "2024-03-20T07:58:25.744971+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_undegraded": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "last_unstale": "2024-03-20T07:59:06.595407+0000", 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "log_dups_size": 0, 2024-03-20T08:01:14.124 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "log_start": "0'0", 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "manifest_stats_invalid": false, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "mapping_epoch": 61, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "object_location_counts": [], 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_scrubbed": 0, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "objects_trimmed": 0, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_stats_invalid": false, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_start": "0'0", 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "parent": "0.0", 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "parent_split_bits": 0, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "pgid": "3.2c", 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "pin_stats_invalid": false, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "purged_snaps": [], 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_epoch": 70, 2024-03-20T08:01:14.125 INFO:tasks.workunit.client.0.smithi118.stdout: "reported_seq": 19, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_duration": 0.0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "scrub_schedule": "periodic scrub scheduled @ 2024-03-21T11:59:44.210159+0000", 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrim_duration": 0.0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "snaptrimq_len": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.126 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.127 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.128 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "state": "active+clean", 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_invalid": false, 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "up": [ 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: 6, 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: 4, 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: 0 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "up_primary": 6, 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "version": "0'0" 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_per_pool": [ 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 24, 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:14.129 INFO:tasks.workunit.client.0.smithi118.stdout: "rbd" 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pg": 8, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 8, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "poolid": 4, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.130 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.131 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.132 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:14.133 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 24 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 192, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 0, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pg": 64, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 8, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 0, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "poolid": 3, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 0, 2024-03-20T08:01:14.134 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.135 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.136 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 0 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:14.137 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 0, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 192 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 3, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "mgr" 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 148, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pg": 1, 2024-03-20T08:01:14.138 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 3, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 148, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "poolid": 1, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 590368, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 1771104, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.139 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 6, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 2, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 2, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.140 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 6, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 394, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 584, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 219, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 2416 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:14.141 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 1781760, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 1771104, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 3 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.142 INFO:tasks.workunit.client.0.smithi118.stdout: { 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "acting": 96, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "application": [ 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "cephfs" 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "log_size": 45, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_pg": 32, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_store_stats": 8, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "ondisk_log_size": 45, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "poolid": 2, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "stat_sum": { 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes": 3428, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_hit_set_archive": 0, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_bytes_recovered": 0, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_deep_scrub_errors": 0, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict": 0, 2024-03-20T08:01:14.143 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_kb": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_full": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_evict_mode_some": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_kb": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_high": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_flush_mode_low": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_keys_recovered": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_large_omap_objects": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_legacy_snapsets": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_clones": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_object_copies": 66, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects": 22, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_degraded": 0, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_dirty": 22, 2024-03-20T08:01:14.144 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_hit_set_archive": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_manifest": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_misplaced": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_missing_on_primary": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_omap": 14, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_pinned": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_recovered": 1, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_repaired": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_objects_unfound": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_bytes": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_omap_keys": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_promote": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_read_kb": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_scrub_errors": 0, 2024-03-20T08:01:14.145 INFO:tasks.workunit.client.0.smithi118.stdout: "num_shallow_scrub_errors": 0, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "num_whiteouts": 0, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write": 51, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "num_write_kb": 20 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "store_stats": { 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "allocated": 98304, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "available": 0, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed": 0, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_allocated": 0, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "data_compressed_original": 0, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "data_stored": 10284, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "internal_metadata": 0, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "internally_reserved": 0, 2024-03-20T08:01:14.146 INFO:tasks.workunit.client.0.smithi118.stdout: "omap_allocated": 0, 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "total": 0 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "up": 96 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: ], 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "usage": { 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "pg_num": 105, 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "pools": 4, 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "stats_by_class": { 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "ssd": { 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "total_avail_bytes": 767731343360, 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "total_bytes": 767960285184, 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_bytes": 228941824, 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_raw_bytes": 228941824, 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_raw_ratio": 0.00029811676358804107 2024-03-20T08:01:14.147 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:14.148 INFO:tasks.workunit.client.0.smithi118.stdout: }, 2024-03-20T08:01:14.148 INFO:tasks.workunit.client.0.smithi118.stdout: "total_avail_bytes": 767731343360, 2024-03-20T08:01:14.148 INFO:tasks.workunit.client.0.smithi118.stdout: "total_bytes": 767960285184, 2024-03-20T08:01:14.148 INFO:tasks.workunit.client.0.smithi118.stdout: "total_used_bytes": 228941824 2024-03-20T08:01:14.148 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:14.148 INFO:tasks.workunit.client.0.smithi118.stdout: } 2024-03-20T08:01:14.148 INFO:tasks.workunit.client.0.smithi118.stdout:} 2024-03-20T08:01:14.148 INFO:tasks.workunit.client.0.smithi118.stdout:OK 2024-03-20T08:01:14.148 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.148 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.149 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.151 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-20T08:01:14.151 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-20T08:01:14.152 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.152 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.152 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: pgmap v73: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.153 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.156 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:12 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "telemetry show", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.157 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.158 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:11 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: pgmap v73: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.159 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "telemetry show-device", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: pgmap v73: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.162 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config-key dump", "format": "json"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.166 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.167 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.167 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd='heap' args=[stats]: dispatch 2024-03-20T08:01:14.167 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='admin socket' entity='admin socket' cmd=heap args=[stats]: finished 2024-03-20T08:01:14.167 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:01:14.167 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:01:14.167 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:12 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:01:14.231 INFO:tasks.workunit:Stopping ['test_telemetry_reef.sh'] on client.0... 2024-03-20T08:01:14.231 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-20T08:01:14.297 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:14 smithi118 ceph-mon[27391]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.297 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:14 smithi118 ceph-mon[38981]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:14 smithi196 ceph-mon[31033]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "telemetry show-all", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:14.516 DEBUG:teuthology.parallel:result is None 2024-03-20T08:01:14.517 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-20T08:01:14.584 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-20T08:01:14.584 DEBUG:teuthology.orchestra.run.smithi118:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-20T08:01:14.643 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-20T08:01:14.643 INFO:teuthology.run_tasks:Running task print... 2024-03-20T08:01:14.654 INFO:teuthology.task.print:**** done end telemetry reef... 2024-03-20T08:01:14.654 INFO:teuthology.run_tasks:Running task print... 2024-03-20T08:01:14.665 INFO:teuthology.task.print:**** done start parallel 2024-03-20T08:01:14.666 INFO:teuthology.run_tasks:Running task parallel... 2024-03-20T08:01:14.677 INFO:teuthology.task.parallel:starting parallel... 2024-03-20T08:01:14.677 INFO:teuthology.task.parallel:In parallel, running task full_sequential... 2024-03-20T08:01:14.677 INFO:teuthology.task.full_sequential:In full_sequential, running task print... 2024-03-20T08:01:14.678 INFO:teuthology.task.print:**** done start ec-rados-default.yaml 2024-03-20T08:01:14.678 INFO:teuthology.task.full_sequential:In full_sequential, running task rados... 2024-03-20T08:01:14.680 INFO:tasks.rados:Beginning rados... 2024-03-20T08:01:14.680 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 4000, 'write_append_excl': False} 2024-03-20T08:01:14.680 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-yuri5-testing-2024-03-18-1144'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'create_rbd_pool': True, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'default', 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-yuri5-testing-2024-03-18-1144', 'sha1': 'e7e80a6ac9849f0f23432d0d9832f096730308ae'}} 2024-03-20T08:01:14.681 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 4000, 'write_append_excl': False} 2024-03-20T08:01:14.681 INFO:tasks.rados:joining rados 2024-03-20T08:01:14.681 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-20T08:01:14.681 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-20T08:01:14.681 INFO:teuthology.task.print:**** done start upgrade, wait 2024-03-20T08:01:14.681 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-20T08:01:14.682 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi118.front.sepia.ceph.com 2024-03-20T08:01:14.682 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -e sha1=e7e80a6ac9849f0f23432d0d9832f096730308ae -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-20T08:01:14.683 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2024-03-20T08:01:14.683 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd erasure-code-profile set teuthologyprofile k=2 m=1 crush-failure-domain=osd 2024-03-20T08:01:14.899 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:01:14.902 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:01:15.265 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:15 smithi118 ceph-mon[27391]: pgmap v74: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-20T08:01:15.265 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:15 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:15.265 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:15 smithi118 ceph-mon[38981]: pgmap v74: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-20T08:01:15.265 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:15 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:15.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:15 smithi196 ceph-mon[31033]: pgmap v74: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-20T08:01:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:15 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:17.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:17 smithi118 ceph-mon[27391]: pgmap v75: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 33 op/s 2024-03-20T08:01:17.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:17 smithi118 ceph-mon[38981]: pgmap v75: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 33 op/s 2024-03-20T08:01:17.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:17 smithi196 ceph-mon[31033]: pgmap v75: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 33 op/s 2024-03-20T08:01:19.051 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -e sha1=e7e80a6ac9849f0f23432d0d9832f096730308ae -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-20T08:01:19.365 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:19 smithi118 ceph-mon[27391]: pgmap v76: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 33 op/s 2024-03-20T08:01:19.365 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:19 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/876307409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-20T08:01:19.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:19 smithi118 ceph-mon[38981]: pgmap v76: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 33 op/s 2024-03-20T08:01:19.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:19 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/876307409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-20T08:01:19.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:19 smithi196 ceph-mon[31033]: pgmap v76: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 33 op/s 2024-03-20T08:01:19.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:19 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/876307409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-20T08:01:19.817 INFO:tasks.rados:starting run 0 out of 1 2024-03-20T08:01:19.817 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-03-20T08:01:19.818 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd pool create unique_pool_0 16 16 erasure teuthologyprofile 2024-03-20T08:01:19.831 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:01:20.073 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:20 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/876307409' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-20T08:01:20.073 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:20 smithi118 ceph-mon[38981]: osdmap e71: 8 total, 8 up, 8 in 2024-03-20T08:01:20.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:20 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/876307409' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-20T08:01:20.074 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:20 smithi118 ceph-mon[27391]: osdmap e71: 8 total, 8 up, 8 in 2024-03-20T08:01:20.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:20 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/876307409' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "teuthologyprofile", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-20T08:01:20.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:20 smithi196 ceph-mon[31033]: osdmap e71: 8 total, 8 up, 8 in 2024-03-20T08:01:20.530 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:01:21.377 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:21 smithi118 ceph-mon[27391]: pgmap v78: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 0 B/s wr, 39 op/s 2024-03-20T08:01:21.378 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:21 smithi118 ceph-mon[38981]: pgmap v78: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 0 B/s wr, 39 op/s 2024-03-20T08:01:21.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:21 smithi196 ceph-mon[31033]: pgmap v78: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 0 B/s wr, 39 op/s 2024-03-20T08:01:21.914 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:01:21] "GET /metrics HTTP/1.1" 200 33654 "" "Prometheus/2.43.0" 2024-03-20T08:01:23.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:23 smithi118 ceph-mon[27391]: pgmap v79: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-20T08:01:23.138 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:23 smithi118 ceph-mon[38981]: pgmap v79: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-20T08:01:23.160 INFO:teuthology.orchestra.run.smithi118.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:01:23.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:23 smithi196 ceph-mon[31033]: pgmap v79: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-20T08:01:23.922 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -e sha1=e7e80a6ac9849f0f23432d0d9832f096730308ae -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true ; do ceph orch ps ; ceph versions ; sleep 30 ; done' 2024-03-20T08:01:24.162 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2463312788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]: dispatch 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: pgmap v80: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:24.163 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:24.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:01:24.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:24.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:24.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2463312788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]: dispatch 2024-03-20T08:01:24.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: pgmap v80: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-20T08:01:24.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:24.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:24.164 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:24.190 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:01:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: from='client.24914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:01:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2463312788' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]: dispatch 2024-03-20T08:01:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: pgmap v80: 105 pgs: 105 active+clean; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-20T08:01:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:01:24.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:01:24.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:25.176 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'unique_pool_0' created 2024-03-20T08:01:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:25 smithi118 ceph-mon[27391]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:01:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:25 smithi118 ceph-mon[27391]: osdmap e72: 8 total, 8 up, 8 in 2024-03-20T08:01:25.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:25 smithi118 ceph-mon[38981]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:01:25.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:25 smithi118 ceph-mon[38981]: osdmap e72: 8 total, 8 up, 8 in 2024-03-20T08:01:25.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:25 smithi196 ceph-mon[31033]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:e7e80a6ac9849f0f23432d0d9832f096730308ae 2024-03-20T08:01:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:25 smithi196 ceph-mon[31033]: osdmap e72: 8 total, 8 up, 8 in 2024-03-20T08:01:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:26 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2463312788' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]': finished 2024-03-20T08:01:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:26 smithi118 ceph-mon[27391]: osdmap e73: 8 total, 8 up, 8 in 2024-03-20T08:01:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:26 smithi118 ceph-mon[27391]: pgmap v83: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:26.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:26 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2463312788' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]': finished 2024-03-20T08:01:26.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:26 smithi118 ceph-mon[38981]: osdmap e73: 8 total, 8 up, 8 in 2024-03-20T08:01:26.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:26 smithi118 ceph-mon[38981]: pgmap v83: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:26.634 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-03-20T08:01:26.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:26 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2463312788' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16, "pgp_num": 16, "pool_type": "erasure", "erasure_code_profile": "teuthologyprofile"}]': finished 2024-03-20T08:01:26.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:26 smithi196 ceph-mon[31033]: osdmap e73: 8 total, 8 up, 8 in 2024-03-20T08:01:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:26 smithi196 ceph-mon[31033]: pgmap v83: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:26.977 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:01:27.308 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:01:27.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:27 smithi118 ceph-mon[27391]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T08:01:27.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:27 smithi118 ceph-mon[27391]: osdmap e74: 8 total, 8 up, 8 in 2024-03-20T08:01:27.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:27 smithi118 ceph-mon[38981]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T08:01:27.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:27 smithi118 ceph-mon[38981]: osdmap e74: 8 total, 8 up, 8 in 2024-03-20T08:01:27.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:27 smithi196 ceph-mon[31033]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-20T08:01:27.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:27 smithi196 ceph-mon[31033]: osdmap e74: 8 total, 8 up, 8 in 2024-03-20T08:01:27.771 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:01:27.771 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (69s) 18s ago 2m 24.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:01:27.771 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (110s) 18s ago 110s 83.2M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:01:27.771 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (3m) 18s ago 3m 12.6M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:01:27.771 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (3m) 18s ago 3m 16.7M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:01:27.771 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,8765 running (6m) 18s ago 6m 448M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 ef306191c58d 2024-03-20T08:01:27.771 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:9283,8765 running (8m) 18s ago 8m 494M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 64dace5519ed 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (8m) 18s ago 8m 48.5M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 89e74e5abc84 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (7m) 18s ago 7m 41.0M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 fd26544c725b 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (6m) 18s ago 6m 44.3M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 8193e23658db 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (90s) 18s ago 89s 14.7M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (80s) 18s ago 80s 8938k - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (6m) 18s ago 6m 47.7M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 481bba84ab31 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (5m) 18s ago 5m 46.5M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (5m) 18s ago 5m 41.9M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (5m) 18s ago 5m 40.8M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (4m) 18s ago 4m 43.5M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (4m) 18s ago 4m 42.4M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (3m) 18s ago 3m 41.5M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (3m) 18s ago 3m 42.2M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:01:27.772 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (63s) 18s ago 2m 34.9M - 2.43.0 a07b618ecd1d d713b2506adb 2024-03-20T08:01:28.355 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:01:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:01:28.355 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 15 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:01:28.356 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:01:28.526 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:28 smithi118 ceph-mon[27391]: osdmap e75: 8 total, 8 up, 8 in 2024-03-20T08:01:28.526 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:28 smithi118 ceph-mon[27391]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:28.526 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:28 smithi118 ceph-mon[27391]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:28.527 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:28 smithi118 ceph-mon[27391]: pgmap v86: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:28.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:28 smithi118 ceph-mon[38981]: osdmap e75: 8 total, 8 up, 8 in 2024-03-20T08:01:28.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:28 smithi118 ceph-mon[38981]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:28.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:28 smithi118 ceph-mon[38981]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:28.527 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:28 smithi118 ceph-mon[38981]: pgmap v86: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:28.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:28 smithi196 ceph-mon[31033]: osdmap e75: 8 total, 8 up, 8 in 2024-03-20T08:01:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:28 smithi196 ceph-mon[31033]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:28 smithi196 ceph-mon[31033]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:01:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:28 smithi196 ceph-mon[31033]: pgmap v86: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:29.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:29 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2590217616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:01:29.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:29 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2590217616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:01:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:29 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2590217616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:01:30.195 INFO:teuthology.orchestra.run.smithi118.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-03-20T08:01:30.457 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/704721641' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:01:30.457 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:01:30.457 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[27391]: pgmap v87: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:30.457 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:30.457 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:30.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/704721641' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:01:30.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:01:30.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[38981]: pgmap v87: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:30.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:30.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:30 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:30 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/704721641' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:01:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:30 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:01:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:30 smithi196 ceph-mon[31033]: pgmap v87: 121 pgs: 16 creating+peering, 105 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:30 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:30 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:31.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:31 smithi196 ceph-mon[31033]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-03-20T08:01:31.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:31 smithi196 ceph-mon[31033]: osdmap e76: 8 total, 8 up, 8 in 2024-03-20T08:01:31.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:31 smithi118 ceph-mon[27391]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-03-20T08:01:31.646 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:31 smithi118 ceph-mon[27391]: osdmap e76: 8 total, 8 up, 8 in 2024-03-20T08:01:31.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:31 smithi118 ceph-mon[38981]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-03-20T08:01:31.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:31 smithi118 ceph-mon[38981]: osdmap e76: 8 total, 8 up, 8 in 2024-03-20T08:01:31.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:01:31] "GET /metrics HTTP/1.1" 200 36010 "" "Prometheus/2.43.0" 2024-03-20T08:01:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:32 smithi118 ceph-mon[27391]: pgmap v89: 121 pgs: 121 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:32.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:32 smithi118 ceph-mon[38981]: pgmap v89: 121 pgs: 121 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:32.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:32 smithi196 ceph-mon[31033]: pgmap v89: 121 pgs: 121 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:33.503 DEBUG:teuthology.orchestra.run.smithi118:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-03-20T08:01:33.531 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight read -> 100 2024-03-20T08:01:33.531 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight delete -> 50 2024-03-20T08:01:33.531 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight snap_create -> 50 2024-03-20T08:01:33.531 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight snap_remove -> 50 2024-03-20T08:01:33.531 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight rollback -> 50 2024-03-20T08:01:33.531 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight setattr -> 25 2024-03-20T08:01:33.531 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight rmattr -> 25 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight copy_from -> 50 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout:adding op weight append -> 100 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout:ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable) 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout:Configuration: 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout: Number of operations: 4000 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout: Number of objects: 50 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout: Max in flight operations: 16 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout: Object size (in bytes): 4000000 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout: Write stride min: 400000 2024-03-20T08:01:33.532 INFO:tasks.rados.rados.0.smithi118.stdout: Write stride max: 800000 2024-03-20T08:01:33.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:33 smithi196 ceph-mon[31033]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T08:01:33.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:33 smithi196 ceph-mon[31033]: Cluster is now healthy 2024-03-20T08:01:33.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:33 smithi118 ceph-mon[27391]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T08:01:33.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:33 smithi118 ceph-mon[27391]: Cluster is now healthy 2024-03-20T08:01:33.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:33 smithi118 ceph-mon[38981]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-20T08:01:33.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:33 smithi118 ceph-mon[38981]: Cluster is now healthy 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:1: write initial oid 1 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:1: seq_num 0 ranges {0=24576} 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:1: writing smithi11881747-1 from 0 to 24576 tid 1 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:2: write initial oid 2 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:2: seq_num 1 ranges {0=24576} 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:2: writing smithi11881747-2 from 0 to 24576 tid 1 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:3: write initial oid 3 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:3: seq_num 2 ranges {0=40960} 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:3: writing smithi11881747-3 from 0 to 40960 tid 1 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:4: write initial oid 4 2024-03-20T08:01:34.203 INFO:tasks.rados.rados.0.smithi118.stdout:4: seq_num 3 ranges {0=57344} 2024-03-20T08:01:34.204 INFO:tasks.rados.rados.0.smithi118.stdout:4: writing smithi11881747-4 from 0 to 57344 tid 1 2024-03-20T08:01:34.205 INFO:tasks.rados.rados.0.smithi118.stdout:5: write initial oid 5 2024-03-20T08:01:34.205 INFO:tasks.rados.rados.0.smithi118.stdout:5: seq_num 4 ranges {0=73728} 2024-03-20T08:01:34.205 INFO:tasks.rados.rados.0.smithi118.stdout:5: writing smithi11881747-5 from 0 to 73728 tid 1 2024-03-20T08:01:34.205 INFO:tasks.rados.rados.0.smithi118.stdout:6: write initial oid 6 2024-03-20T08:01:34.205 INFO:tasks.rados.rados.0.smithi118.stdout:6: seq_num 5 ranges {0=90112} 2024-03-20T08:01:34.205 INFO:tasks.rados.rados.0.smithi118.stdout:6: writing smithi11881747-6 from 0 to 90112 tid 1 2024-03-20T08:01:34.205 INFO:tasks.rados.rados.0.smithi118.stdout:7: write initial oid 7 2024-03-20T08:01:34.205 INFO:tasks.rados.rados.0.smithi118.stdout:7: seq_num 6 ranges {0=106496} 2024-03-20T08:01:34.206 INFO:tasks.rados.rados.0.smithi118.stdout:7: writing smithi11881747-7 from 0 to 106496 tid 1 2024-03-20T08:01:34.206 INFO:tasks.rados.rados.0.smithi118.stdout:8: write initial oid 8 2024-03-20T08:01:34.206 INFO:tasks.rados.rados.0.smithi118.stdout:8: seq_num 7 ranges {0=122880} 2024-03-20T08:01:34.207 INFO:tasks.rados.rados.0.smithi118.stdout:8: writing smithi11881747-8 from 0 to 122880 tid 1 2024-03-20T08:01:34.207 INFO:tasks.rados.rados.0.smithi118.stdout:9: write initial oid 9 2024-03-20T08:01:34.208 INFO:tasks.rados.rados.0.smithi118.stdout:9: seq_num 8 ranges {0=139264} 2024-03-20T08:01:34.208 INFO:tasks.rados.rados.0.smithi118.stdout:9: writing smithi11881747-9 from 0 to 139264 tid 1 2024-03-20T08:01:34.208 INFO:tasks.rados.rados.0.smithi118.stdout:10: write initial oid 10 2024-03-20T08:01:34.208 INFO:tasks.rados.rados.0.smithi118.stdout:10: seq_num 9 ranges {0=155648} 2024-03-20T08:01:34.209 INFO:tasks.rados.rados.0.smithi118.stdout:10: writing smithi11881747-10 from 0 to 155648 tid 1 2024-03-20T08:01:34.209 INFO:tasks.rados.rados.0.smithi118.stdout:1: finishing write tid 1 to smithi11881747-1 2024-03-20T08:01:34.209 INFO:tasks.rados.rados.0.smithi118.stdout:1: finishing write tid 2 to smithi11881747-1 2024-03-20T08:01:34.209 INFO:tasks.rados.rados.0.smithi118.stdout:11: write initial oid 11 2024-03-20T08:01:34.209 INFO:tasks.rados.rados.0.smithi118.stdout:11: seq_num 10 ranges {0=172032} 2024-03-20T08:01:34.210 INFO:tasks.rados.rados.0.smithi118.stdout:11: writing smithi11881747-11 from 0 to 172032 tid 1 2024-03-20T08:01:34.211 INFO:tasks.rados.rados.0.smithi118.stdout:4: finishing write tid 1 to smithi11881747-4 2024-03-20T08:01:34.211 INFO:tasks.rados.rados.0.smithi118.stdout:2: finishing write tid 1 to smithi11881747-2 2024-03-20T08:01:34.211 INFO:tasks.rados.rados.0.smithi118.stdout:2: finishing write tid 2 to smithi11881747-2 2024-03-20T08:01:34.211 INFO:tasks.rados.rados.0.smithi118.stdout:4: finishing write tid 2 to smithi11881747-4 2024-03-20T08:01:34.211 INFO:tasks.rados.rados.0.smithi118.stdout:3: finishing write tid 1 to smithi11881747-3 2024-03-20T08:01:34.211 INFO:tasks.rados.rados.0.smithi118.stdout:12: write initial oid 12 2024-03-20T08:01:34.211 INFO:tasks.rados.rados.0.smithi118.stdout:12: seq_num 11 ranges {0=188416} 2024-03-20T08:01:34.212 INFO:tasks.rados.rados.0.smithi118.stdout:12: writing smithi11881747-12 from 0 to 188416 tid 1 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:13: write initial oid 13 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:3: finishing write tid 2 to smithi11881747-3 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:1: finishing write tid 3 to smithi11881747-1 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:1: oid 1 updating version 0 to 1 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:1: oid 1 updating version 1 to 2 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:5: finishing write tid 1 to smithi11881747-5 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:4: finishing write tid 3 to smithi11881747-4 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:4: oid 4 updating version 0 to 1 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:4: oid 4 updating version 1 to 2 2024-03-20T08:01:34.213 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-03-20T08:01:34.214 INFO:tasks.rados.rados.0.smithi118.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-03-20T08:01:34.214 INFO:tasks.rados.rados.0.smithi118.stdout:8: finishing write tid 1 to smithi11881747-8 2024-03-20T08:01:34.214 INFO:tasks.rados.rados.0.smithi118.stdout:5: finishing write tid 2 to smithi11881747-5 2024-03-20T08:01:34.214 INFO:tasks.rados.rados.0.smithi118.stdout:13: seq_num 12 ranges {0=204800} 2024-03-20T08:01:34.214 INFO:tasks.rados.rados.0.smithi118.stdout:13: writing smithi11881747-13 from 0 to 204800 tid 1 2024-03-20T08:01:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:1: done (12 left) 2024-03-20T08:01:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:4: done (11 left) 2024-03-20T08:01:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:14: write initial oid 14 2024-03-20T08:01:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:14: seq_num 13 ranges {0=221184} 2024-03-20T08:01:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:14: writing smithi11881747-14 from 0 to 221184 tid 1 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:15: write initial oid 15 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:2: finishing write tid 3 to smithi11881747-2 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:2: oid 2 updating version 0 to 1 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:2: oid 2 updating version 1 to 2 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:6: finishing write tid 1 to smithi11881747-6 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:3: finishing write tid 3 to smithi11881747-3 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:3: oid 3 updating version 0 to 1 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:3: oid 3 updating version 1 to 2 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:10: finishing write tid 1 to smithi11881747-10 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:6: finishing write tid 2 to smithi11881747-6 2024-03-20T08:01:34.217 INFO:tasks.rados.rados.0.smithi118.stdout:10: finishing write tid 2 to smithi11881747-10 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:8: finishing write tid 2 to smithi11881747-8 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:6: finishing write tid 3 to smithi11881747-6 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:6: oid 6 updating version 0 to 3 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:6: oid 6 updating version 3 to 4 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 4 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:10: finishing write tid 3 to smithi11881747-10 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:10: oid 10 updating version 0 to 3 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:10: oid 10 updating version 3 to 4 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:9: finishing write tid 1 to smithi11881747-9 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:9: finishing write tid 2 to smithi11881747-9 2024-03-20T08:01:34.218 INFO:tasks.rados.rados.0.smithi118.stdout:8: finishing write tid 3 to smithi11881747-8 2024-03-20T08:01:34.219 INFO:tasks.rados.rados.0.smithi118.stdout:8: oid 8 updating version 0 to 1 2024-03-20T08:01:34.219 INFO:tasks.rados.rados.0.smithi118.stdout:8: oid 8 updating version 1 to 2 2024-03-20T08:01:34.219 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-03-20T08:01:34.219 INFO:tasks.rados.rados.0.smithi118.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-03-20T08:01:34.219 INFO:tasks.rados.rados.0.smithi118.stdout:7: finishing write tid 1 to smithi11881747-7 2024-03-20T08:01:34.219 INFO:tasks.rados.rados.0.smithi118.stdout:7: finishing write tid 2 to smithi11881747-7 2024-03-20T08:01:34.219 INFO:tasks.rados.rados.0.smithi118.stdout:12: finishing write tid 1 to smithi11881747-12 2024-03-20T08:01:34.219 INFO:tasks.rados.rados.0.smithi118.stdout:15: seq_num 14 ranges {0=237568} 2024-03-20T08:01:34.221 INFO:tasks.rados.rados.0.smithi118.stdout:15: writing smithi11881747-15 from 0 to 237568 tid 1 2024-03-20T08:01:34.221 INFO:tasks.rados.rados.0.smithi118.stdout:2: done (12 left) 2024-03-20T08:01:34.221 INFO:tasks.rados.rados.0.smithi118.stdout:3: done (11 left) 2024-03-20T08:01:34.221 INFO:tasks.rados.rados.0.smithi118.stdout:6: done (10 left) 2024-03-20T08:01:34.221 INFO:tasks.rados.rados.0.smithi118.stdout:8: done (9 left) 2024-03-20T08:01:34.221 INFO:tasks.rados.rados.0.smithi118.stdout:10: done (8 left) 2024-03-20T08:01:34.221 INFO:tasks.rados.rados.0.smithi118.stdout:16: write initial oid 16 2024-03-20T08:01:34.221 INFO:tasks.rados.rados.0.smithi118.stdout:12: finishing write tid 2 to smithi11881747-12 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:7: finishing write tid 3 to smithi11881747-7 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:7: oid 7 updating version 0 to 1 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:7: oid 7 updating version 1 to 2 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:12: finishing write tid 3 to smithi11881747-12 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:12: oid 12 updating version 0 to 4 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:12: oid 12 version 4 is already newer than 3 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 4 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-03-20T08:01:34.222 INFO:tasks.rados.rados.0.smithi118.stdout:11: finishing write tid 1 to smithi11881747-11 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:11: finishing write tid 2 to smithi11881747-11 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:13: finishing write tid 1 to smithi11881747-13 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:5: finishing write tid 3 to smithi11881747-5 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:5: oid 5 updating version 0 to 1 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:5: oid 5 updating version 1 to 2 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:16: seq_num 15 ranges {0=253952} 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:16: writing smithi11881747-16 from 0 to 253952 tid 1 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:9: finishing write tid 3 to smithi11881747-9 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:9: oid 9 updating version 0 to 3 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:9: oid 9 updating version 3 to 4 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 4 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:11: finishing write tid 3 to smithi11881747-11 2024-03-20T08:01:34.223 INFO:tasks.rados.rados.0.smithi118.stdout:11: oid 11 updating version 0 to 6 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:11: oid 11 version 6 is already newer than 5 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:14: finishing write tid 1 to smithi11881747-14 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:14: finishing write tid 2 to smithi11881747-14 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:5: done (8 left) 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:7: done (7 left) 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:9: done (6 left) 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:11: done (5 left) 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:12: done (4 left) 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:17: write initial oid 17 2024-03-20T08:01:34.224 INFO:tasks.rados.rados.0.smithi118.stdout:17: seq_num 16 ranges {0=270336} 2024-03-20T08:01:34.225 INFO:tasks.rados.rados.0.smithi118.stdout:17: writing smithi11881747-17 from 0 to 270336 tid 1 2024-03-20T08:01:34.225 INFO:tasks.rados.rados.0.smithi118.stdout:13: finishing write tid 2 to smithi11881747-13 2024-03-20T08:01:34.225 INFO:tasks.rados.rados.0.smithi118.stdout:15: finishing write tid 1 to smithi11881747-15 2024-03-20T08:01:34.225 INFO:tasks.rados.rados.0.smithi118.stdout:14: finishing write tid 3 to smithi11881747-14 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:14: oid 14 updating version 0 to 4 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:14: oid 14 version 4 is already newer than 3 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 4 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:15: finishing write tid 2 to smithi11881747-15 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:14: done (4 left) 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:18: write initial oid 18 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:18: seq_num 17 ranges {0=286720} 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:18: writing smithi11881747-18 from 0 to 286720 tid 1 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:19: write initial oid 19 2024-03-20T08:01:34.226 INFO:tasks.rados.rados.0.smithi118.stdout:19: seq_num 18 ranges {0=303104} 2024-03-20T08:01:34.227 INFO:tasks.rados.rados.0.smithi118.stdout:19: writing smithi11881747-19 from 0 to 303104 tid 1 2024-03-20T08:01:34.227 INFO:tasks.rados.rados.0.smithi118.stdout:20: write initial oid 20 2024-03-20T08:01:34.227 INFO:tasks.rados.rados.0.smithi118.stdout:20: seq_num 19 ranges {0=319488} 2024-03-20T08:01:34.229 INFO:tasks.rados.rados.0.smithi118.stdout:20: writing smithi11881747-20 from 0 to 319488 tid 1 2024-03-20T08:01:34.229 INFO:tasks.rados.rados.0.smithi118.stdout:15: finishing write tid 3 to smithi11881747-15 2024-03-20T08:01:34.229 INFO:tasks.rados.rados.0.smithi118.stdout:15: oid 15 updating version 0 to 1 2024-03-20T08:01:34.229 INFO:tasks.rados.rados.0.smithi118.stdout:15: oid 15 updating version 1 to 2 2024-03-20T08:01:34.229 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:13: finishing write tid 3 to smithi11881747-13 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:13: oid 13 updating version 0 to 7 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:13: oid 13 updating version 7 to 8 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 8 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:16: finishing write tid 1 to smithi11881747-16 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:16: finishing write tid 2 to smithi11881747-16 2024-03-20T08:01:34.230 INFO:tasks.rados.rados.0.smithi118.stdout:16: finishing write tid 3 to smithi11881747-16 2024-03-20T08:01:34.231 INFO:tasks.rados.rados.0.smithi118.stdout:16: oid 16 updating version 0 to 2 2024-03-20T08:01:34.231 INFO:tasks.rados.rados.0.smithi118.stdout:16: oid 16 version 2 is already newer than 1 2024-03-20T08:01:34.231 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-03-20T08:01:34.231 INFO:tasks.rados.rados.0.smithi118.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-03-20T08:01:34.231 INFO:tasks.rados.rados.0.smithi118.stdout:17: finishing write tid 1 to smithi11881747-17 2024-03-20T08:01:34.231 INFO:tasks.rados.rados.0.smithi118.stdout:17: finishing write tid 2 to smithi11881747-17 2024-03-20T08:01:34.231 INFO:tasks.rados.rados.0.smithi118.stdout:17: finishing write tid 3 to smithi11881747-17 2024-03-20T08:01:34.232 INFO:tasks.rados.rados.0.smithi118.stdout:17: oid 17 updating version 0 to 1 2024-03-20T08:01:34.232 INFO:tasks.rados.rados.0.smithi118.stdout:17: oid 17 updating version 1 to 2 2024-03-20T08:01:34.232 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 2 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-03-20T08:01:34.232 INFO:tasks.rados.rados.0.smithi118.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-03-20T08:01:34.232 INFO:tasks.rados.rados.0.smithi118.stdout:18: finishing write tid 1 to smithi11881747-18 2024-03-20T08:01:34.232 INFO:tasks.rados.rados.0.smithi118.stdout:18: finishing write tid 2 to smithi11881747-18 2024-03-20T08:01:34.232 INFO:tasks.rados.rados.0.smithi118.stdout:18: finishing write tid 3 to smithi11881747-18 2024-03-20T08:01:34.232 INFO:tasks.rados.rados.0.smithi118.stdout:18: oid 18 updating version 0 to 1 2024-03-20T08:01:34.233 INFO:tasks.rados.rados.0.smithi118.stdout:18: oid 18 updating version 1 to 2 2024-03-20T08:01:34.233 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 2 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-03-20T08:01:34.233 INFO:tasks.rados.rados.0.smithi118.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-03-20T08:01:34.233 INFO:tasks.rados.rados.0.smithi118.stdout:13: done (6 left) 2024-03-20T08:01:34.233 INFO:tasks.rados.rados.0.smithi118.stdout:15: done (5 left) 2024-03-20T08:01:34.233 INFO:tasks.rados.rados.0.smithi118.stdout:16: done (4 left) 2024-03-20T08:01:34.233 INFO:tasks.rados.rados.0.smithi118.stdout:17: done (3 left) 2024-03-20T08:01:34.233 INFO:tasks.rados.rados.0.smithi118.stdout:18: done (2 left) 2024-03-20T08:01:34.234 INFO:tasks.rados.rados.0.smithi118.stdout:21: write initial oid 21 2024-03-20T08:01:34.234 INFO:tasks.rados.rados.0.smithi118.stdout:21: seq_num 20 ranges {0=344064} 2024-03-20T08:01:34.235 INFO:tasks.rados.rados.0.smithi118.stdout:21: writing smithi11881747-21 from 0 to 344064 tid 1 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:19: finishing write tid 1 to smithi11881747-19 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:19: finishing write tid 2 to smithi11881747-19 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:19: finishing write tid 3 to smithi11881747-19 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:19: oid 19 updating version 0 to 3 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:19: oid 19 updating version 3 to 4 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 4 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:19: done (2 left) 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:22: write initial oid 22 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:22: seq_num 21 ranges {0=360448} 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:22: writing smithi11881747-22 from 0 to 360448 tid 1 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:23: write initial oid 23 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:23: seq_num 22 ranges {0=376832} 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:23: writing smithi11881747-23 from 0 to 376832 tid 1 2024-03-20T08:01:34.236 INFO:tasks.rados.rados.0.smithi118.stdout:24: write initial oid 24 2024-03-20T08:01:34.237 INFO:tasks.rados.rados.0.smithi118.stdout:24: seq_num 23 ranges {0=393216} 2024-03-20T08:01:34.238 INFO:tasks.rados.rados.0.smithi118.stdout:24: writing smithi11881747-24 from 0 to 393216 tid 1 2024-03-20T08:01:34.238 INFO:tasks.rados.rados.0.smithi118.stdout:25: write initial oid 25 2024-03-20T08:01:34.238 INFO:tasks.rados.rados.0.smithi118.stdout:25: seq_num 24 ranges {0=409600} 2024-03-20T08:01:34.241 INFO:tasks.rados.rados.0.smithi118.stdout:25: writing smithi11881747-25 from 0 to 409600 tid 1 2024-03-20T08:01:34.241 INFO:tasks.rados.rados.0.smithi118.stdout:26: write initial oid 26 2024-03-20T08:01:34.241 INFO:tasks.rados.rados.0.smithi118.stdout:26: seq_num 25 ranges {0=425984} 2024-03-20T08:01:34.245 INFO:tasks.rados.rados.0.smithi118.stdout:26: writing smithi11881747-26 from 0 to 425984 tid 1 2024-03-20T08:01:34.245 INFO:tasks.rados.rados.0.smithi118.stdout:20: finishing write tid 1 to smithi11881747-20 2024-03-20T08:01:34.245 INFO:tasks.rados.rados.0.smithi118.stdout:20: finishing write tid 2 to smithi11881747-20 2024-03-20T08:01:34.245 INFO:tasks.rados.rados.0.smithi118.stdout:20: finishing write tid 3 to smithi11881747-20 2024-03-20T08:01:34.245 INFO:tasks.rados.rados.0.smithi118.stdout:20: oid 20 updating version 0 to 4 2024-03-20T08:01:34.245 INFO:tasks.rados.rados.0.smithi118.stdout:20: oid 20 version 4 is already newer than 3 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 4 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:21: finishing write tid 1 to smithi11881747-21 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:21: finishing write tid 2 to smithi11881747-21 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:22: finishing write tid 1 to smithi11881747-22 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:22: finishing write tid 2 to smithi11881747-22 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:21: finishing write tid 3 to smithi11881747-21 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:21: oid 21 updating version 0 to 2 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:21: oid 21 version 2 is already newer than 1 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 2 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:23: finishing write tid 1 to smithi11881747-23 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:23: finishing write tid 2 to smithi11881747-23 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:22: finishing write tid 3 to smithi11881747-22 2024-03-20T08:01:34.246 INFO:tasks.rados.rados.0.smithi118.stdout:22: oid 22 updating version 0 to 4 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:22: oid 22 version 4 is already newer than 3 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:24: finishing write tid 1 to smithi11881747-24 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:23: finishing write tid 3 to smithi11881747-23 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:23: oid 23 updating version 0 to 3 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:23: oid 23 updating version 3 to 4 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 4 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:24: finishing write tid 2 to smithi11881747-24 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:24: finishing write tid 3 to smithi11881747-24 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:24: oid 24 updating version 0 to 2 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:24: oid 24 version 2 is already newer than 1 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 2 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-03-20T08:01:34.247 INFO:tasks.rados.rados.0.smithi118.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:25: finishing write tid 1 to smithi11881747-25 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:25: finishing write tid 2 to smithi11881747-25 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:25: finishing write tid 3 to smithi11881747-25 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:25: oid 25 updating version 0 to 10 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:25: oid 25 version 10 is already newer than 9 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 10 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:20: done (6 left) 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:21: done (5 left) 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:22: done (4 left) 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:23: done (3 left) 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:24: done (2 left) 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:25: done (1 left) 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:27: write initial oid 27 2024-03-20T08:01:34.248 INFO:tasks.rados.rados.0.smithi118.stdout:27: seq_num 26 ranges {0=442368} 2024-03-20T08:01:34.249 INFO:tasks.rados.rados.0.smithi118.stdout:27: writing smithi11881747-27 from 0 to 442368 tid 1 2024-03-20T08:01:34.249 INFO:tasks.rados.rados.0.smithi118.stdout:28: write initial oid 28 2024-03-20T08:01:34.249 INFO:tasks.rados.rados.0.smithi118.stdout:28: seq_num 27 ranges {0=458752} 2024-03-20T08:01:34.250 INFO:tasks.rados.rados.0.smithi118.stdout:28: writing smithi11881747-28 from 0 to 458752 tid 1 2024-03-20T08:01:34.251 INFO:tasks.rados.rados.0.smithi118.stdout:29: write initial oid 29 2024-03-20T08:01:34.251 INFO:tasks.rados.rados.0.smithi118.stdout:29: seq_num 28 ranges {0=475136} 2024-03-20T08:01:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:29: writing smithi11881747-29 from 0 to 475136 tid 1 2024-03-20T08:01:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:30: write initial oid 30 2024-03-20T08:01:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:30: seq_num 29 ranges {0=491520} 2024-03-20T08:01:34.257 INFO:tasks.rados.rados.0.smithi118.stdout:30: writing smithi11881747-30 from 0 to 491520 tid 1 2024-03-20T08:01:34.257 INFO:tasks.rados.rados.0.smithi118.stdout:31: write initial oid 31 2024-03-20T08:01:34.257 INFO:tasks.rados.rados.0.smithi118.stdout:31: seq_num 30 ranges {0=507904} 2024-03-20T08:01:34.260 INFO:tasks.rados.rados.0.smithi118.stdout:31: writing smithi11881747-31 from 0 to 507904 tid 1 2024-03-20T08:01:34.260 INFO:tasks.rados.rados.0.smithi118.stdout:32: write initial oid 32 2024-03-20T08:01:34.260 INFO:tasks.rados.rados.0.smithi118.stdout:32: seq_num 31 ranges {0=524288} 2024-03-20T08:01:34.263 INFO:tasks.rados.rados.0.smithi118.stdout:32: writing smithi11881747-32 from 0 to 524288 tid 1 2024-03-20T08:01:34.263 INFO:tasks.rados.rados.0.smithi118.stdout:33: write initial oid 33 2024-03-20T08:01:34.263 INFO:tasks.rados.rados.0.smithi118.stdout:33: seq_num 32 ranges {0=540672} 2024-03-20T08:01:34.267 INFO:tasks.rados.rados.0.smithi118.stdout:33: writing smithi11881747-33 from 0 to 540672 tid 1 2024-03-20T08:01:34.267 INFO:tasks.rados.rados.0.smithi118.stdout:34: write initial oid 34 2024-03-20T08:01:34.267 INFO:tasks.rados.rados.0.smithi118.stdout:34: seq_num 33 ranges {0=557056} 2024-03-20T08:01:34.271 INFO:tasks.rados.rados.0.smithi118.stdout:34: writing smithi11881747-34 from 0 to 557056 tid 1 2024-03-20T08:01:34.271 INFO:tasks.rados.rados.0.smithi118.stdout:35: write initial oid 35 2024-03-20T08:01:34.271 INFO:tasks.rados.rados.0.smithi118.stdout:35: seq_num 34 ranges {0=573440} 2024-03-20T08:01:34.274 INFO:tasks.rados.rados.0.smithi118.stdout:35: writing smithi11881747-35 from 0 to 573440 tid 1 2024-03-20T08:01:34.274 INFO:tasks.rados.rados.0.smithi118.stdout:36: write initial oid 36 2024-03-20T08:01:34.275 INFO:tasks.rados.rados.0.smithi118.stdout:36: seq_num 35 ranges {0=589824} 2024-03-20T08:01:34.278 INFO:tasks.rados.rados.0.smithi118.stdout:36: writing smithi11881747-36 from 0 to 589824 tid 1 2024-03-20T08:01:34.278 INFO:tasks.rados.rados.0.smithi118.stdout:37: write initial oid 37 2024-03-20T08:01:34.278 INFO:tasks.rados.rados.0.smithi118.stdout:37: seq_num 36 ranges {0=606208} 2024-03-20T08:01:34.282 INFO:tasks.rados.rados.0.smithi118.stdout:37: writing smithi11881747-37 from 0 to 606208 tid 1 2024-03-20T08:01:34.282 INFO:tasks.rados.rados.0.smithi118.stdout:38: write initial oid 38 2024-03-20T08:01:34.282 INFO:tasks.rados.rados.0.smithi118.stdout:38: seq_num 37 ranges {0=622592} 2024-03-20T08:01:34.286 INFO:tasks.rados.rados.0.smithi118.stdout:38: writing smithi11881747-38 from 0 to 622592 tid 1 2024-03-20T08:01:34.286 INFO:tasks.rados.rados.0.smithi118.stdout:39: write initial oid 39 2024-03-20T08:01:34.286 INFO:tasks.rados.rados.0.smithi118.stdout:39: seq_num 38 ranges {0=638976} 2024-03-20T08:01:34.290 INFO:tasks.rados.rados.0.smithi118.stdout:39: writing smithi11881747-39 from 0 to 638976 tid 1 2024-03-20T08:01:34.290 INFO:tasks.rados.rados.0.smithi118.stdout:40: write initial oid 40 2024-03-20T08:01:34.290 INFO:tasks.rados.rados.0.smithi118.stdout:40: seq_num 39 ranges {0=663552} 2024-03-20T08:01:34.294 INFO:tasks.rados.rados.0.smithi118.stdout:40: writing smithi11881747-40 from 0 to 663552 tid 1 2024-03-20T08:01:34.294 INFO:tasks.rados.rados.0.smithi118.stdout:41: write initial oid 41 2024-03-20T08:01:34.294 INFO:tasks.rados.rados.0.smithi118.stdout:41: seq_num 40 ranges {0=679936} 2024-03-20T08:01:34.300 INFO:tasks.rados.rados.0.smithi118.stdout:41: writing smithi11881747-41 from 0 to 679936 tid 1 2024-03-20T08:01:34.300 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 16 2024-03-20T08:01:34.300 INFO:tasks.rados.rados.0.smithi118.stdout:26: finishing write tid 1 to smithi11881747-26 2024-03-20T08:01:34.300 INFO:tasks.rados.rados.0.smithi118.stdout:26: finishing write tid 2 to smithi11881747-26 2024-03-20T08:01:34.300 INFO:tasks.rados.rados.0.smithi118.stdout:26: finishing write tid 3 to smithi11881747-26 2024-03-20T08:01:34.300 INFO:tasks.rados.rados.0.smithi118.stdout:26: oid 26 updating version 0 to 4 2024-03-20T08:01:34.300 INFO:tasks.rados.rados.0.smithi118.stdout:26: oid 26 version 4 is already newer than 3 2024-03-20T08:01:34.301 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 4 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-03-20T08:01:34.301 INFO:tasks.rados.rados.0.smithi118.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:01:34.301 INFO:tasks.rados.rados.0.smithi118.stdout:27: finishing write tid 1 to smithi11881747-27 2024-03-20T08:01:34.301 INFO:tasks.rados.rados.0.smithi118.stdout:26: done (15 left) 2024-03-20T08:01:34.301 INFO:tasks.rados.rados.0.smithi118.stdout:42: write initial oid 42 2024-03-20T08:01:34.301 INFO:tasks.rados.rados.0.smithi118.stdout:42: seq_num 41 ranges {0=696320} 2024-03-20T08:01:34.303 INFO:tasks.rados.rados.0.smithi118.stdout:42: writing smithi11881747-42 from 0 to 696320 tid 1 2024-03-20T08:01:34.303 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 16 2024-03-20T08:01:34.304 INFO:tasks.rados.rados.0.smithi118.stdout:27: finishing write tid 2 to smithi11881747-27 2024-03-20T08:01:34.304 INFO:tasks.rados.rados.0.smithi118.stdout:27: finishing write tid 3 to smithi11881747-27 2024-03-20T08:01:34.304 INFO:tasks.rados.rados.0.smithi118.stdout:27: oid 27 updating version 0 to 6 2024-03-20T08:01:34.304 INFO:tasks.rados.rados.0.smithi118.stdout:27: oid 27 version 6 is already newer than 5 2024-03-20T08:01:34.304 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-03-20T08:01:34.304 INFO:tasks.rados.rados.0.smithi118.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-03-20T08:01:34.305 INFO:tasks.rados.rados.0.smithi118.stdout:28: finishing write tid 1 to smithi11881747-28 2024-03-20T08:01:34.305 INFO:tasks.rados.rados.0.smithi118.stdout:27: done (15 left) 2024-03-20T08:01:34.305 INFO:tasks.rados.rados.0.smithi118.stdout:43: write initial oid 43 2024-03-20T08:01:34.305 INFO:tasks.rados.rados.0.smithi118.stdout:43: seq_num 42 ranges {0=712704} 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:43: writing smithi11881747-43 from 0 to 712704 tid 1 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 16 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:28: finishing write tid 2 to smithi11881747-28 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:28: finishing write tid 3 to smithi11881747-28 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:28: oid 28 updating version 0 to 6 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:28: oid 28 version 6 is already newer than 5 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:29: finishing write tid 1 to smithi11881747-29 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:29: finishing write tid 2 to smithi11881747-29 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:29: finishing write tid 3 to smithi11881747-29 2024-03-20T08:01:34.308 INFO:tasks.rados.rados.0.smithi118.stdout:29: oid 29 updating version 0 to 7 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:29: oid 29 updating version 7 to 8 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 8 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:30: finishing write tid 1 to smithi11881747-30 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:30: finishing write tid 2 to smithi11881747-30 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:28: done (15 left) 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:29: done (14 left) 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:44: write initial oid 44 2024-03-20T08:01:34.309 INFO:tasks.rados.rados.0.smithi118.stdout:44: seq_num 43 ranges {0=729088} 2024-03-20T08:01:34.311 INFO:tasks.rados.rados.0.smithi118.stdout:44: writing smithi11881747-44 from 0 to 729088 tid 1 2024-03-20T08:01:34.311 INFO:tasks.rados.rados.0.smithi118.stdout:45: write initial oid 45 2024-03-20T08:01:34.311 INFO:tasks.rados.rados.0.smithi118.stdout:45: seq_num 44 ranges {0=745472} 2024-03-20T08:01:34.316 INFO:tasks.rados.rados.0.smithi118.stdout:45: writing smithi11881747-45 from 0 to 745472 tid 1 2024-03-20T08:01:34.316 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 16 2024-03-20T08:01:34.316 INFO:tasks.rados.rados.0.smithi118.stdout:30: finishing write tid 3 to smithi11881747-30 2024-03-20T08:01:34.316 INFO:tasks.rados.rados.0.smithi118.stdout:30: oid 30 updating version 0 to 4 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:30: oid 30 version 4 is already newer than 3 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:31: finishing write tid 1 to smithi11881747-31 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:31: finishing write tid 2 to smithi11881747-31 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:31: finishing write tid 3 to smithi11881747-31 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:31: oid 31 updating version 0 to 6 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:31: oid 31 version 6 is already newer than 5 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:32: finishing write tid 1 to smithi11881747-32 2024-03-20T08:01:34.317 INFO:tasks.rados.rados.0.smithi118.stdout:30: done (15 left) 2024-03-20T08:01:34.318 INFO:tasks.rados.rados.0.smithi118.stdout:31: done (14 left) 2024-03-20T08:01:34.318 INFO:tasks.rados.rados.0.smithi118.stdout:46: write initial oid 46 2024-03-20T08:01:34.318 INFO:tasks.rados.rados.0.smithi118.stdout:46: seq_num 45 ranges {0=761856} 2024-03-20T08:01:34.320 INFO:tasks.rados.rados.0.smithi118.stdout:46: writing smithi11881747-46 from 0 to 761856 tid 1 2024-03-20T08:01:34.320 INFO:tasks.rados.rados.0.smithi118.stdout:47: write initial oid 47 2024-03-20T08:01:34.320 INFO:tasks.rados.rados.0.smithi118.stdout:47: seq_num 46 ranges {0=778240} 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:47: writing smithi11881747-47 from 0 to 778240 tid 1 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 16 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:32: finishing write tid 2 to smithi11881747-32 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:32: finishing write tid 3 to smithi11881747-32 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:32: oid 32 updating version 0 to 6 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:32: oid 32 version 6 is already newer than 5 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 6 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:33: finishing write tid 1 to smithi11881747-33 2024-03-20T08:01:34.325 INFO:tasks.rados.rados.0.smithi118.stdout:32: done (15 left) 2024-03-20T08:01:34.326 INFO:tasks.rados.rados.0.smithi118.stdout:48: write initial oid 48 2024-03-20T08:01:34.326 INFO:tasks.rados.rados.0.smithi118.stdout:48: seq_num 47 ranges {0=794624} 2024-03-20T08:01:34.329 INFO:tasks.rados.rados.0.smithi118.stdout:48: writing smithi11881747-48 from 0 to 794624 tid 1 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 16 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:33: finishing write tid 2 to smithi11881747-33 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:33: finishing write tid 3 to smithi11881747-33 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:33: oid 33 updating version 0 to 4 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:33: oid 33 version 4 is already newer than 3 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 4 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:34: finishing write tid 1 to smithi11881747-34 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:34: finishing write tid 2 to smithi11881747-34 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:33: done (15 left) 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:49: write initial oid 49 2024-03-20T08:01:34.330 INFO:tasks.rados.rados.0.smithi118.stdout:49: seq_num 48 ranges {0=409600,409600=401408} 2024-03-20T08:01:34.332 INFO:tasks.rados.rados.0.smithi118.stdout:49: writing smithi11881747-49 from 0 to 409600 tid 1 2024-03-20T08:01:34.335 INFO:tasks.rados.rados.0.smithi118.stdout:49: writing smithi11881747-49 from 409600 to 811008 tid 2 2024-03-20T08:01:34.335 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 16 2024-03-20T08:01:34.335 INFO:tasks.rados.rados.0.smithi118.stdout:34: finishing write tid 3 to smithi11881747-34 2024-03-20T08:01:34.335 INFO:tasks.rados.rados.0.smithi118.stdout:34: oid 34 updating version 0 to 2 2024-03-20T08:01:34.335 INFO:tasks.rados.rados.0.smithi118.stdout:34: oid 34 version 2 is already newer than 1 2024-03-20T08:01:34.335 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 2 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-03-20T08:01:34.335 INFO:tasks.rados.rados.0.smithi118.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-03-20T08:01:34.336 INFO:tasks.rados.rados.0.smithi118.stdout:34: done (15 left) 2024-03-20T08:01:34.336 INFO:tasks.rados.rados.0.smithi118.stdout:50: write initial oid 50 2024-03-20T08:01:34.336 INFO:tasks.rados.rados.0.smithi118.stdout:50: seq_num 49 ranges {0=425984,425984=401408} 2024-03-20T08:01:34.337 INFO:tasks.rados.rados.0.smithi118.stdout:50: writing smithi11881747-50 from 0 to 425984 tid 1 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout:50: writing smithi11881747-50 from 425984 to 827392 tid 2 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 16 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout:35: finishing write tid 1 to smithi11881747-35 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout:35: finishing write tid 2 to smithi11881747-35 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout:35: finishing write tid 3 to smithi11881747-35 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout:35: oid 35 updating version 0 to 6 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout:35: oid 35 version 6 is already newer than 5 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-03-20T08:01:34.341 INFO:tasks.rados.rados.0.smithi118.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:36: finishing write tid 1 to smithi11881747-36 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:36: finishing write tid 2 to smithi11881747-36 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:36: finishing write tid 3 to smithi11881747-36 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:36: oid 36 updating version 0 to 4 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:36: oid 36 version 4 is already newer than 3 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 4 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:37: finishing write tid 1 to smithi11881747-37 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:37: finishing write tid 2 to smithi11881747-37 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:37: finishing write tid 3 to smithi11881747-37 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:37: oid 37 updating version 0 to 1 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:37: oid 37 updating version 1 to 2 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 2 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-03-20T08:01:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:38: finishing write tid 1 to smithi11881747-38 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:35: done (15 left) 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:36: done (14 left) 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:37: done (13 left) 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:51: copy_from oid 8 from oid 14 current snap is 0 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:38: finishing write tid 2 to smithi11881747-38 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 0 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:38: finishing write tid 3 to smithi11881747-38 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:38: oid 38 updating version 0 to 6 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:38: oid 38 version 6 is already newer than 5 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 6 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-03-20T08:01:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:39: finishing write tid 1 to smithi11881747-39 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:39: finishing write tid 2 to smithi11881747-39 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:39: finishing write tid 3 to smithi11881747-39 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:39: oid 39 updating version 0 to 8 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:39: oid 39 version 8 is already newer than 7 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 8 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:40: finishing write tid 1 to smithi11881747-40 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:40: finishing write tid 2 to smithi11881747-40 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:40: finishing write tid 3 to smithi11881747-40 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:40: oid 40 updating version 0 to 4 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:40: oid 40 version 4 is already newer than 3 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 4 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-03-20T08:01:34.344 INFO:tasks.rados.rados.0.smithi118.stdout:52: seq_num 50 ranges {286720=442368,729088=401408} 2024-03-20T08:01:34.345 INFO:tasks.rados.rados.0.smithi118.stdout:52: writing smithi11881747-18 from 286720 to 729088 tid 1 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:52: writing smithi11881747-18 from 729088 to 1130496 tid 2 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:38: done (14 left) 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:39: done (13 left) 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:40: done (12 left) 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:53: rmattr oid 38 current snap is 0 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:53: done (12 left) 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:54: delete oid 4 current snap is 0 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:41: finishing write tid 1 to smithi11881747-41 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:41: finishing write tid 2 to smithi11881747-41 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:41: finishing write tid 3 to smithi11881747-41 2024-03-20T08:01:34.351 INFO:tasks.rados.rados.0.smithi118.stdout:41: oid 41 updating version 0 to 9 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:41: oid 41 updating version 9 to 10 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 10 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:42: finishing write tid 1 to smithi11881747-42 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:42: finishing write tid 2 to smithi11881747-42 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:42: finishing write tid 3 to smithi11881747-42 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:42: oid 42 updating version 0 to 12 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:42: oid 42 version 12 is already newer than 11 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:43: finishing write tid 1 to smithi11881747-43 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:43: finishing write tid 2 to smithi11881747-43 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:43: finishing write tid 3 to smithi11881747-43 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:43: oid 43 updating version 0 to 8 2024-03-20T08:01:34.352 INFO:tasks.rados.rados.0.smithi118.stdout:43: oid 43 version 8 is already newer than 7 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 8 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:44: finishing write tid 1 to smithi11881747-44 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:44: finishing write tid 2 to smithi11881747-44 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:44: finishing write tid 3 to smithi11881747-44 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:44: oid 44 updating version 0 to 6 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:44: oid 44 version 6 is already newer than 5 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:45: finishing write tid 1 to smithi11881747-45 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:45: finishing write tid 2 to smithi11881747-45 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:45: finishing write tid 3 to smithi11881747-45 2024-03-20T08:01:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:45: oid 45 updating version 0 to 5 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:45: oid 45 updating version 5 to 6 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 6 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:46: finishing write tid 1 to smithi11881747-46 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:46: finishing write tid 2 to smithi11881747-46 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:46: finishing write tid 3 to smithi11881747-46 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:46: oid 46 updating version 0 to 8 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:46: oid 46 version 8 is already newer than 7 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 8 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:47: finishing write tid 1 to smithi11881747-47 2024-03-20T08:01:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:47: finishing write tid 2 to smithi11881747-47 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:47: finishing write tid 3 to smithi11881747-47 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:47: oid 47 updating version 0 to 11 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:47: oid 47 updating version 11 to 12 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 12 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:48: finishing write tid 1 to smithi11881747-48 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:48: finishing write tid 2 to smithi11881747-48 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:48: finishing write tid 3 to smithi11881747-48 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:48: oid 48 updating version 0 to 8 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:48: oid 48 version 8 is already newer than 7 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 8 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:49: finishing write tid 1 to smithi11881747-49 2024-03-20T08:01:34.355 INFO:tasks.rados.rados.0.smithi118.stdout:49: finishing write tid 2 to smithi11881747-49 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:49: finishing write tid 3 to smithi11881747-49 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:49: finishing write tid 4 to smithi11881747-49 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:49: oid 49 updating version 0 to 11 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:49: oid 49 version 11 is already newer than 10 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:49: oid 49 version 11 is already newer than 9 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 11 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:50: finishing write tid 1 to smithi11881747-50 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:50: finishing write tid 2 to smithi11881747-50 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:50: finishing write tid 3 to smithi11881747-50 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:50: finishing write tid 4 to smithi11881747-50 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:50: oid 50 updating version 0 to 9 2024-03-20T08:01:34.356 INFO:tasks.rados.rados.0.smithi118.stdout:50: oid 50 version 9 is already newer than 8 2024-03-20T08:01:34.357 INFO:tasks.rados.rados.0.smithi118.stdout:50: oid 50 version 9 is already newer than 7 2024-03-20T08:01:34.357 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-03-20T08:01:34.357 INFO:tasks.rados.rados.0.smithi118.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-03-20T08:01:34.357 INFO:tasks.rados.rados.0.smithi118.stdout:51: finishing copy_from racing read to smithi11881747-8 2024-03-20T08:01:34.357 INFO:tasks.rados.rados.0.smithi118.stdout:51: finishing copy_from to smithi11881747-8 2024-03-20T08:01:34.357 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 4 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-03-20T08:01:34.358 INFO:tasks.rados.rados.0.smithi118.stdout:52: finishing write tid 1 to smithi11881747-18 2024-03-20T08:01:34.358 INFO:tasks.rados.rados.0.smithi118.stdout:41: done (12 left) 2024-03-20T08:01:34.358 INFO:tasks.rados.rados.0.smithi118.stdout:42: done (11 left) 2024-03-20T08:01:34.358 INFO:tasks.rados.rados.0.smithi118.stdout:43: done (10 left) 2024-03-20T08:01:34.358 INFO:tasks.rados.rados.0.smithi118.stdout:44: done (9 left) 2024-03-20T08:01:34.358 INFO:tasks.rados.rados.0.smithi118.stdout:45: done (8 left) 2024-03-20T08:01:34.358 INFO:tasks.rados.rados.0.smithi118.stdout:46: done (7 left) 2024-03-20T08:01:34.358 INFO:tasks.rados.rados.0.smithi118.stdout:47: done (6 left) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:48: done (5 left) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:49: done (4 left) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:50: done (3 left) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:51: done (2 left) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:54: done (1 left) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:55: read oid 15 snap -1 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:55: expect (ObjNum 14 snap 0 seq_num 14) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:56: rollback oid 31 current snap is 0 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:56: done (2 left) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:57: read oid 28 snap -1 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:57: expect (ObjNum 27 snap 0 seq_num 27) 2024-03-20T08:01:34.359 INFO:tasks.rados.rados.0.smithi118.stdout:58: copy_from oid 29 from oid 47 current snap is 0 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:59: rollback oid 45 current snap is 0 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:59: done (4 left) 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:60: copy_from oid 23 from oid 48 current snap is 0 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:61: delete oid 1 current snap is 0 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:52: finishing write tid 2 to smithi11881747-18 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:52: finishing write tid 3 to smithi11881747-18 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:52: finishing write tid 4 to smithi11881747-18 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:52: oid 18 updating version 0 to 14 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:52: oid 18 version 14 is already newer than 13 2024-03-20T08:01:34.360 INFO:tasks.rados.rados.0.smithi118.stdout:52: oid 18 version 14 is already newer than 12 2024-03-20T08:01:34.361 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 14 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-03-20T08:01:34.361 INFO:tasks.rados.rados.0.smithi118.stdout:52: left oid 18 (ObjNum 50 snap 0 seq_num 50) 2024-03-20T08:01:34.361 INFO:tasks.rados.rados.0.smithi118.stdout:55: expect (ObjNum 14 snap 0 seq_num 14) 2024-03-20T08:01:34.362 INFO:tasks.rados.rados.0.smithi118.stdout:52: done (5 left) 2024-03-20T08:01:34.362 INFO:tasks.rados.rados.0.smithi118.stdout:55: done (4 left) 2024-03-20T08:01:34.362 INFO:tasks.rados.rados.0.smithi118.stdout:61: done (3 left) 2024-03-20T08:01:34.362 INFO:tasks.rados.rados.0.smithi118.stdout:62: read oid 12 snap -1 2024-03-20T08:01:34.362 INFO:tasks.rados.rados.0.smithi118.stdout:62: expect (ObjNum 11 snap 0 seq_num 11) 2024-03-20T08:01:34.362 INFO:tasks.rados.rados.0.smithi118.stdout:63: snap_create 2024-03-20T08:01:34.362 INFO:tasks.rados.rados.0.smithi118.stdout:57: expect (ObjNum 27 snap 0 seq_num 27) 2024-03-20T08:01:34.372 INFO:tasks.rados.rados.0.smithi118.stdout:60: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:01:34.372 INFO:tasks.rados.rados.0.smithi118.stdout:60: finishing copy_from to smithi11881747-23 2024-03-20T08:01:34.372 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 8 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-03-20T08:01:34.372 INFO:tasks.rados.rados.0.smithi118.stdout:58: finishing copy_from racing read to smithi11881747-29 2024-03-20T08:01:34.372 INFO:tasks.rados.rados.0.smithi118.stdout:58: finishing copy_from to smithi11881747-29 2024-03-20T08:01:34.372 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 12 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:01:34.372 INFO:tasks.rados.rados.0.smithi118.stdout:62: expect (ObjNum 11 snap 0 seq_num 11) 2024-03-20T08:01:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:34 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/987064373' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-03-20T08:01:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:34 smithi196 ceph-mon[31033]: pgmap v90: 121 pgs: 121 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:34 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/987064373' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-03-20T08:01:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:34 smithi118 ceph-mon[27391]: pgmap v90: 121 pgs: 121 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:34.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:34 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/987064373' entity='client.0' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]: dispatch 2024-03-20T08:01:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:34 smithi118 ceph-mon[38981]: pgmap v90: 121 pgs: 121 active+clean; 580 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:35.207 INFO:tasks.rados.rados.0.smithi118.stdout:57: done (4 left) 2024-03-20T08:01:35.207 INFO:tasks.rados.rados.0.smithi118.stdout:58: done (3 left) 2024-03-20T08:01:35.207 INFO:tasks.rados.rados.0.smithi118.stdout:60: done (2 left) 2024-03-20T08:01:35.207 INFO:tasks.rados.rados.0.smithi118.stdout:62: done (1 left) 2024-03-20T08:01:35.207 INFO:tasks.rados.rados.0.smithi118.stdout:63: done (0 left) 2024-03-20T08:01:35.208 INFO:tasks.rados.rados.0.smithi118.stdout:64: copy_from oid 15 from oid 2 current snap is 1 2024-03-20T08:01:35.208 INFO:tasks.rados.rados.0.smithi118.stdout:65: read oid 30 snap -1 2024-03-20T08:01:35.208 INFO:tasks.rados.rados.0.smithi118.stdout:65: expect (ObjNum 29 snap 0 seq_num 29) 2024-03-20T08:01:35.208 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 1 2024-03-20T08:01:35.208 INFO:tasks.rados.rados.0.smithi118.stdout:66: seq_num 53 ranges {794624=491520,1286144=401408} 2024-03-20T08:01:35.214 INFO:tasks.rados.rados.0.smithi118.stdout:66: writing smithi11881747-23 from 794624 to 1286144 tid 1 2024-03-20T08:01:35.217 INFO:tasks.rados.rados.0.smithi118.stdout:66: writing smithi11881747-23 from 1286144 to 1687552 tid 2 2024-03-20T08:01:35.217 INFO:tasks.rados.rados.0.smithi118.stdout:67: rollback oid 10 current snap is 1 2024-03-20T08:01:35.218 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 0 2024-03-20T08:01:35.218 INFO:tasks.rados.rados.0.smithi118.stdout:68: setattr oid 32 current snap is 1 2024-03-20T08:01:35.218 INFO:tasks.rados.rados.0.smithi118.stdout:65: expect (ObjNum 29 snap 0 seq_num 29) 2024-03-20T08:01:35.236 INFO:tasks.rados.rados.0.smithi118.stdout:64: finishing copy_from racing read to smithi11881747-15 2024-03-20T08:01:35.236 INFO:tasks.rados.rados.0.smithi118.stdout:64: finishing copy_from to smithi11881747-15 2024-03-20T08:01:35.236 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-03-20T08:01:35.236 INFO:tasks.rados.rados.0.smithi118.stdout:66: finishing write tid 1 to smithi11881747-23 2024-03-20T08:01:35.236 INFO:tasks.rados.rados.0.smithi118.stdout:67: finishing rollback tid 0 to smithi11881747-10 2024-03-20T08:01:35.236 INFO:tasks.rados.rados.0.smithi118.stdout:66: finishing write tid 2 to smithi11881747-23 2024-03-20T08:01:35.236 INFO:tasks.rados.rados.0.smithi118.stdout:67: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:01:35.236 INFO:tasks.rados.rados.0.smithi118.stdout:66: finishing write tid 3 to smithi11881747-23 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:67: finishing rollback tid 2 to smithi11881747-10 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 13 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:66: finishing write tid 4 to smithi11881747-23 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:66: oid 23 updating version 0 to 18 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:66: oid 23 version 18 is already newer than 17 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:66: oid 23 version 18 is already newer than 16 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 18 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:66: left oid 23 (ObjNum 53 snap 1 seq_num 53) 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:64: done (4 left) 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:65: done (3 left) 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:66: done (2 left) 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:67: done (1 left) 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:69: copy_from oid 45 from oid 34 current snap is 1 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:70: read oid 23 snap 0 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:70: expect (ObjNum 47 snap 0 seq_num 47) 2024-03-20T08:01:35.237 INFO:tasks.rados.rados.0.smithi118.stdout:71: copy_from oid 36 from oid 27 current snap is 1 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:72: rollback oid 49 current snap is 1 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 49 to 0 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:73: read oid 4 snap 0 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:73: expect deleted 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:74: rmattr oid 7 current snap is 1 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:74: done (6 left) 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:75: copy_from oid 20 from oid 12 current snap is 1 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:76: read oid 39 snap -1 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:76: expect (ObjNum 38 snap 0 seq_num 38) 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:77: copy_from oid 6 from oid 37 current snap is 1 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:78: read oid 2 snap -1 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:78: expect (ObjNum 1 snap 0 seq_num 1) 2024-03-20T08:01:35.238 INFO:tasks.rados.rados.0.smithi118.stdout:79: setattr oid 41 current snap is 1 2024-03-20T08:01:35.239 INFO:tasks.rados.rados.0.smithi118.stdout:73: done (10 left) 2024-03-20T08:01:35.239 INFO:tasks.rados.rados.0.smithi118.stdout:80: snap_create 2024-03-20T08:01:35.239 INFO:tasks.rados.rados.0.smithi118.stdout:78: expect (ObjNum 1 snap 0 seq_num 1) 2024-03-20T08:01:35.240 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 9 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-03-20T08:01:35.242 INFO:tasks.rados.rados.0.smithi118.stdout:70: expect (ObjNum 47 snap 0 seq_num 47) 2024-03-20T08:01:35.266 INFO:tasks.rados.rados.0.smithi118.stdout:72: finishing rollback tid 0 to smithi11881747-49 2024-03-20T08:01:35.266 INFO:tasks.rados.rados.0.smithi118.stdout:72: finishing rollback tid 1 to smithi11881747-49 2024-03-20T08:01:35.266 INFO:tasks.rados.rados.0.smithi118.stdout:76: expect (ObjNum 38 snap 0 seq_num 38) 2024-03-20T08:01:35.283 INFO:tasks.rados.rados.0.smithi118.stdout:72: finishing rollback tid 2 to smithi11881747-49 2024-03-20T08:01:35.283 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-03-20T08:01:35.283 INFO:tasks.rados.rados.0.smithi118.stdout:69: finishing copy_from racing read to smithi11881747-45 2024-03-20T08:01:35.283 INFO:tasks.rados.rados.0.smithi118.stdout:77: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:01:35.283 INFO:tasks.rados.rados.0.smithi118.stdout:77: finishing copy_from to smithi11881747-6 2024-03-20T08:01:35.283 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 2 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-03-20T08:01:35.283 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 20 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-03-20T08:01:35.284 INFO:tasks.rados.rados.0.smithi118.stdout:71: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:01:35.284 INFO:tasks.rados.rados.0.smithi118.stdout:75: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:01:35.284 INFO:tasks.rados.rados.0.smithi118.stdout:69: finishing copy_from to smithi11881747-45 2024-03-20T08:01:35.284 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 2 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-03-20T08:01:35.284 INFO:tasks.rados.rados.0.smithi118.stdout:71: finishing copy_from to smithi11881747-36 2024-03-20T08:01:35.284 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-03-20T08:01:35.284 INFO:tasks.rados.rados.0.smithi118.stdout:75: finishing copy_from to smithi11881747-20 2024-03-20T08:01:35.284 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 4 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-03-20T08:01:35.573 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:35 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/987064373' entity='client.0' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]': finished 2024-03-20T08:01:35.573 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:35 smithi118 ceph-mon[27391]: osdmap e77: 8 total, 8 up, 8 in 2024-03-20T08:01:35.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:35 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/987064373' entity='client.0' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]': finished 2024-03-20T08:01:35.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:35 smithi118 ceph-mon[38981]: osdmap e77: 8 total, 8 up, 8 in 2024-03-20T08:01:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:35 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/987064373' entity='client.0' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "write_fadvise_dontneed", "val": "false"}]': finished 2024-03-20T08:01:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:35 smithi196 ceph-mon[31033]: osdmap e77: 8 total, 8 up, 8 in 2024-03-20T08:01:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:68: done (10 left) 2024-03-20T08:01:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:69: done (9 left) 2024-03-20T08:01:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:70: done (8 left) 2024-03-20T08:01:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:71: done (7 left) 2024-03-20T08:01:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:72: done (6 left) 2024-03-20T08:01:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:75: done (5 left) 2024-03-20T08:01:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:76: done (4 left) 2024-03-20T08:01:36.213 INFO:tasks.rados.rados.0.smithi118.stdout:77: done (3 left) 2024-03-20T08:01:36.213 INFO:tasks.rados.rados.0.smithi118.stdout:78: done (2 left) 2024-03-20T08:01:36.213 INFO:tasks.rados.rados.0.smithi118.stdout:79: done (1 left) 2024-03-20T08:01:36.213 INFO:tasks.rados.rados.0.smithi118.stdout:80: done (0 left) 2024-03-20T08:01:36.213 INFO:tasks.rados.rados.0.smithi118.stdout:81: rollback oid 40 current snap is 2 2024-03-20T08:01:36.213 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 1 2024-03-20T08:01:36.213 INFO:tasks.rados.rados.0.smithi118.stdout:82: snap_create 2024-03-20T08:01:36.216 INFO:tasks.rados.rados.0.smithi118.stdout:81: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:01:36.216 INFO:tasks.rados.rados.0.smithi118.stdout:81: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:01:36.216 INFO:tasks.rados.rados.0.smithi118.stdout:81: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:01:36.216 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 12 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-03-20T08:01:36.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:36 smithi196 ceph-mon[31033]: osdmap e78: 8 total, 8 up, 8 in 2024-03-20T08:01:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:36 smithi196 ceph-mon[31033]: pgmap v93: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 49 op/s 2024-03-20T08:01:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:36 smithi118 ceph-mon[27391]: osdmap e78: 8 total, 8 up, 8 in 2024-03-20T08:01:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:36 smithi118 ceph-mon[27391]: pgmap v93: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 49 op/s 2024-03-20T08:01:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:36 smithi118 ceph-mon[38981]: osdmap e78: 8 total, 8 up, 8 in 2024-03-20T08:01:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:36 smithi118 ceph-mon[38981]: pgmap v93: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 49 op/s 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:81: done (1 left) 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:82: done (0 left) 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:83: rollback oid 9 current snap is 3 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 9 to 0 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:84: read oid 20 snap 1 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:84: expect (ObjNum 11 snap 0 seq_num 11) 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:85: read oid 41 snap -1 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:01:37.213 INFO:tasks.rados.rados.0.smithi118.stdout:86: snap_create 2024-03-20T08:01:37.217 INFO:tasks.rados.rados.0.smithi118.stdout:83: finishing rollback tid 0 to smithi11881747-9 2024-03-20T08:01:37.217 INFO:tasks.rados.rados.0.smithi118.stdout:83: finishing rollback tid 1 to smithi11881747-9 2024-03-20T08:01:37.218 INFO:tasks.rados.rados.0.smithi118.stdout:83: finishing rollback tid 2 to smithi11881747-9 2024-03-20T08:01:37.218 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 16 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-03-20T08:01:37.219 INFO:tasks.rados.rados.0.smithi118.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:01:37.242 INFO:tasks.rados.rados.0.smithi118.stdout:84: expect (ObjNum 11 snap 0 seq_num 11) 2024-03-20T08:01:37.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:37 smithi196 ceph-mon[31033]: osdmap e79: 8 total, 8 up, 8 in 2024-03-20T08:01:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:37 smithi118 ceph-mon[27391]: osdmap e79: 8 total, 8 up, 8 in 2024-03-20T08:01:37.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:37 smithi118 ceph-mon[38981]: osdmap e79: 8 total, 8 up, 8 in 2024-03-20T08:01:38.221 INFO:tasks.rados.rados.0.smithi118.stdout:83: done (3 left) 2024-03-20T08:01:38.222 INFO:tasks.rados.rados.0.smithi118.stdout:84: done (2 left) 2024-03-20T08:01:38.222 INFO:tasks.rados.rados.0.smithi118.stdout:85: done (1 left) 2024-03-20T08:01:38.222 INFO:tasks.rados.rados.0.smithi118.stdout:86: done (0 left) 2024-03-20T08:01:38.222 INFO:tasks.rados.rados.0.smithi118.stdout:87: snap_remove snap 0 2024-03-20T08:01:38.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:38 smithi196 ceph-mon[31033]: osdmap e80: 8 total, 8 up, 8 in 2024-03-20T08:01:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:38 smithi196 ceph-mon[31033]: pgmap v96: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 74 op/s 2024-03-20T08:01:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:38 smithi118 ceph-mon[27391]: osdmap e80: 8 total, 8 up, 8 in 2024-03-20T08:01:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:38 smithi118 ceph-mon[27391]: pgmap v96: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 74 op/s 2024-03-20T08:01:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:38 smithi118 ceph-mon[38981]: osdmap e80: 8 total, 8 up, 8 in 2024-03-20T08:01:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:38 smithi118 ceph-mon[38981]: pgmap v96: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 74 op/s 2024-03-20T08:01:39.229 INFO:tasks.rados.rados.0.smithi118.stdout:87: done (0 left) 2024-03-20T08:01:39.229 INFO:tasks.rados.rados.0.smithi118.stdout:88: read oid 29 snap 3 2024-03-20T08:01:39.229 INFO:tasks.rados.rados.0.smithi118.stdout:88: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:01:39.229 INFO:tasks.rados.rados.0.smithi118.stdout:89: read oid 46 snap -1 2024-03-20T08:01:39.230 INFO:tasks.rados.rados.0.smithi118.stdout:89: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:01:39.230 INFO:tasks.rados.rados.0.smithi118.stdout:90: copy_from oid 36 from oid 2 current snap is 4 2024-03-20T08:01:39.230 INFO:tasks.rados.rados.0.smithi118.stdout:91: rmattr oid 49 current snap is 4 2024-03-20T08:01:39.230 INFO:tasks.rados.rados.0.smithi118.stdout:91: done (3 left) 2024-03-20T08:01:39.230 INFO:tasks.rados.rados.0.smithi118.stdout:92: snap_remove snap 2 2024-03-20T08:01:39.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:39 smithi196 ceph-mon[31033]: osdmap e81: 8 total, 8 up, 8 in 2024-03-20T08:01:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:39 smithi118 ceph-mon[27391]: osdmap e81: 8 total, 8 up, 8 in 2024-03-20T08:01:39.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:39 smithi118 ceph-mon[38981]: osdmap e81: 8 total, 8 up, 8 in 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:92: done (3 left) 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:93: rmattr oid 30 current snap is 4 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:93: done (3 left) 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:94: read oid 42 snap 1 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:94: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:95: rmattr oid 14 current snap is 4 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:95: done (4 left) 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:96: read oid 9 snap 1 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:96: expect (ObjNum 8 snap 0 seq_num 8) 2024-03-20T08:01:40.232 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 4 2024-03-20T08:01:40.233 INFO:tasks.rados.rados.0.smithi118.stdout:97: seq_num 54 ranges {393216=507904,901120=401408} 2024-03-20T08:01:40.237 INFO:tasks.rados.rados.0.smithi118.stdout:97: writing smithi11881747-24 from 393216 to 901120 tid 1 2024-03-20T08:01:40.240 INFO:tasks.rados.rados.0.smithi118.stdout:97: writing smithi11881747-24 from 901120 to 1302528 tid 2 2024-03-20T08:01:40.240 INFO:tasks.rados.rados.0.smithi118.stdout:98: copy_from oid 31 from oid 6 current snap is 4 2024-03-20T08:01:40.240 INFO:tasks.rados.rados.0.smithi118.stdout:88: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:01:40.262 INFO:tasks.rados.rados.0.smithi118.stdout:89: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:01:40.283 INFO:tasks.rados.rados.0.smithi118.stdout:90: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:01:40.283 INFO:tasks.rados.rados.0.smithi118.stdout:90: finishing copy_from to smithi11881747-36 2024-03-20T08:01:40.283 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-03-20T08:01:40.283 INFO:tasks.rados.rados.0.smithi118.stdout:88: done (6 left) 2024-03-20T08:01:40.283 INFO:tasks.rados.rados.0.smithi118.stdout:89: done (5 left) 2024-03-20T08:01:40.283 INFO:tasks.rados.rados.0.smithi118.stdout:90: done (4 left) 2024-03-20T08:01:40.283 INFO:tasks.rados.rados.0.smithi118.stdout:99: snap_remove snap 3 2024-03-20T08:01:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:40 smithi196 ceph-mon[31033]: osdmap e82: 8 total, 8 up, 8 in 2024-03-20T08:01:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:40 smithi196 ceph-mon[31033]: pgmap v99: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:40 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:40 smithi118 ceph-mon[27391]: osdmap e82: 8 total, 8 up, 8 in 2024-03-20T08:01:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:40 smithi118 ceph-mon[27391]: pgmap v99: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:40 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:40 smithi118 ceph-mon[38981]: osdmap e82: 8 total, 8 up, 8 in 2024-03-20T08:01:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:40 smithi118 ceph-mon[38981]: pgmap v99: 121 pgs: 121 active+clean; 25 MiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:40 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:41.240 INFO:tasks.rados.rados.0.smithi118.stdout:99: done (4 left) 2024-03-20T08:01:41.240 INFO:tasks.rados.rados.0.smithi118.stdout:100: copy_from oid 25 from oid 30 current snap is 4 2024-03-20T08:01:41.240 INFO:tasks.rados.rados.0.smithi118.stdout:101: read oid 35 snap 1 2024-03-20T08:01:41.240 INFO:tasks.rados.rados.0.smithi118.stdout:101: expect (ObjNum 34 snap 0 seq_num 34) 2024-03-20T08:01:41.240 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 4 2024-03-20T08:01:41.240 INFO:tasks.rados.rados.0.smithi118.stdout:102: seq_num 55 ranges {188416=524288,712704=401408} 2024-03-20T08:01:41.245 INFO:tasks.rados.rados.0.smithi118.stdout:102: writing smithi11881747-20 from 188416 to 712704 tid 1 2024-03-20T08:01:41.248 INFO:tasks.rados.rados.0.smithi118.stdout:102: writing smithi11881747-20 from 712704 to 1114112 tid 2 2024-03-20T08:01:41.248 INFO:tasks.rados.rados.0.smithi118.stdout:94: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:01:41.270 INFO:tasks.rados.rados.0.smithi118.stdout:97: finishing write tid 1 to smithi11881747-24 2024-03-20T08:01:41.270 INFO:tasks.rados.rados.0.smithi118.stdout:96: expect (ObjNum 8 snap 0 seq_num 8) 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:97: finishing write tid 2 to smithi11881747-24 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:97: finishing write tid 3 to smithi11881747-24 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:97: finishing write tid 4 to smithi11881747-24 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:97: oid 24 updating version 0 to 12 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:97: oid 24 updating version 12 to 13 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:97: oid 24 version 13 is already newer than 11 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 13 (ObjNum 54 snap 4 seq_num 54) dirty exists 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:97: left oid 24 (ObjNum 54 snap 4 seq_num 54) 2024-03-20T08:01:41.274 INFO:tasks.rados.rados.0.smithi118.stdout:98: finishing copy_from racing read to smithi11881747-31 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:98: finishing copy_from to smithi11881747-31 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 2 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:102: finishing write tid 1 to smithi11881747-20 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:94: done (6 left) 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:96: done (5 left) 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:97: done (4 left) 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:98: done (3 left) 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:103: delete oid 44 current snap is 4 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:100: finishing copy_from racing read to smithi11881747-25 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:102: finishing write tid 2 to smithi11881747-20 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:102: finishing write tid 3 to smithi11881747-20 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:100: finishing copy_from to smithi11881747-25 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-03-20T08:01:41.275 INFO:tasks.rados.rados.0.smithi118.stdout:101: expect (ObjNum 34 snap 0 seq_num 34) 2024-03-20T08:01:41.289 INFO:tasks.rados.rados.0.smithi118.stdout:102: finishing write tid 4 to smithi11881747-20 2024-03-20T08:01:41.289 INFO:tasks.rados.rados.0.smithi118.stdout:102: oid 20 updating version 0 to 28 2024-03-20T08:01:41.289 INFO:tasks.rados.rados.0.smithi118.stdout:102: oid 20 version 28 is already newer than 27 2024-03-20T08:01:41.289 INFO:tasks.rados.rados.0.smithi118.stdout:102: oid 20 version 28 is already newer than 26 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 28 (ObjNum 55 snap 4 seq_num 55) dirty exists 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:102: left oid 20 (ObjNum 55 snap 4 seq_num 55) 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:100: done (3 left) 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:101: done (2 left) 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:102: done (1 left) 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:103: done (0 left) 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:104: read oid 43 snap -1 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:104: expect (ObjNum 42 snap 0 seq_num 42) 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:105: rollback oid 38 current snap is 4 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 38 to 1 2024-03-20T08:01:41.290 INFO:tasks.rados.rados.0.smithi118.stdout:106: delete oid 12 current snap is 4 2024-03-20T08:01:41.291 INFO:tasks.rados.rados.0.smithi118.stdout:106: done (2 left) 2024-03-20T08:01:41.291 INFO:tasks.rados.rados.0.smithi118.stdout:107: read oid 17 snap 1 2024-03-20T08:01:41.291 INFO:tasks.rados.rados.0.smithi118.stdout:107: expect (ObjNum 16 snap 0 seq_num 16) 2024-03-20T08:01:41.291 INFO:tasks.rados.rados.0.smithi118.stdout:108: rollback oid 40 current snap is 4 2024-03-20T08:01:41.291 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 1 2024-03-20T08:01:41.291 INFO:tasks.rados.rados.0.smithi118.stdout:109: delete oid 8 current snap is 4 2024-03-20T08:01:41.292 INFO:tasks.rados.rados.0.smithi118.stdout:105: finishing rollback tid 0 to smithi11881747-38 2024-03-20T08:01:41.292 INFO:tasks.rados.rados.0.smithi118.stdout:105: finishing rollback tid 1 to smithi11881747-38 2024-03-20T08:01:41.293 INFO:tasks.rados.rados.0.smithi118.stdout:105: finishing rollback tid 2 to smithi11881747-38 2024-03-20T08:01:41.293 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-03-20T08:01:41.293 INFO:tasks.rados.rados.0.smithi118.stdout:105: done (4 left) 2024-03-20T08:01:41.293 INFO:tasks.rados.rados.0.smithi118.stdout:109: done (3 left) 2024-03-20T08:01:41.293 INFO:tasks.rados.rados.0.smithi118.stdout:110: setattr oid 46 current snap is 4 2024-03-20T08:01:41.293 INFO:tasks.rados.rados.0.smithi118.stdout:108: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:01:41.294 INFO:tasks.rados.rados.0.smithi118.stdout:108: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:01:41.294 INFO:tasks.rados.rados.0.smithi118.stdout:108: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:01:41.294 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 19 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-03-20T08:01:41.295 INFO:tasks.rados.rados.0.smithi118.stdout:108: done (3 left) 2024-03-20T08:01:41.295 INFO:tasks.rados.rados.0.smithi118.stdout:111: snap_create 2024-03-20T08:01:41.295 INFO:tasks.rados.rados.0.smithi118.stdout:104: expect (ObjNum 42 snap 0 seq_num 42) 2024-03-20T08:01:41.316 INFO:tasks.rados.rados.0.smithi118.stdout:107: expect (ObjNum 16 snap 0 seq_num 16) 2024-03-20T08:01:41.324 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 14 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-03-20T08:01:41.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:41 smithi196 ceph-mon[31033]: osdmap e83: 8 total, 8 up, 8 in 2024-03-20T08:01:41.646 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:41 smithi118 ceph-mon[27391]: osdmap e83: 8 total, 8 up, 8 in 2024-03-20T08:01:41.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:41 smithi118 ceph-mon[38981]: osdmap e83: 8 total, 8 up, 8 in 2024-03-20T08:01:41.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:01:41] "GET /metrics HTTP/1.1" 200 36010 "" "Prometheus/2.43.0" 2024-03-20T08:01:42.244 INFO:tasks.rados.rados.0.smithi118.stdout:104: done (3 left) 2024-03-20T08:01:42.244 INFO:tasks.rados.rados.0.smithi118.stdout:107: done (2 left) 2024-03-20T08:01:42.244 INFO:tasks.rados.rados.0.smithi118.stdout:110: done (1 left) 2024-03-20T08:01:42.244 INFO:tasks.rados.rados.0.smithi118.stdout:111: done (0 left) 2024-03-20T08:01:42.244 INFO:tasks.rados.rados.0.smithi118.stdout:112: copy_from oid 25 from oid 6 current snap is 5 2024-03-20T08:01:42.244 INFO:tasks.rados.rados.0.smithi118.stdout:113: setattr oid 23 current snap is 5 2024-03-20T08:01:42.245 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 5 2024-03-20T08:01:42.245 INFO:tasks.rados.rados.0.smithi118.stdout:114: seq_num 59 ranges {24576=598016,622592=401408} 2024-03-20T08:01:42.249 INFO:tasks.rados.rados.0.smithi118.stdout:114: writing smithi11881747-15 from 24576 to 622592 tid 1 2024-03-20T08:01:42.253 INFO:tasks.rados.rados.0.smithi118.stdout:114: writing smithi11881747-15 from 622592 to 1024000 tid 2 2024-03-20T08:01:42.253 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 5 2024-03-20T08:01:42.253 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 26 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-03-20T08:01:42.253 INFO:tasks.rados.rados.0.smithi118.stdout:115: seq_num 60 ranges {1114112=614400,1728512=401408} 2024-03-20T08:01:42.262 INFO:tasks.rados.rados.0.smithi118.stdout:115: writing smithi11881747-20 from 1114112 to 1728512 tid 1 2024-03-20T08:01:42.265 INFO:tasks.rados.rados.0.smithi118.stdout:115: writing smithi11881747-20 from 1728512 to 2129920 tid 2 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:112: finishing copy_from racing read to smithi11881747-25 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:112: finishing copy_from to smithi11881747-25 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 2 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:114: finishing write tid 1 to smithi11881747-15 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:114: finishing write tid 2 to smithi11881747-15 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:114: finishing write tid 3 to smithi11881747-15 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:114: finishing write tid 4 to smithi11881747-15 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:114: oid 15 updating version 0 to 11 2024-03-20T08:01:42.266 INFO:tasks.rados.rados.0.smithi118.stdout:114: oid 15 updating version 11 to 12 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:114: oid 15 version 12 is already newer than 10 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 12 (ObjNum 59 snap 5 seq_num 59) dirty exists 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:114: left oid 15 (ObjNum 59 snap 5 seq_num 59) 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:112: done (3 left) 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:113: done (2 left) 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:114: done (1 left) 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:116: rollback oid 30 current snap is 5 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 1 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:117: read oid 26 snap -1 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:117: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 5 2024-03-20T08:01:42.267 INFO:tasks.rados.rados.0.smithi118.stdout:118: seq_num 61 ranges {24576=630784,655360=401408} 2024-03-20T08:01:42.269 INFO:tasks.rados.rados.0.smithi118.stdout:118: writing smithi11881747-2 from 24576 to 655360 tid 1 2024-03-20T08:01:42.271 INFO:tasks.rados.rados.0.smithi118.stdout:118: writing smithi11881747-2 from 655360 to 1056768 tid 2 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:119: snap_create 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:115: finishing write tid 1 to smithi11881747-20 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:115: finishing write tid 2 to smithi11881747-20 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:115: finishing write tid 3 to smithi11881747-20 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:116: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:116: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:115: finishing write tid 4 to smithi11881747-20 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:115: oid 20 updating version 0 to 32 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:115: oid 20 version 32 is already newer than 30 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:115: oid 20 version 32 is already newer than 31 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 32 (ObjNum 60 snap 5 seq_num 60) dirty exists 2024-03-20T08:01:42.272 INFO:tasks.rados.rados.0.smithi118.stdout:115: left oid 20 (ObjNum 60 snap 5 seq_num 60) 2024-03-20T08:01:42.273 INFO:tasks.rados.rados.0.smithi118.stdout:116: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:01:42.273 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 9 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-03-20T08:01:42.273 INFO:tasks.rados.rados.0.smithi118.stdout:117: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:118: finishing write tid 1 to smithi11881747-2 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:118: finishing write tid 2 to smithi11881747-2 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:118: finishing write tid 3 to smithi11881747-2 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:118: finishing write tid 4 to smithi11881747-2 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:118: oid 2 updating version 0 to 18 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:118: oid 2 version 18 is already newer than 17 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:118: oid 2 version 18 is already newer than 16 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 18 (ObjNum 61 snap 5 seq_num 61) dirty exists 2024-03-20T08:01:42.284 INFO:tasks.rados.rados.0.smithi118.stdout:118: left oid 2 (ObjNum 61 snap 5 seq_num 61) 2024-03-20T08:01:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:42 smithi196 ceph-mon[31033]: osdmap e84: 8 total, 8 up, 8 in 2024-03-20T08:01:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:42 smithi196 ceph-mon[31033]: pgmap v102: 121 pgs: 121 active+clean; 24 MiB data, 255 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 376 KiB/s wr, 8 op/s 2024-03-20T08:01:42.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:42 smithi118 ceph-mon[27391]: osdmap e84: 8 total, 8 up, 8 in 2024-03-20T08:01:42.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:42 smithi118 ceph-mon[27391]: pgmap v102: 121 pgs: 121 active+clean; 24 MiB data, 255 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 376 KiB/s wr, 8 op/s 2024-03-20T08:01:42.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:42 smithi118 ceph-mon[38981]: osdmap e84: 8 total, 8 up, 8 in 2024-03-20T08:01:42.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:42 smithi118 ceph-mon[38981]: pgmap v102: 121 pgs: 121 active+clean; 24 MiB data, 255 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 376 KiB/s wr, 8 op/s 2024-03-20T08:01:43.248 INFO:tasks.rados.rados.0.smithi118.stdout:115: done (4 left) 2024-03-20T08:01:43.248 INFO:tasks.rados.rados.0.smithi118.stdout:116: done (3 left) 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:117: done (2 left) 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:118: done (1 left) 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:119: done (0 left) 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:120: read oid 48 snap 4 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:120: expect (ObjNum 47 snap 0 seq_num 47) 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:121: read oid 49 snap 5 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:121: expect (ObjNum 48 snap 0 seq_num 48) 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:122: delete oid 44 current snap is 6 2024-03-20T08:01:43.249 INFO:tasks.rados.rados.0.smithi118.stdout:122: done (2 left) 2024-03-20T08:01:43.250 INFO:tasks.rados.rados.0.smithi118.stdout:123: rollback oid 23 current snap is 6 2024-03-20T08:01:43.250 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 23 to 4 2024-03-20T08:01:43.250 INFO:tasks.rados.rados.0.smithi118.stdout:124: read oid 30 snap -1 2024-03-20T08:01:43.250 INFO:tasks.rados.rados.0.smithi118.stdout:124: expect (ObjNum 29 snap 0 seq_num 29) 2024-03-20T08:01:43.250 INFO:tasks.rados.rados.0.smithi118.stdout:125: setattr oid 35 current snap is 6 2024-03-20T08:01:43.251 INFO:tasks.rados.rados.0.smithi118.stdout:126: delete oid 25 current snap is 6 2024-03-20T08:01:43.253 INFO:tasks.rados.rados.0.smithi118.stdout:121: expect (ObjNum 48 snap 0 seq_num 48) 2024-03-20T08:01:43.277 INFO:tasks.rados.rados.0.smithi118.stdout:124: expect (ObjNum 29 snap 0 seq_num 29) 2024-03-20T08:01:43.291 INFO:tasks.rados.rados.0.smithi118.stdout:123: finishing rollback tid 0 to smithi11881747-23 2024-03-20T08:01:43.291 INFO:tasks.rados.rados.0.smithi118.stdout:123: finishing rollback tid 1 to smithi11881747-23 2024-03-20T08:01:43.291 INFO:tasks.rados.rados.0.smithi118.stdout:121: done (5 left) 2024-03-20T08:01:43.291 INFO:tasks.rados.rados.0.smithi118.stdout:124: done (4 left) 2024-03-20T08:01:43.291 INFO:tasks.rados.rados.0.smithi118.stdout:126: done (3 left) 2024-03-20T08:01:43.291 INFO:tasks.rados.rados.0.smithi118.stdout:127: snap_remove snap 5 2024-03-20T08:01:43.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:43 smithi196 ceph-mon[31033]: osdmap e85: 8 total, 8 up, 8 in 2024-03-20T08:01:43.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:43 smithi118 ceph-mon[27391]: osdmap e85: 8 total, 8 up, 8 in 2024-03-20T08:01:43.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:43 smithi118 ceph-mon[38981]: osdmap e85: 8 total, 8 up, 8 in 2024-03-20T08:01:44.254 INFO:tasks.rados.rados.0.smithi118.stdout:127: done (3 left) 2024-03-20T08:01:44.254 INFO:tasks.rados.rados.0.smithi118.stdout:128: setattr oid 50 current snap is 6 2024-03-20T08:01:44.255 INFO:tasks.rados.rados.0.smithi118.stdout:123: finishing rollback tid 2 to smithi11881747-23 2024-03-20T08:01:44.255 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 30 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-03-20T08:01:44.255 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 20 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-03-20T08:01:44.255 INFO:tasks.rados.rados.0.smithi118.stdout:120: expect (ObjNum 47 snap 0 seq_num 47) 2024-03-20T08:01:44.279 INFO:tasks.rados.rados.0.smithi118.stdout:120: done (3 left) 2024-03-20T08:01:44.279 INFO:tasks.rados.rados.0.smithi118.stdout:123: done (2 left) 2024-03-20T08:01:44.280 INFO:tasks.rados.rados.0.smithi118.stdout:125: done (1 left) 2024-03-20T08:01:44.280 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 6 2024-03-20T08:01:44.280 INFO:tasks.rados.rados.0.smithi118.stdout:129: seq_num 64 ranges {794624=679936,1474560=401408} 2024-03-20T08:01:44.287 INFO:tasks.rados.rados.0.smithi118.stdout:129: writing smithi11881747-48 from 794624 to 1474560 tid 1 2024-03-20T08:01:44.290 INFO:tasks.rados.rados.0.smithi118.stdout:129: writing smithi11881747-48 from 1474560 to 1875968 tid 2 2024-03-20T08:01:44.290 INFO:tasks.rados.rados.0.smithi118.stdout:130: read oid 47 snap 1 2024-03-20T08:01:44.290 INFO:tasks.rados.rados.0.smithi118.stdout:130: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:01:44.291 INFO:tasks.rados.rados.0.smithi118.stdout:131: rollback oid 27 current snap is 6 2024-03-20T08:01:44.291 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 1 2024-03-20T08:01:44.291 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 20 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-03-20T08:01:44.291 INFO:tasks.rados.rados.0.smithi118.stdout:128: done (3 left) 2024-03-20T08:01:44.291 INFO:tasks.rados.rados.0.smithi118.stdout:132: delete oid 14 current snap is 6 2024-03-20T08:01:44.292 INFO:tasks.rados.rados.0.smithi118.stdout:132: done (3 left) 2024-03-20T08:01:44.292 INFO:tasks.rados.rados.0.smithi118.stdout:133: delete oid 19 current snap is 6 2024-03-20T08:01:44.293 INFO:tasks.rados.rados.0.smithi118.stdout:131: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:01:44.293 INFO:tasks.rados.rados.0.smithi118.stdout:131: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:01:44.294 INFO:tasks.rados.rados.0.smithi118.stdout:131: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:01:44.294 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 36 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-03-20T08:01:44.294 INFO:tasks.rados.rados.0.smithi118.stdout:130: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:130: done (3 left) 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:131: done (2 left) 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:133: done (1 left) 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:134: copy_from oid 27 from oid 38 current snap is 6 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:129: finishing write tid 1 to smithi11881747-48 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:129: finishing write tid 2 to smithi11881747-48 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:129: finishing write tid 3 to smithi11881747-48 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:129: finishing write tid 4 to smithi11881747-48 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:129: oid 48 updating version 0 to 15 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:129: oid 48 updating version 15 to 16 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:129: oid 48 version 16 is already newer than 14 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 16 (ObjNum 64 snap 6 seq_num 64) dirty exists 2024-03-20T08:01:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:129: left oid 48 (ObjNum 64 snap 6 seq_num 64) 2024-03-20T08:01:44.318 INFO:tasks.rados.rados.0.smithi118.stdout:129: done (1 left) 2024-03-20T08:01:44.318 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 6 2024-03-20T08:01:44.318 INFO:tasks.rados.rados.0.smithi118.stdout:135: seq_num 67 ranges {606208=729088,1335296=401408} 2024-03-20T08:01:44.324 INFO:tasks.rados.rados.0.smithi118.stdout:135: writing smithi11881747-31 from 606208 to 1335296 tid 1 2024-03-20T08:01:44.327 INFO:tasks.rados.rados.0.smithi118.stdout:135: writing smithi11881747-31 from 1335296 to 1736704 tid 2 2024-03-20T08:01:44.327 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 6 2024-03-20T08:01:44.327 INFO:tasks.rados.rados.0.smithi118.stdout:134: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:01:44.327 INFO:tasks.rados.rados.0.smithi118.stdout:134: finishing copy_from to smithi11881747-27 2024-03-20T08:01:44.327 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-03-20T08:01:44.327 INFO:tasks.rados.rados.0.smithi118.stdout:136: seq_num 68 ranges {0=745472,745472=401408} 2024-03-20T08:01:44.330 INFO:tasks.rados.rados.0.smithi118.stdout:136: writing smithi11881747-8 from 0 to 745472 tid 1 2024-03-20T08:01:44.333 INFO:tasks.rados.rados.0.smithi118.stdout:136: writing smithi11881747-8 from 745472 to 1146880 tid 2 2024-03-20T08:01:44.333 INFO:tasks.rados.rados.0.smithi118.stdout:134: done (2 left) 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:137: copy_from oid 41 from oid 44 current snap is 6 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: finishing write tid 1 to smithi11881747-31 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: finishing write tid 2 to smithi11881747-31 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: finishing write tid 3 to smithi11881747-31 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: finishing write tid 4 to smithi11881747-31 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: oid 31 updating version 0 to 22 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: oid 31 version 22 is already newer than 21 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: oid 31 updating version 22 to 23 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 23 (ObjNum 67 snap 6 seq_num 67) dirty exists 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: left oid 31 (ObjNum 67 snap 6 seq_num 67) 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:135: done (2 left) 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:138: copy_from oid 28 from oid 10 current snap is 6 2024-03-20T08:01:44.334 INFO:tasks.rados.rados.0.smithi118.stdout:139: snap_create 2024-03-20T08:01:44.335 INFO:tasks.rados.rados.0.smithi118.stdout:137: finishing copy_from to smithi11881747-41 2024-03-20T08:01:44.335 INFO:tasks.rados.rados.0.smithi118.stdout:137: got expected ENOENT (src dne) 2024-03-20T08:01:44.335 INFO:tasks.rados.rados.0.smithi118.stdout:137: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:01:44.336 INFO:tasks.rados.rados.0.smithi118.stdout:136: finishing write tid 1 to smithi11881747-8 2024-03-20T08:01:44.336 INFO:tasks.rados.rados.0.smithi118.stdout:136: finishing write tid 2 to smithi11881747-8 2024-03-20T08:01:44.336 INFO:tasks.rados.rados.0.smithi118.stdout:136: finishing write tid 3 to smithi11881747-8 2024-03-20T08:01:44.337 INFO:tasks.rados.rados.0.smithi118.stdout:136: finishing write tid 4 to smithi11881747-8 2024-03-20T08:01:44.337 INFO:tasks.rados.rados.0.smithi118.stdout:136: oid 8 updating version 0 to 18 2024-03-20T08:01:44.337 INFO:tasks.rados.rados.0.smithi118.stdout:136: oid 8 version 18 is already newer than 17 2024-03-20T08:01:44.337 INFO:tasks.rados.rados.0.smithi118.stdout:136: oid 8 version 18 is already newer than 16 2024-03-20T08:01:44.337 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 18 (ObjNum 68 snap 6 seq_num 68) dirty exists 2024-03-20T08:01:44.337 INFO:tasks.rados.rados.0.smithi118.stdout:136: left oid 8 (ObjNum 68 snap 6 seq_num 68) 2024-03-20T08:01:44.338 INFO:tasks.rados.rados.0.smithi118.stdout:138: finishing copy_from racing read to smithi11881747-28 2024-03-20T08:01:44.338 INFO:tasks.rados.rados.0.smithi118.stdout:138: finishing copy_from to smithi11881747-28 2024-03-20T08:01:44.338 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 13 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-03-20T08:01:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:44 smithi196 ceph-mon[31033]: osdmap e86: 8 total, 8 up, 8 in 2024-03-20T08:01:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:44 smithi196 ceph-mon[31033]: pgmap v105: 121 pgs: 121 active+clean; 24 MiB data, 255 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 376 KiB/s wr, 8 op/s 2024-03-20T08:01:44.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:44 smithi118 ceph-mon[27391]: osdmap e86: 8 total, 8 up, 8 in 2024-03-20T08:01:44.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:44 smithi118 ceph-mon[27391]: pgmap v105: 121 pgs: 121 active+clean; 24 MiB data, 255 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 376 KiB/s wr, 8 op/s 2024-03-20T08:01:44.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:44 smithi118 ceph-mon[38981]: osdmap e86: 8 total, 8 up, 8 in 2024-03-20T08:01:44.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:44 smithi118 ceph-mon[38981]: pgmap v105: 121 pgs: 121 active+clean; 24 MiB data, 255 MiB used, 715 GiB / 715 GiB avail; 2.3 MiB/s rd, 376 KiB/s wr, 8 op/s 2024-03-20T08:01:45.257 INFO:tasks.rados.rados.0.smithi118.stdout:136: done (3 left) 2024-03-20T08:01:45.257 INFO:tasks.rados.rados.0.smithi118.stdout:137: done (2 left) 2024-03-20T08:01:45.258 INFO:tasks.rados.rados.0.smithi118.stdout:138: done (1 left) 2024-03-20T08:01:45.258 INFO:tasks.rados.rados.0.smithi118.stdout:139: done (0 left) 2024-03-20T08:01:45.258 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 7 2024-03-20T08:01:45.258 INFO:tasks.rados.rados.0.smithi118.stdout:140: seq_num 69 ranges {139264=761856,901120=401408} 2024-03-20T08:01:45.262 INFO:tasks.rados.rados.0.smithi118.stdout:140: writing smithi11881747-9 from 139264 to 901120 tid 1 2024-03-20T08:01:45.264 INFO:tasks.rados.rados.0.smithi118.stdout:140: writing smithi11881747-9 from 901120 to 1302528 tid 2 2024-03-20T08:01:45.264 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 7 2024-03-20T08:01:45.264 INFO:tasks.rados.rados.0.smithi118.stdout:141: seq_num 70 ranges {0=778240,778240=401408} 2024-03-20T08:01:45.269 INFO:tasks.rados.rados.0.smithi118.stdout:141: writing smithi11881747-12 from 0 to 778240 tid 1 2024-03-20T08:01:45.272 INFO:tasks.rados.rados.0.smithi118.stdout:141: writing smithi11881747-12 from 778240 to 1179648 tid 2 2024-03-20T08:01:45.272 INFO:tasks.rados.rados.0.smithi118.stdout:142: rollback oid 22 current snap is 7 2024-03-20T08:01:45.272 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 22 to 1 2024-03-20T08:01:45.272 INFO:tasks.rados.rados.0.smithi118.stdout:140: finishing write tid 1 to smithi11881747-9 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:140: finishing write tid 2 to smithi11881747-9 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:140: finishing write tid 3 to smithi11881747-9 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:140: finishing write tid 4 to smithi11881747-9 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:140: oid 9 updating version 0 to 32 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:140: oid 9 version 32 is already newer than 31 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:140: oid 9 version 32 is already newer than 30 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 32 (ObjNum 69 snap 7 seq_num 69) dirty exists 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:140: left oid 9 (ObjNum 69 snap 7 seq_num 69) 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:140: done (2 left) 2024-03-20T08:01:45.273 INFO:tasks.rados.rados.0.smithi118.stdout:143: snap_create 2024-03-20T08:01:45.275 INFO:tasks.rados.rados.0.smithi118.stdout:141: finishing write tid 1 to smithi11881747-12 2024-03-20T08:01:45.275 INFO:tasks.rados.rados.0.smithi118.stdout:141: finishing write tid 2 to smithi11881747-12 2024-03-20T08:01:45.275 INFO:tasks.rados.rados.0.smithi118.stdout:141: finishing write tid 3 to smithi11881747-12 2024-03-20T08:01:45.277 INFO:tasks.rados.rados.0.smithi118.stdout:141: finishing write tid 4 to smithi11881747-12 2024-03-20T08:01:45.277 INFO:tasks.rados.rados.0.smithi118.stdout:141: oid 12 updating version 0 to 9 2024-03-20T08:01:45.277 INFO:tasks.rados.rados.0.smithi118.stdout:141: oid 12 version 9 is already newer than 8 2024-03-20T08:01:45.277 INFO:tasks.rados.rados.0.smithi118.stdout:141: oid 12 version 9 is already newer than 7 2024-03-20T08:01:45.277 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 9 (ObjNum 70 snap 7 seq_num 70) dirty exists 2024-03-20T08:01:45.277 INFO:tasks.rados.rados.0.smithi118.stdout:141: left oid 12 (ObjNum 70 snap 7 seq_num 70) 2024-03-20T08:01:45.278 INFO:tasks.rados.rados.0.smithi118.stdout:142: finishing rollback tid 0 to smithi11881747-22 2024-03-20T08:01:45.278 INFO:tasks.rados.rados.0.smithi118.stdout:142: finishing rollback tid 1 to smithi11881747-22 2024-03-20T08:01:45.278 INFO:tasks.rados.rados.0.smithi118.stdout:142: finishing rollback tid 2 to smithi11881747-22 2024-03-20T08:01:45.278 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 20 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-03-20T08:01:45.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:45 smithi196 ceph-mon[31033]: osdmap e87: 8 total, 8 up, 8 in 2024-03-20T08:01:45.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:45 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:45.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:45 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:45.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:45 smithi118 ceph-mon[27391]: osdmap e87: 8 total, 8 up, 8 in 2024-03-20T08:01:45.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:45 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:45.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:45 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:45.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:45 smithi118 ceph-mon[38981]: osdmap e87: 8 total, 8 up, 8 in 2024-03-20T08:01:45.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:45 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:45.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:45 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:01:45.999 INFO:tasks.rados.rados.0.smithi118.stdout:141: done (2 left) 2024-03-20T08:01:45.999 INFO:tasks.rados.rados.0.smithi118.stdout:142: done (1 left) 2024-03-20T08:01:45.999 INFO:tasks.rados.rados.0.smithi118.stdout:143: done (0 left) 2024-03-20T08:01:45.999 INFO:tasks.rados.rados.0.smithi118.stdout:144: read oid 33 snap -1 2024-03-20T08:01:45.999 INFO:tasks.rados.rados.0.smithi118.stdout:144: expect (ObjNum 32 snap 0 seq_num 32) 2024-03-20T08:01:46.000 INFO:tasks.rados.rados.0.smithi118.stdout:145: read oid 45 snap -1 2024-03-20T08:01:46.000 INFO:tasks.rados.rados.0.smithi118.stdout:145: expect (ObjNum 33 snap 0 seq_num 33) 2024-03-20T08:01:46.000 INFO:tasks.rados.rados.0.smithi118.stdout:146: snap_create 2024-03-20T08:01:46.003 INFO:tasks.rados.rados.0.smithi118.stdout:145: expect (ObjNum 33 snap 0 seq_num 33) 2024-03-20T08:01:46.021 INFO:tasks.rados.rados.0.smithi118.stdout:144: expect (ObjNum 32 snap 0 seq_num 32) 2024-03-20T08:01:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:46 smithi196 ceph-mon[31033]: osdmap e88: 8 total, 8 up, 8 in 2024-03-20T08:01:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:46 smithi196 ceph-mon[31033]: pgmap v108: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 44 op/s 2024-03-20T08:01:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:46 smithi196 ceph-mon[31033]: osdmap e89: 8 total, 8 up, 8 in 2024-03-20T08:01:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:46 smithi118 ceph-mon[27391]: osdmap e88: 8 total, 8 up, 8 in 2024-03-20T08:01:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:46 smithi118 ceph-mon[27391]: pgmap v108: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 44 op/s 2024-03-20T08:01:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:46 smithi118 ceph-mon[27391]: osdmap e89: 8 total, 8 up, 8 in 2024-03-20T08:01:46.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:46 smithi118 ceph-mon[38981]: osdmap e88: 8 total, 8 up, 8 in 2024-03-20T08:01:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:46 smithi118 ceph-mon[38981]: pgmap v108: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 44 op/s 2024-03-20T08:01:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:46 smithi118 ceph-mon[38981]: osdmap e89: 8 total, 8 up, 8 in 2024-03-20T08:01:46.999 INFO:tasks.rados.rados.0.smithi118.stdout:144: done (2 left) 2024-03-20T08:01:47.000 INFO:tasks.rados.rados.0.smithi118.stdout:145: done (1 left) 2024-03-20T08:01:47.000 INFO:tasks.rados.rados.0.smithi118.stdout:146: done (0 left) 2024-03-20T08:01:47.000 INFO:tasks.rados.rados.0.smithi118.stdout:147: copy_from oid 48 from oid 42 current snap is 9 2024-03-20T08:01:47.000 INFO:tasks.rados.rados.0.smithi118.stdout:148: delete oid 29 current snap is 9 2024-03-20T08:01:47.002 INFO:tasks.rados.rados.0.smithi118.stdout:148: done (1 left) 2024-03-20T08:01:47.002 INFO:tasks.rados.rados.0.smithi118.stdout:149: snap_create 2024-03-20T08:01:47.007 INFO:tasks.rados.rados.0.smithi118.stdout:147: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:01:47.008 INFO:tasks.rados.rados.0.smithi118.stdout:147: finishing copy_from to smithi11881747-48 2024-03-20T08:01:47.008 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-03-20T08:01:48.005 INFO:tasks.rados.rados.0.smithi118.stdout:147: done (1 left) 2024-03-20T08:01:48.005 INFO:tasks.rados.rados.0.smithi118.stdout:149: done (0 left) 2024-03-20T08:01:48.005 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 10 2024-03-20T08:01:48.005 INFO:tasks.rados.rados.0.smithi118.stdout:150: seq_num 72 ranges {712704=409600,1122304=524288,1646592=278528} 2024-03-20T08:01:48.011 INFO:tasks.rados.rados.0.smithi118.stdout:150: writing smithi11881747-43 from 712704 to 1122304 tid 1 2024-03-20T08:01:48.013 INFO:tasks.rados.rados.0.smithi118.stdout:150: writing smithi11881747-43 from 1122304 to 1646592 tid 2 2024-03-20T08:01:48.015 INFO:tasks.rados.rados.0.smithi118.stdout:150: writing smithi11881747-43 from 1646592 to 1925120 tid 3 2024-03-20T08:01:48.015 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 10 2024-03-20T08:01:48.015 INFO:tasks.rados.rados.0.smithi118.stdout:151: seq_num 73 ranges {155648=425984,581632=409600,991232=393216} 2024-03-20T08:01:48.018 INFO:tasks.rados.rados.0.smithi118.stdout:151: writing smithi11881747-10 from 155648 to 581632 tid 1 2024-03-20T08:01:48.021 INFO:tasks.rados.rados.0.smithi118.stdout:151: writing smithi11881747-10 from 581632 to 991232 tid 2 2024-03-20T08:01:48.023 INFO:tasks.rados.rados.0.smithi118.stdout:151: writing smithi11881747-10 from 991232 to 1384448 tid 3 2024-03-20T08:01:48.023 INFO:tasks.rados.rados.0.smithi118.stdout:152: snap_create 2024-03-20T08:01:48.023 INFO:tasks.rados.rados.0.smithi118.stdout:150: finishing write tid 1 to smithi11881747-43 2024-03-20T08:01:48.023 INFO:tasks.rados.rados.0.smithi118.stdout:150: finishing write tid 2 to smithi11881747-43 2024-03-20T08:01:48.023 INFO:tasks.rados.rados.0.smithi118.stdout:150: finishing write tid 3 to smithi11881747-43 2024-03-20T08:01:48.024 INFO:tasks.rados.rados.0.smithi118.stdout:150: finishing write tid 4 to smithi11881747-43 2024-03-20T08:01:48.024 INFO:tasks.rados.rados.0.smithi118.stdout:150: finishing write tid 5 to smithi11881747-43 2024-03-20T08:01:48.024 INFO:tasks.rados.rados.0.smithi118.stdout:150: oid 43 updating version 0 to 25 2024-03-20T08:01:48.024 INFO:tasks.rados.rados.0.smithi118.stdout:150: oid 43 updating version 25 to 27 2024-03-20T08:01:48.024 INFO:tasks.rados.rados.0.smithi118.stdout:150: oid 43 version 27 is already newer than 26 2024-03-20T08:01:48.025 INFO:tasks.rados.rados.0.smithi118.stdout:150: oid 43 updating version 27 to 28 2024-03-20T08:01:48.025 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 28 (ObjNum 72 snap 10 seq_num 72) dirty exists 2024-03-20T08:01:48.025 INFO:tasks.rados.rados.0.smithi118.stdout:150: left oid 43 (ObjNum 72 snap 10 seq_num 72) 2024-03-20T08:01:48.025 INFO:tasks.rados.rados.0.smithi118.stdout:151: finishing write tid 1 to smithi11881747-10 2024-03-20T08:01:48.026 INFO:tasks.rados.rados.0.smithi118.stdout:151: finishing write tid 2 to smithi11881747-10 2024-03-20T08:01:48.026 INFO:tasks.rados.rados.0.smithi118.stdout:151: finishing write tid 3 to smithi11881747-10 2024-03-20T08:01:48.026 INFO:tasks.rados.rados.0.smithi118.stdout:151: finishing write tid 4 to smithi11881747-10 2024-03-20T08:01:48.027 INFO:tasks.rados.rados.0.smithi118.stdout:151: finishing write tid 5 to smithi11881747-10 2024-03-20T08:01:48.027 INFO:tasks.rados.rados.0.smithi118.stdout:151: oid 10 updating version 0 to 33 2024-03-20T08:01:48.027 INFO:tasks.rados.rados.0.smithi118.stdout:151: oid 10 version 33 is already newer than 32 2024-03-20T08:01:48.027 INFO:tasks.rados.rados.0.smithi118.stdout:151: oid 10 version 33 is already newer than 31 2024-03-20T08:01:48.027 INFO:tasks.rados.rados.0.smithi118.stdout:151: oid 10 version 33 is already newer than 30 2024-03-20T08:01:48.027 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 33 (ObjNum 73 snap 10 seq_num 73) dirty exists 2024-03-20T08:01:48.028 INFO:tasks.rados.rados.0.smithi118.stdout:151: left oid 10 (ObjNum 73 snap 10 seq_num 73) 2024-03-20T08:01:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:48 smithi196 ceph-mon[31033]: osdmap e90: 8 total, 8 up, 8 in 2024-03-20T08:01:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:48 smithi118 ceph-mon[27391]: osdmap e90: 8 total, 8 up, 8 in 2024-03-20T08:01:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:48 smithi118 ceph-mon[38981]: osdmap e90: 8 total, 8 up, 8 in 2024-03-20T08:01:49.009 INFO:tasks.rados.rados.0.smithi118.stdout:150: done (2 left) 2024-03-20T08:01:49.009 INFO:tasks.rados.rados.0.smithi118.stdout:151: done (1 left) 2024-03-20T08:01:49.009 INFO:tasks.rados.rados.0.smithi118.stdout:152: done (0 left) 2024-03-20T08:01:49.009 INFO:tasks.rados.rados.0.smithi118.stdout:153: snap_create 2024-03-20T08:01:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:49 smithi196 ceph-mon[31033]: pgmap v111: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 44 op/s 2024-03-20T08:01:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:49 smithi196 ceph-mon[31033]: osdmap e91: 8 total, 8 up, 8 in 2024-03-20T08:01:49.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:49 smithi118 ceph-mon[27391]: pgmap v111: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 44 op/s 2024-03-20T08:01:49.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:49 smithi118 ceph-mon[27391]: osdmap e91: 8 total, 8 up, 8 in 2024-03-20T08:01:49.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:49 smithi118 ceph-mon[38981]: pgmap v111: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 44 op/s 2024-03-20T08:01:49.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:49 smithi118 ceph-mon[38981]: osdmap e91: 8 total, 8 up, 8 in 2024-03-20T08:01:50.014 INFO:tasks.rados.rados.0.smithi118.stdout:153: done (0 left) 2024-03-20T08:01:50.014 INFO:tasks.rados.rados.0.smithi118.stdout:154: read oid 42 snap -1 2024-03-20T08:01:50.014 INFO:tasks.rados.rados.0.smithi118.stdout:154: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:01:50.015 INFO:tasks.rados.rados.0.smithi118.stdout:155: rmattr oid 17 current snap is 12 2024-03-20T08:01:50.015 INFO:tasks.rados.rados.0.smithi118.stdout:155: done (1 left) 2024-03-20T08:01:50.015 INFO:tasks.rados.rados.0.smithi118.stdout:156: setattr oid 28 current snap is 12 2024-03-20T08:01:50.017 INFO:tasks.rados.rados.0.smithi118.stdout:157: copy_from oid 33 from oid 16 current snap is 12 2024-03-20T08:01:50.017 INFO:tasks.rados.rados.0.smithi118.stdout:158: setattr oid 32 current snap is 12 2024-03-20T08:01:50.018 INFO:tasks.rados.rados.0.smithi118.stdout:154: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:01:50.041 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 38 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-03-20T08:01:50.041 INFO:tasks.rados.rados.0.smithi118.stdout:157: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:01:50.041 INFO:tasks.rados.rados.0.smithi118.stdout:154: done (3 left) 2024-03-20T08:01:50.041 INFO:tasks.rados.rados.0.smithi118.stdout:156: done (2 left) 2024-03-20T08:01:50.041 INFO:tasks.rados.rados.0.smithi118.stdout:159: snap_remove snap 1 2024-03-20T08:01:50.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:50 smithi196 ceph-mon[31033]: osdmap e92: 8 total, 8 up, 8 in 2024-03-20T08:01:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:50 smithi118 ceph-mon[27391]: osdmap e92: 8 total, 8 up, 8 in 2024-03-20T08:01:50.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:50 smithi118 ceph-mon[38981]: osdmap e92: 8 total, 8 up, 8 in 2024-03-20T08:01:51.021 INFO:tasks.rados.rados.0.smithi118.stdout:159: done (2 left) 2024-03-20T08:01:51.021 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 12 2024-03-20T08:01:51.021 INFO:tasks.rados.rados.0.smithi118.stdout:160: seq_num 74 ranges {1687552=442368,2129920=696320,2826240=106496} 2024-03-20T08:01:51.032 INFO:tasks.rados.rados.0.smithi118.stdout:160: writing smithi11881747-23 from 1687552 to 2129920 tid 1 2024-03-20T08:01:51.037 INFO:tasks.rados.rados.0.smithi118.stdout:160: writing smithi11881747-23 from 2129920 to 2826240 tid 2 2024-03-20T08:01:51.038 INFO:tasks.rados.rados.0.smithi118.stdout:160: writing smithi11881747-23 from 2826240 to 2932736 tid 3 2024-03-20T08:01:51.038 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 12 2024-03-20T08:01:51.038 INFO:tasks.rados.rados.0.smithi118.stdout:157: finishing copy_from to smithi11881747-33 2024-03-20T08:01:51.038 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-03-20T08:01:51.039 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 20 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-03-20T08:01:51.039 INFO:tasks.rados.rados.0.smithi118.stdout:160: finishing write tid 1 to smithi11881747-23 2024-03-20T08:01:51.039 INFO:tasks.rados.rados.0.smithi118.stdout:161: seq_num 75 ranges {811008=458752,1269760=581632,1851392=221184} 2024-03-20T08:01:51.045 INFO:tasks.rados.rados.0.smithi118.stdout:161: writing smithi11881747-49 from 811008 to 1269760 tid 1 2024-03-20T08:01:51.048 INFO:tasks.rados.rados.0.smithi118.stdout:161: writing smithi11881747-49 from 1269760 to 1851392 tid 2 2024-03-20T08:01:51.050 INFO:tasks.rados.rados.0.smithi118.stdout:161: writing smithi11881747-49 from 1851392 to 2072576 tid 3 2024-03-20T08:01:51.050 INFO:tasks.rados.rados.0.smithi118.stdout:157: done (3 left) 2024-03-20T08:01:51.051 INFO:tasks.rados.rados.0.smithi118.stdout:158: done (2 left) 2024-03-20T08:01:51.051 INFO:tasks.rados.rados.0.smithi118.stdout:162: snap_remove snap 11 2024-03-20T08:01:51.355 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:51 smithi118 ceph-mon[27391]: pgmap v114: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:51.356 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:51 smithi118 ceph-mon[27391]: osdmap e93: 8 total, 8 up, 8 in 2024-03-20T08:01:51.356 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:51 smithi118 ceph-mon[38981]: pgmap v114: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:51.356 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:51 smithi118 ceph-mon[38981]: osdmap e93: 8 total, 8 up, 8 in 2024-03-20T08:01:51.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:51 smithi196 ceph-mon[31033]: pgmap v114: 121 pgs: 121 active+clean; 35 MiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:01:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:51 smithi196 ceph-mon[31033]: osdmap e93: 8 total, 8 up, 8 in 2024-03-20T08:01:51.902 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:01:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:01:51] "GET /metrics HTTP/1.1" 200 36073 "" "Prometheus/2.43.0" 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:162: done (2 left) 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:163: delete oid 22 current snap is 12 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:160: finishing write tid 2 to smithi11881747-23 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:160: finishing write tid 3 to smithi11881747-23 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:160: finishing write tid 4 to smithi11881747-23 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:160: finishing write tid 5 to smithi11881747-23 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:160: oid 23 updating version 0 to 47 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:160: oid 23 updating version 47 to 49 2024-03-20T08:01:52.025 INFO:tasks.rados.rados.0.smithi118.stdout:160: oid 23 version 49 is already newer than 48 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:160: oid 23 version 49 is already newer than 46 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 49 (ObjNum 74 snap 12 seq_num 74) dirty exists 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:160: left oid 23 (ObjNum 74 snap 12 seq_num 74) 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:161: finishing write tid 1 to smithi11881747-49 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:161: finishing write tid 2 to smithi11881747-49 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:161: finishing write tid 3 to smithi11881747-49 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:161: finishing write tid 4 to smithi11881747-49 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:161: finishing write tid 5 to smithi11881747-49 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:161: oid 49 updating version 0 to 44 2024-03-20T08:01:52.026 INFO:tasks.rados.rados.0.smithi118.stdout:161: oid 49 updating version 44 to 45 2024-03-20T08:01:52.027 INFO:tasks.rados.rados.0.smithi118.stdout:161: oid 49 version 45 is already newer than 43 2024-03-20T08:01:52.027 INFO:tasks.rados.rados.0.smithi118.stdout:161: oid 49 updating version 45 to 46 2024-03-20T08:01:52.027 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 46 (ObjNum 75 snap 12 seq_num 75) dirty exists 2024-03-20T08:01:52.027 INFO:tasks.rados.rados.0.smithi118.stdout:161: left oid 49 (ObjNum 75 snap 12 seq_num 75) 2024-03-20T08:01:52.028 INFO:tasks.rados.rados.0.smithi118.stdout:160: done (2 left) 2024-03-20T08:01:52.028 INFO:tasks.rados.rados.0.smithi118.stdout:161: done (1 left) 2024-03-20T08:01:52.028 INFO:tasks.rados.rados.0.smithi118.stdout:163: done (0 left) 2024-03-20T08:01:52.028 INFO:tasks.rados.rados.0.smithi118.stdout:164: snap_create 2024-03-20T08:01:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:52 smithi118 ceph-mon[27391]: osdmap e94: 8 total, 8 up, 8 in 2024-03-20T08:01:52.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:52 smithi118 ceph-mon[38981]: osdmap e94: 8 total, 8 up, 8 in 2024-03-20T08:01:52.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:52 smithi196 ceph-mon[31033]: osdmap e94: 8 total, 8 up, 8 in 2024-03-20T08:01:53.025 INFO:tasks.rados.rados.0.smithi118.stdout:164: done (0 left) 2024-03-20T08:01:53.025 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 13 2024-03-20T08:01:53.025 INFO:tasks.rados.rados.0.smithi118.stdout:165: seq_num 77 ranges {1736704=491520,2228224=794624,3022848=8192} 2024-03-20T08:01:53.037 INFO:tasks.rados.rados.0.smithi118.stdout:165: writing smithi11881747-31 from 1736704 to 2228224 tid 1 2024-03-20T08:01:53.042 INFO:tasks.rados.rados.0.smithi118.stdout:165: writing smithi11881747-31 from 2228224 to 3022848 tid 2 2024-03-20T08:01:53.042 INFO:tasks.rados.rados.0.smithi118.stdout:165: writing smithi11881747-31 from 3022848 to 3031040 tid 3 2024-03-20T08:01:53.042 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 13 2024-03-20T08:01:53.042 INFO:tasks.rados.rados.0.smithi118.stdout:166: seq_num 78 ranges {253952=516096,770048=679936,1449984=122880} 2024-03-20T08:01:53.046 INFO:tasks.rados.rados.0.smithi118.stdout:166: writing smithi11881747-16 from 253952 to 770048 tid 1 2024-03-20T08:01:53.050 INFO:tasks.rados.rados.0.smithi118.stdout:166: writing smithi11881747-16 from 770048 to 1449984 tid 2 2024-03-20T08:01:53.051 INFO:tasks.rados.rados.0.smithi118.stdout:166: writing smithi11881747-16 from 1449984 to 1572864 tid 3 2024-03-20T08:01:53.051 INFO:tasks.rados.rados.0.smithi118.stdout:167: copy_from oid 10 from oid 39 current snap is 13 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: finishing write tid 1 to smithi11881747-31 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: finishing write tid 2 to smithi11881747-31 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: finishing write tid 3 to smithi11881747-31 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: finishing write tid 4 to smithi11881747-31 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: finishing write tid 5 to smithi11881747-31 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: oid 31 updating version 0 to 41 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: oid 31 updating version 41 to 44 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: oid 31 version 44 is already newer than 42 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: oid 31 version 44 is already newer than 43 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 44 (ObjNum 77 snap 13 seq_num 77) dirty exists 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: left oid 31 (ObjNum 77 snap 13 seq_num 77) 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:166: finishing write tid 1 to smithi11881747-16 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:165: done (2 left) 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:168: read oid 25 snap 8 2024-03-20T08:01:53.052 INFO:tasks.rados.rados.0.smithi118.stdout:168: expect deleted 2024-03-20T08:01:53.053 INFO:tasks.rados.rados.0.smithi118.stdout:169: copy_from oid 21 from oid 35 current snap is 13 2024-03-20T08:01:53.053 INFO:tasks.rados.rados.0.smithi118.stdout:170: snap_remove snap 7 2024-03-20T08:01:53.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:53 smithi196 ceph-mon[31033]: pgmap v117: 121 pgs: 121 active+clean; 38 MiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.3 MiB/s rd, 828 KiB/s wr, 16 op/s 2024-03-20T08:01:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:53 smithi196 ceph-mon[31033]: osdmap e95: 8 total, 8 up, 8 in 2024-03-20T08:01:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:53 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:53 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:53 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[38981]: pgmap v117: 121 pgs: 121 active+clean; 38 MiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.3 MiB/s rd, 828 KiB/s wr, 16 op/s 2024-03-20T08:01:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[38981]: osdmap e95: 8 total, 8 up, 8 in 2024-03-20T08:01:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[27391]: pgmap v117: 121 pgs: 121 active+clean; 38 MiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.3 MiB/s rd, 828 KiB/s wr, 16 op/s 2024-03-20T08:01:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[27391]: osdmap e95: 8 total, 8 up, 8 in 2024-03-20T08:01:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:01:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:53 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:01:54.036 INFO:tasks.rados.rados.0.smithi118.stdout:166: finishing write tid 2 to smithi11881747-16 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:166: finishing write tid 3 to smithi11881747-16 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:166: finishing write tid 4 to smithi11881747-16 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:166: finishing write tid 5 to smithi11881747-16 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:166: oid 16 updating version 0 to 58 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:166: oid 16 version 58 is already newer than 56 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:166: oid 16 version 58 is already newer than 57 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:166: oid 16 version 58 is already newer than 55 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 58 (ObjNum 78 snap 13 seq_num 78) dirty exists 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:166: left oid 16 (ObjNum 78 snap 13 seq_num 78) 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:167: finishing copy_from racing read to smithi11881747-10 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:167: finishing copy_from to smithi11881747-10 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 8 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-03-20T08:01:54.037 INFO:tasks.rados.rados.0.smithi118.stdout:169: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:01:54.038 INFO:tasks.rados.rados.0.smithi118.stdout:169: finishing copy_from to smithi11881747-21 2024-03-20T08:01:54.038 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 20 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-03-20T08:01:54.038 INFO:tasks.rados.rados.0.smithi118.stdout:166: done (4 left) 2024-03-20T08:01:54.038 INFO:tasks.rados.rados.0.smithi118.stdout:167: done (3 left) 2024-03-20T08:01:54.038 INFO:tasks.rados.rados.0.smithi118.stdout:168: done (2 left) 2024-03-20T08:01:54.038 INFO:tasks.rados.rados.0.smithi118.stdout:169: done (1 left) 2024-03-20T08:01:54.038 INFO:tasks.rados.rados.0.smithi118.stdout:170: done (0 left) 2024-03-20T08:01:54.038 INFO:tasks.rados.rados.0.smithi118.stdout:171: delete oid 48 current snap is 13 2024-03-20T08:01:54.042 INFO:tasks.rados.rados.0.smithi118.stdout:171: done (0 left) 2024-03-20T08:01:54.042 INFO:tasks.rados.rados.0.smithi118.stdout:172: snap_remove snap 8 2024-03-20T08:01:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:54 smithi196 ceph-mon[31033]: Upgrade: Target is version 19.0.0-2153-ge7e80a6a (unknown) 2024-03-20T08:01:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:54 smithi196 ceph-mon[31033]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, digests ['quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912'] 2024-03-20T08:01:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:54 smithi196 ceph-mon[31033]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-20T08:01:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:54 smithi196 ceph-mon[31033]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 on smithi196 2024-03-20T08:01:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:54 smithi196 ceph-mon[31033]: osdmap e96: 8 total, 8 up, 8 in 2024-03-20T08:01:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[27391]: Upgrade: Target is version 19.0.0-2153-ge7e80a6a (unknown) 2024-03-20T08:01:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[27391]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, digests ['quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912'] 2024-03-20T08:01:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[27391]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-20T08:01:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[27391]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 on smithi196 2024-03-20T08:01:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[27391]: osdmap e96: 8 total, 8 up, 8 in 2024-03-20T08:01:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[38981]: Upgrade: Target is version 19.0.0-2153-ge7e80a6a (unknown) 2024-03-20T08:01:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[38981]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, digests ['quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912'] 2024-03-20T08:01:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[38981]: Upgrade: Need to upgrade myself (mgr.y) 2024-03-20T08:01:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[38981]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 on smithi196 2024-03-20T08:01:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:54 smithi118 ceph-mon[38981]: osdmap e96: 8 total, 8 up, 8 in 2024-03-20T08:01:55.040 INFO:tasks.rados.rados.0.smithi118.stdout:172: done (0 left) 2024-03-20T08:01:55.040 INFO:tasks.rados.rados.0.smithi118.stdout:173: snap_create 2024-03-20T08:01:55.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:55 smithi196 ceph-mon[31033]: pgmap v120: 121 pgs: 121 active+clean; 38 MiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.3 MiB/s rd, 828 KiB/s wr, 16 op/s 2024-03-20T08:01:55.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:55 smithi196 ceph-mon[31033]: osdmap e97: 8 total, 8 up, 8 in 2024-03-20T08:01:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:55 smithi118 ceph-mon[27391]: pgmap v120: 121 pgs: 121 active+clean; 38 MiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.3 MiB/s rd, 828 KiB/s wr, 16 op/s 2024-03-20T08:01:55.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:55 smithi118 ceph-mon[27391]: osdmap e97: 8 total, 8 up, 8 in 2024-03-20T08:01:55.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:55 smithi118 ceph-mon[38981]: pgmap v120: 121 pgs: 121 active+clean; 38 MiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.3 MiB/s rd, 828 KiB/s wr, 16 op/s 2024-03-20T08:01:55.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:55 smithi118 ceph-mon[38981]: osdmap e97: 8 total, 8 up, 8 in 2024-03-20T08:01:56.002 INFO:tasks.rados.rados.0.smithi118.stdout:173: done (0 left) 2024-03-20T08:01:56.002 INFO:tasks.rados.rados.0.smithi118.stdout:174: read oid 46 snap -1 2024-03-20T08:01:56.002 INFO:tasks.rados.rados.0.smithi118.stdout:174: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:01:56.002 INFO:tasks.rados.rados.0.smithi118.stdout:175: copy_from oid 3 from oid 30 current snap is 14 2024-03-20T08:01:56.003 INFO:tasks.rados.rados.0.smithi118.stdout:176: copy_from oid 17 from oid 9 current snap is 14 2024-03-20T08:01:56.003 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 14 2024-03-20T08:01:56.003 INFO:tasks.rados.rados.0.smithi118.stdout:177: seq_num 80 ranges {3031040=548864,3579904=450560,4030464=352256} 2024-03-20T08:01:56.022 INFO:tasks.rados.rados.0.smithi118.stdout:177: writing smithi11881747-31 from 3031040 to 3579904 tid 1 2024-03-20T08:01:56.024 INFO:tasks.rados.rados.0.smithi118.stdout:177: writing smithi11881747-31 from 3579904 to 4030464 tid 2 2024-03-20T08:01:56.026 INFO:tasks.rados.rados.0.smithi118.stdout:177: writing smithi11881747-31 from 4030464 to 4382720 tid 3 2024-03-20T08:01:56.027 INFO:tasks.rados.rados.0.smithi118.stdout:178: setattr oid 32 current snap is 14 2024-03-20T08:01:56.027 INFO:tasks.rados.rados.0.smithi118.stdout:174: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:01:56.049 INFO:tasks.rados.rados.0.smithi118.stdout:176: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:01:56.049 INFO:tasks.rados.rados.0.smithi118.stdout:176: finishing copy_from to smithi11881747-17 2024-03-20T08:01:56.049 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 32 (ObjNum 69 snap 7 seq_num 69) dirty exists 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:175: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:175: finishing copy_from to smithi11881747-3 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 9 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: finishing write tid 1 to smithi11881747-31 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: finishing write tid 2 to smithi11881747-31 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: finishing write tid 3 to smithi11881747-31 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: finishing write tid 4 to smithi11881747-31 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: finishing write tid 5 to smithi11881747-31 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: oid 31 updating version 0 to 63 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: oid 31 version 63 is already newer than 61 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: oid 31 version 63 is already newer than 60 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: oid 31 version 63 is already newer than 62 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 63 (ObjNum 80 snap 14 seq_num 80) dirty exists 2024-03-20T08:01:56.050 INFO:tasks.rados.rados.0.smithi118.stdout:177: left oid 31 (ObjNum 80 snap 14 seq_num 80) 2024-03-20T08:01:56.051 INFO:tasks.rados.rados.0.smithi118.stdout:174: done (4 left) 2024-03-20T08:01:56.051 INFO:tasks.rados.rados.0.smithi118.stdout:175: done (3 left) 2024-03-20T08:01:56.051 INFO:tasks.rados.rados.0.smithi118.stdout:176: done (2 left) 2024-03-20T08:01:56.051 INFO:tasks.rados.rados.0.smithi118.stdout:177: done (1 left) 2024-03-20T08:01:56.051 INFO:tasks.rados.rados.0.smithi118.stdout:179: snap_remove snap 10 2024-03-20T08:01:56.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:56 smithi196 ceph-mon[31033]: osdmap e98: 8 total, 8 up, 8 in 2024-03-20T08:01:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:56 smithi196 ceph-mon[31033]: osdmap e99: 8 total, 8 up, 8 in 2024-03-20T08:01:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:56 smithi118 ceph-mon[27391]: osdmap e98: 8 total, 8 up, 8 in 2024-03-20T08:01:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:56 smithi118 ceph-mon[27391]: osdmap e99: 8 total, 8 up, 8 in 2024-03-20T08:01:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:56 smithi118 ceph-mon[38981]: osdmap e98: 8 total, 8 up, 8 in 2024-03-20T08:01:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:56 smithi118 ceph-mon[38981]: osdmap e99: 8 total, 8 up, 8 in 2024-03-20T08:01:57.046 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-03-20T08:01:57.047 INFO:tasks.rados.rados.0.smithi118.stdout:178: done (1 left) 2024-03-20T08:01:57.047 INFO:tasks.rados.rados.0.smithi118.stdout:179: done (0 left) 2024-03-20T08:01:57.047 INFO:tasks.rados.rados.0.smithi118.stdout:180: rmattr oid 4 current snap is 14 2024-03-20T08:01:57.047 INFO:tasks.rados.rados.0.smithi118.stdout:180: done (0 left) 2024-03-20T08:01:57.047 INFO:tasks.rados.rados.0.smithi118.stdout:181: copy_from oid 13 from oid 25 current snap is 14 2024-03-20T08:01:57.047 INFO:tasks.rados.rados.0.smithi118.stdout:182: snap_remove snap 12 2024-03-20T08:01:57.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:57 smithi196 ceph-mon[31033]: pgmap v123: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 MiB/s wr, 6 op/s 2024-03-20T08:01:57.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:57 smithi118 ceph-mon[27391]: pgmap v123: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 MiB/s wr, 6 op/s 2024-03-20T08:01:57.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:57 smithi118 ceph-mon[38981]: pgmap v123: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 MiB/s wr, 6 op/s 2024-03-20T08:01:58.048 INFO:tasks.rados.rados.0.smithi118.stdout:182: done (1 left) 2024-03-20T08:01:58.048 INFO:tasks.rados.rados.0.smithi118.stdout:183: read oid 3 snap -1 2024-03-20T08:01:58.048 INFO:tasks.rados.rados.0.smithi118.stdout:183: expect (ObjNum 29 snap 0 seq_num 29) 2024-03-20T08:01:58.049 INFO:tasks.rados.rados.0.smithi118.stdout:184: rollback oid 36 current snap is 14 2024-03-20T08:01:58.049 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 36 to 6 2024-03-20T08:01:58.049 INFO:tasks.rados.rados.0.smithi118.stdout:181: finishing copy_from to smithi11881747-13 2024-03-20T08:01:58.049 INFO:tasks.rados.rados.0.smithi118.stdout:181: got expected ENOENT (src dne) 2024-03-20T08:01:58.049 INFO:tasks.rados.rados.0.smithi118.stdout:181: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:01:58.049 INFO:tasks.rados.rados.0.smithi118.stdout:181: done (2 left) 2024-03-20T08:01:58.049 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 14 2024-03-20T08:01:58.049 INFO:tasks.rados.rados.0.smithi118.stdout:185: seq_num 81 ranges {1572864=565248,2138112=737280,2875392=65536} 2024-03-20T08:01:58.060 INFO:tasks.rados.rados.0.smithi118.stdout:185: writing smithi11881747-16 from 1572864 to 2138112 tid 1 2024-03-20T08:01:58.065 INFO:tasks.rados.rados.0.smithi118.stdout:185: writing smithi11881747-16 from 2138112 to 2875392 tid 2 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:185: writing smithi11881747-16 from 2875392 to 2940928 tid 3 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:186: read oid 27 snap -1 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:186: expect (ObjNum 37 snap 0 seq_num 37) 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:187: rollback oid 40 current snap is 14 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 9 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:184: finishing rollback tid 0 to smithi11881747-36 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:184: finishing rollback tid 1 to smithi11881747-36 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:184: finishing rollback tid 2 to smithi11881747-36 2024-03-20T08:01:58.066 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 20 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-03-20T08:01:58.067 INFO:tasks.rados.rados.0.smithi118.stdout:183: expect (ObjNum 29 snap 0 seq_num 29) 2024-03-20T08:01:58.081 INFO:tasks.rados.rados.0.smithi118.stdout:185: finishing write tid 1 to smithi11881747-16 2024-03-20T08:01:58.081 INFO:tasks.rados.rados.0.smithi118.stdout:185: finishing write tid 2 to smithi11881747-16 2024-03-20T08:01:58.081 INFO:tasks.rados.rados.0.smithi118.stdout:185: finishing write tid 3 to smithi11881747-16 2024-03-20T08:01:58.081 INFO:tasks.rados.rados.0.smithi118.stdout:185: finishing write tid 4 to smithi11881747-16 2024-03-20T08:01:58.081 INFO:tasks.rados.rados.0.smithi118.stdout:187: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:01:58.081 INFO:tasks.rados.rados.0.smithi118.stdout:187: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:01:58.081 INFO:tasks.rados.rados.0.smithi118.stdout:187: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:01:58.081 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 25 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-03-20T08:01:58.082 INFO:tasks.rados.rados.0.smithi118.stdout:186: expect (ObjNum 37 snap 0 seq_num 37) 2024-03-20T08:01:58.097 INFO:tasks.rados.rados.0.smithi118.stdout:185: finishing write tid 5 to smithi11881747-16 2024-03-20T08:01:58.097 INFO:tasks.rados.rados.0.smithi118.stdout:185: oid 16 updating version 0 to 84 2024-03-20T08:01:58.097 INFO:tasks.rados.rados.0.smithi118.stdout:185: oid 16 updating version 84 to 85 2024-03-20T08:01:58.097 INFO:tasks.rados.rados.0.smithi118.stdout:185: oid 16 updating version 85 to 87 2024-03-20T08:01:58.097 INFO:tasks.rados.rados.0.smithi118.stdout:185: oid 16 version 87 is already newer than 86 2024-03-20T08:01:58.097 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 87 (ObjNum 81 snap 14 seq_num 81) dirty exists 2024-03-20T08:01:58.098 INFO:tasks.rados.rados.0.smithi118.stdout:185: left oid 16 (ObjNum 81 snap 14 seq_num 81) 2024-03-20T08:01:58.098 INFO:tasks.rados.rados.0.smithi118.stdout:183: done (4 left) 2024-03-20T08:01:58.098 INFO:tasks.rados.rados.0.smithi118.stdout:184: done (3 left) 2024-03-20T08:01:58.098 INFO:tasks.rados.rados.0.smithi118.stdout:185: done (2 left) 2024-03-20T08:01:58.098 INFO:tasks.rados.rados.0.smithi118.stdout:186: done (1 left) 2024-03-20T08:01:58.098 INFO:tasks.rados.rados.0.smithi118.stdout:187: done (0 left) 2024-03-20T08:01:58.098 INFO:tasks.rados.rados.0.smithi118.stdout:188: delete oid 11 current snap is 14 2024-03-20T08:01:58.099 INFO:tasks.rados.rados.0.smithi118.stdout:188: done (0 left) 2024-03-20T08:01:58.099 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 14 2024-03-20T08:01:58.099 INFO:tasks.rados.rados.0.smithi118.stdout:189: seq_num 83 ranges {0=598016,598016=499712,1097728=303104} 2024-03-20T08:01:58.102 INFO:tasks.rados.rados.0.smithi118.stdout:189: writing smithi11881747-44 from 0 to 598016 tid 1 2024-03-20T08:01:58.105 INFO:tasks.rados.rados.0.smithi118.stdout:189: writing smithi11881747-44 from 598016 to 1097728 tid 2 2024-03-20T08:01:58.107 INFO:tasks.rados.rados.0.smithi118.stdout:189: writing smithi11881747-44 from 1097728 to 1400832 tid 3 2024-03-20T08:01:58.107 INFO:tasks.rados.rados.0.smithi118.stdout:190: snap_create 2024-03-20T08:01:58.107 INFO:tasks.rados.rados.0.smithi118.stdout:189: finishing write tid 1 to smithi11881747-44 2024-03-20T08:01:58.110 INFO:tasks.rados.rados.0.smithi118.stdout:189: finishing write tid 2 to smithi11881747-44 2024-03-20T08:01:58.110 INFO:tasks.rados.rados.0.smithi118.stdout:189: finishing write tid 3 to smithi11881747-44 2024-03-20T08:01:58.110 INFO:tasks.rados.rados.0.smithi118.stdout:189: finishing write tid 4 to smithi11881747-44 2024-03-20T08:01:58.111 INFO:tasks.rados.rados.0.smithi118.stdout:189: finishing write tid 5 to smithi11881747-44 2024-03-20T08:01:58.111 INFO:tasks.rados.rados.0.smithi118.stdout:189: oid 44 updating version 0 to 27 2024-03-20T08:01:58.111 INFO:tasks.rados.rados.0.smithi118.stdout:189: oid 44 updating version 27 to 30 2024-03-20T08:01:58.111 INFO:tasks.rados.rados.0.smithi118.stdout:189: oid 44 version 30 is already newer than 28 2024-03-20T08:01:58.111 INFO:tasks.rados.rados.0.smithi118.stdout:189: oid 44 version 30 is already newer than 29 2024-03-20T08:01:58.111 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 30 (ObjNum 83 snap 14 seq_num 83) dirty exists 2024-03-20T08:01:58.111 INFO:tasks.rados.rados.0.smithi118.stdout:189: left oid 44 (ObjNum 83 snap 14 seq_num 83) 2024-03-20T08:01:58.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:58 smithi196 ceph-mon[31033]: osdmap e100: 8 total, 8 up, 8 in 2024-03-20T08:01:58.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:58 smithi196 ceph-mon[31033]: osdmap e101: 8 total, 8 up, 8 in 2024-03-20T08:01:58.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:58 smithi118 ceph-mon[27391]: osdmap e100: 8 total, 8 up, 8 in 2024-03-20T08:01:58.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:58 smithi118 ceph-mon[27391]: osdmap e101: 8 total, 8 up, 8 in 2024-03-20T08:01:58.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:58 smithi118 ceph-mon[38981]: osdmap e100: 8 total, 8 up, 8 in 2024-03-20T08:01:58.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:58 smithi118 ceph-mon[38981]: osdmap e101: 8 total, 8 up, 8 in 2024-03-20T08:01:58.709 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:01:59.021 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (100s) 49s ago 2m 24.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (2m) 49s ago 2m 83.2M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (3m) 49s ago 3m 12.6M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (3m) 49s ago 3m 16.7M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,8765 running (7m) 49s ago 7m 448M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 ef306191c58d 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:9283,8765 running (8m) 49s ago 8m 494M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 64dace5519ed 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (9m) 49s ago 9m 48.5M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 89e74e5abc84 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (7m) 49s ago 7m 41.0M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 fd26544c725b 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (7m) 49s ago 7m 44.3M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 8193e23658db 2024-03-20T08:01:59.022 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (2m) 49s ago 2m 14.7M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (111s) 49s ago 111s 8938k - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (6m) 49s ago 6m 47.7M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 481bba84ab31 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (6m) 49s ago 6m 46.5M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (6m) 49s ago 6m 41.9M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (5m) 49s ago 5m 40.8M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (5m) 49s ago 5m 43.5M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (4m) 49s ago 4m 42.4M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (4m) 49s ago 4m 41.5M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (3m) 49s ago 3m 42.2M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:01:59.023 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (95s) 49s ago 3m 34.9M - 2.43.0 a07b618ecd1d d713b2506adb 2024-03-20T08:01:59.058 INFO:tasks.rados.rados.0.smithi118.stdout:189: done (1 left) 2024-03-20T08:01:59.058 INFO:tasks.rados.rados.0.smithi118.stdout:190: done (0 left) 2024-03-20T08:01:59.058 INFO:tasks.rados.rados.0.smithi118.stdout:191: snap_remove snap 6 2024-03-20T08:01:59.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:01:59 smithi196 ceph-mon[31033]: pgmap v126: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 MiB/s wr, 6 op/s 2024-03-20T08:01:59.415 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:01:59 smithi118 ceph-mon[27391]: pgmap v126: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 MiB/s wr, 6 op/s 2024-03-20T08:01:59.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:01:59 smithi118 ceph-mon[38981]: pgmap v126: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 MiB/s wr, 6 op/s 2024-03-20T08:01:59.415 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:01:59.415 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:01:59.415 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:01:59.415 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 15 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:01:59.416 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:02:00.061 INFO:tasks.rados.rados.0.smithi118.stdout:191: done (0 left) 2024-03-20T08:02:00.061 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 15 2024-03-20T08:02:00.061 INFO:tasks.rados.rados.0.smithi118.stdout:192: seq_num 84 ranges {0=614400,614400=434176,1048576=368640} 2024-03-20T08:02:00.064 INFO:tasks.rados.rados.0.smithi118.stdout:192: writing smithi11881747-22 from 0 to 614400 tid 1 2024-03-20T08:02:00.067 INFO:tasks.rados.rados.0.smithi118.stdout:192: writing smithi11881747-22 from 614400 to 1048576 tid 2 2024-03-20T08:02:00.071 INFO:tasks.rados.rados.0.smithi118.stdout:192: writing smithi11881747-22 from 1048576 to 1417216 tid 3 2024-03-20T08:02:00.071 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 15 2024-03-20T08:02:00.071 INFO:tasks.rados.rados.0.smithi118.stdout:193: seq_num 85 ranges {638976=630784,1269760=720896,1990656=81920} 2024-03-20T08:02:00.079 INFO:tasks.rados.rados.0.smithi118.stdout:193: writing smithi11881747-10 from 638976 to 1269760 tid 1 2024-03-20T08:02:00.083 INFO:tasks.rados.rados.0.smithi118.stdout:193: writing smithi11881747-10 from 1269760 to 1990656 tid 2 2024-03-20T08:02:00.084 INFO:tasks.rados.rados.0.smithi118.stdout:193: writing smithi11881747-10 from 1990656 to 2072576 tid 3 2024-03-20T08:02:00.084 INFO:tasks.rados.rados.0.smithi118.stdout:194: setattr oid 6 current snap is 15 2024-03-20T08:02:00.084 INFO:tasks.rados.rados.0.smithi118.stdout:192: finishing write tid 1 to smithi11881747-22 2024-03-20T08:02:00.084 INFO:tasks.rados.rados.0.smithi118.stdout:192: finishing write tid 2 to smithi11881747-22 2024-03-20T08:02:00.085 INFO:tasks.rados.rados.0.smithi118.stdout:192: finishing write tid 3 to smithi11881747-22 2024-03-20T08:02:00.085 INFO:tasks.rados.rados.0.smithi118.stdout:192: finishing write tid 4 to smithi11881747-22 2024-03-20T08:02:00.085 INFO:tasks.rados.rados.0.smithi118.stdout:192: finishing write tid 5 to smithi11881747-22 2024-03-20T08:02:00.085 INFO:tasks.rados.rados.0.smithi118.stdout:192: oid 22 updating version 0 to 63 2024-03-20T08:02:00.085 INFO:tasks.rados.rados.0.smithi118.stdout:192: oid 22 version 63 is already newer than 62 2024-03-20T08:02:00.085 INFO:tasks.rados.rados.0.smithi118.stdout:192: oid 22 updating version 63 to 64 2024-03-20T08:02:00.085 INFO:tasks.rados.rados.0.smithi118.stdout:192: oid 22 version 64 is already newer than 61 2024-03-20T08:02:00.085 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 64 (ObjNum 84 snap 15 seq_num 84) dirty exists 2024-03-20T08:02:00.086 INFO:tasks.rados.rados.0.smithi118.stdout:192: left oid 22 (ObjNum 84 snap 15 seq_num 84) 2024-03-20T08:02:00.086 INFO:tasks.rados.rados.0.smithi118.stdout:193: finishing write tid 1 to smithi11881747-10 2024-03-20T08:02:00.086 INFO:tasks.rados.rados.0.smithi118.stdout:192: done (2 left) 2024-03-20T08:02:00.086 INFO:tasks.rados.rados.0.smithi118.stdout:195: read oid 12 snap 4 2024-03-20T08:02:00.086 INFO:tasks.rados.rados.0.smithi118.stdout:195: expect deleted 2024-03-20T08:02:00.086 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 15 2024-03-20T08:02:00.087 INFO:tasks.rados.rados.0.smithi118.stdout:196: seq_num 86 ranges {491520=647168,1138688=606208,1744896=196608} 2024-03-20T08:02:00.090 INFO:tasks.rados.rados.0.smithi118.stdout:196: writing smithi11881747-3 from 491520 to 1138688 tid 1 2024-03-20T08:02:00.094 INFO:tasks.rados.rados.0.smithi118.stdout:196: writing smithi11881747-3 from 1138688 to 1744896 tid 2 2024-03-20T08:02:00.096 INFO:tasks.rados.rados.0.smithi118.stdout:196: writing smithi11881747-3 from 1744896 to 1941504 tid 3 2024-03-20T08:02:00.096 INFO:tasks.rados.rados.0.smithi118.stdout:197: snap_create 2024-03-20T08:02:00.096 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 30 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: finishing write tid 2 to smithi11881747-10 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: finishing write tid 3 to smithi11881747-10 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: finishing write tid 4 to smithi11881747-10 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: finishing write tid 5 to smithi11881747-10 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: oid 10 updating version 0 to 82 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: oid 10 version 82 is already newer than 80 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: oid 10 version 82 is already newer than 79 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: oid 10 version 82 is already newer than 81 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 82 (ObjNum 85 snap 15 seq_num 85) dirty exists 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:193: left oid 10 (ObjNum 85 snap 15 seq_num 85) 2024-03-20T08:02:00.097 INFO:tasks.rados.rados.0.smithi118.stdout:196: finishing write tid 1 to smithi11881747-3 2024-03-20T08:02:00.098 INFO:tasks.rados.rados.0.smithi118.stdout:196: finishing write tid 2 to smithi11881747-3 2024-03-20T08:02:00.098 INFO:tasks.rados.rados.0.smithi118.stdout:196: finishing write tid 3 to smithi11881747-3 2024-03-20T08:02:00.098 INFO:tasks.rados.rados.0.smithi118.stdout:196: finishing write tid 4 to smithi11881747-3 2024-03-20T08:02:00.099 INFO:tasks.rados.rados.0.smithi118.stdout:196: finishing write tid 5 to smithi11881747-3 2024-03-20T08:02:00.099 INFO:tasks.rados.rados.0.smithi118.stdout:196: oid 3 updating version 0 to 28 2024-03-20T08:02:00.099 INFO:tasks.rados.rados.0.smithi118.stdout:196: oid 3 updating version 28 to 30 2024-03-20T08:02:00.099 INFO:tasks.rados.rados.0.smithi118.stdout:196: oid 3 updating version 30 to 31 2024-03-20T08:02:00.099 INFO:tasks.rados.rados.0.smithi118.stdout:196: oid 3 version 31 is already newer than 29 2024-03-20T08:02:00.099 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 31 (ObjNum 86 snap 15 seq_num 86) dirty exists 2024-03-20T08:02:00.099 INFO:tasks.rados.rados.0.smithi118.stdout:196: left oid 3 (ObjNum 86 snap 15 seq_num 86) 2024-03-20T08:02:00.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:00 smithi196 ceph-mon[31033]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:00 smithi196 ceph-mon[31033]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:00 smithi196 ceph-mon[31033]: osdmap e102: 8 total, 8 up, 8 in 2024-03-20T08:02:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:00 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/2283038179' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:02:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:00 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[27391]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[27391]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[27391]: osdmap e102: 8 total, 8 up, 8 in 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/2283038179' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[38981]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[38981]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[38981]: osdmap e102: 8 total, 8 up, 8 in 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/2283038179' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:02:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:00 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:01.003 INFO:tasks.rados.rados.0.smithi118.stdout:193: done (4 left) 2024-03-20T08:02:01.003 INFO:tasks.rados.rados.0.smithi118.stdout:194: done (3 left) 2024-03-20T08:02:01.004 INFO:tasks.rados.rados.0.smithi118.stdout:195: done (2 left) 2024-03-20T08:02:01.004 INFO:tasks.rados.rados.0.smithi118.stdout:196: done (1 left) 2024-03-20T08:02:01.004 INFO:tasks.rados.rados.0.smithi118.stdout:197: done (0 left) 2024-03-20T08:02:01.004 INFO:tasks.rados.rados.0.smithi118.stdout:198: snap_create 2024-03-20T08:02:01.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:01 smithi196 ceph-mon[31033]: pgmap v129: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:01 smithi196 ceph-mon[31033]: osdmap e103: 8 total, 8 up, 8 in 2024-03-20T08:02:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:01 smithi196 ceph-mon[31033]: osdmap e104: 8 total, 8 up, 8 in 2024-03-20T08:02:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:01 smithi118 ceph-mon[27391]: pgmap v129: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:01 smithi118 ceph-mon[27391]: osdmap e103: 8 total, 8 up, 8 in 2024-03-20T08:02:01.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:01 smithi118 ceph-mon[27391]: osdmap e104: 8 total, 8 up, 8 in 2024-03-20T08:02:01.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:01 smithi118 ceph-mon[38981]: pgmap v129: 121 pgs: 121 active+clean; 41 MiB data, 282 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:01 smithi118 ceph-mon[38981]: osdmap e103: 8 total, 8 up, 8 in 2024-03-20T08:02:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:01 smithi118 ceph-mon[38981]: osdmap e104: 8 total, 8 up, 8 in 2024-03-20T08:02:01.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:02:01] "GET /metrics HTTP/1.1" 200 36065 "" "Prometheus/2.43.0" 2024-03-20T08:02:02.005 INFO:tasks.rados.rados.0.smithi118.stdout:198: done (0 left) 2024-03-20T08:02:02.005 INFO:tasks.rados.rados.0.smithi118.stdout:199: read oid 45 snap -1 2024-03-20T08:02:02.005 INFO:tasks.rados.rados.0.smithi118.stdout:199: expect (ObjNum 33 snap 0 seq_num 33) 2024-03-20T08:02:02.005 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 17 2024-03-20T08:02:02.005 INFO:tasks.rados.rados.0.smithi118.stdout:200: seq_num 87 ranges {4382720=663552,5046272=491520,5537792=311296} 2024-03-20T08:02:02.032 INFO:tasks.rados.rados.0.smithi118.stdout:200: writing smithi11881747-31 from 4382720 to 5046272 tid 1 2024-03-20T08:02:02.034 INFO:tasks.rados.rados.0.smithi118.stdout:200: writing smithi11881747-31 from 5046272 to 5537792 tid 2 2024-03-20T08:02:02.037 INFO:tasks.rados.rados.0.smithi118.stdout:200: writing smithi11881747-31 from 5537792 to 5849088 tid 3 2024-03-20T08:02:02.037 INFO:tasks.rados.rados.0.smithi118.stdout:201: read oid 49 snap -1 2024-03-20T08:02:02.037 INFO:tasks.rados.rados.0.smithi118.stdout:201: expect (ObjNum 75 snap 12 seq_num 75) 2024-03-20T08:02:02.037 INFO:tasks.rados.rados.0.smithi118.stdout:202: read oid 38 snap -1 2024-03-20T08:02:02.038 INFO:tasks.rados.rados.0.smithi118.stdout:202: expect (ObjNum 37 snap 0 seq_num 37) 2024-03-20T08:02:02.038 INFO:tasks.rados.rados.0.smithi118.stdout:203: read oid 40 snap -1 2024-03-20T08:02:02.038 INFO:tasks.rados.rados.0.smithi118.stdout:203: expect (ObjNum 39 snap 0 seq_num 39) 2024-03-20T08:02:02.038 INFO:tasks.rados.rados.0.smithi118.stdout:204: snap_create 2024-03-20T08:02:02.038 INFO:tasks.rados.rados.0.smithi118.stdout:199: expect (ObjNum 33 snap 0 seq_num 33) 2024-03-20T08:02:02.054 INFO:tasks.rados.rados.0.smithi118.stdout:200: finishing write tid 1 to smithi11881747-31 2024-03-20T08:02:02.054 INFO:tasks.rados.rados.0.smithi118.stdout:200: finishing write tid 2 to smithi11881747-31 2024-03-20T08:02:02.055 INFO:tasks.rados.rados.0.smithi118.stdout:200: finishing write tid 3 to smithi11881747-31 2024-03-20T08:02:02.055 INFO:tasks.rados.rados.0.smithi118.stdout:203: expect (ObjNum 39 snap 0 seq_num 39) 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:200: finishing write tid 4 to smithi11881747-31 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:200: finishing write tid 5 to smithi11881747-31 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:200: oid 31 updating version 0 to 85 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:200: oid 31 updating version 85 to 86 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:200: oid 31 updating version 86 to 87 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:200: oid 31 version 87 is already newer than 84 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 87 (ObjNum 87 snap 17 seq_num 87) dirty exists 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:200: left oid 31 (ObjNum 87 snap 17 seq_num 87) 2024-03-20T08:02:02.072 INFO:tasks.rados.rados.0.smithi118.stdout:202: expect (ObjNum 37 snap 0 seq_num 37) 2024-03-20T08:02:02.088 INFO:tasks.rados.rados.0.smithi118.stdout:201: expect (ObjNum 75 snap 12 seq_num 75) 2024-03-20T08:02:03.008 INFO:tasks.rados.rados.0.smithi118.stdout:199: done (5 left) 2024-03-20T08:02:03.009 INFO:tasks.rados.rados.0.smithi118.stdout:200: done (4 left) 2024-03-20T08:02:03.009 INFO:tasks.rados.rados.0.smithi118.stdout:201: done (3 left) 2024-03-20T08:02:03.009 INFO:tasks.rados.rados.0.smithi118.stdout:202: done (2 left) 2024-03-20T08:02:03.009 INFO:tasks.rados.rados.0.smithi118.stdout:203: done (1 left) 2024-03-20T08:02:03.009 INFO:tasks.rados.rados.0.smithi118.stdout:204: done (0 left) 2024-03-20T08:02:03.009 INFO:tasks.rados.rados.0.smithi118.stdout:205: snap_remove snap 13 2024-03-20T08:02:03.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:03 smithi196 ceph-mon[31033]: pgmap v132: 121 pgs: 121 active+clean; 49 MiB data, 298 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-03-20T08:02:03.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:03 smithi196 ceph-mon[31033]: osdmap e105: 8 total, 8 up, 8 in 2024-03-20T08:02:03.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:03 smithi118 ceph-mon[27391]: pgmap v132: 121 pgs: 121 active+clean; 49 MiB data, 298 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-03-20T08:02:03.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:03 smithi118 ceph-mon[27391]: osdmap e105: 8 total, 8 up, 8 in 2024-03-20T08:02:03.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:03 smithi118 ceph-mon[38981]: pgmap v132: 121 pgs: 121 active+clean; 49 MiB data, 298 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-03-20T08:02:03.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:03 smithi118 ceph-mon[38981]: osdmap e105: 8 total, 8 up, 8 in 2024-03-20T08:02:04.015 INFO:tasks.rados.rados.0.smithi118.stdout:205: done (0 left) 2024-03-20T08:02:04.015 INFO:tasks.rados.rados.0.smithi118.stdout:206: read oid 36 snap -1 2024-03-20T08:02:04.015 INFO:tasks.rados.rados.0.smithi118.stdout:206: expect (ObjNum 1 snap 0 seq_num 1) 2024-03-20T08:02:04.015 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 18 2024-03-20T08:02:04.015 INFO:tasks.rados.rados.0.smithi118.stdout:207: seq_num 88 ranges {0=679936,679936=778240,1458176=24576} 2024-03-20T08:02:04.019 INFO:tasks.rados.rados.0.smithi118.stdout:207: writing smithi11881747-1 from 0 to 679936 tid 1 2024-03-20T08:02:04.025 INFO:tasks.rados.rados.0.smithi118.stdout:207: writing smithi11881747-1 from 679936 to 1458176 tid 2 2024-03-20T08:02:04.025 INFO:tasks.rados.rados.0.smithi118.stdout:207: writing smithi11881747-1 from 1458176 to 1482752 tid 3 2024-03-20T08:02:04.025 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 18 2024-03-20T08:02:04.025 INFO:tasks.rados.rados.0.smithi118.stdout:206: expect (ObjNum 1 snap 0 seq_num 1) 2024-03-20T08:02:04.026 INFO:tasks.rados.rados.0.smithi118.stdout:207: finishing write tid 1 to smithi11881747-1 2024-03-20T08:02:04.026 INFO:tasks.rados.rados.0.smithi118.stdout:208: seq_num 89 ranges {1302528=696320,1998848=655360,2654208=147456} 2024-03-20T08:02:04.037 INFO:tasks.rados.rados.0.smithi118.stdout:208: writing smithi11881747-9 from 1302528 to 1998848 tid 1 2024-03-20T08:02:04.040 INFO:tasks.rados.rados.0.smithi118.stdout:208: writing smithi11881747-9 from 1998848 to 2654208 tid 2 2024-03-20T08:02:04.042 INFO:tasks.rados.rados.0.smithi118.stdout:208: writing smithi11881747-9 from 2654208 to 2801664 tid 3 2024-03-20T08:02:04.042 INFO:tasks.rados.rados.0.smithi118.stdout:206: done (2 left) 2024-03-20T08:02:04.042 INFO:tasks.rados.rados.0.smithi118.stdout:209: rmattr oid 19 current snap is 18 2024-03-20T08:02:04.042 INFO:tasks.rados.rados.0.smithi118.stdout:209: done (2 left) 2024-03-20T08:02:04.042 INFO:tasks.rados.rados.0.smithi118.stdout:210: snap_remove snap 17 2024-03-20T08:02:04.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:04 smithi196 ceph-mon[31033]: osdmap e106: 8 total, 8 up, 8 in 2024-03-20T08:02:04.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:04 smithi118 ceph-mon[27391]: osdmap e106: 8 total, 8 up, 8 in 2024-03-20T08:02:04.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:04 smithi118 ceph-mon[38981]: osdmap e106: 8 total, 8 up, 8 in 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: finishing write tid 2 to smithi11881747-1 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: finishing write tid 3 to smithi11881747-1 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: finishing write tid 4 to smithi11881747-1 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: finishing write tid 5 to smithi11881747-1 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: oid 1 updating version 0 to 93 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: oid 1 updating version 93 to 94 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: oid 1 updating version 94 to 95 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: oid 1 version 95 is already newer than 92 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 95 (ObjNum 88 snap 18 seq_num 88) dirty exists 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:207: left oid 1 (ObjNum 88 snap 18 seq_num 88) 2024-03-20T08:02:05.020 INFO:tasks.rados.rados.0.smithi118.stdout:208: finishing write tid 1 to smithi11881747-9 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: finishing write tid 2 to smithi11881747-9 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: finishing write tid 3 to smithi11881747-9 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: finishing write tid 4 to smithi11881747-9 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: finishing write tid 5 to smithi11881747-9 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: oid 9 updating version 0 to 47 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: oid 9 updating version 47 to 49 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: oid 9 version 49 is already newer than 48 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: oid 9 version 49 is already newer than 46 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 49 (ObjNum 89 snap 18 seq_num 89) dirty exists 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: left oid 9 (ObjNum 89 snap 18 seq_num 89) 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:207: done (2 left) 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:208: done (1 left) 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:210: done (0 left) 2024-03-20T08:02:05.021 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 18 2024-03-20T08:02:05.022 INFO:tasks.rados.rados.0.smithi118.stdout:211: seq_num 90 ranges {557056=712704,1269760=540672,1810432=262144} 2024-03-20T08:02:05.026 INFO:tasks.rados.rados.0.smithi118.stdout:211: writing smithi11881747-45 from 557056 to 1269760 tid 1 2024-03-20T08:02:05.029 INFO:tasks.rados.rados.0.smithi118.stdout:211: writing smithi11881747-45 from 1269760 to 1810432 tid 2 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:211: writing smithi11881747-45 from 1810432 to 2072576 tid 3 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:212: copy_from oid 28 from oid 4 current snap is 18 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:213: rollback oid 37 current snap is 18 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 16 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:214: rmattr oid 8 current snap is 18 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:214: done (3 left) 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:215: rmattr oid 23 current snap is 18 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:215: done (3 left) 2024-03-20T08:02:05.033 INFO:tasks.rados.rados.0.smithi118.stdout:216: rmattr oid 39 current snap is 18 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:216: done (3 left) 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:217: read oid 1 snap -1 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:217: expect (ObjNum 88 snap 18 seq_num 88) 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:218: read oid 6 snap -1 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:218: expect (ObjNum 36 snap 0 seq_num 36) 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:219: delete oid 23 current snap is 18 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:211: finishing write tid 1 to smithi11881747-45 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:212: finishing copy_from to smithi11881747-28 2024-03-20T08:02:05.034 INFO:tasks.rados.rados.0.smithi118.stdout:212: got expected ENOENT (src dne) 2024-03-20T08:02:05.036 INFO:tasks.rados.rados.0.smithi118.stdout:212: finishing copy_from racing read to smithi11881747-28 2024-03-20T08:02:05.037 INFO:tasks.rados.rados.0.smithi118.stdout:211: finishing write tid 2 to smithi11881747-45 2024-03-20T08:02:05.038 INFO:tasks.rados.rados.0.smithi118.stdout:212: done (5 left) 2024-03-20T08:02:05.038 INFO:tasks.rados.rados.0.smithi118.stdout:219: done (4 left) 2024-03-20T08:02:05.038 INFO:tasks.rados.rados.0.smithi118.stdout:220: read oid 41 snap -1 2024-03-20T08:02:05.038 INFO:tasks.rados.rados.0.smithi118.stdout:220: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:05.038 INFO:tasks.rados.rados.0.smithi118.stdout:211: finishing write tid 3 to smithi11881747-45 2024-03-20T08:02:05.038 INFO:tasks.rados.rados.0.smithi118.stdout:211: finishing write tid 4 to smithi11881747-45 2024-03-20T08:02:05.038 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 18 2024-03-20T08:02:05.038 INFO:tasks.rados.rados.0.smithi118.stdout:221: seq_num 92 ranges {1130496=745472,1875968=761856,2637824=40960} 2024-03-20T08:02:05.048 INFO:tasks.rados.rados.0.smithi118.stdout:221: writing smithi11881747-18 from 1130496 to 1875968 tid 1 2024-03-20T08:02:05.053 INFO:tasks.rados.rados.0.smithi118.stdout:221: writing smithi11881747-18 from 1875968 to 2637824 tid 2 2024-03-20T08:02:05.053 INFO:tasks.rados.rados.0.smithi118.stdout:221: writing smithi11881747-18 from 2637824 to 2678784 tid 3 2024-03-20T08:02:05.053 INFO:tasks.rados.rados.0.smithi118.stdout:222: delete oid 7 current snap is 18 2024-03-20T08:02:05.053 INFO:tasks.rados.rados.0.smithi118.stdout:218: expect (ObjNum 36 snap 0 seq_num 36) 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:213: finishing rollback tid 0 to smithi11881747-37 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:213: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:213: finishing rollback tid 2 to smithi11881747-37 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 38 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:211: finishing write tid 5 to smithi11881747-45 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:211: oid 45 updating version 0 to 31 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:211: oid 45 updating version 31 to 33 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:211: oid 45 updating version 33 to 34 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:211: oid 45 version 34 is already newer than 32 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 34 (ObjNum 90 snap 18 seq_num 90) dirty exists 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:211: left oid 45 (ObjNum 90 snap 18 seq_num 90) 2024-03-20T08:02:05.073 INFO:tasks.rados.rados.0.smithi118.stdout:211: done (6 left) 2024-03-20T08:02:05.074 INFO:tasks.rados.rados.0.smithi118.stdout:213: done (5 left) 2024-03-20T08:02:05.074 INFO:tasks.rados.rados.0.smithi118.stdout:218: done (4 left) 2024-03-20T08:02:05.074 INFO:tasks.rados.rados.0.smithi118.stdout:222: done (3 left) 2024-03-20T08:02:05.074 INFO:tasks.rados.rados.0.smithi118.stdout:223: delete oid 40 current snap is 18 2024-03-20T08:02:05.074 INFO:tasks.rados.rados.0.smithi118.stdout:220: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:05.092 INFO:tasks.rados.rados.0.smithi118.stdout:217: expect (ObjNum 88 snap 18 seq_num 88) 2024-03-20T08:02:05.130 INFO:tasks.rados.rados.0.smithi118.stdout:221: finishing write tid 1 to smithi11881747-18 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:221: finishing write tid 2 to smithi11881747-18 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:221: finishing write tid 3 to smithi11881747-18 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:221: finishing write tid 4 to smithi11881747-18 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:217: done (3 left) 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:220: done (2 left) 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:223: done (1 left) 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:224: setattr oid 48 current snap is 18 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:221: finishing write tid 5 to smithi11881747-18 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:221: oid 18 updating version 0 to 60 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:221: oid 18 version 60 is already newer than 59 2024-03-20T08:02:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:221: oid 18 version 60 is already newer than 58 2024-03-20T08:02:05.132 INFO:tasks.rados.rados.0.smithi118.stdout:221: oid 18 updating version 60 to 61 2024-03-20T08:02:05.132 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 61 (ObjNum 92 snap 18 seq_num 92) dirty exists 2024-03-20T08:02:05.132 INFO:tasks.rados.rados.0.smithi118.stdout:221: left oid 18 (ObjNum 92 snap 18 seq_num 92) 2024-03-20T08:02:05.132 INFO:tasks.rados.rados.0.smithi118.stdout:221: done (1 left) 2024-03-20T08:02:05.132 INFO:tasks.rados.rados.0.smithi118.stdout:225: snap_remove snap 4 2024-03-20T08:02:05.323 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:05 smithi118 ceph-mon[27391]: pgmap v135: 121 pgs: 121 active+clean; 49 MiB data, 298 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-03-20T08:02:05.324 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:05 smithi118 ceph-mon[27391]: osdmap e107: 8 total, 8 up, 8 in 2024-03-20T08:02:05.324 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:05 smithi118 ceph-mon[38981]: pgmap v135: 121 pgs: 121 active+clean; 49 MiB data, 298 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-03-20T08:02:05.324 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:05 smithi118 ceph-mon[38981]: osdmap e107: 8 total, 8 up, 8 in 2024-03-20T08:02:05.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:05 smithi196 ceph-mon[31033]: pgmap v135: 121 pgs: 121 active+clean; 49 MiB data, 298 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-03-20T08:02:05.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:05 smithi196 ceph-mon[31033]: osdmap e107: 8 total, 8 up, 8 in 2024-03-20T08:02:06.024 INFO:tasks.rados.rados.0.smithi118.stdout:225: done (1 left) 2024-03-20T08:02:06.024 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 18 2024-03-20T08:02:06.024 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 67 (ObjNum 129 snap 0 seq_num 321910176) dirty exists 2024-03-20T08:02:06.024 INFO:tasks.rados.rados.0.smithi118.stdout:226: seq_num 95 ranges {606208=794624,1400832=409600,1810432=393216} 2024-03-20T08:02:06.032 INFO:tasks.rados.rados.0.smithi118.stdout:226: writing smithi11881747-37 from 606208 to 1400832 tid 1 2024-03-20T08:02:06.036 INFO:tasks.rados.rados.0.smithi118.stdout:226: writing smithi11881747-37 from 1400832 to 1810432 tid 2 2024-03-20T08:02:06.039 INFO:tasks.rados.rados.0.smithi118.stdout:226: writing smithi11881747-37 from 1810432 to 2203648 tid 3 2024-03-20T08:02:06.039 INFO:tasks.rados.rados.0.smithi118.stdout:224: done (1 left) 2024-03-20T08:02:06.039 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 18 2024-03-20T08:02:06.039 INFO:tasks.rados.rados.0.smithi118.stdout:227: seq_num 96 ranges {1056768=409600,1466368=696320,2162688=507904} 2024-03-20T08:02:06.047 INFO:tasks.rados.rados.0.smithi118.stdout:227: writing smithi11881747-2 from 1056768 to 1466368 tid 1 2024-03-20T08:02:06.051 INFO:tasks.rados.rados.0.smithi118.stdout:227: writing smithi11881747-2 from 1466368 to 2162688 tid 2 2024-03-20T08:02:06.055 INFO:tasks.rados.rados.0.smithi118.stdout:227: writing smithi11881747-2 from 2162688 to 2670592 tid 3 2024-03-20T08:02:06.055 INFO:tasks.rados.rados.0.smithi118.stdout:228: copy_from oid 27 from oid 26 current snap is 18 2024-03-20T08:02:06.055 INFO:tasks.rados.rados.0.smithi118.stdout:226: finishing write tid 1 to smithi11881747-37 2024-03-20T08:02:06.055 INFO:tasks.rados.rados.0.smithi118.stdout:226: finishing write tid 2 to smithi11881747-37 2024-03-20T08:02:06.055 INFO:tasks.rados.rados.0.smithi118.stdout:226: finishing write tid 3 to smithi11881747-37 2024-03-20T08:02:06.055 INFO:tasks.rados.rados.0.smithi118.stdout:226: finishing write tid 4 to smithi11881747-37 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:226: finishing write tid 5 to smithi11881747-37 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:226: oid 37 updating version 0 to 50 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:226: oid 37 version 50 is already newer than 46 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:226: oid 37 version 50 is already newer than 45 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:226: oid 37 version 50 is already newer than 49 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 50 (ObjNum 95 snap 18 seq_num 95) dirty exists 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:226: left oid 37 (ObjNum 95 snap 18 seq_num 95) 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:227: finishing write tid 1 to smithi11881747-2 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:227: finishing write tid 2 to smithi11881747-2 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:226: done (2 left) 2024-03-20T08:02:06.056 INFO:tasks.rados.rados.0.smithi118.stdout:229: snap_create 2024-03-20T08:02:06.057 INFO:tasks.rados.rados.0.smithi118.stdout:227: finishing write tid 3 to smithi11881747-2 2024-03-20T08:02:06.057 INFO:tasks.rados.rados.0.smithi118.stdout:227: finishing write tid 4 to smithi11881747-2 2024-03-20T08:02:06.058 INFO:tasks.rados.rados.0.smithi118.stdout:227: finishing write tid 5 to smithi11881747-2 2024-03-20T08:02:06.058 INFO:tasks.rados.rados.0.smithi118.stdout:227: oid 2 updating version 0 to 43 2024-03-20T08:02:06.058 INFO:tasks.rados.rados.0.smithi118.stdout:227: oid 2 version 43 is already newer than 41 2024-03-20T08:02:06.058 INFO:tasks.rados.rados.0.smithi118.stdout:227: oid 2 version 43 is already newer than 40 2024-03-20T08:02:06.058 INFO:tasks.rados.rados.0.smithi118.stdout:227: oid 2 version 43 is already newer than 42 2024-03-20T08:02:06.058 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 43 (ObjNum 96 snap 18 seq_num 96) dirty exists 2024-03-20T08:02:06.058 INFO:tasks.rados.rados.0.smithi118.stdout:227: left oid 2 (ObjNum 96 snap 18 seq_num 96) 2024-03-20T08:02:06.059 INFO:tasks.rados.rados.0.smithi118.stdout:228: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:02:06.059 INFO:tasks.rados.rados.0.smithi118.stdout:228: finishing copy_from to smithi11881747-27 2024-03-20T08:02:06.060 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 4 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-03-20T08:02:06.340 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:06 smithi118 ceph-mon[27391]: osdmap e108: 8 total, 8 up, 8 in 2024-03-20T08:02:06.340 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:06 smithi118 ceph-mon[38981]: osdmap e108: 8 total, 8 up, 8 in 2024-03-20T08:02:06.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:06 smithi196 ceph-mon[31033]: osdmap e108: 8 total, 8 up, 8 in 2024-03-20T08:02:07.026 INFO:tasks.rados.rados.0.smithi118.stdout:227: done (2 left) 2024-03-20T08:02:07.026 INFO:tasks.rados.rados.0.smithi118.stdout:228: done (1 left) 2024-03-20T08:02:07.026 INFO:tasks.rados.rados.0.smithi118.stdout:229: done (0 left) 2024-03-20T08:02:07.027 INFO:tasks.rados.rados.0.smithi118.stdout:230: snap_create 2024-03-20T08:02:07.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:07 smithi196 ceph-mon[31033]: pgmap v138: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 21 op/s 2024-03-20T08:02:07.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:07 smithi196 ceph-mon[31033]: osdmap e109: 8 total, 8 up, 8 in 2024-03-20T08:02:07.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:07 smithi118 ceph-mon[27391]: pgmap v138: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 21 op/s 2024-03-20T08:02:07.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:07 smithi118 ceph-mon[27391]: osdmap e109: 8 total, 8 up, 8 in 2024-03-20T08:02:07.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:07 smithi118 ceph-mon[38981]: pgmap v138: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 21 op/s 2024-03-20T08:02:07.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:07 smithi118 ceph-mon[38981]: osdmap e109: 8 total, 8 up, 8 in 2024-03-20T08:02:08.033 INFO:tasks.rados.rados.0.smithi118.stdout:230: done (0 left) 2024-03-20T08:02:08.033 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 20 2024-03-20T08:02:08.033 INFO:tasks.rados.rados.0.smithi118.stdout:231: seq_num 97 ranges {1302528=434176,1736704=581632,2318336=540672,2859008=81920} 2024-03-20T08:02:08.042 INFO:tasks.rados.rados.0.smithi118.stdout:231: writing smithi11881747-17 from 1302528 to 1736704 tid 1 2024-03-20T08:02:08.045 INFO:tasks.rados.rados.0.smithi118.stdout:231: writing smithi11881747-17 from 1736704 to 2318336 tid 2 2024-03-20T08:02:08.048 INFO:tasks.rados.rados.0.smithi118.stdout:231: writing smithi11881747-17 from 2318336 to 2859008 tid 3 2024-03-20T08:02:08.049 INFO:tasks.rados.rados.0.smithi118.stdout:231: writing smithi11881747-17 from 2859008 to 2940928 tid 4 2024-03-20T08:02:08.049 INFO:tasks.rados.rados.0.smithi118.stdout:232: rollback oid 8 current snap is 20 2024-03-20T08:02:08.049 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 8 to 14 2024-03-20T08:02:08.049 INFO:tasks.rados.rados.0.smithi118.stdout:233: rmattr oid 26 current snap is 20 2024-03-20T08:02:08.049 INFO:tasks.rados.rados.0.smithi118.stdout:233: done (2 left) 2024-03-20T08:02:08.049 INFO:tasks.rados.rados.0.smithi118.stdout:234: delete oid 2 current snap is 20 2024-03-20T08:02:08.050 INFO:tasks.rados.rados.0.smithi118.stdout:231: finishing write tid 1 to smithi11881747-17 2024-03-20T08:02:08.050 INFO:tasks.rados.rados.0.smithi118.stdout:231: finishing write tid 2 to smithi11881747-17 2024-03-20T08:02:08.052 INFO:tasks.rados.rados.0.smithi118.stdout:234: done (2 left) 2024-03-20T08:02:08.052 INFO:tasks.rados.rados.0.smithi118.stdout:235: rmattr oid 24 current snap is 20 2024-03-20T08:02:08.052 INFO:tasks.rados.rados.0.smithi118.stdout:235: done (2 left) 2024-03-20T08:02:08.052 INFO:tasks.rados.rados.0.smithi118.stdout:236: snap_create 2024-03-20T08:02:08.053 INFO:tasks.rados.rados.0.smithi118.stdout:231: finishing write tid 3 to smithi11881747-17 2024-03-20T08:02:08.053 INFO:tasks.rados.rados.0.smithi118.stdout:231: finishing write tid 4 to smithi11881747-17 2024-03-20T08:02:08.053 INFO:tasks.rados.rados.0.smithi118.stdout:231: finishing write tid 5 to smithi11881747-17 2024-03-20T08:02:08.054 INFO:tasks.rados.rados.0.smithi118.stdout:231: finishing write tid 6 to smithi11881747-17 2024-03-20T08:02:08.055 INFO:tasks.rados.rados.0.smithi118.stdout:231: oid 17 updating version 0 to 33 2024-03-20T08:02:08.055 INFO:tasks.rados.rados.0.smithi118.stdout:231: oid 17 updating version 33 to 36 2024-03-20T08:02:08.055 INFO:tasks.rados.rados.0.smithi118.stdout:231: oid 17 updating version 36 to 37 2024-03-20T08:02:08.055 INFO:tasks.rados.rados.0.smithi118.stdout:231: oid 17 version 37 is already newer than 35 2024-03-20T08:02:08.055 INFO:tasks.rados.rados.0.smithi118.stdout:231: oid 17 version 37 is already newer than 34 2024-03-20T08:02:08.055 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 37 (ObjNum 97 snap 20 seq_num 97) dirty exists 2024-03-20T08:02:08.055 INFO:tasks.rados.rados.0.smithi118.stdout:231: left oid 17 (ObjNum 97 snap 20 seq_num 97) 2024-03-20T08:02:08.056 INFO:tasks.rados.rados.0.smithi118.stdout:232: finishing rollback tid 0 to smithi11881747-8 2024-03-20T08:02:08.056 INFO:tasks.rados.rados.0.smithi118.stdout:232: finishing rollback tid 1 to smithi11881747-8 2024-03-20T08:02:08.056 INFO:tasks.rados.rados.0.smithi118.stdout:232: finishing rollback tid 2 to smithi11881747-8 2024-03-20T08:02:08.056 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 50 (ObjNum 68 snap 6 seq_num 68) dirty exists 2024-03-20T08:02:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:08 smithi196 ceph-mon[31033]: osdmap e110: 8 total, 8 up, 8 in 2024-03-20T08:02:08.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:08 smithi118 ceph-mon[27391]: osdmap e110: 8 total, 8 up, 8 in 2024-03-20T08:02:08.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:08 smithi118 ceph-mon[38981]: osdmap e110: 8 total, 8 up, 8 in 2024-03-20T08:02:09.036 INFO:tasks.rados.rados.0.smithi118.stdout:231: done (2 left) 2024-03-20T08:02:09.037 INFO:tasks.rados.rados.0.smithi118.stdout:232: done (1 left) 2024-03-20T08:02:09.037 INFO:tasks.rados.rados.0.smithi118.stdout:236: done (0 left) 2024-03-20T08:02:09.037 INFO:tasks.rados.rados.0.smithi118.stdout:237: copy_from oid 48 from oid 47 current snap is 21 2024-03-20T08:02:09.037 INFO:tasks.rados.rados.0.smithi118.stdout:238: read oid 34 snap -1 2024-03-20T08:02:09.037 INFO:tasks.rados.rados.0.smithi118.stdout:238: expect (ObjNum 33 snap 0 seq_num 33) 2024-03-20T08:02:09.037 INFO:tasks.rados.rados.0.smithi118.stdout:239: snap_create 2024-03-20T08:02:09.042 INFO:tasks.rados.rados.0.smithi118.stdout:238: expect (ObjNum 33 snap 0 seq_num 33) 2024-03-20T08:02:09.057 INFO:tasks.rados.rados.0.smithi118.stdout:237: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:02:09.058 INFO:tasks.rados.rados.0.smithi118.stdout:237: finishing copy_from to smithi11881747-48 2024-03-20T08:02:09.058 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 12 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:02:09.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:09 smithi196 ceph-mon[31033]: pgmap v141: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 21 op/s 2024-03-20T08:02:09.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:09 smithi196 ceph-mon[31033]: osdmap e111: 8 total, 8 up, 8 in 2024-03-20T08:02:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:09 smithi118 ceph-mon[27391]: pgmap v141: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 21 op/s 2024-03-20T08:02:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:09 smithi118 ceph-mon[27391]: osdmap e111: 8 total, 8 up, 8 in 2024-03-20T08:02:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:09 smithi118 ceph-mon[38981]: pgmap v141: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail; 4.8 MiB/s rd, 1.8 MiB/s wr, 21 op/s 2024-03-20T08:02:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:09 smithi118 ceph-mon[38981]: osdmap e111: 8 total, 8 up, 8 in 2024-03-20T08:02:10.042 INFO:tasks.rados.rados.0.smithi118.stdout:237: done (2 left) 2024-03-20T08:02:10.042 INFO:tasks.rados.rados.0.smithi118.stdout:238: done (1 left) 2024-03-20T08:02:10.042 INFO:tasks.rados.rados.0.smithi118.stdout:239: done (0 left) 2024-03-20T08:02:10.042 INFO:tasks.rados.rados.0.smithi118.stdout:240: read oid 47 snap -1 2024-03-20T08:02:10.042 INFO:tasks.rados.rados.0.smithi118.stdout:240: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:10.042 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 22 2024-03-20T08:02:10.043 INFO:tasks.rados.rados.0.smithi118.stdout:241: seq_num 99 ranges {2072576=466944,2539520=802816,3342336=401408} 2024-03-20T08:02:10.055 INFO:tasks.rados.rados.0.smithi118.stdout:241: writing smithi11881747-45 from 2072576 to 2539520 tid 1 2024-03-20T08:02:10.060 INFO:tasks.rados.rados.0.smithi118.stdout:241: writing smithi11881747-45 from 2539520 to 3342336 tid 2 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:241: writing smithi11881747-45 from 3342336 to 3743744 tid 3 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:242: read oid 49 snap 20 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:242: expect (ObjNum 75 snap 12 seq_num 75) 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:243: read oid 28 snap -1 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:243: expect (ObjNum 9 snap 0 seq_num 9) 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:244: read oid 11 snap -1 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:244: expect deleted 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:245: read oid 5 snap 21 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:245: expect (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:02:10.064 INFO:tasks.rados.rados.0.smithi118.stdout:246: setattr oid 48 current snap is 22 2024-03-20T08:02:10.065 INFO:tasks.rados.rados.0.smithi118.stdout:240: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:10.088 INFO:tasks.rados.rados.0.smithi118.stdout:241: finishing write tid 1 to smithi11881747-45 2024-03-20T08:02:10.088 INFO:tasks.rados.rados.0.smithi118.stdout:241: finishing write tid 2 to smithi11881747-45 2024-03-20T08:02:10.088 INFO:tasks.rados.rados.0.smithi118.stdout:243: expect (ObjNum 9 snap 0 seq_num 9) 2024-03-20T08:02:10.095 INFO:tasks.rados.rados.0.smithi118.stdout:241: finishing write tid 3 to smithi11881747-45 2024-03-20T08:02:10.095 INFO:tasks.rados.rados.0.smithi118.stdout:245: expect (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:02:10.097 INFO:tasks.rados.rados.0.smithi118.stdout:241: finishing write tid 4 to smithi11881747-45 2024-03-20T08:02:10.097 INFO:tasks.rados.rados.0.smithi118.stdout:241: finishing write tid 5 to smithi11881747-45 2024-03-20T08:02:10.097 INFO:tasks.rados.rados.0.smithi118.stdout:241: oid 45 updating version 0 to 52 2024-03-20T08:02:10.097 INFO:tasks.rados.rados.0.smithi118.stdout:241: oid 45 updating version 52 to 55 2024-03-20T08:02:10.097 INFO:tasks.rados.rados.0.smithi118.stdout:241: oid 45 version 55 is already newer than 53 2024-03-20T08:02:10.097 INFO:tasks.rados.rados.0.smithi118.stdout:241: oid 45 version 55 is already newer than 54 2024-03-20T08:02:10.097 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 55 (ObjNum 99 snap 22 seq_num 99) dirty exists 2024-03-20T08:02:10.098 INFO:tasks.rados.rados.0.smithi118.stdout:241: left oid 45 (ObjNum 99 snap 22 seq_num 99) 2024-03-20T08:02:10.098 INFO:tasks.rados.rados.0.smithi118.stdout:242: expect (ObjNum 75 snap 12 seq_num 75) 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:240: done (6 left) 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:241: done (5 left) 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:242: done (4 left) 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:243: done (3 left) 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:244: done (2 left) 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:245: done (1 left) 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:247: rollback oid 15 current snap is 22 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 15 to 16 2024-03-20T08:02:10.167 INFO:tasks.rados.rados.0.smithi118.stdout:248: delete oid 44 current snap is 22 2024-03-20T08:02:10.171 INFO:tasks.rados.rados.0.smithi118.stdout:248: done (2 left) 2024-03-20T08:02:10.171 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 22 2024-03-20T08:02:10.171 INFO:tasks.rados.rados.0.smithi118.stdout:249: seq_num 101 ranges {3743744=499712,4243456=573440,4816896=524288,5341184=106496} 2024-03-20T08:02:10.193 INFO:tasks.rados.rados.0.smithi118.stdout:249: writing smithi11881747-45 from 3743744 to 4243456 tid 1 2024-03-20T08:02:10.196 INFO:tasks.rados.rados.0.smithi118.stdout:249: writing smithi11881747-45 from 4243456 to 4816896 tid 2 2024-03-20T08:02:10.200 INFO:tasks.rados.rados.0.smithi118.stdout:249: writing smithi11881747-45 from 4816896 to 5341184 tid 3 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:249: writing smithi11881747-45 from 5341184 to 5447680 tid 4 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:250: setattr oid 10 current snap is 22 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 77 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:247: finishing rollback tid 0 to smithi11881747-15 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:247: finishing rollback tid 1 to smithi11881747-15 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:247: finishing rollback tid 2 to smithi11881747-15 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 26 (ObjNum 59 snap 5 seq_num 59) dirty exists 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:249: finishing write tid 1 to smithi11881747-45 2024-03-20T08:02:10.201 INFO:tasks.rados.rados.0.smithi118.stdout:249: finishing write tid 2 to smithi11881747-45 2024-03-20T08:02:10.203 INFO:tasks.rados.rados.0.smithi118.stdout:249: finishing write tid 3 to smithi11881747-45 2024-03-20T08:02:10.203 INFO:tasks.rados.rados.0.smithi118.stdout:249: finishing write tid 4 to smithi11881747-45 2024-03-20T08:02:10.203 INFO:tasks.rados.rados.0.smithi118.stdout:249: finishing write tid 5 to smithi11881747-45 2024-03-20T08:02:10.203 INFO:tasks.rados.rados.0.smithi118.stdout:246: done (3 left) 2024-03-20T08:02:10.203 INFO:tasks.rados.rados.0.smithi118.stdout:247: done (2 left) 2024-03-20T08:02:10.203 INFO:tasks.rados.rados.0.smithi118.stdout:251: setattr oid 30 current snap is 22 2024-03-20T08:02:10.204 INFO:tasks.rados.rados.0.smithi118.stdout:249: finishing write tid 6 to smithi11881747-45 2024-03-20T08:02:10.204 INFO:tasks.rados.rados.0.smithi118.stdout:249: oid 45 updating version 0 to 60 2024-03-20T08:02:10.204 INFO:tasks.rados.rados.0.smithi118.stdout:249: oid 45 version 60 is already newer than 57 2024-03-20T08:02:10.204 INFO:tasks.rados.rados.0.smithi118.stdout:249: oid 45 version 60 is already newer than 58 2024-03-20T08:02:10.204 INFO:tasks.rados.rados.0.smithi118.stdout:249: oid 45 version 60 is already newer than 56 2024-03-20T08:02:10.204 INFO:tasks.rados.rados.0.smithi118.stdout:249: oid 45 version 60 is already newer than 59 2024-03-20T08:02:10.204 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 60 (ObjNum 101 snap 22 seq_num 101) dirty exists 2024-03-20T08:02:10.204 INFO:tasks.rados.rados.0.smithi118.stdout:249: left oid 45 (ObjNum 101 snap 22 seq_num 101) 2024-03-20T08:02:10.205 INFO:tasks.rados.rados.0.smithi118.stdout:249: done (2 left) 2024-03-20T08:02:10.205 INFO:tasks.rados.rados.0.smithi118.stdout:252: snap_remove snap 20 2024-03-20T08:02:10.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:10 smithi196 ceph-mon[31033]: osdmap e112: 8 total, 8 up, 8 in 2024-03-20T08:02:10.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:10 smithi118 ceph-mon[27391]: osdmap e112: 8 total, 8 up, 8 in 2024-03-20T08:02:10.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:10 smithi118 ceph-mon[38981]: osdmap e112: 8 total, 8 up, 8 in 2024-03-20T08:02:11.005 INFO:tasks.rados.rados.0.smithi118.stdout:252: done (2 left) 2024-03-20T08:02:11.006 INFO:tasks.rados.rados.0.smithi118.stdout:253: read oid 46 snap -1 2024-03-20T08:02:11.006 INFO:tasks.rados.rados.0.smithi118.stdout:253: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:02:11.009 INFO:tasks.rados.rados.0.smithi118.stdout:254: read oid 39 snap -1 2024-03-20T08:02:11.009 INFO:tasks.rados.rados.0.smithi118.stdout:254: expect (ObjNum 38 snap 0 seq_num 38) 2024-03-20T08:02:11.009 INFO:tasks.rados.rados.0.smithi118.stdout:255: snap_create 2024-03-20T08:02:11.009 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 103 (ObjNum 85 snap 15 seq_num 85) dirty exists 2024-03-20T08:02:11.009 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 15 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-03-20T08:02:11.016 INFO:tasks.rados.rados.0.smithi118.stdout:253: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:02:11.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:11 smithi196 ceph-mon[31033]: pgmap v144: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:11.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:11 smithi196 ceph-mon[31033]: osdmap e113: 8 total, 8 up, 8 in 2024-03-20T08:02:11.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:11 smithi196 ceph-mon[31033]: osdmap e114: 8 total, 8 up, 8 in 2024-03-20T08:02:11.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:11 smithi118 ceph-mon[38981]: pgmap v144: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:11.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:11 smithi118 ceph-mon[38981]: osdmap e113: 8 total, 8 up, 8 in 2024-03-20T08:02:11.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:11 smithi118 ceph-mon[38981]: osdmap e114: 8 total, 8 up, 8 in 2024-03-20T08:02:11.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:11 smithi118 ceph-mon[27391]: pgmap v144: 121 pgs: 121 active+clean; 56 MiB data, 311 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:11.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:11 smithi118 ceph-mon[27391]: osdmap e113: 8 total, 8 up, 8 in 2024-03-20T08:02:11.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:11 smithi118 ceph-mon[27391]: osdmap e114: 8 total, 8 up, 8 in 2024-03-20T08:02:11.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:02:11] "GET /metrics HTTP/1.1" 200 36065 "" "Prometheus/2.43.0" 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:250: done (4 left) 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:251: done (3 left) 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:253: done (2 left) 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:255: done (1 left) 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:256: rollback oid 43 current snap is 23 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 43 to 18 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:257: copy_from oid 36 from oid 41 current snap is 23 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:258: copy_from oid 37 from oid 27 current snap is 23 2024-03-20T08:02:12.007 INFO:tasks.rados.rados.0.smithi118.stdout:259: read oid 2 snap -1 2024-03-20T08:02:12.008 INFO:tasks.rados.rados.0.smithi118.stdout:259: expect deleted 2024-03-20T08:02:12.008 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 23 2024-03-20T08:02:12.008 INFO:tasks.rados.rados.0.smithi118.stdout:260: seq_num 102 ranges {827392=516096,1343488=450560,1794048=729088,2523136=24576} 2024-03-20T08:02:12.013 INFO:tasks.rados.rados.0.smithi118.stdout:260: writing smithi11881747-50 from 827392 to 1343488 tid 1 2024-03-20T08:02:12.016 INFO:tasks.rados.rados.0.smithi118.stdout:260: writing smithi11881747-50 from 1343488 to 1794048 tid 2 2024-03-20T08:02:12.020 INFO:tasks.rados.rados.0.smithi118.stdout:260: writing smithi11881747-50 from 1794048 to 2523136 tid 3 2024-03-20T08:02:12.021 INFO:tasks.rados.rados.0.smithi118.stdout:260: writing smithi11881747-50 from 2523136 to 2547712 tid 4 2024-03-20T08:02:12.021 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 23 2024-03-20T08:02:12.021 INFO:tasks.rados.rados.0.smithi118.stdout:261: seq_num 103 ranges {5447680=532480,5980160=737280,6717440=466944} 2024-03-20T08:02:12.053 INFO:tasks.rados.rados.0.smithi118.stdout:261: writing smithi11881747-45 from 5447680 to 5980160 tid 1 2024-03-20T08:02:12.058 INFO:tasks.rados.rados.0.smithi118.stdout:261: writing smithi11881747-45 from 5980160 to 6717440 tid 2 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:261: writing smithi11881747-45 from 6717440 to 7184384 tid 3 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:262: rollback oid 13 current snap is 23 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 13 to 18 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:256: finishing rollback tid 0 to smithi11881747-43 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:256: finishing rollback tid 1 to smithi11881747-43 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:256: finishing rollback tid 2 to smithi11881747-43 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 109 (ObjNum 72 snap 10 seq_num 72) dirty exists 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:257: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:260: finishing write tid 1 to smithi11881747-50 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:260: finishing write tid 2 to smithi11881747-50 2024-03-20T08:02:12.062 INFO:tasks.rados.rados.0.smithi118.stdout:260: finishing write tid 3 to smithi11881747-50 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: finishing write tid 4 to smithi11881747-50 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: finishing write tid 5 to smithi11881747-50 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: finishing write tid 6 to smithi11881747-50 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: oid 50 updating version 0 to 39 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: oid 50 version 39 is already newer than 38 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: oid 50 updating version 39 to 40 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: oid 50 updating version 40 to 41 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: oid 50 updating version 41 to 42 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 42 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: left oid 50 (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:261: finishing write tid 1 to smithi11881747-45 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:261: finishing write tid 2 to smithi11881747-45 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:256: done (7 left) 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:259: done (6 left) 2024-03-20T08:02:12.063 INFO:tasks.rados.rados.0.smithi118.stdout:260: done (5 left) 2024-03-20T08:02:12.064 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 23 2024-03-20T08:02:12.064 INFO:tasks.rados.rados.0.smithi118.stdout:263: seq_num 104 ranges {0=548864,548864=622592,1171456=581632} 2024-03-20T08:02:12.065 INFO:tasks.rados.rados.0.smithi118.stdout:263: writing smithi11881747-14 from 0 to 548864 tid 1 2024-03-20T08:02:12.068 INFO:tasks.rados.rados.0.smithi118.stdout:263: writing smithi11881747-14 from 548864 to 1171456 tid 2 2024-03-20T08:02:12.072 INFO:tasks.rados.rados.0.smithi118.stdout:263: writing smithi11881747-14 from 1171456 to 1753088 tid 3 2024-03-20T08:02:12.072 INFO:tasks.rados.rados.0.smithi118.stdout:264: read oid 46 snap -1 2024-03-20T08:02:12.072 INFO:tasks.rados.rados.0.smithi118.stdout:264: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:02:12.072 INFO:tasks.rados.rados.0.smithi118.stdout:265: snap_remove snap 15 2024-03-20T08:02:13.046 INFO:tasks.rados.rados.0.smithi118.stdout:265: done (7 left) 2024-03-20T08:02:13.046 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 23 2024-03-20T08:02:13.046 INFO:tasks.rados.rados.0.smithi118.stdout:266: seq_num 105 ranges {2072576=565248,2637824=507904,3145728=507904,3653632=188416} 2024-03-20T08:02:13.061 INFO:tasks.rados.rados.0.smithi118.stdout:266: writing smithi11881747-49 from 2072576 to 2637824 tid 1 2024-03-20T08:02:13.064 INFO:tasks.rados.rados.0.smithi118.stdout:266: writing smithi11881747-49 from 2637824 to 3145728 tid 2 2024-03-20T08:02:13.067 INFO:tasks.rados.rados.0.smithi118.stdout:266: writing smithi11881747-49 from 3145728 to 3653632 tid 3 2024-03-20T08:02:13.068 INFO:tasks.rados.rados.0.smithi118.stdout:266: writing smithi11881747-49 from 3653632 to 3842048 tid 4 2024-03-20T08:02:13.068 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 23 2024-03-20T08:02:13.068 INFO:tasks.rados.rados.0.smithi118.stdout:261: finishing write tid 3 to smithi11881747-45 2024-03-20T08:02:13.069 INFO:tasks.rados.rados.0.smithi118.stdout:267: seq_num 106 ranges {1482752=581632,2064384=794624,2859008=409600} 2024-03-20T08:02:13.079 INFO:tasks.rados.rados.0.smithi118.stdout:267: writing smithi11881747-1 from 1482752 to 2064384 tid 1 2024-03-20T08:02:13.084 INFO:tasks.rados.rados.0.smithi118.stdout:267: writing smithi11881747-1 from 2064384 to 2859008 tid 2 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:267: writing smithi11881747-1 from 2859008 to 3268608 tid 3 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:268: rollback oid 24 current snap is 23 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 9 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:262: finishing rollback tid 0 to smithi11881747-13 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:262: finishing rollback tid 1 to smithi11881747-13 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:261: finishing write tid 4 to smithi11881747-45 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:262: finishing rollback tid 2 to smithi11881747-13 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 61 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-03-20T08:02:13.087 INFO:tasks.rados.rados.0.smithi118.stdout:261: finishing write tid 5 to smithi11881747-45 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:261: oid 45 updating version 0 to 64 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:261: oid 45 updating version 64 to 67 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:261: oid 45 version 67 is already newer than 66 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:261: oid 45 version 67 is already newer than 65 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 67 (ObjNum 103 snap 23 seq_num 103) dirty exists 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:261: left oid 45 (ObjNum 103 snap 23 seq_num 103) 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: finishing write tid 1 to smithi11881747-14 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: finishing write tid 2 to smithi11881747-14 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: finishing write tid 3 to smithi11881747-14 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: finishing write tid 4 to smithi11881747-14 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: finishing write tid 5 to smithi11881747-14 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: oid 14 updating version 0 to 52 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: oid 14 updating version 52 to 53 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: oid 14 version 53 is already newer than 51 2024-03-20T08:02:13.088 INFO:tasks.rados.rados.0.smithi118.stdout:263: oid 14 updating version 53 to 54 2024-03-20T08:02:13.089 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 54 (ObjNum 104 snap 23 seq_num 104) dirty exists 2024-03-20T08:02:13.089 INFO:tasks.rados.rados.0.smithi118.stdout:263: left oid 14 (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:02:13.089 INFO:tasks.rados.rados.0.smithi118.stdout:261: done (9 left) 2024-03-20T08:02:13.089 INFO:tasks.rados.rados.0.smithi118.stdout:262: done (8 left) 2024-03-20T08:02:13.089 INFO:tasks.rados.rados.0.smithi118.stdout:263: done (7 left) 2024-03-20T08:02:13.089 INFO:tasks.rados.rados.0.smithi118.stdout:269: rollback oid 11 current snap is 23 2024-03-20T08:02:13.089 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 14 2024-03-20T08:02:13.089 INFO:tasks.rados.rados.0.smithi118.stdout:264: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:02:13.108 INFO:tasks.rados.rados.0.smithi118.stdout:257: finishing copy_from to smithi11881747-36 2024-03-20T08:02:13.108 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 20 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-03-20T08:02:13.108 INFO:tasks.rados.rados.0.smithi118.stdout:254: expect (ObjNum 38 snap 0 seq_num 38) 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:258: finishing copy_from racing read to smithi11881747-37 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:258: finishing copy_from to smithi11881747-37 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 4 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:266: finishing write tid 1 to smithi11881747-49 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:266: finishing write tid 2 to smithi11881747-49 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:266: finishing write tid 3 to smithi11881747-49 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:266: finishing write tid 4 to smithi11881747-49 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:266: finishing write tid 5 to smithi11881747-49 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:266: finishing write tid 6 to smithi11881747-49 2024-03-20T08:02:13.126 INFO:tasks.rados.rados.0.smithi118.stdout:266: oid 49 updating version 0 to 77 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:266: oid 49 updating version 77 to 80 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:266: oid 49 version 80 is already newer than 78 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:266: oid 49 updating version 80 to 81 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:266: oid 49 version 81 is already newer than 79 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 81 (ObjNum 105 snap 23 seq_num 105) dirty exists 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:266: left oid 49 (ObjNum 105 snap 23 seq_num 105) 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:267: finishing write tid 1 to smithi11881747-1 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:269: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 38 (ObjNum 33 snap 0 seq_num 291747632) dirty dne 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:268: finishing rollback tid 0 to smithi11881747-24 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:267: finishing write tid 2 to smithi11881747-1 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:254: done (7 left) 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:257: done (6 left) 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:258: done (5 left) 2024-03-20T08:02:13.127 INFO:tasks.rados.rados.0.smithi118.stdout:264: done (4 left) 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:266: done (3 left) 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:269: done (2 left) 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:270: rmattr oid 50 current snap is 23 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:267: finishing write tid 3 to smithi11881747-1 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:268: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:267: finishing write tid 4 to smithi11881747-1 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:268: finishing rollback tid 2 to smithi11881747-24 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 50 (ObjNum 54 snap 4 seq_num 54) dirty exists 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:267: finishing write tid 5 to smithi11881747-1 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:267: oid 1 updating version 0 to 120 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:267: oid 1 version 120 is already newer than 119 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:267: oid 1 version 120 is already newer than 118 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:267: oid 1 version 120 is already newer than 117 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 120 (ObjNum 106 snap 23 seq_num 106) dirty exists 2024-03-20T08:02:13.128 INFO:tasks.rados.rados.0.smithi118.stdout:267: left oid 1 (ObjNum 106 snap 23 seq_num 106) 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:267: done (2 left) 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:268: done (1 left) 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:271: rollback oid 47 current snap is 23 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 18 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:272: rmattr oid 7 current snap is 23 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:272: done (2 left) 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:273: copy_from oid 34 from oid 27 current snap is 23 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:274: rollback oid 38 current snap is 23 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 38 to 14 2024-03-20T08:02:13.129 INFO:tasks.rados.rados.0.smithi118.stdout:275: snap_remove snap 16 2024-03-20T08:02:13.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:13 smithi196 ceph-mon[31033]: pgmap v147: 121 pgs: 121 active+clean; 62 MiB data, 325 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 29 op/s 2024-03-20T08:02:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:13 smithi196 ceph-mon[31033]: osdmap e115: 8 total, 8 up, 8 in 2024-03-20T08:02:13.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:13 smithi118 ceph-mon[27391]: pgmap v147: 121 pgs: 121 active+clean; 62 MiB data, 325 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 29 op/s 2024-03-20T08:02:13.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:13 smithi118 ceph-mon[27391]: osdmap e115: 8 total, 8 up, 8 in 2024-03-20T08:02:13.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:13 smithi118 ceph-mon[38981]: pgmap v147: 121 pgs: 121 active+clean; 62 MiB data, 325 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 29 op/s 2024-03-20T08:02:13.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:13 smithi118 ceph-mon[38981]: osdmap e115: 8 total, 8 up, 8 in 2024-03-20T08:02:14.051 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 51 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:02:14.051 INFO:tasks.rados.rados.0.smithi118.stdout:274: finishing rollback tid 0 to smithi11881747-38 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:274: finishing rollback tid 1 to smithi11881747-38 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:274: finishing rollback tid 2 to smithi11881747-38 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 83 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:271: finishing rollback tid 0 to smithi11881747-47 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:271: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:271: finishing rollback tid 2 to smithi11881747-47 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 113 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:273: finishing copy_from racing read to smithi11881747-34 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:273: finishing copy_from to smithi11881747-34 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 4 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:270: done (4 left) 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:271: done (3 left) 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:273: done (2 left) 2024-03-20T08:02:14.052 INFO:tasks.rados.rados.0.smithi118.stdout:274: done (1 left) 2024-03-20T08:02:14.053 INFO:tasks.rados.rados.0.smithi118.stdout:275: done (0 left) 2024-03-20T08:02:14.053 INFO:tasks.rados.rados.0.smithi118.stdout:276: snap_create 2024-03-20T08:02:14.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:14 smithi196 ceph-mon[31033]: osdmap e116: 8 total, 8 up, 8 in 2024-03-20T08:02:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:14 smithi118 ceph-mon[27391]: osdmap e116: 8 total, 8 up, 8 in 2024-03-20T08:02:14.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:14 smithi118 ceph-mon[38981]: osdmap e116: 8 total, 8 up, 8 in 2024-03-20T08:02:15.051 INFO:tasks.rados.rados.0.smithi118.stdout:276: done (0 left) 2024-03-20T08:02:15.052 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 24 2024-03-20T08:02:15.052 INFO:tasks.rados.rados.0.smithi118.stdout:277: seq_num 107 ranges {2940928=598016,3538944=729088,4268032=466944,4734976=8192} 2024-03-20T08:02:15.070 INFO:tasks.rados.rados.0.smithi118.stdout:277: writing smithi11881747-16 from 2940928 to 3538944 tid 1 2024-03-20T08:02:15.075 INFO:tasks.rados.rados.0.smithi118.stdout:277: writing smithi11881747-16 from 3538944 to 4268032 tid 2 2024-03-20T08:02:15.078 INFO:tasks.rados.rados.0.smithi118.stdout:277: writing smithi11881747-16 from 4268032 to 4734976 tid 3 2024-03-20T08:02:15.078 INFO:tasks.rados.rados.0.smithi118.stdout:277: writing smithi11881747-16 from 4734976 to 4743168 tid 4 2024-03-20T08:02:15.078 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 24 2024-03-20T08:02:15.078 INFO:tasks.rados.rados.0.smithi118.stdout:277: finishing write tid 1 to smithi11881747-16 2024-03-20T08:02:15.078 INFO:tasks.rados.rados.0.smithi118.stdout:277: finishing write tid 2 to smithi11881747-16 2024-03-20T08:02:15.078 INFO:tasks.rados.rados.0.smithi118.stdout:278: seq_num 108 ranges {2678784=614400,3293184=606208,3899392=598016} 2024-03-20T08:02:15.096 INFO:tasks.rados.rados.0.smithi118.stdout:278: writing smithi11881747-18 from 2678784 to 3293184 tid 1 2024-03-20T08:02:15.099 INFO:tasks.rados.rados.0.smithi118.stdout:278: writing smithi11881747-18 from 3293184 to 3899392 tid 2 2024-03-20T08:02:15.103 INFO:tasks.rados.rados.0.smithi118.stdout:278: writing smithi11881747-18 from 3899392 to 4497408 tid 3 2024-03-20T08:02:15.103 INFO:tasks.rados.rados.0.smithi118.stdout:279: setattr oid 45 current snap is 24 2024-03-20T08:02:15.103 INFO:tasks.rados.rados.0.smithi118.stdout:277: finishing write tid 3 to smithi11881747-16 2024-03-20T08:02:15.103 INFO:tasks.rados.rados.0.smithi118.stdout:277: finishing write tid 4 to smithi11881747-16 2024-03-20T08:02:15.103 INFO:tasks.rados.rados.0.smithi118.stdout:277: finishing write tid 5 to smithi11881747-16 2024-03-20T08:02:15.103 INFO:tasks.rados.rados.0.smithi118.stdout:277: finishing write tid 6 to smithi11881747-16 2024-03-20T08:02:15.103 INFO:tasks.rados.rados.0.smithi118.stdout:277: oid 16 updating version 0 to 122 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:277: oid 16 version 122 is already newer than 121 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:277: oid 16 version 122 is already newer than 119 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:277: oid 16 version 122 is already newer than 120 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:277: oid 16 updating version 122 to 123 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 123 (ObjNum 107 snap 24 seq_num 107) dirty exists 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:277: left oid 16 (ObjNum 107 snap 24 seq_num 107) 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:278: finishing write tid 1 to smithi11881747-18 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:278: finishing write tid 2 to smithi11881747-18 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:277: done (2 left) 2024-03-20T08:02:15.104 INFO:tasks.rados.rados.0.smithi118.stdout:280: snap_remove snap 9 2024-03-20T08:02:15.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:15 smithi196 ceph-mon[31033]: pgmap v150: 121 pgs: 121 active+clean; 62 MiB data, 325 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 29 op/s 2024-03-20T08:02:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:15 smithi196 ceph-mon[31033]: osdmap e117: 8 total, 8 up, 8 in 2024-03-20T08:02:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:15 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:15 smithi118 ceph-mon[27391]: pgmap v150: 121 pgs: 121 active+clean; 62 MiB data, 325 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 29 op/s 2024-03-20T08:02:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:15 smithi118 ceph-mon[27391]: osdmap e117: 8 total, 8 up, 8 in 2024-03-20T08:02:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:15 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:15 smithi118 ceph-mon[38981]: pgmap v150: 121 pgs: 121 active+clean; 62 MiB data, 325 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 29 op/s 2024-03-20T08:02:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:15 smithi118 ceph-mon[38981]: osdmap e117: 8 total, 8 up, 8 in 2024-03-20T08:02:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:15 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:16.008 INFO:tasks.rados.rados.0.smithi118.stdout:278: finishing write tid 3 to smithi11881747-18 2024-03-20T08:02:16.008 INFO:tasks.rados.rados.0.smithi118.stdout:278: finishing write tid 4 to smithi11881747-18 2024-03-20T08:02:16.008 INFO:tasks.rados.rados.0.smithi118.stdout:278: finishing write tid 5 to smithi11881747-18 2024-03-20T08:02:16.008 INFO:tasks.rados.rados.0.smithi118.stdout:278: oid 18 updating version 0 to 91 2024-03-20T08:02:16.008 INFO:tasks.rados.rados.0.smithi118.stdout:278: oid 18 version 91 is already newer than 90 2024-03-20T08:02:16.008 INFO:tasks.rados.rados.0.smithi118.stdout:278: oid 18 updating version 91 to 92 2024-03-20T08:02:16.008 INFO:tasks.rados.rados.0.smithi118.stdout:278: oid 18 version 92 is already newer than 89 2024-03-20T08:02:16.009 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 92 (ObjNum 108 snap 24 seq_num 108) dirty exists 2024-03-20T08:02:16.009 INFO:tasks.rados.rados.0.smithi118.stdout:278: left oid 18 (ObjNum 108 snap 24 seq_num 108) 2024-03-20T08:02:16.009 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 83 (ObjNum 103 snap 23 seq_num 103) dirty exists 2024-03-20T08:02:16.009 INFO:tasks.rados.rados.0.smithi118.stdout:278: done (2 left) 2024-03-20T08:02:16.009 INFO:tasks.rados.rados.0.smithi118.stdout:279: done (1 left) 2024-03-20T08:02:16.009 INFO:tasks.rados.rados.0.smithi118.stdout:280: done (0 left) 2024-03-20T08:02:16.009 INFO:tasks.rados.rados.0.smithi118.stdout:281: setattr oid 40 current snap is 24 2024-03-20T08:02:16.010 INFO:tasks.rados.rados.0.smithi118.stdout:282: snap_create 2024-03-20T08:02:16.021 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 84 (ObjNum 129 snap 0 seq_num 322167064) dirty exists 2024-03-20T08:02:16.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:16 smithi196 ceph-mon[31033]: osdmap e118: 8 total, 8 up, 8 in 2024-03-20T08:02:16.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:16 smithi196 ceph-mon[31033]: osdmap e119: 8 total, 8 up, 8 in 2024-03-20T08:02:16.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:16 smithi118 ceph-mon[38981]: osdmap e118: 8 total, 8 up, 8 in 2024-03-20T08:02:16.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:16 smithi118 ceph-mon[38981]: osdmap e119: 8 total, 8 up, 8 in 2024-03-20T08:02:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:16 smithi118 ceph-mon[27391]: osdmap e118: 8 total, 8 up, 8 in 2024-03-20T08:02:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:16 smithi118 ceph-mon[27391]: osdmap e119: 8 total, 8 up, 8 in 2024-03-20T08:02:17.014 INFO:tasks.rados.rados.0.smithi118.stdout:281: done (1 left) 2024-03-20T08:02:17.014 INFO:tasks.rados.rados.0.smithi118.stdout:282: done (0 left) 2024-03-20T08:02:17.014 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 25 2024-03-20T08:02:17.014 INFO:tasks.rados.rados.0.smithi118.stdout:283: seq_num 109 ranges {253952=630784,884736=491520,1376256=483328,1859584=229376} 2024-03-20T08:02:17.018 INFO:tasks.rados.rados.0.smithi118.stdout:283: writing smithi11881747-33 from 253952 to 884736 tid 1 2024-03-20T08:02:17.021 INFO:tasks.rados.rados.0.smithi118.stdout:283: writing smithi11881747-33 from 884736 to 1376256 tid 2 2024-03-20T08:02:17.025 INFO:tasks.rados.rados.0.smithi118.stdout:283: writing smithi11881747-33 from 1376256 to 1859584 tid 3 2024-03-20T08:02:17.026 INFO:tasks.rados.rados.0.smithi118.stdout:283: writing smithi11881747-33 from 1859584 to 2088960 tid 4 2024-03-20T08:02:17.026 INFO:tasks.rados.rados.0.smithi118.stdout:284: copy_from oid 18 from oid 13 current snap is 25 2024-03-20T08:02:17.026 INFO:tasks.rados.rados.0.smithi118.stdout:283: finishing write tid 1 to smithi11881747-33 2024-03-20T08:02:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:283: finishing write tid 2 to smithi11881747-33 2024-03-20T08:02:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:285: snap_create 2024-03-20T08:02:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:283: finishing write tid 3 to smithi11881747-33 2024-03-20T08:02:17.030 INFO:tasks.rados.rados.0.smithi118.stdout:283: finishing write tid 4 to smithi11881747-33 2024-03-20T08:02:17.030 INFO:tasks.rados.rados.0.smithi118.stdout:283: finishing write tid 5 to smithi11881747-33 2024-03-20T08:02:17.031 INFO:tasks.rados.rados.0.smithi118.stdout:283: finishing write tid 6 to smithi11881747-33 2024-03-20T08:02:17.031 INFO:tasks.rados.rados.0.smithi118.stdout:283: oid 33 updating version 0 to 47 2024-03-20T08:02:17.031 INFO:tasks.rados.rados.0.smithi118.stdout:283: oid 33 version 47 is already newer than 46 2024-03-20T08:02:17.031 INFO:tasks.rados.rados.0.smithi118.stdout:283: oid 33 version 47 is already newer than 45 2024-03-20T08:02:17.031 INFO:tasks.rados.rados.0.smithi118.stdout:283: oid 33 version 47 is already newer than 44 2024-03-20T08:02:17.031 INFO:tasks.rados.rados.0.smithi118.stdout:283: oid 33 version 47 is already newer than 43 2024-03-20T08:02:17.032 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 47 (ObjNum 109 snap 25 seq_num 109) dirty exists 2024-03-20T08:02:17.032 INFO:tasks.rados.rados.0.smithi118.stdout:283: left oid 33 (ObjNum 109 snap 25 seq_num 109) 2024-03-20T08:02:17.035 INFO:tasks.rados.rados.0.smithi118.stdout:284: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:02:17.035 INFO:tasks.rados.rados.0.smithi118.stdout:284: finishing copy_from to smithi11881747-18 2024-03-20T08:02:17.035 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 61 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-03-20T08:02:17.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:17 smithi196 ceph-mon[31033]: pgmap v153: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-03-20T08:02:17.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:17 smithi196 ceph-mon[31033]: osdmap e120: 8 total, 8 up, 8 in 2024-03-20T08:02:17.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:17 smithi118 ceph-mon[27391]: pgmap v153: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-03-20T08:02:17.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:17 smithi118 ceph-mon[27391]: osdmap e120: 8 total, 8 up, 8 in 2024-03-20T08:02:17.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:17 smithi118 ceph-mon[38981]: pgmap v153: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-03-20T08:02:17.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:17 smithi118 ceph-mon[38981]: osdmap e120: 8 total, 8 up, 8 in 2024-03-20T08:02:18.016 INFO:tasks.rados.rados.0.smithi118.stdout:283: done (2 left) 2024-03-20T08:02:18.016 INFO:tasks.rados.rados.0.smithi118.stdout:284: done (1 left) 2024-03-20T08:02:18.016 INFO:tasks.rados.rados.0.smithi118.stdout:285: done (0 left) 2024-03-20T08:02:18.016 INFO:tasks.rados.rados.0.smithi118.stdout:286: snap_create 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:286: done (0 left) 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:287: read oid 15 snap -1 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:287: expect (ObjNum 59 snap 5 seq_num 59) 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:288: read oid 1 snap -1 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:288: expect (ObjNum 106 snap 23 seq_num 106) 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:289: read oid 10 snap -1 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:289: expect (ObjNum 85 snap 15 seq_num 85) 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:290: copy_from oid 23 from oid 47 current snap is 27 2024-03-20T08:02:19.018 INFO:tasks.rados.rados.0.smithi118.stdout:291: read oid 5 snap -1 2024-03-20T08:02:19.019 INFO:tasks.rados.rados.0.smithi118.stdout:291: expect (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:02:19.019 INFO:tasks.rados.rados.0.smithi118.stdout:292: read oid 19 snap -1 2024-03-20T08:02:19.019 INFO:tasks.rados.rados.0.smithi118.stdout:292: expect deleted 2024-03-20T08:02:19.019 INFO:tasks.rados.rados.0.smithi118.stdout:293: snap_remove snap 24 2024-03-20T08:02:19.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:19 smithi196 ceph-mon[31033]: pgmap v156: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-03-20T08:02:19.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:19 smithi196 ceph-mon[31033]: osdmap e121: 8 total, 8 up, 8 in 2024-03-20T08:02:19.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:19 smithi118 ceph-mon[27391]: pgmap v156: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-03-20T08:02:19.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:19 smithi118 ceph-mon[27391]: osdmap e121: 8 total, 8 up, 8 in 2024-03-20T08:02:19.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:19 smithi118 ceph-mon[38981]: pgmap v156: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 23 op/s 2024-03-20T08:02:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:19 smithi118 ceph-mon[38981]: osdmap e121: 8 total, 8 up, 8 in 2024-03-20T08:02:20.020 INFO:tasks.rados.rados.0.smithi118.stdout:293: done (6 left) 2024-03-20T08:02:20.021 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 27 2024-03-20T08:02:20.021 INFO:tasks.rados.rados.0.smithi118.stdout:294: seq_num 110 ranges {4743168=647168,5390336=778240,6168576=425984} 2024-03-20T08:02:20.049 INFO:tasks.rados.rados.0.smithi118.stdout:294: writing smithi11881747-16 from 4743168 to 5390336 tid 1 2024-03-20T08:02:20.054 INFO:tasks.rados.rados.0.smithi118.stdout:294: writing smithi11881747-16 from 5390336 to 6168576 tid 2 2024-03-20T08:02:20.057 INFO:tasks.rados.rados.0.smithi118.stdout:294: writing smithi11881747-16 from 6168576 to 6594560 tid 3 2024-03-20T08:02:20.057 INFO:tasks.rados.rados.0.smithi118.stdout:295: rollback oid 4 current snap is 27 2024-03-20T08:02:20.057 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 4 to 14 2024-03-20T08:02:20.057 INFO:tasks.rados.rados.0.smithi118.stdout:291: expect (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:02:20.059 INFO:tasks.rados.rados.0.smithi118.stdout:290: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:02:20.059 INFO:tasks.rados.rados.0.smithi118.stdout:290: finishing copy_from to smithi11881747-23 2024-03-20T08:02:20.059 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 113 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:02:20.059 INFO:tasks.rados.rados.0.smithi118.stdout:287: expect (ObjNum 59 snap 5 seq_num 59) 2024-03-20T08:02:20.087 INFO:tasks.rados.rados.0.smithi118.stdout:287: done (7 left) 2024-03-20T08:02:20.087 INFO:tasks.rados.rados.0.smithi118.stdout:290: done (6 left) 2024-03-20T08:02:20.087 INFO:tasks.rados.rados.0.smithi118.stdout:291: done (5 left) 2024-03-20T08:02:20.087 INFO:tasks.rados.rados.0.smithi118.stdout:292: done (4 left) 2024-03-20T08:02:20.087 INFO:tasks.rados.rados.0.smithi118.stdout:296: snap_create 2024-03-20T08:02:20.087 INFO:tasks.rados.rados.0.smithi118.stdout:288: expect (ObjNum 106 snap 23 seq_num 106) 2024-03-20T08:02:20.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:20 smithi196 ceph-mon[31033]: osdmap e122: 8 total, 8 up, 8 in 2024-03-20T08:02:20.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:20.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:20 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:02:20.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:20 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:02:20.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:20 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:02:20.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:20 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:20.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:20 smithi196 ceph-mon[31033]: osdmap e123: 8 total, 8 up, 8 in 2024-03-20T08:02:20.193 INFO:tasks.rados.rados.0.smithi118.stdout:289: expect (ObjNum 85 snap 15 seq_num 85) 2024-03-20T08:02:20.259 INFO:tasks.rados.rados.0.smithi118.stdout:294: finishing write tid 1 to smithi11881747-16 2024-03-20T08:02:20.259 INFO:tasks.rados.rados.0.smithi118.stdout:294: finishing write tid 2 to smithi11881747-16 2024-03-20T08:02:20.259 INFO:tasks.rados.rados.0.smithi118.stdout:295: finishing rollback tid 1 to smithi11881747-4 2024-03-20T08:02:20.259 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 0 (ObjNum 129 snap 0 seq_num 1267910735) dirty dne 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:294: finishing write tid 3 to smithi11881747-16 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:294: finishing write tid 4 to smithi11881747-16 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:294: finishing write tid 5 to smithi11881747-16 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:294: oid 16 updating version 0 to 137 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:294: oid 16 version 137 is already newer than 134 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:294: oid 16 version 137 is already newer than 136 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:294: oid 16 version 137 is already newer than 135 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 137 (ObjNum 110 snap 27 seq_num 110) dirty exists 2024-03-20T08:02:20.260 INFO:tasks.rados.rados.0.smithi118.stdout:294: left oid 16 (ObjNum 110 snap 27 seq_num 110) 2024-03-20T08:02:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[27391]: osdmap e122: 8 total, 8 up, 8 in 2024-03-20T08:02:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:02:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:02:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[27391]: osdmap e123: 8 total, 8 up, 8 in 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[38981]: osdmap e122: 8 total, 8 up, 8 in 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:20 smithi118 ceph-mon[38981]: osdmap e123: 8 total, 8 up, 8 in 2024-03-20T08:02:21.027 INFO:tasks.rados.rados.0.smithi118.stdout:288: done (4 left) 2024-03-20T08:02:21.027 INFO:tasks.rados.rados.0.smithi118.stdout:289: done (3 left) 2024-03-20T08:02:21.027 INFO:tasks.rados.rados.0.smithi118.stdout:294: done (2 left) 2024-03-20T08:02:21.027 INFO:tasks.rados.rados.0.smithi118.stdout:295: done (1 left) 2024-03-20T08:02:21.027 INFO:tasks.rados.rados.0.smithi118.stdout:296: done (0 left) 2024-03-20T08:02:21.027 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 28 2024-03-20T08:02:21.027 INFO:tasks.rados.rados.0.smithi118.stdout:297: seq_num 111 ranges {204800=663552,868352=663552,1531904=450560,1982464=90112} 2024-03-20T08:02:21.031 INFO:tasks.rados.rados.0.smithi118.stdout:297: writing smithi11881747-18 from 204800 to 868352 tid 1 2024-03-20T08:02:21.035 INFO:tasks.rados.rados.0.smithi118.stdout:297: writing smithi11881747-18 from 868352 to 1531904 tid 2 2024-03-20T08:02:21.038 INFO:tasks.rados.rados.0.smithi118.stdout:297: writing smithi11881747-18 from 1531904 to 1982464 tid 3 2024-03-20T08:02:21.039 INFO:tasks.rados.rados.0.smithi118.stdout:297: writing smithi11881747-18 from 1982464 to 2072576 tid 4 2024-03-20T08:02:21.039 INFO:tasks.rados.rados.0.smithi118.stdout:298: read oid 31 snap -1 2024-03-20T08:02:21.039 INFO:tasks.rados.rados.0.smithi118.stdout:298: expect (ObjNum 87 snap 17 seq_num 87) 2024-03-20T08:02:21.039 INFO:tasks.rados.rados.0.smithi118.stdout:299: read oid 5 snap -1 2024-03-20T08:02:21.039 INFO:tasks.rados.rados.0.smithi118.stdout:299: expect (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:02:21.039 INFO:tasks.rados.rados.0.smithi118.stdout:300: copy_from oid 21 from oid 32 current snap is 28 2024-03-20T08:02:21.039 INFO:tasks.rados.rados.0.smithi118.stdout:297: finishing write tid 1 to smithi11881747-18 2024-03-20T08:02:21.040 INFO:tasks.rados.rados.0.smithi118.stdout:297: finishing write tid 2 to smithi11881747-18 2024-03-20T08:02:21.040 INFO:tasks.rados.rados.0.smithi118.stdout:301: snap_create 2024-03-20T08:02:21.042 INFO:tasks.rados.rados.0.smithi118.stdout:297: finishing write tid 3 to smithi11881747-18 2024-03-20T08:02:21.042 INFO:tasks.rados.rados.0.smithi118.stdout:297: finishing write tid 4 to smithi11881747-18 2024-03-20T08:02:21.043 INFO:tasks.rados.rados.0.smithi118.stdout:297: finishing write tid 5 to smithi11881747-18 2024-03-20T08:02:21.044 INFO:tasks.rados.rados.0.smithi118.stdout:297: finishing write tid 6 to smithi11881747-18 2024-03-20T08:02:21.044 INFO:tasks.rados.rados.0.smithi118.stdout:297: oid 18 updating version 0 to 108 2024-03-20T08:02:21.044 INFO:tasks.rados.rados.0.smithi118.stdout:297: oid 18 version 108 is already newer than 107 2024-03-20T08:02:21.044 INFO:tasks.rados.rados.0.smithi118.stdout:297: oid 18 version 108 is already newer than 105 2024-03-20T08:02:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:297: oid 18 version 108 is already newer than 104 2024-03-20T08:02:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:297: oid 18 version 108 is already newer than 106 2024-03-20T08:02:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 108 (ObjNum 111 snap 28 seq_num 111) dirty exists 2024-03-20T08:02:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:297: left oid 18 (ObjNum 111 snap 28 seq_num 111) 2024-03-20T08:02:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:299: expect (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:02:21.049 INFO:tasks.rados.rados.0.smithi118.stdout:300: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:02:21.050 INFO:tasks.rados.rados.0.smithi118.stdout:300: finishing copy_from to smithi11881747-21 2024-03-20T08:02:21.050 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-03-20T08:02:21.071 INFO:tasks.rados.rados.0.smithi118.stdout:298: expect (ObjNum 87 snap 17 seq_num 87) 2024-03-20T08:02:21.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:21 smithi196 ceph-mon[31033]: Upgrade: Updating mgr.x 2024-03-20T08:02:21.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:21 smithi196 ceph-mon[31033]: Deploying daemon mgr.x on smithi196 2024-03-20T08:02:21.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:21 smithi196 ceph-mon[31033]: pgmap v159: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:21.308 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:21 smithi196 systemd[1]: Stopping Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:02:21.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:21 smithi118 ceph-mon[27391]: Upgrade: Updating mgr.x 2024-03-20T08:02:21.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:21 smithi118 ceph-mon[27391]: Deploying daemon mgr.x on smithi196 2024-03-20T08:02:21.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:21 smithi118 ceph-mon[27391]: pgmap v159: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:21.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:21 smithi118 ceph-mon[38981]: Upgrade: Updating mgr.x 2024-03-20T08:02:21.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:21 smithi118 ceph-mon[38981]: Deploying daemon mgr.x on smithi196 2024-03-20T08:02:21.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:21 smithi118 ceph-mon[38981]: pgmap v159: 121 pgs: 121 active+clean; 75 MiB data, 346 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:21.583 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:21 smithi196 podman[66133]: 2024-03-20 08:02:21.308476708 +0000 UTC m=+0.144787876 container died ef306191c58d106e82e4862eb9524ff40994b0cceb09e5da29b7ae5aa64c5f0e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-78dbc53, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:02:21.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:21 smithi196 podman[66133]: 2024-03-20 08:02:21.741440492 +0000 UTC m=+0.577751702 container cleanup ef306191c58d106e82e4862eb9524ff40994b0cceb09e5da29b7ae5aa64c5f0e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=reef-78dbc53, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-20T08:02:21.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:21 smithi196 bash[66133]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x 2024-03-20T08:02:21.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:02:21] "GET /metrics HTTP/1.1" 200 36069 "" "Prometheus/2.43.0" 2024-03-20T08:02:22.038 INFO:tasks.rados.rados.0.smithi118.stdout:297: done (4 left) 2024-03-20T08:02:22.038 INFO:tasks.rados.rados.0.smithi118.stdout:298: done (3 left) 2024-03-20T08:02:22.038 INFO:tasks.rados.rados.0.smithi118.stdout:299: done (2 left) 2024-03-20T08:02:22.038 INFO:tasks.rados.rados.0.smithi118.stdout:300: done (1 left) 2024-03-20T08:02:22.038 INFO:tasks.rados.rados.0.smithi118.stdout:301: done (0 left) 2024-03-20T08:02:22.038 INFO:tasks.rados.rados.0.smithi118.stdout:302: read oid 41 snap -1 2024-03-20T08:02:22.038 INFO:tasks.rados.rados.0.smithi118.stdout:302: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:22.038 INFO:tasks.rados.rados.0.smithi118.stdout:303: delete oid 13 current snap is 29 2024-03-20T08:02:22.041 INFO:tasks.rados.rados.0.smithi118.stdout:302: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:22.060 INFO:tasks.rados.rados.0.smithi118.stdout:302: done (1 left) 2024-03-20T08:02:22.061 INFO:tasks.rados.rados.0.smithi118.stdout:303: done (0 left) 2024-03-20T08:02:22.061 INFO:tasks.rados.rados.0.smithi118.stdout:304: snap_create 2024-03-20T08:02:22.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:22 smithi196 ceph-mon[31033]: osdmap e124: 8 total, 8 up, 8 in 2024-03-20T08:02:22.356 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:21 smithi196 podman[66152]: 2024-03-20 08:02:21.926146853 +0000 UTC m=+0.622195395 container remove ef306191c58d106e82e4862eb9524ff40994b0cceb09e5da29b7ae5aa64c5f0e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=reef-78dbc53, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-20T08:02:22.356 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:21 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:02:22.356 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-03-20T08:02:22.357 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 systemd[1]: Stopped Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:02:22.357 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x.service: Consumed 18.289s CPU time. 2024-03-20T08:02:22.357 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 systemd[1]: Starting Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:02:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:22 smithi118 ceph-mon[27391]: osdmap e124: 8 total, 8 up, 8 in 2024-03-20T08:02:22.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:22 smithi118 ceph-mon[38981]: osdmap e124: 8 total, 8 up, 8 in 2024-03-20T08:02:22.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 podman[66285]: 2024-03-20 08:02:22.475160256 +0000 UTC m=+0.018208342 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:02:22.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 podman[66285]: 2024-03-20 08:02:22.586010289 +0000 UTC m=+0.129058353 container create 838b6245d898211f970d446b8339ab52a6c2517e9c9ee51a5fdc8741e0bedb8d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:02:23.038 INFO:tasks.rados.rados.0.smithi118.stdout:304: done (0 left) 2024-03-20T08:02:23.038 INFO:tasks.rados.rados.0.smithi118.stdout:305: delete oid 19 current snap is 30 2024-03-20T08:02:23.042 INFO:tasks.rados.rados.0.smithi118.stdout:305: done (0 left) 2024-03-20T08:02:23.042 INFO:tasks.rados.rados.0.smithi118.stdout:306: delete oid 8 current snap is 30 2024-03-20T08:02:23.044 INFO:tasks.rados.rados.0.smithi118.stdout:306: done (0 left) 2024-03-20T08:02:23.044 INFO:tasks.rados.rados.0.smithi118.stdout:307: delete oid 49 current snap is 30 2024-03-20T08:02:23.048 INFO:tasks.rados.rados.0.smithi118.stdout:307: done (0 left) 2024-03-20T08:02:23.048 INFO:tasks.rados.rados.0.smithi118.stdout:308: copy_from oid 33 from oid 9 current snap is 30 2024-03-20T08:02:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:309: read oid 27 snap -1 2024-03-20T08:02:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:309: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:02:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:310: rollback oid 42 current snap is 30 2024-03-20T08:02:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 14 2024-03-20T08:02:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:311: read oid 29 snap -1 2024-03-20T08:02:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:311: expect deleted 2024-03-20T08:02:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:312: delete oid 19 current snap is 30 2024-03-20T08:02:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:311: done (4 left) 2024-03-20T08:02:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:312: done (3 left) 2024-03-20T08:02:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:313: rollback oid 37 current snap is 30 2024-03-20T08:02:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 27 2024-03-20T08:02:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:314: rollback oid 29 current snap is 30 2024-03-20T08:02:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 28 2024-03-20T08:02:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 30 2024-03-20T08:02:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:315: seq_num 117 ranges {1417216=770048,2187264=417792,2605056=442368,3047424=344064} 2024-03-20T08:02:23.063 INFO:tasks.rados.rados.0.smithi118.stdout:315: writing smithi11881747-22 from 1417216 to 2187264 tid 1 2024-03-20T08:02:23.066 INFO:tasks.rados.rados.0.smithi118.stdout:315: writing smithi11881747-22 from 2187264 to 2605056 tid 2 2024-03-20T08:02:23.068 INFO:tasks.rados.rados.0.smithi118.stdout:315: writing smithi11881747-22 from 2605056 to 3047424 tid 3 2024-03-20T08:02:23.071 INFO:tasks.rados.rados.0.smithi118.stdout:315: writing smithi11881747-22 from 3047424 to 3391488 tid 4 2024-03-20T08:02:23.071 INFO:tasks.rados.rados.0.smithi118.stdout:316: read oid 48 snap -1 2024-03-20T08:02:23.071 INFO:tasks.rados.rados.0.smithi118.stdout:316: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:23.071 INFO:tasks.rados.rados.0.smithi118.stdout:317: copy_from oid 49 from oid 23 current snap is 30 2024-03-20T08:02:23.071 INFO:tasks.rados.rados.0.smithi118.stdout:310: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:02:23.072 INFO:tasks.rados.rados.0.smithi118.stdout:309: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:02:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:314: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:02:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 0 (ObjNum 33 snap 0 seq_num 291742968) dirty dne 2024-03-20T08:02:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:310: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:02:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:313: finishing rollback tid 0 to smithi11881747-37 2024-03-20T08:02:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:313: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:02:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:309: done (7 left) 2024-03-20T08:02:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:314: done (6 left) 2024-03-20T08:02:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:318: rollback oid 16 current snap is 30 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 27 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:313: finishing rollback tid 2 to smithi11881747-37 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 94 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:310: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 82 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:308: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:308: finishing copy_from to smithi11881747-33 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 49 (ObjNum 89 snap 18 seq_num 89) dirty exists 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:315: finishing write tid 1 to smithi11881747-22 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:315: finishing write tid 2 to smithi11881747-22 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:315: finishing write tid 3 to smithi11881747-22 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:315: finishing write tid 4 to smithi11881747-22 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:315: finishing write tid 5 to smithi11881747-22 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:317: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:02:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:317: finishing copy_from to smithi11881747-49 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 113 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:308: done (6 left) 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:310: done (5 left) 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:313: done (4 left) 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:317: done (3 left) 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:319: read oid 6 snap -1 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:319: expect (ObjNum 36 snap 0 seq_num 36) 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:320: snap_create 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:315: finishing write tid 6 to smithi11881747-22 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:315: oid 22 updating version 0 to 101 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:315: oid 22 version 101 is already newer than 99 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:315: oid 22 version 101 is already newer than 98 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:315: oid 22 version 101 is already newer than 100 2024-03-20T08:02:23.087 INFO:tasks.rados.rados.0.smithi118.stdout:315: oid 22 version 101 is already newer than 97 2024-03-20T08:02:23.088 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:02:23.088 INFO:tasks.rados.rados.0.smithi118.stdout:315: left oid 22 (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:02:23.088 INFO:tasks.rados.rados.0.smithi118.stdout:316: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:23.107 INFO:tasks.rados.rados.0.smithi118.stdout:318: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:02:23.107 INFO:tasks.rados.rados.0.smithi118.stdout:318: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:02:23.107 INFO:tasks.rados.rados.0.smithi118.stdout:318: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:02:23.107 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 141 (ObjNum 110 snap 27 seq_num 110) dirty exists 2024-03-20T08:02:23.108 INFO:tasks.rados.rados.0.smithi118.stdout:319: expect (ObjNum 36 snap 0 seq_num 36) 2024-03-20T08:02:23.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:23 smithi196 ceph-mon[31033]: pgmap v162: 121 pgs: 121 active+clean; 75 MiB data, 356 MiB used, 715 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-20T08:02:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:23 smithi196 ceph-mon[31033]: osdmap e125: 8 total, 8 up, 8 in 2024-03-20T08:02:23.140 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 podman[66285]: 2024-03-20 08:02:22.872486284 +0000 UTC m=+0.415534360 container init 838b6245d898211f970d446b8339ab52a6c2517e9c9ee51a5fdc8741e0bedb8d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-03-20T08:02:23.140 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 podman[66285]: 2024-03-20 08:02:22.878273671 +0000 UTC m=+0.421321737 container start 838b6245d898211f970d446b8339ab52a6c2517e9c9ee51a5fdc8741e0bedb8d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, GIT_CLEAN=True, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-20T08:02:23.140 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 ceph-mgr[66322]: -- 172.21.15.196:0/1661731223 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e9c73b65a0 con 0x55e9c738ac00 2024-03-20T08:02:23.140 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 bash[66285]: 838b6245d898211f970d446b8339ab52a6c2517e9c9ee51a5fdc8741e0bedb8d 2024-03-20T08:02:23.140 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:22 smithi196 systemd[1]: Started Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:02:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:23 smithi118 ceph-mon[27391]: pgmap v162: 121 pgs: 121 active+clean; 75 MiB data, 356 MiB used, 715 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-20T08:02:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:23 smithi118 ceph-mon[27391]: osdmap e125: 8 total, 8 up, 8 in 2024-03-20T08:02:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:23 smithi118 ceph-mon[38981]: pgmap v162: 121 pgs: 121 active+clean; 75 MiB data, 356 MiB used, 715 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-20T08:02:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:23 smithi118 ceph-mon[38981]: osdmap e125: 8 total, 8 up, 8 in 2024-03-20T08:02:23.782 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:23.463+0000 7f7a404fc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T08:02:23.782 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:23.572+0000 7f7a404fc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T08:02:24.080 INFO:tasks.rados.rados.0.smithi118.stdout:315: done (4 left) 2024-03-20T08:02:24.081 INFO:tasks.rados.rados.0.smithi118.stdout:316: done (3 left) 2024-03-20T08:02:24.081 INFO:tasks.rados.rados.0.smithi118.stdout:318: done (2 left) 2024-03-20T08:02:24.081 INFO:tasks.rados.rados.0.smithi118.stdout:319: done (1 left) 2024-03-20T08:02:24.081 INFO:tasks.rados.rados.0.smithi118.stdout:320: done (0 left) 2024-03-20T08:02:24.081 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 5 current snap is 31 2024-03-20T08:02:24.081 INFO:tasks.rados.rados.0.smithi118.stdout:321: seq_num 118 ranges {73728=786432,860160=704512,1564672=499712} 2024-03-20T08:02:24.085 INFO:tasks.rados.rados.0.smithi118.stdout:321: writing smithi11881747-5 from 73728 to 860160 tid 1 2024-03-20T08:02:24.089 INFO:tasks.rados.rados.0.smithi118.stdout:321: writing smithi11881747-5 from 860160 to 1564672 tid 2 2024-03-20T08:02:24.092 INFO:tasks.rados.rados.0.smithi118.stdout:321: writing smithi11881747-5 from 1564672 to 2064384 tid 3 2024-03-20T08:02:24.092 INFO:tasks.rados.rados.0.smithi118.stdout:322: rollback oid 31 current snap is 31 2024-03-20T08:02:24.092 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 29 2024-03-20T08:02:24.093 INFO:tasks.rados.rados.0.smithi118.stdout:321: finishing write tid 1 to smithi11881747-5 2024-03-20T08:02:24.093 INFO:tasks.rados.rados.0.smithi118.stdout:321: finishing write tid 2 to smithi11881747-5 2024-03-20T08:02:24.093 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 31 2024-03-20T08:02:24.093 INFO:tasks.rados.rados.0.smithi118.stdout:323: seq_num 119 ranges {778240=802816,1581056=589824,2170880=409600,2580480=204800} 2024-03-20T08:02:24.101 INFO:tasks.rados.rados.0.smithi118.stdout:323: writing smithi11881747-49 from 778240 to 1581056 tid 1 2024-03-20T08:02:24.104 INFO:tasks.rados.rados.0.smithi118.stdout:323: writing smithi11881747-49 from 1581056 to 2170880 tid 2 2024-03-20T08:02:24.107 INFO:tasks.rados.rados.0.smithi118.stdout:323: writing smithi11881747-49 from 2170880 to 2580480 tid 3 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:323: writing smithi11881747-49 from 2580480 to 2785280 tid 4 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:324: rmattr oid 41 current snap is 31 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:321: finishing write tid 3 to smithi11881747-5 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:321: finishing write tid 4 to smithi11881747-5 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:321: finishing write tid 5 to smithi11881747-5 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:321: oid 5 updating version 0 to 84 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:321: oid 5 updating version 84 to 87 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:321: oid 5 version 87 is already newer than 85 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:321: oid 5 version 87 is already newer than 86 2024-03-20T08:02:24.109 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 87 (ObjNum 118 snap 31 seq_num 118) dirty exists 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:321: left oid 5 (ObjNum 118 snap 31 seq_num 118) 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:322: finishing rollback tid 0 to smithi11881747-31 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:322: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:322: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 136 (ObjNum 87 snap 17 seq_num 87) dirty exists 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:323: finishing write tid 1 to smithi11881747-49 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:323: finishing write tid 2 to smithi11881747-49 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:321: done (3 left) 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:322: done (2 left) 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:325: read oid 32 snap -1 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:325: expect (ObjNum 31 snap 0 seq_num 31) 2024-03-20T08:02:24.110 INFO:tasks.rados.rados.0.smithi118.stdout:326: snap_remove snap 19 2024-03-20T08:02:24.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-mon[31033]: osdmap e126: 8 total, 8 up, 8 in 2024-03-20T08:02:24.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:24.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:24.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:24.347 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:24.249+0000 7f7a404fc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T08:02:24.347 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:24.347+0000 7f7a404fc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T08:02:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:24 smithi118 ceph-mon[27391]: osdmap e126: 8 total, 8 up, 8 in 2024-03-20T08:02:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:24 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:24 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:24.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:24 smithi118 ceph-mon[38981]: osdmap e126: 8 total, 8 up, 8 in 2024-03-20T08:02:24.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:24 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:24 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:24.638 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:24.494+0000 7f7a404fc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T08:02:24.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:24.594+0000 7f7a404fc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T08:02:24.953 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:24.681+0000 7f7a404fc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T08:02:24.953 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:24.953+0000 7f7a404fc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T08:02:25.086 INFO:tasks.rados.rados.0.smithi118.stdout:326: done (3 left) 2024-03-20T08:02:25.086 INFO:tasks.rados.rados.0.smithi118.stdout:327: read oid 17 snap -1 2024-03-20T08:02:25.086 INFO:tasks.rados.rados.0.smithi118.stdout:327: expect (ObjNum 97 snap 20 seq_num 97) 2024-03-20T08:02:25.086 INFO:tasks.rados.rados.0.smithi118.stdout:328: snap_remove snap 21 2024-03-20T08:02:25.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:25 smithi196 ceph-mon[31033]: pgmap v165: 121 pgs: 121 active+clean; 75 MiB data, 356 MiB used, 715 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-20T08:02:25.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:25 smithi196 ceph-mon[31033]: osdmap e127: 8 total, 8 up, 8 in 2024-03-20T08:02:25.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:25.076+0000 7f7a404fc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T08:02:25.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:25.184+0000 7f7a404fc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T08:02:25.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:25.284+0000 7f7a404fc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T08:02:25.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:25 smithi118 ceph-mon[38981]: pgmap v165: 121 pgs: 121 active+clean; 75 MiB data, 356 MiB used, 715 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-20T08:02:25.524 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:25 smithi118 ceph-mon[38981]: osdmap e127: 8 total, 8 up, 8 in 2024-03-20T08:02:25.524 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:25 smithi118 ceph-mon[27391]: pgmap v165: 121 pgs: 121 active+clean; 75 MiB data, 356 MiB used, 715 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-20T08:02:25.525 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:25 smithi118 ceph-mon[27391]: osdmap e127: 8 total, 8 up, 8 in 2024-03-20T08:02:25.590 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:25.467+0000 7f7a404fc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T08:02:26.010 INFO:tasks.rados.rados.0.smithi118.stdout:328: done (4 left) 2024-03-20T08:02:26.010 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 40 current snap is 31 2024-03-20T08:02:26.010 INFO:tasks.rados.rados.0.smithi118.stdout:329: seq_num 120 ranges {0=417792,417792=475136,892928=622592,1515520=507904} 2024-03-20T08:02:26.012 INFO:tasks.rados.rados.0.smithi118.stdout:329: writing smithi11881747-40 from 0 to 417792 tid 1 2024-03-20T08:02:26.016 INFO:tasks.rados.rados.0.smithi118.stdout:329: writing smithi11881747-40 from 417792 to 892928 tid 2 2024-03-20T08:02:26.020 INFO:tasks.rados.rados.0.smithi118.stdout:329: writing smithi11881747-40 from 892928 to 1515520 tid 3 2024-03-20T08:02:26.024 INFO:tasks.rados.rados.0.smithi118.stdout:329: writing smithi11881747-40 from 1515520 to 2023424 tid 4 2024-03-20T08:02:26.024 INFO:tasks.rados.rados.0.smithi118.stdout:330: delete oid 10 current snap is 31 2024-03-20T08:02:26.024 INFO:tasks.rados.rados.0.smithi118.stdout:323: finishing write tid 3 to smithi11881747-49 2024-03-20T08:02:26.024 INFO:tasks.rados.rados.0.smithi118.stdout:323: finishing write tid 4 to smithi11881747-49 2024-03-20T08:02:26.024 INFO:tasks.rados.rados.0.smithi118.stdout:323: finishing write tid 5 to smithi11881747-49 2024-03-20T08:02:26.025 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 143 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-03-20T08:02:26.025 INFO:tasks.rados.rados.0.smithi118.stdout:325: expect (ObjNum 31 snap 0 seq_num 31) 2024-03-20T08:02:26.046 INFO:tasks.rados.rados.0.smithi118.stdout:323: finishing write tid 6 to smithi11881747-49 2024-03-20T08:02:26.046 INFO:tasks.rados.rados.0.smithi118.stdout:323: oid 49 updating version 0 to 115 2024-03-20T08:02:26.046 INFO:tasks.rados.rados.0.smithi118.stdout:323: oid 49 updating version 115 to 116 2024-03-20T08:02:26.046 INFO:tasks.rados.rados.0.smithi118.stdout:323: oid 49 updating version 116 to 117 2024-03-20T08:02:26.046 INFO:tasks.rados.rados.0.smithi118.stdout:323: oid 49 updating version 117 to 118 2024-03-20T08:02:26.047 INFO:tasks.rados.rados.0.smithi118.stdout:323: oid 49 version 118 is already newer than 114 2024-03-20T08:02:26.047 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 118 (ObjNum 119 snap 31 seq_num 119) dirty exists 2024-03-20T08:02:26.047 INFO:tasks.rados.rados.0.smithi118.stdout:323: left oid 49 (ObjNum 119 snap 31 seq_num 119) 2024-03-20T08:02:26.047 INFO:tasks.rados.rados.0.smithi118.stdout:327: expect (ObjNum 97 snap 20 seq_num 97) 2024-03-20T08:02:26.149 INFO:tasks.rados.rados.0.smithi118.stdout:329: finishing write tid 1 to smithi11881747-40 2024-03-20T08:02:26.149 INFO:tasks.rados.rados.0.smithi118.stdout:329: finishing write tid 2 to smithi11881747-40 2024-03-20T08:02:26.149 INFO:tasks.rados.rados.0.smithi118.stdout:323: done (5 left) 2024-03-20T08:02:26.149 INFO:tasks.rados.rados.0.smithi118.stdout:324: done (4 left) 2024-03-20T08:02:26.149 INFO:tasks.rados.rados.0.smithi118.stdout:325: done (3 left) 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:327: done (2 left) 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:330: done (1 left) 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:331: rollback oid 46 current snap is 31 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 28 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: finishing write tid 3 to smithi11881747-40 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: finishing write tid 4 to smithi11881747-40 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: finishing write tid 5 to smithi11881747-40 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: finishing write tid 6 to smithi11881747-40 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: oid 40 updating version 0 to 108 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: oid 40 version 108 is already newer than 106 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: oid 40 version 108 is already newer than 100 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: oid 40 version 108 is already newer than 101 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:329: oid 40 version 108 is already newer than 107 2024-03-20T08:02:26.150 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 108 (ObjNum 120 snap 31 seq_num 120) dirty exists 2024-03-20T08:02:26.151 INFO:tasks.rados.rados.0.smithi118.stdout:329: left oid 40 (ObjNum 120 snap 31 seq_num 120) 2024-03-20T08:02:26.151 INFO:tasks.rados.rados.0.smithi118.stdout:329: done (1 left) 2024-03-20T08:02:26.151 INFO:tasks.rados.rados.0.smithi118.stdout:332: rmattr oid 33 current snap is 31 2024-03-20T08:02:26.151 INFO:tasks.rados.rados.0.smithi118.stdout:333: snap_remove snap 26 2024-03-20T08:02:26.376 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:26 smithi196 ceph-mon[31033]: osdmap e128: 8 total, 8 up, 8 in 2024-03-20T08:02:26.376 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:26 smithi196 ceph-mon[31033]: pgmap v168: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-03-20T08:02:26.376 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:26 smithi196 ceph-mon[31033]: osdmap e129: 8 total, 8 up, 8 in 2024-03-20T08:02:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:26 smithi118 ceph-mon[27391]: osdmap e128: 8 total, 8 up, 8 in 2024-03-20T08:02:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:26 smithi118 ceph-mon[27391]: pgmap v168: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-03-20T08:02:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:26 smithi118 ceph-mon[27391]: osdmap e129: 8 total, 8 up, 8 in 2024-03-20T08:02:26.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:26 smithi118 ceph-mon[38981]: osdmap e128: 8 total, 8 up, 8 in 2024-03-20T08:02:26.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:26 smithi118 ceph-mon[38981]: pgmap v168: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-03-20T08:02:26.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:26 smithi118 ceph-mon[38981]: osdmap e129: 8 total, 8 up, 8 in 2024-03-20T08:02:26.630 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:26.493+0000 7f7a404fc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T08:02:26.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:26.630+0000 7f7a404fc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 0 (ObjNum 89 snap 18 seq_num 89) dirty exists 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:331: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:331: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:331: finishing rollback tid 2 to smithi11881747-46 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 59 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:331: done (2 left) 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:332: done (1 left) 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:333: done (0 left) 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:334: copy_from oid 31 from oid 44 current snap is 31 2024-03-20T08:02:27.093 INFO:tasks.rados.rados.0.smithi118.stdout:335: snap_create 2024-03-20T08:02:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:334: finishing copy_from to smithi11881747-31 2024-03-20T08:02:27.098 INFO:tasks.rados.rados.0.smithi118.stdout:334: got expected ENOENT (src dne) 2024-03-20T08:02:27.098 INFO:tasks.rados.rados.0.smithi118.stdout:334: finishing copy_from racing read to smithi11881747-31 2024-03-20T08:02:27.284 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:26.981+0000 7f7a404fc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T08:02:27.284 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:27.071+0000 7f7a404fc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T08:02:27.557 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:27.284+0000 7f7a404fc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:02:27.557 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:27.483+0000 7f7a404fc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T08:02:27.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:27.789+0000 7f7a404fc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:02:28.093 INFO:tasks.rados.rados.0.smithi118.stdout:334: done (1 left) 2024-03-20T08:02:28.094 INFO:tasks.rados.rados.0.smithi118.stdout:335: done (0 left) 2024-03-20T08:02:28.094 INFO:tasks.rados.rados.0.smithi118.stdout:336: copy_from oid 27 from oid 39 current snap is 32 2024-03-20T08:02:28.094 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 32 2024-03-20T08:02:28.094 INFO:tasks.rados.rados.0.smithi118.stdout:337: seq_num 122 ranges {0=450560,450560=688128,1138688=630784,1769472=286720} 2024-03-20T08:02:28.096 INFO:tasks.rados.rados.0.smithi118.stdout:337: writing smithi11881747-8 from 0 to 450560 tid 1 2024-03-20T08:02:28.101 INFO:tasks.rados.rados.0.smithi118.stdout:337: writing smithi11881747-8 from 450560 to 1138688 tid 2 2024-03-20T08:02:28.105 INFO:tasks.rados.rados.0.smithi118.stdout:337: writing smithi11881747-8 from 1138688 to 1769472 tid 3 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:337: writing smithi11881747-8 from 1769472 to 2056192 tid 4 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:338: copy_from oid 45 from oid 50 current snap is 32 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:337: finishing write tid 1 to smithi11881747-8 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:336: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:336: finishing copy_from to smithi11881747-27 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 8 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:337: finishing write tid 2 to smithi11881747-8 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:336: done (2 left) 2024-03-20T08:02:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:339: snap_create 2024-03-20T08:02:28.108 INFO:tasks.rados.rados.0.smithi118.stdout:337: finishing write tid 3 to smithi11881747-8 2024-03-20T08:02:28.108 INFO:tasks.rados.rados.0.smithi118.stdout:337: finishing write tid 4 to smithi11881747-8 2024-03-20T08:02:28.109 INFO:tasks.rados.rados.0.smithi118.stdout:337: finishing write tid 5 to smithi11881747-8 2024-03-20T08:02:28.110 INFO:tasks.rados.rados.0.smithi118.stdout:337: finishing write tid 6 to smithi11881747-8 2024-03-20T08:02:28.110 INFO:tasks.rados.rados.0.smithi118.stdout:337: oid 8 updating version 0 to 74 2024-03-20T08:02:28.110 INFO:tasks.rados.rados.0.smithi118.stdout:337: oid 8 version 74 is already newer than 72 2024-03-20T08:02:28.110 INFO:tasks.rados.rados.0.smithi118.stdout:337: oid 8 version 74 is already newer than 73 2024-03-20T08:02:28.110 INFO:tasks.rados.rados.0.smithi118.stdout:337: oid 8 updating version 74 to 75 2024-03-20T08:02:28.111 INFO:tasks.rados.rados.0.smithi118.stdout:337: oid 8 version 75 is already newer than 71 2024-03-20T08:02:28.111 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 75 (ObjNum 122 snap 32 seq_num 122) dirty exists 2024-03-20T08:02:28.111 INFO:tasks.rados.rados.0.smithi118.stdout:337: left oid 8 (ObjNum 122 snap 32 seq_num 122) 2024-03-20T08:02:28.120 INFO:tasks.rados.rados.0.smithi118.stdout:338: finishing copy_from racing read to smithi11881747-45 2024-03-20T08:02:28.123 INFO:tasks.rados.rados.0.smithi118.stdout:338: finishing copy_from to smithi11881747-45 2024-03-20T08:02:28.123 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 51 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:02:28.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:28 smithi196 ceph-mon[31033]: osdmap e130: 8 total, 8 up, 8 in 2024-03-20T08:02:28.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:28 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:28.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:28 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:28.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:28 smithi196 ceph-mon[31033]: pgmap v171: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-03-20T08:02:28.372 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:28.071+0000 7f7a404fc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T08:02:28.372 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:28.371+0000 7f7a404fc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T08:02:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:28 smithi118 ceph-mon[27391]: osdmap e130: 8 total, 8 up, 8 in 2024-03-20T08:02:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:28 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:28 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:28 smithi118 ceph-mon[27391]: pgmap v171: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-03-20T08:02:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:28 smithi118 ceph-mon[38981]: osdmap e130: 8 total, 8 up, 8 in 2024-03-20T08:02:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:28 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:28 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:28 smithi118 ceph-mon[38981]: pgmap v171: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail; 8.6 MiB/s rd, 2.8 MiB/s wr, 20 op/s 2024-03-20T08:02:29.108 INFO:tasks.rados.rados.0.smithi118.stdout:337: done (2 left) 2024-03-20T08:02:29.108 INFO:tasks.rados.rados.0.smithi118.stdout:338: done (1 left) 2024-03-20T08:02:29.108 INFO:tasks.rados.rados.0.smithi118.stdout:339: done (0 left) 2024-03-20T08:02:29.108 INFO:tasks.rados.rados.0.smithi118.stdout:340: read oid 36 snap -1 2024-03-20T08:02:29.108 INFO:tasks.rados.rados.0.smithi118.stdout:340: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:29.109 INFO:tasks.rados.rados.0.smithi118.stdout:341: read oid 26 snap -1 2024-03-20T08:02:29.109 INFO:tasks.rados.rados.0.smithi118.stdout:341: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:02:29.109 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 33 2024-03-20T08:02:29.109 INFO:tasks.rados.rados.0.smithi118.stdout:342: seq_num 123 ranges {5849088=466944,6316032=573440,6889472=794624,7684096=237568} 2024-03-20T08:02:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:342: writing smithi11881747-31 from 5849088 to 6316032 tid 1 2024-03-20T08:02:29.145 INFO:tasks.rados.rados.0.smithi118.stdout:342: writing smithi11881747-31 from 6316032 to 6889472 tid 2 2024-03-20T08:02:29.149 INFO:tasks.rados.rados.0.smithi118.stdout:342: writing smithi11881747-31 from 6889472 to 7684096 tid 3 2024-03-20T08:02:29.151 INFO:tasks.rados.rados.0.smithi118.stdout:342: writing smithi11881747-31 from 7684096 to 7921664 tid 4 2024-03-20T08:02:29.151 INFO:tasks.rados.rados.0.smithi118.stdout:343: setattr oid 8 current snap is 33 2024-03-20T08:02:29.151 INFO:tasks.rados.rados.0.smithi118.stdout:341: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:02:29.162 INFO:tasks.rados.rados.0.smithi118.stdout:340: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: finishing write tid 1 to smithi11881747-31 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: finishing write tid 2 to smithi11881747-31 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: finishing write tid 3 to smithi11881747-31 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: finishing write tid 4 to smithi11881747-31 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: finishing write tid 5 to smithi11881747-31 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: finishing write tid 6 to smithi11881747-31 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: oid 31 updating version 0 to 161 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: oid 31 updating version 161 to 163 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: oid 31 version 163 is already newer than 160 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: oid 31 version 163 is already newer than 162 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:342: oid 31 updating version 163 to 164 2024-03-20T08:02:29.182 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 164 (ObjNum 123 snap 33 seq_num 123) dirty exists 2024-03-20T08:02:29.183 INFO:tasks.rados.rados.0.smithi118.stdout:342: left oid 31 (ObjNum 123 snap 33 seq_num 123) 2024-03-20T08:02:29.183 INFO:tasks.rados.rados.0.smithi118.stdout:340: done (3 left) 2024-03-20T08:02:29.183 INFO:tasks.rados.rados.0.smithi118.stdout:341: done (2 left) 2024-03-20T08:02:29.183 INFO:tasks.rados.rados.0.smithi118.stdout:342: done (1 left) 2024-03-20T08:02:29.183 INFO:tasks.rados.rados.0.smithi118.stdout:344: read oid 47 snap -1 2024-03-20T08:02:29.183 INFO:tasks.rados.rados.0.smithi118.stdout:344: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:29.183 INFO:tasks.rados.rados.0.smithi118.stdout:345: snap_remove snap 25 2024-03-20T08:02:29.350 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:29.254+0000 7f7a404fc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T08:02:29.350 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:29 smithi196 ceph-mon[31033]: osdmap e131: 8 total, 8 up, 8 in 2024-03-20T08:02:29.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:29 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:29.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:29 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:29 smithi118 ceph-mon[27391]: osdmap e131: 8 total, 8 up, 8 in 2024-03-20T08:02:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:29 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:29 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:29.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:29 smithi118 ceph-mon[38981]: osdmap e131: 8 total, 8 up, 8 in 2024-03-20T08:02:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:29 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:29 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:29.762 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:02:30.091 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:02:30.091 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (2m) 80s ago 3m 24.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:02:30.091 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (2m) 2s ago 2m 88.7M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (4m) 80s ago 4m 12.6M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (4m) 2s ago 4m 17.5M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (7s) 2s ago 7m 59.7M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 838b6245d898 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:9283,8765 running (9m) 80s ago 9m 494M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 64dace5519ed 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (9m) 80s ago 9m 48.5M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 89e74e5abc84 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (8m) 2s ago 8m 40.9M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 fd26544c725b 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (8m) 80s ago 8m 44.3M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 8193e23658db 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (2m) 80s ago 2m 14.7M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (2m) 2s ago 2m 16.2M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (7m) 80s ago 7m 47.7M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 481bba84ab31 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (7m) 80s ago 6m 46.5M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (6m) 80s ago 6m 41.9M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (6m) 80s ago 6m 40.8M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:02:30.092 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (5m) 2s ago 5m 77.0M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:02:30.093 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (5m) 2s ago 5m 66.3M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:02:30.093 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (4m) 2s ago 4m 59.5M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:02:30.093 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (4m) 2s ago 4m 55.7M 2992M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:02:30.093 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (2m) 2s ago 3m 42.5M - 2.43.0 a07b618ecd1d d713b2506adb 2024-03-20T08:02:30.109 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:29.844+0000 7f7a404fc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T08:02:30.109 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:29.948+0000 7f7a404fc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T08:02:30.111 INFO:tasks.rados.rados.0.smithi118.stdout:345: done (2 left) 2024-03-20T08:02:30.112 INFO:tasks.rados.rados.0.smithi118.stdout:346: snap_remove snap 18 2024-03-20T08:02:30.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-mon[31033]: osdmap e132: 8 total, 8 up, 8 in 2024-03-20T08:02:30.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-mon[31033]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:30.365 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-mon[31033]: pgmap v174: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:30.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:30.366 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:30.181+0000 7f7a404fc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T08:02:30.366 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:30] ENGINE Bus STARTING 2024-03-20T08:02:30.366 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: CherryPy Checker: 2024-03-20T08:02:30.366 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: The Application mounted at '' has an empty config. 2024-03-20T08:02:30.366 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:30.366 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:30] ENGINE Serving on http://:::9283 2024-03-20T08:02:30.366 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:30] ENGINE Bus STARTED 2024-03-20T08:02:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:30 smithi118 ceph-mon[27391]: osdmap e132: 8 total, 8 up, 8 in 2024-03-20T08:02:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:30 smithi118 ceph-mon[27391]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:30 smithi118 ceph-mon[27391]: pgmap v174: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:30 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:30.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:30 smithi118 ceph-mon[38981]: osdmap e132: 8 total, 8 up, 8 in 2024-03-20T08:02:30.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:30 smithi118 ceph-mon[38981]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:30 smithi118 ceph-mon[38981]: pgmap v174: 121 pgs: 121 active+clean; 87 MiB data, 373 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:30 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:30.493 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:02:30.493 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:02:30.493 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:02:30.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:02:30.493 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:02:30.493 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:02:30.493 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:02:30.494 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:02:30.494 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:02:30.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:02:30.494 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:02:30.494 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:02:30.494 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:02:30.494 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:02:30.494 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 15 2024-03-20T08:02:30.495 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:02:30.495 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:02:31.115 INFO:tasks.rados.rados.0.smithi118.stdout:346: done (2 left) 2024-03-20T08:02:31.115 INFO:tasks.rados.rados.0.smithi118.stdout:347: read oid 14 snap -1 2024-03-20T08:02:31.115 INFO:tasks.rados.rados.0.smithi118.stdout:347: expect (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:02:31.115 INFO:tasks.rados.rados.0.smithi118.stdout:348: setattr oid 15 current snap is 33 2024-03-20T08:02:31.115 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 77 (ObjNum 122 snap 32 seq_num 122) dirty exists 2024-03-20T08:02:31.115 INFO:tasks.rados.rados.0.smithi118.stdout:344: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:31.147 INFO:tasks.rados.rados.0.smithi118.stdout:347: expect (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:02:31.192 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 44 (ObjNum 59 snap 5 seq_num 59) dirty exists 2024-03-20T08:02:31.192 INFO:tasks.rados.rados.0.smithi118.stdout:343: done (3 left) 2024-03-20T08:02:31.193 INFO:tasks.rados.rados.0.smithi118.stdout:344: done (2 left) 2024-03-20T08:02:31.193 INFO:tasks.rados.rados.0.smithi118.stdout:347: done (1 left) 2024-03-20T08:02:31.193 INFO:tasks.rados.rados.0.smithi118.stdout:348: done (0 left) 2024-03-20T08:02:31.193 INFO:tasks.rados.rados.0.smithi118.stdout:349: setattr oid 3 current snap is 33 2024-03-20T08:02:31.194 INFO:tasks.rados.rados.0.smithi118.stdout:350: delete oid 40 current snap is 33 2024-03-20T08:02:31.196 INFO:tasks.rados.rados.0.smithi118.stdout:350: done (1 left) 2024-03-20T08:02:31.196 INFO:tasks.rados.rados.0.smithi118.stdout:351: copy_from oid 6 from oid 19 current snap is 33 2024-03-20T08:02:31.196 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 33 2024-03-20T08:02:31.197 INFO:tasks.rados.rados.0.smithi118.stdout:352: seq_num 125 ranges {2940928=499712,3440640=745472,4186112=409600,4595712=450560} 2024-03-20T08:02:31.214 INFO:tasks.rados.rados.0.smithi118.stdout:352: writing smithi11881747-17 from 2940928 to 3440640 tid 1 2024-03-20T08:02:31.219 INFO:tasks.rados.rados.0.smithi118.stdout:352: writing smithi11881747-17 from 3440640 to 4186112 tid 2 2024-03-20T08:02:31.221 INFO:tasks.rados.rados.0.smithi118.stdout:352: writing smithi11881747-17 from 4186112 to 4595712 tid 3 2024-03-20T08:02:31.224 INFO:tasks.rados.rados.0.smithi118.stdout:352: writing smithi11881747-17 from 4595712 to 5046272 tid 4 2024-03-20T08:02:31.224 INFO:tasks.rados.rados.0.smithi118.stdout:353: read oid 45 snap -1 2024-03-20T08:02:31.224 INFO:tasks.rados.rados.0.smithi118.stdout:353: expect (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:02:31.224 INFO:tasks.rados.rados.0.smithi118.stdout:354: snap_create 2024-03-20T08:02:31.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 63 (ObjNum 86 snap 15 seq_num 86) dirty exists 2024-03-20T08:02:31.224 INFO:tasks.rados.rados.0.smithi118.stdout:351: finishing copy_from to smithi11881747-6 2024-03-20T08:02:31.224 INFO:tasks.rados.rados.0.smithi118.stdout:351: got expected ENOENT (src dne) 2024-03-20T08:02:31.224 INFO:tasks.rados.rados.0.smithi118.stdout:351: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:02:31.225 INFO:tasks.rados.rados.0.smithi118.stdout:352: finishing write tid 1 to smithi11881747-17 2024-03-20T08:02:31.225 INFO:tasks.rados.rados.0.smithi118.stdout:352: finishing write tid 2 to smithi11881747-17 2024-03-20T08:02:31.225 INFO:tasks.rados.rados.0.smithi118.stdout:352: finishing write tid 3 to smithi11881747-17 2024-03-20T08:02:31.227 INFO:tasks.rados.rados.0.smithi118.stdout:352: finishing write tid 4 to smithi11881747-17 2024-03-20T08:02:31.227 INFO:tasks.rados.rados.0.smithi118.stdout:352: finishing write tid 5 to smithi11881747-17 2024-03-20T08:02:31.228 INFO:tasks.rados.rados.0.smithi118.stdout:352: finishing write tid 6 to smithi11881747-17 2024-03-20T08:02:31.228 INFO:tasks.rados.rados.0.smithi118.stdout:352: oid 17 updating version 0 to 41 2024-03-20T08:02:31.228 INFO:tasks.rados.rados.0.smithi118.stdout:352: oid 17 version 41 is already newer than 39 2024-03-20T08:02:31.228 INFO:tasks.rados.rados.0.smithi118.stdout:352: oid 17 version 41 is already newer than 40 2024-03-20T08:02:31.228 INFO:tasks.rados.rados.0.smithi118.stdout:352: oid 17 version 41 is already newer than 38 2024-03-20T08:02:31.229 INFO:tasks.rados.rados.0.smithi118.stdout:352: oid 17 updating version 41 to 42 2024-03-20T08:02:31.229 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 42 (ObjNum 125 snap 33 seq_num 125) dirty exists 2024-03-20T08:02:31.229 INFO:tasks.rados.rados.0.smithi118.stdout:352: left oid 17 (ObjNum 125 snap 33 seq_num 125) 2024-03-20T08:02:31.230 INFO:tasks.rados.rados.0.smithi118.stdout:353: expect (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:02:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: osdmap e133: 8 total, 8 up, 8 in 2024-03-20T08:02:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: Standby manager daemon x restarted 2024-03-20T08:02:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: Standby manager daemon x started 2024-03-20T08:02:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:02:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:02:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:02:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:02:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:31 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1674243595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:02:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: osdmap e133: 8 total, 8 up, 8 in 2024-03-20T08:02:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: Standby manager daemon x restarted 2024-03-20T08:02:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: Standby manager daemon x started 2024-03-20T08:02:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:02:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1674243595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: osdmap e133: 8 total, 8 up, 8 in 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: Standby manager daemon x restarted 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: Standby manager daemon x started 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:02:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1674243595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:02:31.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:02:31] "GET /metrics HTTP/1.1" 200 36075 "" "Prometheus/2.43.0" 2024-03-20T08:02:32.127 INFO:tasks.rados.rados.0.smithi118.stdout:349: done (4 left) 2024-03-20T08:02:32.127 INFO:tasks.rados.rados.0.smithi118.stdout:351: done (3 left) 2024-03-20T08:02:32.127 INFO:tasks.rados.rados.0.smithi118.stdout:352: done (2 left) 2024-03-20T08:02:32.127 INFO:tasks.rados.rados.0.smithi118.stdout:353: done (1 left) 2024-03-20T08:02:32.127 INFO:tasks.rados.rados.0.smithi118.stdout:354: done (0 left) 2024-03-20T08:02:32.127 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 34 2024-03-20T08:02:32.128 INFO:tasks.rados.rados.0.smithi118.stdout:355: seq_num 126 ranges {0=516096,516096=630784,1146880=614400,1761280=360448} 2024-03-20T08:02:32.129 INFO:tasks.rados.rados.0.smithi118.stdout:355: writing smithi11881747-13 from 0 to 516096 tid 1 2024-03-20T08:02:32.133 INFO:tasks.rados.rados.0.smithi118.stdout:355: writing smithi11881747-13 from 516096 to 1146880 tid 2 2024-03-20T08:02:32.137 INFO:tasks.rados.rados.0.smithi118.stdout:355: writing smithi11881747-13 from 1146880 to 1761280 tid 3 2024-03-20T08:02:32.139 INFO:tasks.rados.rados.0.smithi118.stdout:355: writing smithi11881747-13 from 1761280 to 2121728 tid 4 2024-03-20T08:02:32.140 INFO:tasks.rados.rados.0.smithi118.stdout:356: delete oid 37 current snap is 34 2024-03-20T08:02:32.140 INFO:tasks.rados.rados.0.smithi118.stdout:355: finishing write tid 1 to smithi11881747-13 2024-03-20T08:02:32.140 INFO:tasks.rados.rados.0.smithi118.stdout:355: finishing write tid 2 to smithi11881747-13 2024-03-20T08:02:32.140 INFO:tasks.rados.rados.0.smithi118.stdout:355: finishing write tid 3 to smithi11881747-13 2024-03-20T08:02:32.142 INFO:tasks.rados.rados.0.smithi118.stdout:356: done (1 left) 2024-03-20T08:02:32.142 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 34 2024-03-20T08:02:32.142 INFO:tasks.rados.rados.0.smithi118.stdout:357: seq_num 128 ranges {622592=548864,1171456=794624,1966080=581632,2547712=229376} 2024-03-20T08:02:32.148 INFO:tasks.rados.rados.0.smithi118.stdout:357: writing smithi11881747-38 from 622592 to 1171456 tid 1 2024-03-20T08:02:32.153 INFO:tasks.rados.rados.0.smithi118.stdout:357: writing smithi11881747-38 from 1171456 to 1966080 tid 2 2024-03-20T08:02:32.156 INFO:tasks.rados.rados.0.smithi118.stdout:357: writing smithi11881747-38 from 1966080 to 2547712 tid 3 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:357: writing smithi11881747-38 from 2547712 to 2777088 tid 4 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:358: setattr oid 20 current snap is 34 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: finishing write tid 4 to smithi11881747-13 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: finishing write tid 5 to smithi11881747-13 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: finishing write tid 6 to smithi11881747-13 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: oid 13 updating version 0 to 122 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: oid 13 version 122 is already newer than 118 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: oid 13 version 122 is already newer than 121 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: oid 13 version 122 is already newer than 119 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: oid 13 version 122 is already newer than 120 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 122 (ObjNum 126 snap 34 seq_num 126) dirty exists 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:355: left oid 13 (ObjNum 126 snap 34 seq_num 126) 2024-03-20T08:02:32.158 INFO:tasks.rados.rados.0.smithi118.stdout:357: finishing write tid 1 to smithi11881747-38 2024-03-20T08:02:32.159 INFO:tasks.rados.rados.0.smithi118.stdout:357: finishing write tid 2 to smithi11881747-38 2024-03-20T08:02:32.159 INFO:tasks.rados.rados.0.smithi118.stdout:355: done (2 left) 2024-03-20T08:02:32.160 INFO:tasks.rados.rados.0.smithi118.stdout:359: read oid 14 snap -1 2024-03-20T08:02:32.160 INFO:tasks.rados.rados.0.smithi118.stdout:359: expect (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:02:32.160 INFO:tasks.rados.rados.0.smithi118.stdout:360: snap_create 2024-03-20T08:02:32.160 INFO:tasks.rados.rados.0.smithi118.stdout:357: finishing write tid 3 to smithi11881747-38 2024-03-20T08:02:32.160 INFO:tasks.rados.rados.0.smithi118.stdout:357: finishing write tid 4 to smithi11881747-38 2024-03-20T08:02:32.160 INFO:tasks.rados.rados.0.smithi118.stdout:357: finishing write tid 5 to smithi11881747-38 2024-03-20T08:02:32.161 INFO:tasks.rados.rados.0.smithi118.stdout:357: finishing write tid 6 to smithi11881747-38 2024-03-20T08:02:32.161 INFO:tasks.rados.rados.0.smithi118.stdout:357: oid 38 updating version 0 to 139 2024-03-20T08:02:32.161 INFO:tasks.rados.rados.0.smithi118.stdout:357: oid 38 version 139 is already newer than 135 2024-03-20T08:02:32.161 INFO:tasks.rados.rados.0.smithi118.stdout:357: oid 38 version 139 is already newer than 137 2024-03-20T08:02:32.161 INFO:tasks.rados.rados.0.smithi118.stdout:357: oid 38 version 139 is already newer than 136 2024-03-20T08:02:32.161 INFO:tasks.rados.rados.0.smithi118.stdout:357: oid 38 version 139 is already newer than 138 2024-03-20T08:02:32.161 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 139 (ObjNum 128 snap 34 seq_num 128) dirty exists 2024-03-20T08:02:32.161 INFO:tasks.rados.rados.0.smithi118.stdout:357: left oid 38 (ObjNum 128 snap 34 seq_num 128) 2024-03-20T08:02:32.164 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 145 (ObjNum 60 snap 5 seq_num 60) dirty exists 2024-03-20T08:02:32.165 INFO:tasks.rados.rados.0.smithi118.stdout:359: expect (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:02:32.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:32 smithi196 ceph-mon[31033]: osdmap e134: 8 total, 8 up, 8 in 2024-03-20T08:02:32.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:32 smithi196 ceph-mon[31033]: mgrmap e23: y(active, since 3m), standbys: x 2024-03-20T08:02:32.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:32 smithi196 ceph-mon[31033]: pgmap v177: 121 pgs: 121 active+clean; 96 MiB data, 387 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 14 op/s 2024-03-20T08:02:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:32 smithi118 ceph-mon[27391]: osdmap e134: 8 total, 8 up, 8 in 2024-03-20T08:02:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:32 smithi118 ceph-mon[27391]: mgrmap e23: y(active, since 3m), standbys: x 2024-03-20T08:02:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:32 smithi118 ceph-mon[27391]: pgmap v177: 121 pgs: 121 active+clean; 96 MiB data, 387 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 14 op/s 2024-03-20T08:02:32.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:32 smithi118 ceph-mon[38981]: osdmap e134: 8 total, 8 up, 8 in 2024-03-20T08:02:32.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:32 smithi118 ceph-mon[38981]: mgrmap e23: y(active, since 3m), standbys: x 2024-03-20T08:02:32.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:32 smithi118 ceph-mon[38981]: pgmap v177: 121 pgs: 121 active+clean; 96 MiB data, 387 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 14 op/s 2024-03-20T08:02:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:357: done (3 left) 2024-03-20T08:02:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:358: done (2 left) 2024-03-20T08:02:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:359: done (1 left) 2024-03-20T08:02:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:360: done (0 left) 2024-03-20T08:02:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:361: delete oid 27 current snap is 35 2024-03-20T08:02:33.135 INFO:tasks.rados.rados.0.smithi118.stdout:361: done (0 left) 2024-03-20T08:02:33.135 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 35 2024-03-20T08:02:33.135 INFO:tasks.rados.rados.0.smithi118.stdout:362: seq_num 130 ranges {1941504=581632,2523136=614400,3137536=598016,3735552=393216} 2024-03-20T08:02:33.148 INFO:tasks.rados.rados.0.smithi118.stdout:362: writing smithi11881747-3 from 1941504 to 2523136 tid 1 2024-03-20T08:02:33.152 INFO:tasks.rados.rados.0.smithi118.stdout:362: writing smithi11881747-3 from 2523136 to 3137536 tid 2 2024-03-20T08:02:33.156 INFO:tasks.rados.rados.0.smithi118.stdout:362: writing smithi11881747-3 from 3137536 to 3735552 tid 3 2024-03-20T08:02:33.158 INFO:tasks.rados.rados.0.smithi118.stdout:362: writing smithi11881747-3 from 3735552 to 4128768 tid 4 2024-03-20T08:02:33.158 INFO:tasks.rados.rados.0.smithi118.stdout:363: snap_create 2024-03-20T08:02:33.158 INFO:tasks.rados.rados.0.smithi118.stdout:362: finishing write tid 1 to smithi11881747-3 2024-03-20T08:02:33.158 INFO:tasks.rados.rados.0.smithi118.stdout:362: finishing write tid 2 to smithi11881747-3 2024-03-20T08:02:33.159 INFO:tasks.rados.rados.0.smithi118.stdout:362: finishing write tid 3 to smithi11881747-3 2024-03-20T08:02:33.160 INFO:tasks.rados.rados.0.smithi118.stdout:362: finishing write tid 4 to smithi11881747-3 2024-03-20T08:02:33.161 INFO:tasks.rados.rados.0.smithi118.stdout:362: finishing write tid 5 to smithi11881747-3 2024-03-20T08:02:33.162 INFO:tasks.rados.rados.0.smithi118.stdout:362: finishing write tid 6 to smithi11881747-3 2024-03-20T08:02:33.162 INFO:tasks.rados.rados.0.smithi118.stdout:362: oid 3 updating version 0 to 69 2024-03-20T08:02:33.162 INFO:tasks.rados.rados.0.smithi118.stdout:362: oid 3 version 69 is already newer than 68 2024-03-20T08:02:33.162 INFO:tasks.rados.rados.0.smithi118.stdout:362: oid 3 version 69 is already newer than 66 2024-03-20T08:02:33.162 INFO:tasks.rados.rados.0.smithi118.stdout:362: oid 3 version 69 is already newer than 65 2024-03-20T08:02:33.162 INFO:tasks.rados.rados.0.smithi118.stdout:362: oid 3 version 69 is already newer than 67 2024-03-20T08:02:33.162 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 69 (ObjNum 130 snap 35 seq_num 130) dirty exists 2024-03-20T08:02:33.162 INFO:tasks.rados.rados.0.smithi118.stdout:362: left oid 3 (ObjNum 130 snap 35 seq_num 130) 2024-03-20T08:02:33.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:33 smithi196 ceph-mon[31033]: osdmap e135: 8 total, 8 up, 8 in 2024-03-20T08:02:33.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:33 smithi118 ceph-mon[27391]: osdmap e135: 8 total, 8 up, 8 in 2024-03-20T08:02:33.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:33 smithi118 ceph-mon[38981]: osdmap e135: 8 total, 8 up, 8 in 2024-03-20T08:02:34.138 INFO:tasks.rados.rados.0.smithi118.stdout:362: done (1 left) 2024-03-20T08:02:34.138 INFO:tasks.rados.rados.0.smithi118.stdout:363: done (0 left) 2024-03-20T08:02:34.138 INFO:tasks.rados.rados.0.smithi118.stdout:364: rollback oid 27 current snap is 36 2024-03-20T08:02:34.138 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 35 2024-03-20T08:02:34.139 INFO:tasks.rados.rados.0.smithi118.stdout:365: setattr oid 20 current snap is 36 2024-03-20T08:02:34.139 INFO:tasks.rados.rados.0.smithi118.stdout:364: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:02:34.139 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 147 (ObjNum 273 snap 0 seq_num 1864347104) dirty dne 2024-03-20T08:02:34.139 INFO:tasks.rados.rados.0.smithi118.stdout:364: done (1 left) 2024-03-20T08:02:34.140 INFO:tasks.rados.rados.0.smithi118.stdout:366: snap_create 2024-03-20T08:02:34.142 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 150 (ObjNum 60 snap 5 seq_num 60) dirty exists 2024-03-20T08:02:34.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:34 smithi196 ceph-mon[31033]: osdmap e136: 8 total, 8 up, 8 in 2024-03-20T08:02:34.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:34 smithi196 ceph-mon[31033]: pgmap v180: 121 pgs: 121 active+clean; 96 MiB data, 387 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 14 op/s 2024-03-20T08:02:34.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:34 smithi118 ceph-mon[27391]: osdmap e136: 8 total, 8 up, 8 in 2024-03-20T08:02:34.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:34 smithi118 ceph-mon[27391]: pgmap v180: 121 pgs: 121 active+clean; 96 MiB data, 387 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 14 op/s 2024-03-20T08:02:34.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:34 smithi118 ceph-mon[38981]: osdmap e136: 8 total, 8 up, 8 in 2024-03-20T08:02:34.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:34 smithi118 ceph-mon[38981]: pgmap v180: 121 pgs: 121 active+clean; 96 MiB data, 387 MiB used, 715 GiB / 715 GiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 14 op/s 2024-03-20T08:02:35.140 INFO:tasks.rados.rados.0.smithi118.stdout:365: done (1 left) 2024-03-20T08:02:35.140 INFO:tasks.rados.rados.0.smithi118.stdout:366: done (0 left) 2024-03-20T08:02:35.140 INFO:tasks.rados.rados.0.smithi118.stdout:367: read oid 46 snap -1 2024-03-20T08:02:35.140 INFO:tasks.rados.rados.0.smithi118.stdout:367: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:02:35.140 INFO:tasks.rados.rados.0.smithi118.stdout:368: snap_create 2024-03-20T08:02:35.149 INFO:tasks.rados.rados.0.smithi118.stdout:367: expect (ObjNum 45 snap 0 seq_num 45) 2024-03-20T08:02:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:35 smithi118 ceph-mon[27391]: osdmap e137: 8 total, 8 up, 8 in 2024-03-20T08:02:35.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:35 smithi118 ceph-mon[38981]: osdmap e137: 8 total, 8 up, 8 in 2024-03-20T08:02:35.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:35 smithi196 ceph-mon[31033]: osdmap e137: 8 total, 8 up, 8 in 2024-03-20T08:02:36.011 INFO:tasks.rados.rados.0.smithi118.stdout:367: done (1 left) 2024-03-20T08:02:36.011 INFO:tasks.rados.rados.0.smithi118.stdout:368: done (0 left) 2024-03-20T08:02:36.011 INFO:tasks.rados.rados.0.smithi118.stdout:369: delete oid 32 current snap is 38 2024-03-20T08:02:36.016 INFO:tasks.rados.rados.0.smithi118.stdout:369: done (0 left) 2024-03-20T08:02:36.017 INFO:tasks.rados.rados.0.smithi118.stdout:370: read oid 29 snap 23 2024-03-20T08:02:36.017 INFO:tasks.rados.rados.0.smithi118.stdout:370: expect deleted 2024-03-20T08:02:36.017 INFO:tasks.rados.rados.0.smithi118.stdout:371: delete oid 1 current snap is 38 2024-03-20T08:02:36.022 INFO:tasks.rados.rados.0.smithi118.stdout:370: done (1 left) 2024-03-20T08:02:36.022 INFO:tasks.rados.rados.0.smithi118.stdout:371: done (0 left) 2024-03-20T08:02:36.022 INFO:tasks.rados.rados.0.smithi118.stdout:372: copy_from oid 11 from oid 50 current snap is 38 2024-03-20T08:02:36.022 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 38 2024-03-20T08:02:36.022 INFO:tasks.rados.rados.0.smithi118.stdout:373: seq_num 133 ranges {6594560=630784,7225344=663552,7888896=770048,8658944=172032} 2024-03-20T08:02:36.061 INFO:tasks.rados.rados.0.smithi118.stdout:373: writing smithi11881747-16 from 6594560 to 7225344 tid 1 2024-03-20T08:02:36.065 INFO:tasks.rados.rados.0.smithi118.stdout:373: writing smithi11881747-16 from 7225344 to 7888896 tid 2 2024-03-20T08:02:36.069 INFO:tasks.rados.rados.0.smithi118.stdout:373: writing smithi11881747-16 from 7888896 to 8658944 tid 3 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:373: writing smithi11881747-16 from 8658944 to 8830976 tid 4 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:374: rmattr oid 2 current snap is 38 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:374: done (2 left) 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:375: delete oid 35 current snap is 38 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:372: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:372: finishing copy_from to smithi11881747-11 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 51 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:373: finishing write tid 1 to smithi11881747-16 2024-03-20T08:02:36.071 INFO:tasks.rados.rados.0.smithi118.stdout:373: finishing write tid 2 to smithi11881747-16 2024-03-20T08:02:36.072 INFO:tasks.rados.rados.0.smithi118.stdout:373: finishing write tid 3 to smithi11881747-16 2024-03-20T08:02:36.073 INFO:tasks.rados.rados.0.smithi118.stdout:373: finishing write tid 4 to smithi11881747-16 2024-03-20T08:02:36.073 INFO:tasks.rados.rados.0.smithi118.stdout:373: finishing write tid 5 to smithi11881747-16 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:373: finishing write tid 6 to smithi11881747-16 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:373: oid 16 updating version 0 to 174 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:373: oid 16 version 174 is already newer than 173 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:373: oid 16 updating version 174 to 175 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:373: oid 16 version 175 is already newer than 172 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:373: oid 16 version 175 is already newer than 171 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 175 (ObjNum 133 snap 38 seq_num 133) dirty exists 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:373: left oid 16 (ObjNum 133 snap 38 seq_num 133) 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:372: done (2 left) 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:373: done (1 left) 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:375: done (0 left) 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:376: read oid 24 snap -1 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:376: expect (ObjNum 54 snap 4 seq_num 54) 2024-03-20T08:02:36.075 INFO:tasks.rados.rados.0.smithi118.stdout:377: snap_create 2024-03-20T08:02:36.082 INFO:tasks.rados.rados.0.smithi118.stdout:376: expect (ObjNum 54 snap 4 seq_num 54) 2024-03-20T08:02:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:36 smithi118 ceph-mon[27391]: osdmap e138: 8 total, 8 up, 8 in 2024-03-20T08:02:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:36 smithi118 ceph-mon[27391]: pgmap v183: 121 pgs: 121 active+clean; 102 MiB data, 401 MiB used, 715 GiB / 715 GiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 31 op/s 2024-03-20T08:02:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:36 smithi118 ceph-mon[27391]: osdmap e139: 8 total, 8 up, 8 in 2024-03-20T08:02:36.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:36 smithi118 ceph-mon[38981]: osdmap e138: 8 total, 8 up, 8 in 2024-03-20T08:02:36.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:36 smithi118 ceph-mon[38981]: pgmap v183: 121 pgs: 121 active+clean; 102 MiB data, 401 MiB used, 715 GiB / 715 GiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 31 op/s 2024-03-20T08:02:36.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:36 smithi118 ceph-mon[38981]: osdmap e139: 8 total, 8 up, 8 in 2024-03-20T08:02:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:36 smithi196 ceph-mon[31033]: osdmap e138: 8 total, 8 up, 8 in 2024-03-20T08:02:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:36 smithi196 ceph-mon[31033]: pgmap v183: 121 pgs: 121 active+clean; 102 MiB data, 401 MiB used, 715 GiB / 715 GiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 31 op/s 2024-03-20T08:02:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:36 smithi196 ceph-mon[31033]: osdmap e139: 8 total, 8 up, 8 in 2024-03-20T08:02:37.018 INFO:tasks.rados.rados.0.smithi118.stdout:376: done (1 left) 2024-03-20T08:02:37.018 INFO:tasks.rados.rados.0.smithi118.stdout:377: done (0 left) 2024-03-20T08:02:37.018 INFO:tasks.rados.rados.0.smithi118.stdout:378: rmattr oid 6 current snap is 39 2024-03-20T08:02:37.018 INFO:tasks.rados.rados.0.smithi118.stdout:379: rollback oid 3 current snap is 39 2024-03-20T08:02:37.018 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 3 to 32 2024-03-20T08:02:37.018 INFO:tasks.rados.rados.0.smithi118.stdout:380: rollback oid 16 current snap is 39 2024-03-20T08:02:37.018 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 29 2024-03-20T08:02:37.018 INFO:tasks.rados.rados.0.smithi118.stdout:381: snap_create 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:380: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:379: finishing rollback tid 0 to smithi11881747-3 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:380: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:380: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 179 (ObjNum 110 snap 27 seq_num 110) dirty exists 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:379: finishing rollback tid 1 to smithi11881747-3 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:379: finishing rollback tid 2 to smithi11881747-3 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 73 (ObjNum 86 snap 15 seq_num 86) dirty exists 2024-03-20T08:02:37.024 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 71 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-03-20T08:02:37.601 INFO:tasks.rados.rados.0.smithi118.stdout:378: done (3 left) 2024-03-20T08:02:37.602 INFO:tasks.rados.rados.0.smithi118.stdout:379: done (2 left) 2024-03-20T08:02:37.602 INFO:tasks.rados.rados.0.smithi118.stdout:380: done (1 left) 2024-03-20T08:02:37.602 INFO:tasks.rados.rados.0.smithi118.stdout:381: done (0 left) 2024-03-20T08:02:37.602 INFO:tasks.rados.rados.0.smithi118.stdout:382: rollback oid 39 current snap is 40 2024-03-20T08:02:37.602 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 28 2024-03-20T08:02:37.602 INFO:tasks.rados.rados.0.smithi118.stdout:383: rollback oid 21 current snap is 40 2024-03-20T08:02:37.602 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 22 2024-03-20T08:02:37.602 INFO:tasks.rados.rados.0.smithi118.stdout:384: delete oid 24 current snap is 40 2024-03-20T08:02:37.604 INFO:tasks.rados.rados.0.smithi118.stdout:384: done (2 left) 2024-03-20T08:02:37.604 INFO:tasks.rados.rados.0.smithi118.stdout:385: delete oid 3 current snap is 40 2024-03-20T08:02:37.607 INFO:tasks.rados.rados.0.smithi118.stdout:382: finishing rollback tid 0 to smithi11881747-39 2024-03-20T08:02:37.607 INFO:tasks.rados.rados.0.smithi118.stdout:385: done (2 left) 2024-03-20T08:02:37.607 INFO:tasks.rados.rados.0.smithi118.stdout:386: read oid 1 snap 32 2024-03-20T08:02:37.607 INFO:tasks.rados.rados.0.smithi118.stdout:386: expect (ObjNum 106 snap 23 seq_num 106) 2024-03-20T08:02:37.607 INFO:tasks.rados.rados.0.smithi118.stdout:387: setattr oid 2 current snap is 40 2024-03-20T08:02:37.608 INFO:tasks.rados.rados.0.smithi118.stdout:382: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:02:37.608 INFO:tasks.rados.rados.0.smithi118.stdout:382: finishing rollback tid 2 to smithi11881747-39 2024-03-20T08:02:37.608 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 154 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-03-20T08:02:37.610 INFO:tasks.rados.rados.0.smithi118.stdout:382: done (3 left) 2024-03-20T08:02:37.610 INFO:tasks.rados.rados.0.smithi118.stdout:388: setattr oid 3 current snap is 40 2024-03-20T08:02:37.611 INFO:tasks.rados.rados.0.smithi118.stdout:383: finishing rollback tid 0 to smithi11881747-21 2024-03-20T08:02:37.612 INFO:tasks.rados.rados.0.smithi118.stdout:383: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:02:37.612 INFO:tasks.rados.rados.0.smithi118.stdout:383: finishing rollback tid 2 to smithi11881747-21 2024-03-20T08:02:37.612 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 143 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-03-20T08:02:37.612 INFO:tasks.rados.rados.0.smithi118.stdout:383: done (3 left) 2024-03-20T08:02:37.612 INFO:tasks.rados.rados.0.smithi118.stdout:389: delete oid 34 current snap is 40 2024-03-20T08:02:37.616 INFO:tasks.rados.rados.0.smithi118.stdout:389: done (3 left) 2024-03-20T08:02:37.617 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 40 2024-03-20T08:02:37.617 INFO:tasks.rados.rados.0.smithi118.stdout:390: seq_num 138 ranges {0=720896,720896=540672,1261568=557056,1818624=507904} 2024-03-20T08:02:37.621 INFO:tasks.rados.rados.0.smithi118.stdout:390: writing smithi11881747-4 from 0 to 720896 tid 1 2024-03-20T08:02:37.623 INFO:tasks.rados.rados.0.smithi118.stdout:390: writing smithi11881747-4 from 720896 to 1261568 tid 2 2024-03-20T08:02:37.627 INFO:tasks.rados.rados.0.smithi118.stdout:390: writing smithi11881747-4 from 1261568 to 1818624 tid 3 2024-03-20T08:02:37.630 INFO:tasks.rados.rados.0.smithi118.stdout:390: writing smithi11881747-4 from 1818624 to 2326528 tid 4 2024-03-20T08:02:37.630 INFO:tasks.rados.rados.0.smithi118.stdout:391: snap_remove snap 22 2024-03-20T08:02:37.636 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:37] ENGINE Bus STOPPING 2024-03-20T08:02:37.636 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:02:37.636 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:37] ENGINE Bus STOPPED 2024-03-20T08:02:37.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:37] ENGINE Bus STARTING 2024-03-20T08:02:37.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f799e18b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f799e18b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f799e18b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f799e18b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f799e18b700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f79a1191700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.890 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f79a1191700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.890 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f79a1191700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.890 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f79a1191700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.890 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:02:37.754+0000 7f79a1191700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:02:37.890 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:37] ENGINE Serving on http://:::9283 2024-03-20T08:02:37.890 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:02:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:02:37] ENGINE Bus STARTED 2024-03-20T08:02:37.926 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:37.600+0000 7f802af27700 -1 mgr handle_mgr_map I was active but no longer am 2024-03-20T08:02:37.926 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ignoring --setuser ceph since I am not root 2024-03-20T08:02:37.926 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ignoring --setgroup ceph since I am not root 2024-03-20T08:02:38.283 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:38.187+0000 7f3bdfc18200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T08:02:38.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: osdmap e140: 8 total, 8 up, 8 in 2024-03-20T08:02:38.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:38.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: osdmap e141: 8 total, 8 up, 8 in 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: mgrmap e24: x(active, starting, since 0.00898801s) 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:02:38.284 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:02:38.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: Manager daemon x is now available 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: osdmap e140: 8 total, 8 up, 8 in 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-20T08:02:38.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: osdmap e141: 8 total, 8 up, 8 in 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: mgrmap e24: x(active, starting, since 0.00898801s) 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:02:38.287 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: Manager daemon x is now available 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:38.288 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-20T08:02:38.289 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-20T08:02:38.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: osdmap e140: 8 total, 8 up, 8 in 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 172.21.15.118:0/3789105115' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: osdmap e141: 8 total, 8 up, 8 in 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:02:38.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.14583 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: mgrmap e24: x(active, starting, since 0.00898801s) 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:02:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: Manager daemon x is now available 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-20T08:02:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:38 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-20T08:02:38.605 INFO:tasks.rados.rados.0.smithi118.stdout:391: done (4 left) 2024-03-20T08:02:38.605 INFO:tasks.rados.rados.0.smithi118.stdout:392: delete oid 6 current snap is 40 2024-03-20T08:02:38.605 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 72 (ObjNum 33 snap 0 seq_num 322239872) dirty exists 2024-03-20T08:02:38.605 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 76 (ObjNum 65 snap 0 seq_num 322181472) dirty exists 2024-03-20T08:02:38.606 INFO:tasks.rados.rados.0.smithi118.stdout:390: finishing write tid 1 to smithi11881747-4 2024-03-20T08:02:38.606 INFO:tasks.rados.rados.0.smithi118.stdout:386: expect (ObjNum 106 snap 23 seq_num 106) 2024-03-20T08:02:38.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:38.283+0000 7f3bdfc18200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T08:02:38.722 INFO:tasks.rados.rados.0.smithi118.stdout:390: finishing write tid 2 to smithi11881747-4 2024-03-20T08:02:38.722 INFO:tasks.rados.rados.0.smithi118.stdout:390: finishing write tid 3 to smithi11881747-4 2024-03-20T08:02:38.722 INFO:tasks.rados.rados.0.smithi118.stdout:390: finishing write tid 4 to smithi11881747-4 2024-03-20T08:02:38.722 INFO:tasks.rados.rados.0.smithi118.stdout:390: finishing write tid 5 to smithi11881747-4 2024-03-20T08:02:38.722 INFO:tasks.rados.rados.0.smithi118.stdout:390: finishing write tid 6 to smithi11881747-4 2024-03-20T08:02:38.722 INFO:tasks.rados.rados.0.smithi118.stdout:390: oid 4 updating version 0 to 33 2024-03-20T08:02:38.722 INFO:tasks.rados.rados.0.smithi118.stdout:390: oid 4 updating version 33 to 35 2024-03-20T08:02:38.722 INFO:tasks.rados.rados.0.smithi118.stdout:390: oid 4 version 35 is already newer than 32 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:390: oid 4 version 35 is already newer than 34 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:390: oid 4 version 35 is already newer than 31 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 35 (ObjNum 138 snap 40 seq_num 138) dirty exists 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:390: left oid 4 (ObjNum 138 snap 40 seq_num 138) 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:386: done (4 left) 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:387: done (3 left) 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:388: done (2 left) 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:390: done (1 left) 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:392: done (0 left) 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:393: rollback oid 9 current snap is 40 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 9 to 31 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:394: read oid 11 snap 35 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:394: expect deleted 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 40 2024-03-20T08:02:38.723 INFO:tasks.rados.rados.0.smithi118.stdout:395: seq_num 140 ranges {778240=753664,1531904=704512,2236416=524288,2760704=376832} 2024-03-20T08:02:38.730 INFO:tasks.rados.rados.0.smithi118.stdout:395: writing smithi11881747-23 from 778240 to 1531904 tid 1 2024-03-20T08:02:38.734 INFO:tasks.rados.rados.0.smithi118.stdout:395: writing smithi11881747-23 from 1531904 to 2236416 tid 2 2024-03-20T08:02:38.738 INFO:tasks.rados.rados.0.smithi118.stdout:395: writing smithi11881747-23 from 2236416 to 2760704 tid 3 2024-03-20T08:02:38.739 INFO:tasks.rados.rados.0.smithi118.stdout:395: writing smithi11881747-23 from 2760704 to 3137536 tid 4 2024-03-20T08:02:38.739 INFO:tasks.rados.rados.0.smithi118.stdout:396: delete oid 7 current snap is 40 2024-03-20T08:02:38.740 INFO:tasks.rados.rados.0.smithi118.stdout:393: finishing rollback tid 0 to smithi11881747-9 2024-03-20T08:02:38.740 INFO:tasks.rados.rados.0.smithi118.stdout:393: finishing rollback tid 1 to smithi11881747-9 2024-03-20T08:02:38.740 INFO:tasks.rados.rados.0.smithi118.stdout:393: finishing rollback tid 2 to smithi11881747-9 2024-03-20T08:02:38.740 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 133 (ObjNum 89 snap 18 seq_num 89) dirty exists 2024-03-20T08:02:38.740 INFO:tasks.rados.rados.0.smithi118.stdout:395: finishing write tid 1 to smithi11881747-23 2024-03-20T08:02:38.740 INFO:tasks.rados.rados.0.smithi118.stdout:395: finishing write tid 2 to smithi11881747-23 2024-03-20T08:02:38.740 INFO:tasks.rados.rados.0.smithi118.stdout:395: finishing write tid 3 to smithi11881747-23 2024-03-20T08:02:38.741 INFO:tasks.rados.rados.0.smithi118.stdout:393: done (3 left) 2024-03-20T08:02:38.741 INFO:tasks.rados.rados.0.smithi118.stdout:394: done (2 left) 2024-03-20T08:02:38.741 INFO:tasks.rados.rados.0.smithi118.stdout:396: done (1 left) 2024-03-20T08:02:38.741 INFO:tasks.rados.rados.0.smithi118.stdout:397: read oid 26 snap 23 2024-03-20T08:02:38.741 INFO:tasks.rados.rados.0.smithi118.stdout:397: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:02:38.741 INFO:tasks.rados.rados.0.smithi118.stdout:398: read oid 47 snap -1 2024-03-20T08:02:38.741 INFO:tasks.rados.rados.0.smithi118.stdout:398: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:38.742 INFO:tasks.rados.rados.0.smithi118.stdout:399: read oid 41 snap -1 2024-03-20T08:02:38.742 INFO:tasks.rados.rados.0.smithi118.stdout:399: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:38.742 INFO:tasks.rados.rados.0.smithi118.stdout:400: rollback oid 2 current snap is 40 2024-03-20T08:02:38.742 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 28 2024-03-20T08:02:38.742 INFO:tasks.rados.rados.0.smithi118.stdout:401: snap_create 2024-03-20T08:02:38.742 INFO:tasks.rados.rados.0.smithi118.stdout:395: finishing write tid 4 to smithi11881747-23 2024-03-20T08:02:38.745 INFO:tasks.rados.rados.0.smithi118.stdout:395: finishing write tid 5 to smithi11881747-23 2024-03-20T08:02:38.746 INFO:tasks.rados.rados.0.smithi118.stdout:400: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:02:38.746 INFO:tasks.rados.rados.0.smithi118.stdout:400: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:02:38.746 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 82 (ObjNum 33 snap 0 seq_num 1267838831) dirty dne 2024-03-20T08:02:38.746 INFO:tasks.rados.rados.0.smithi118.stdout:395: finishing write tid 6 to smithi11881747-23 2024-03-20T08:02:38.747 INFO:tasks.rados.rados.0.smithi118.stdout:395: oid 23 updating version 0 to 190 2024-03-20T08:02:38.747 INFO:tasks.rados.rados.0.smithi118.stdout:395: oid 23 updating version 190 to 191 2024-03-20T08:02:38.747 INFO:tasks.rados.rados.0.smithi118.stdout:395: oid 23 version 191 is already newer than 189 2024-03-20T08:02:38.747 INFO:tasks.rados.rados.0.smithi118.stdout:395: oid 23 version 191 is already newer than 187 2024-03-20T08:02:38.747 INFO:tasks.rados.rados.0.smithi118.stdout:395: oid 23 version 191 is already newer than 188 2024-03-20T08:02:38.747 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 191 (ObjNum 140 snap 40 seq_num 140) dirty exists 2024-03-20T08:02:38.747 INFO:tasks.rados.rados.0.smithi118.stdout:395: left oid 23 (ObjNum 140 snap 40 seq_num 140) 2024-03-20T08:02:38.747 INFO:tasks.rados.rados.0.smithi118.stdout:397: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:02:38.765 INFO:tasks.rados.rados.0.smithi118.stdout:398: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:399: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:39.261 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:38.941+0000 7f3bdfc18200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T08:02:39.261 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:39.030+0000 7f3bdfc18200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T08:02:39.261 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:39.168+0000 7f3bdfc18200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T08:02:39.261 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:39.262+0000 7f3bdfc18200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T08:02:39.597 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:39.344+0000 7f3bdfc18200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T08:02:39.608 INFO:tasks.rados.rados.0.smithi118.stdout:395: done (5 left) 2024-03-20T08:02:39.609 INFO:tasks.rados.rados.0.smithi118.stdout:397: done (4 left) 2024-03-20T08:02:39.609 INFO:tasks.rados.rados.0.smithi118.stdout:398: done (3 left) 2024-03-20T08:02:39.609 INFO:tasks.rados.rados.0.smithi118.stdout:399: done (2 left) 2024-03-20T08:02:39.609 INFO:tasks.rados.rados.0.smithi118.stdout:400: done (1 left) 2024-03-20T08:02:39.609 INFO:tasks.rados.rados.0.smithi118.stdout:401: done (0 left) 2024-03-20T08:02:39.609 INFO:tasks.rados.rados.0.smithi118.stdout:402: read oid 14 snap -1 2024-03-20T08:02:39.609 INFO:tasks.rados.rados.0.smithi118.stdout:402: expect (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:02:39.609 INFO:tasks.rados.rados.0.smithi118.stdout:403: delete oid 12 current snap is 41 2024-03-20T08:02:39.610 INFO:tasks.rados.rados.0.smithi118.stdout:403: done (1 left) 2024-03-20T08:02:39.610 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 41 2024-03-20T08:02:39.610 INFO:tasks.rados.rados.0.smithi118.stdout:404: seq_num 143 ranges {778240=802816,1581056=761856,2342912=745472,3088384=98304} 2024-03-20T08:02:39.619 INFO:tasks.rados.rados.0.smithi118.stdout:404: writing smithi11881747-47 from 778240 to 1581056 tid 1 2024-03-20T08:02:39.624 INFO:tasks.rados.rados.0.smithi118.stdout:404: writing smithi11881747-47 from 1581056 to 2342912 tid 2 2024-03-20T08:02:39.628 INFO:tasks.rados.rados.0.smithi118.stdout:404: writing smithi11881747-47 from 2342912 to 3088384 tid 3 2024-03-20T08:02:39.629 INFO:tasks.rados.rados.0.smithi118.stdout:404: writing smithi11881747-47 from 3088384 to 3186688 tid 4 2024-03-20T08:02:39.629 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 41 2024-03-20T08:02:39.629 INFO:tasks.rados.rados.0.smithi118.stdout:402: expect (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:404: finishing write tid 1 to smithi11881747-47 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:405: seq_num 144 ranges {0=16384} 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:405: writing smithi11881747-44 from 0 to 16384 tid 1 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:402: done (2 left) 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 41 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:404: finishing write tid 2 to smithi11881747-47 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:404: finishing write tid 3 to smithi11881747-47 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:404: finishing write tid 4 to smithi11881747-47 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:406: seq_num 145 ranges {0=32768} 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:406: writing smithi11881747-7 from 0 to 32768 tid 1 2024-03-20T08:02:39.676 INFO:tasks.rados.rados.0.smithi118.stdout:407: snap_remove snap 28 2024-03-20T08:02:39.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: Deploying cephadm binary to smithi196 2024-03-20T08:02:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: Deploying cephadm binary to smithi118 2024-03-20T08:02:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: [20/Mar/2024:08:02:38] ENGINE Bus STARTING 2024-03-20T08:02:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: mgrmap e25: x(active, since 1.00821s) 2024-03-20T08:02:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: osdmap e142: 8 total, 8 up, 8 in 2024-03-20T08:02:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: pgmap v4: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 18 GiB/s rd, 14 GiB/s wr, 158.06k op/s; 0 B/s, 138 objects/s recovering 2024-03-20T08:02:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: [20/Mar/2024:08:02:38] ENGINE Serving on http://172.21.15.196:8765 2024-03-20T08:02:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: [20/Mar/2024:08:02:38] ENGINE Serving on https://172.21.15.196:7150 2024-03-20T08:02:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:39 smithi196 ceph-mon[31033]: [20/Mar/2024:08:02:38] ENGINE Bus STARTED 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: Deploying cephadm binary to smithi196 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: Deploying cephadm binary to smithi118 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: [20/Mar/2024:08:02:38] ENGINE Bus STARTING 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: mgrmap e25: x(active, since 1.00821s) 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: osdmap e142: 8 total, 8 up, 8 in 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: pgmap v4: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 18 GiB/s rd, 14 GiB/s wr, 158.06k op/s; 0 B/s, 138 objects/s recovering 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: [20/Mar/2024:08:02:38] ENGINE Serving on http://172.21.15.196:8765 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: [20/Mar/2024:08:02:38] ENGINE Serving on https://172.21.15.196:7150 2024-03-20T08:02:39.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[27391]: [20/Mar/2024:08:02:38] ENGINE Bus STARTED 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:39.605+0000 7f3bdfc18200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:39.725+0000 7f3bdfc18200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:39.812+0000 7f3bdfc18200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: Deploying cephadm binary to smithi196 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: Deploying cephadm binary to smithi118 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: [20/Mar/2024:08:02:38] ENGINE Bus STARTING 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: mgrmap e25: x(active, since 1.00821s) 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: osdmap e142: 8 total, 8 up, 8 in 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: pgmap v4: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 18 GiB/s rd, 14 GiB/s wr, 158.06k op/s; 0 B/s, 138 objects/s recovering 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: [20/Mar/2024:08:02:38] ENGINE Serving on http://172.21.15.196:8765 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: [20/Mar/2024:08:02:38] ENGINE Serving on https://172.21.15.196:7150 2024-03-20T08:02:39.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-mon[38981]: [20/Mar/2024:08:02:38] ENGINE Bus STARTED 2024-03-20T08:02:40.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:39.903+0000 7f3bdfc18200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T08:02:40.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:40.076+0000 7f3bdfc18200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T08:02:40.616 INFO:tasks.rados.rados.0.smithi118.stdout:407: done (3 left) 2024-03-20T08:02:40.616 INFO:tasks.rados.rados.0.smithi118.stdout:408: rollback oid 5 current snap is 41 2024-03-20T08:02:40.616 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 37 2024-03-20T08:02:40.616 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 41 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:409: seq_num 146 ranges {0=49152} 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:409: writing smithi11881747-10 from 0 to 49152 tid 1 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:410: copy_from oid 42 from oid 13 current snap is 41 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:404: finishing write tid 5 to smithi11881747-47 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:404: finishing write tid 6 to smithi11881747-47 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:404: oid 47 updating version 0 to 196 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:404: oid 47 version 196 is already newer than 194 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:404: oid 47 updating version 196 to 197 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:404: oid 47 version 197 is already newer than 193 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:404: oid 47 version 197 is already newer than 195 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 197 (ObjNum 143 snap 41 seq_num 143) dirty exists 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:404: left oid 47 (ObjNum 143 snap 41 seq_num 143) 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:405: finishing write tid 1 to smithi11881747-44 2024-03-20T08:02:40.617 INFO:tasks.rados.rados.0.smithi118.stdout:406: finishing write tid 1 to smithi11881747-7 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:405: finishing write tid 2 to smithi11881747-44 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:406: finishing write tid 2 to smithi11881747-7 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:405: finishing write tid 3 to smithi11881747-44 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:405: oid 44 updating version 0 to 87 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:405: oid 44 version 87 is already newer than 86 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 87 (ObjNum 144 snap 41 seq_num 144) dirty exists 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:405: left oid 44 (ObjNum 144 snap 41 seq_num 144) 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:406: finishing write tid 3 to smithi11881747-7 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:406: oid 7 updating version 0 to 26 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:406: oid 7 version 26 is already newer than 25 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 26 (ObjNum 145 snap 41 seq_num 145) dirty exists 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:406: left oid 7 (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:404: done (5 left) 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:405: done (4 left) 2024-03-20T08:02:40.618 INFO:tasks.rados.rados.0.smithi118.stdout:406: done (3 left) 2024-03-20T08:02:40.619 INFO:tasks.rados.rados.0.smithi118.stdout:411: snap_create 2024-03-20T08:02:40.626 INFO:tasks.rados.rados.0.smithi118.stdout:408: finishing rollback tid 0 to smithi11881747-5 2024-03-20T08:02:40.626 INFO:tasks.rados.rados.0.smithi118.stdout:408: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:02:40.626 INFO:tasks.rados.rados.0.smithi118.stdout:409: finishing write tid 1 to smithi11881747-10 2024-03-20T08:02:40.626 INFO:tasks.rados.rados.0.smithi118.stdout:409: finishing write tid 2 to smithi11881747-10 2024-03-20T08:02:40.626 INFO:tasks.rados.rados.0.smithi118.stdout:408: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:02:40.626 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 141 (ObjNum 118 snap 31 seq_num 118) dirty exists 2024-03-20T08:02:40.627 INFO:tasks.rados.rados.0.smithi118.stdout:409: finishing write tid 3 to smithi11881747-10 2024-03-20T08:02:40.627 INFO:tasks.rados.rados.0.smithi118.stdout:409: oid 10 updating version 0 to 187 2024-03-20T08:02:40.627 INFO:tasks.rados.rados.0.smithi118.stdout:409: oid 10 updating version 187 to 188 2024-03-20T08:02:40.628 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 188 (ObjNum 146 snap 41 seq_num 146) dirty exists 2024-03-20T08:02:40.628 INFO:tasks.rados.rados.0.smithi118.stdout:409: left oid 10 (ObjNum 146 snap 41 seq_num 146) 2024-03-20T08:02:40.637 INFO:tasks.rados.rados.0.smithi118.stdout:410: finishing copy_from racing read to smithi11881747-42 2024-03-20T08:02:40.637 INFO:tasks.rados.rados.0.smithi118.stdout:410: finishing copy_from to smithi11881747-42 2024-03-20T08:02:40.638 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 122 (ObjNum 126 snap 34 seq_num 126) dirty exists 2024-03-20T08:02:40.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:40 smithi196 ceph-mon[31033]: osdmap e143: 8 total, 8 up, 8 in 2024-03-20T08:02:40.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:40 smithi196 ceph-mon[31033]: pgmap v6: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 135 MiB/s rd, 102 MiB/s wr, 1.15k op/s; 0 B/s, 1 objects/s recovering 2024-03-20T08:02:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:40 smithi118 ceph-mon[27391]: osdmap e143: 8 total, 8 up, 8 in 2024-03-20T08:02:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:40 smithi118 ceph-mon[27391]: pgmap v6: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 135 MiB/s rd, 102 MiB/s wr, 1.15k op/s; 0 B/s, 1 objects/s recovering 2024-03-20T08:02:40.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:40 smithi118 ceph-mon[38981]: osdmap e143: 8 total, 8 up, 8 in 2024-03-20T08:02:40.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:40 smithi118 ceph-mon[38981]: pgmap v6: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 135 MiB/s rd, 102 MiB/s wr, 1.15k op/s; 0 B/s, 1 objects/s recovering 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:408: done (3 left) 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:409: done (2 left) 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:410: done (1 left) 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:411: done (0 left) 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 42 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:412: seq_num 147 ranges {0=65536} 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:412: writing smithi11881747-12 from 0 to 65536 tid 1 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:413: read oid 9 snap -1 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:413: expect (ObjNum 89 snap 18 seq_num 89) 2024-03-20T08:02:41.013 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 42 2024-03-20T08:02:41.014 INFO:tasks.rados.rados.0.smithi118.stdout:414: seq_num 148 ranges {425984=81920} 2024-03-20T08:02:41.015 INFO:tasks.rados.rados.0.smithi118.stdout:414: writing smithi11881747-26 from 425984 to 507904 tid 1 2024-03-20T08:02:41.015 INFO:tasks.rados.rados.0.smithi118.stdout:415: snap_create 2024-03-20T08:02:41.020 INFO:tasks.rados.rados.0.smithi118.stdout:412: finishing write tid 1 to smithi11881747-12 2024-03-20T08:02:41.021 INFO:tasks.rados.rados.0.smithi118.stdout:414: finishing write tid 1 to smithi11881747-26 2024-03-20T08:02:41.021 INFO:tasks.rados.rados.0.smithi118.stdout:414: finishing write tid 2 to smithi11881747-26 2024-03-20T08:02:41.021 INFO:tasks.rados.rados.0.smithi118.stdout:412: finishing write tid 2 to smithi11881747-12 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:412: finishing write tid 3 to smithi11881747-12 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:412: oid 12 updating version 0 to 29 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:412: oid 12 updating version 29 to 30 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 30 (ObjNum 147 snap 42 seq_num 147) dirty exists 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:412: left oid 12 (ObjNum 147 snap 42 seq_num 147) 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:414: finishing write tid 3 to smithi11881747-26 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:414: oid 26 updating version 0 to 92 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:414: oid 26 version 92 is already newer than 91 2024-03-20T08:02:41.022 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 92 (ObjNum 148 snap 42 seq_num 148) dirty exists 2024-03-20T08:02:41.023 INFO:tasks.rados.rados.0.smithi118.stdout:414: left oid 26 (ObjNum 148 snap 42 seq_num 148) 2024-03-20T08:02:41.025 INFO:tasks.rados.rados.0.smithi118.stdout:413: expect (ObjNum 89 snap 18 seq_num 89) 2024-03-20T08:02:41.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:41.045+0000 7f3bdfc18200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T08:02:41.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:41.177+0000 7f3bdfc18200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T08:02:41.799 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-mon[27391]: mgrmap e26: x(active, since 3s) 2024-03-20T08:02:41.799 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-mon[27391]: osdmap e144: 8 total, 8 up, 8 in 2024-03-20T08:02:41.799 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-mon[27391]: osdmap e145: 8 total, 8 up, 8 in 2024-03-20T08:02:41.799 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-mon[38981]: mgrmap e26: x(active, since 3s) 2024-03-20T08:02:41.800 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-mon[38981]: osdmap e144: 8 total, 8 up, 8 in 2024-03-20T08:02:41.800 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-mon[38981]: osdmap e145: 8 total, 8 up, 8 in 2024-03-20T08:02:41.800 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:41.509+0000 7f3bdfc18200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T08:02:41.800 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:41.600+0000 7f3bdfc18200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T08:02:41.801 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:41.800+0000 7f3bdfc18200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:02:41.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:41 smithi196 ceph-mon[31033]: mgrmap e26: x(active, since 3s) 2024-03-20T08:02:41.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:41 smithi196 ceph-mon[31033]: osdmap e144: 8 total, 8 up, 8 in 2024-03-20T08:02:41.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:41 smithi196 ceph-mon[31033]: osdmap e145: 8 total, 8 up, 8 in 2024-03-20T08:02:42.018 INFO:tasks.rados.rados.0.smithi118.stdout:412: done (3 left) 2024-03-20T08:02:42.018 INFO:tasks.rados.rados.0.smithi118.stdout:413: done (2 left) 2024-03-20T08:02:42.018 INFO:tasks.rados.rados.0.smithi118.stdout:414: done (1 left) 2024-03-20T08:02:42.018 INFO:tasks.rados.rados.0.smithi118.stdout:415: done (0 left) 2024-03-20T08:02:42.018 INFO:tasks.rados.rados.0.smithi118.stdout:416: copy_from oid 15 from oid 40 current snap is 43 2024-03-20T08:02:42.018 INFO:tasks.rados.rados.0.smithi118.stdout:417: read oid 7 snap 42 2024-03-20T08:02:42.019 INFO:tasks.rados.rados.0.smithi118.stdout:417: expect (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:02:42.019 INFO:tasks.rados.rados.0.smithi118.stdout:418: rollback oid 31 current snap is 43 2024-03-20T08:02:42.019 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 35 2024-03-20T08:02:42.019 INFO:tasks.rados.rados.0.smithi118.stdout:419: delete oid 35 current snap is 43 2024-03-20T08:02:42.020 INFO:tasks.rados.rados.0.smithi118.stdout:417: expect (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:02:42.023 INFO:tasks.rados.rados.0.smithi118.stdout:417: done (3 left) 2024-03-20T08:02:42.023 INFO:tasks.rados.rados.0.smithi118.stdout:419: done (2 left) 2024-03-20T08:02:42.023 INFO:tasks.rados.rados.0.smithi118.stdout:420: read oid 8 snap -1 2024-03-20T08:02:42.023 INFO:tasks.rados.rados.0.smithi118.stdout:420: expect (ObjNum 122 snap 32 seq_num 122) 2024-03-20T08:02:42.023 INFO:tasks.rados.rados.0.smithi118.stdout:421: rollback oid 5 current snap is 43 2024-03-20T08:02:42.023 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 35 2024-03-20T08:02:42.023 INFO:tasks.rados.rados.0.smithi118.stdout:422: delete oid 37 current snap is 43 2024-03-20T08:02:42.024 INFO:tasks.rados.rados.0.smithi118.stdout:416: finishing copy_from to smithi11881747-15 2024-03-20T08:02:42.024 INFO:tasks.rados.rados.0.smithi118.stdout:416: got expected ENOENT (src dne) 2024-03-20T08:02:42.024 INFO:tasks.rados.rados.0.smithi118.stdout:422: done (4 left) 2024-03-20T08:02:42.024 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 43 2024-03-20T08:02:42.024 INFO:tasks.rados.rados.0.smithi118.stdout:423: seq_num 151 ranges {1753088=131072} 2024-03-20T08:02:42.035 INFO:tasks.rados.rados.0.smithi118.stdout:423: writing smithi11881747-14 from 1753088 to 1884160 tid 1 2024-03-20T08:02:42.035 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 43 2024-03-20T08:02:42.035 INFO:tasks.rados.rados.0.smithi118.stdout:418: finishing rollback tid 0 to smithi11881747-31 2024-03-20T08:02:42.035 INFO:tasks.rados.rados.0.smithi118.stdout:424: seq_num 152 ranges {0=147456} 2024-03-20T08:02:42.036 INFO:tasks.rados.rados.0.smithi118.stdout:424: writing smithi11881747-34 from 0 to 147456 tid 1 2024-03-20T08:02:42.036 INFO:tasks.rados.rados.0.smithi118.stdout:425: read oid 32 snap -1 2024-03-20T08:02:42.036 INFO:tasks.rados.rados.0.smithi118.stdout:425: expect deleted 2024-03-20T08:02:42.036 INFO:tasks.rados.rados.0.smithi118.stdout:418: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:02:42.036 INFO:tasks.rados.rados.0.smithi118.stdout:416: finishing copy_from racing read to smithi11881747-15 2024-03-20T08:02:42.036 INFO:tasks.rados.rados.0.smithi118.stdout:418: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 198 (ObjNum 123 snap 33 seq_num 123) dirty exists 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:421: finishing rollback tid 0 to smithi11881747-5 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:421: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:421: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 151 (ObjNum 118 snap 31 seq_num 118) dirty exists 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:426: read oid 49 snap -1 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:426: expect (ObjNum 119 snap 31 seq_num 119) 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:416: done (7 left) 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:418: done (6 left) 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:421: done (5 left) 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 43 2024-03-20T08:02:42.037 INFO:tasks.rados.rados.0.smithi118.stdout:427: seq_num 153 ranges {2801664=163840} 2024-03-20T08:02:42.053 INFO:tasks.rados.rados.0.smithi118.stdout:427: writing smithi11881747-9 from 2801664 to 2965504 tid 1 2024-03-20T08:02:42.053 INFO:tasks.rados.rados.0.smithi118.stdout:428: read oid 41 snap -1 2024-03-20T08:02:42.053 INFO:tasks.rados.rados.0.smithi118.stdout:428: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:42.053 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 43 2024-03-20T08:02:42.053 INFO:tasks.rados.rados.0.smithi118.stdout:429: seq_num 154 ranges {0=180224} 2024-03-20T08:02:42.054 INFO:tasks.rados.rados.0.smithi118.stdout:429: writing smithi11881747-19 from 0 to 180224 tid 1 2024-03-20T08:02:42.054 INFO:tasks.rados.rados.0.smithi118.stdout:430: snap_remove snap 35 2024-03-20T08:02:42.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:41.985+0000 7f3bdfc18200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T08:02:42.546 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:42.289+0000 7f3bdfc18200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:02:42.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:42 smithi196 ceph-mon[31033]: pgmap v9: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 45 MiB/s rd, 34 MiB/s wr, 381 op/s; 0 B/s, 0 objects/s recovering 2024-03-20T08:02:42.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:42 smithi196 ceph-mon[31033]: osdmap e146: 8 total, 8 up, 8 in 2024-03-20T08:02:42.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:42 smithi118 ceph-mon[27391]: pgmap v9: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 45 MiB/s rd, 34 MiB/s wr, 381 op/s; 0 B/s, 0 objects/s recovering 2024-03-20T08:02:42.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:42 smithi118 ceph-mon[27391]: osdmap e146: 8 total, 8 up, 8 in 2024-03-20T08:02:42.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:42.547+0000 7f3bdfc18200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T08:02:42.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:42.783+0000 7f3bdfc18200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T08:02:42.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:42 smithi118 ceph-mon[38981]: pgmap v9: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail; 45 MiB/s rd, 34 MiB/s wr, 381 op/s; 0 B/s, 0 objects/s recovering 2024-03-20T08:02:42.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:42 smithi118 ceph-mon[38981]: osdmap e146: 8 total, 8 up, 8 in 2024-03-20T08:02:43.024 INFO:tasks.rados.rados.0.smithi118.stdout:430: done (8 left) 2024-03-20T08:02:43.024 INFO:tasks.rados.rados.0.smithi118.stdout:431: rmattr oid 25 current snap is 43 2024-03-20T08:02:43.024 INFO:tasks.rados.rados.0.smithi118.stdout:431: done (8 left) 2024-03-20T08:02:43.024 INFO:tasks.rados.rados.0.smithi118.stdout:432: copy_from oid 20 from oid 27 current snap is 43 2024-03-20T08:02:43.024 INFO:tasks.rados.rados.0.smithi118.stdout:420: expect (ObjNum 122 snap 32 seq_num 122) 2024-03-20T08:02:43.088 INFO:tasks.rados.rados.0.smithi118.stdout:423: finishing write tid 1 to smithi11881747-14 2024-03-20T08:02:43.088 INFO:tasks.rados.rados.0.smithi118.stdout:423: finishing write tid 2 to smithi11881747-14 2024-03-20T08:02:43.088 INFO:tasks.rados.rados.0.smithi118.stdout:423: finishing write tid 3 to smithi11881747-14 2024-03-20T08:02:43.088 INFO:tasks.rados.rados.0.smithi118.stdout:423: oid 14 updating version 0 to 94 2024-03-20T08:02:43.088 INFO:tasks.rados.rados.0.smithi118.stdout:423: oid 14 version 94 is already newer than 93 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 94 (ObjNum 151 snap 43 seq_num 151) dirty exists 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:423: left oid 14 (ObjNum 151 snap 43 seq_num 151) 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:424: finishing write tid 1 to smithi11881747-34 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:420: done (8 left) 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:423: done (7 left) 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:425: done (6 left) 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:433: copy_from oid 24 from oid 32 current snap is 43 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:424: finishing write tid 2 to smithi11881747-34 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:424: finishing write tid 3 to smithi11881747-34 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:424: oid 34 updating version 0 to 62 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:424: oid 34 version 62 is already newer than 61 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 62 (ObjNum 152 snap 43 seq_num 152) dirty exists 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:424: left oid 34 (ObjNum 152 snap 43 seq_num 152) 2024-03-20T08:02:43.089 INFO:tasks.rados.rados.0.smithi118.stdout:426: expect (ObjNum 119 snap 31 seq_num 119) 2024-03-20T08:02:43.174 INFO:tasks.rados.rados.0.smithi118.stdout:424: done (6 left) 2024-03-20T08:02:43.174 INFO:tasks.rados.rados.0.smithi118.stdout:426: done (5 left) 2024-03-20T08:02:43.174 INFO:tasks.rados.rados.0.smithi118.stdout:434: delete oid 14 current snap is 43 2024-03-20T08:02:43.174 INFO:tasks.rados.rados.0.smithi118.stdout:427: finishing write tid 1 to smithi11881747-9 2024-03-20T08:02:43.174 INFO:tasks.rados.rados.0.smithi118.stdout:428: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:429: finishing write tid 1 to smithi11881747-19 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:427: finishing write tid 2 to smithi11881747-9 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:429: finishing write tid 2 to smithi11881747-19 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:427: finishing write tid 3 to smithi11881747-9 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:427: oid 9 updating version 0 to 154 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:427: oid 9 version 154 is already newer than 153 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 154 (ObjNum 153 snap 43 seq_num 153) dirty exists 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:427: left oid 9 (ObjNum 153 snap 43 seq_num 153) 2024-03-20T08:02:43.194 INFO:tasks.rados.rados.0.smithi118.stdout:429: finishing write tid 3 to smithi11881747-19 2024-03-20T08:02:43.195 INFO:tasks.rados.rados.0.smithi118.stdout:429: oid 19 updating version 0 to 50 2024-03-20T08:02:43.195 INFO:tasks.rados.rados.0.smithi118.stdout:429: oid 19 version 50 is already newer than 49 2024-03-20T08:02:43.195 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 50 (ObjNum 154 snap 43 seq_num 154) dirty exists 2024-03-20T08:02:43.195 INFO:tasks.rados.rados.0.smithi118.stdout:429: left oid 19 (ObjNum 154 snap 43 seq_num 154) 2024-03-20T08:02:43.195 INFO:tasks.rados.rados.0.smithi118.stdout:432: finishing copy_from to smithi11881747-20 2024-03-20T08:02:43.195 INFO:tasks.rados.rados.0.smithi118.stdout:432: got expected ENOENT (src dne) 2024-03-20T08:02:43.195 INFO:tasks.rados.rados.0.smithi118.stdout:432: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:02:43.195 INFO:tasks.rados.rados.0.smithi118.stdout:427: done (5 left) 2024-03-20T08:02:43.196 INFO:tasks.rados.rados.0.smithi118.stdout:428: done (4 left) 2024-03-20T08:02:43.196 INFO:tasks.rados.rados.0.smithi118.stdout:429: done (3 left) 2024-03-20T08:02:43.196 INFO:tasks.rados.rados.0.smithi118.stdout:432: done (2 left) 2024-03-20T08:02:43.196 INFO:tasks.rados.rados.0.smithi118.stdout:434: done (1 left) 2024-03-20T08:02:43.196 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 43 2024-03-20T08:02:43.196 INFO:tasks.rados.rados.0.smithi118.stdout:433: finishing copy_from to smithi11881747-24 2024-03-20T08:02:43.196 INFO:tasks.rados.rados.0.smithi118.stdout:433: got expected ENOENT (src dne) 2024-03-20T08:02:43.196 INFO:tasks.rados.rados.0.smithi118.stdout:435: seq_num 156 ranges {16384=221184} 2024-03-20T08:02:43.197 INFO:tasks.rados.rados.0.smithi118.stdout:435: writing smithi11881747-44 from 16384 to 237568 tid 1 2024-03-20T08:02:43.198 INFO:tasks.rados.rados.0.smithi118.stdout:436: delete oid 18 current snap is 43 2024-03-20T08:02:43.198 INFO:tasks.rados.rados.0.smithi118.stdout:433: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:02:43.198 INFO:tasks.rados.rados.0.smithi118.stdout:433: done (2 left) 2024-03-20T08:02:43.198 INFO:tasks.rados.rados.0.smithi118.stdout:436: done (1 left) 2024-03-20T08:02:43.198 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 43 2024-03-20T08:02:43.198 INFO:tasks.rados.rados.0.smithi118.stdout:437: seq_num 158 ranges {1024000=253952} 2024-03-20T08:02:43.204 INFO:tasks.rados.rados.0.smithi118.stdout:437: writing smithi11881747-15 from 1024000 to 1277952 tid 1 2024-03-20T08:02:43.204 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 43 2024-03-20T08:02:43.204 INFO:tasks.rados.rados.0.smithi118.stdout:435: finishing write tid 1 to smithi11881747-44 2024-03-20T08:02:43.204 INFO:tasks.rados.rados.0.smithi118.stdout:438: seq_num 159 ranges {155648=270336} 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:438: writing smithi11881747-28 from 155648 to 425984 tid 1 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:435: finishing write tid 2 to smithi11881747-44 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:435: finishing write tid 3 to smithi11881747-44 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:435: oid 44 updating version 0 to 99 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:435: oid 44 version 99 is already newer than 98 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 99 (ObjNum 156 snap 43 seq_num 156) dirty exists 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:435: left oid 44 (ObjNum 156 snap 43 seq_num 156) 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:435: done (2 left) 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 43 2024-03-20T08:02:43.207 INFO:tasks.rados.rados.0.smithi118.stdout:439: seq_num 160 ranges {65536=286720} 2024-03-20T08:02:43.209 INFO:tasks.rados.rados.0.smithi118.stdout:439: writing smithi11881747-12 from 65536 to 352256 tid 1 2024-03-20T08:02:43.209 INFO:tasks.rados.rados.0.smithi118.stdout:440: read oid 34 snap -1 2024-03-20T08:02:43.209 INFO:tasks.rados.rados.0.smithi118.stdout:440: expect (ObjNum 152 snap 43 seq_num 152) 2024-03-20T08:02:43.209 INFO:tasks.rados.rados.0.smithi118.stdout:441: snap_remove snap 30 2024-03-20T08:02:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:43.663+0000 7f3bdfc18200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:441: done (4 left) 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:442: rollback oid 1 current snap is 43 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 1 to 41 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:437: finishing write tid 1 to smithi11881747-15 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:437: finishing write tid 2 to smithi11881747-15 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:437: finishing write tid 3 to smithi11881747-15 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:437: oid 15 updating version 0 to 53 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:437: oid 15 version 53 is already newer than 52 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 53 (ObjNum 158 snap 43 seq_num 158) dirty exists 2024-03-20T08:02:44.029 INFO:tasks.rados.rados.0.smithi118.stdout:437: left oid 15 (ObjNum 158 snap 43 seq_num 158) 2024-03-20T08:02:44.030 INFO:tasks.rados.rados.0.smithi118.stdout:438: finishing write tid 1 to smithi11881747-28 2024-03-20T08:02:44.030 INFO:tasks.rados.rados.0.smithi118.stdout:438: finishing write tid 2 to smithi11881747-28 2024-03-20T08:02:44.030 INFO:tasks.rados.rados.0.smithi118.stdout:440: expect (ObjNum 152 snap 43 seq_num 152) 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:437: done (4 left) 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:440: done (3 left) 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:443: delete oid 25 current snap is 43 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:438: finishing write tid 3 to smithi11881747-28 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:438: oid 28 updating version 0 to 213 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:438: oid 28 updating version 213 to 214 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 214 (ObjNum 159 snap 43 seq_num 159) dirty exists 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:438: left oid 28 (ObjNum 159 snap 43 seq_num 159) 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:439: finishing write tid 1 to smithi11881747-12 2024-03-20T08:02:44.037 INFO:tasks.rados.rados.0.smithi118.stdout:439: finishing write tid 2 to smithi11881747-12 2024-03-20T08:02:44.038 INFO:tasks.rados.rados.0.smithi118.stdout:439: finishing write tid 3 to smithi11881747-12 2024-03-20T08:02:44.038 INFO:tasks.rados.rados.0.smithi118.stdout:439: oid 12 updating version 0 to 34 2024-03-20T08:02:44.038 INFO:tasks.rados.rados.0.smithi118.stdout:439: oid 12 updating version 34 to 35 2024-03-20T08:02:44.038 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 35 (ObjNum 160 snap 43 seq_num 160) dirty exists 2024-03-20T08:02:44.038 INFO:tasks.rados.rados.0.smithi118.stdout:439: left oid 12 (ObjNum 160 snap 43 seq_num 160) 2024-03-20T08:02:44.038 INFO:tasks.rados.rados.0.smithi118.stdout:442: finishing rollback tid 1 to smithi11881747-1 2024-03-20T08:02:44.038 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 178 (ObjNum 129 snap 0 seq_num 322284720) dirty dne 2024-03-20T08:02:44.042 INFO:tasks.rados.rados.0.smithi118.stdout:438: done (3 left) 2024-03-20T08:02:44.042 INFO:tasks.rados.rados.0.smithi118.stdout:439: done (2 left) 2024-03-20T08:02:44.042 INFO:tasks.rados.rados.0.smithi118.stdout:442: done (1 left) 2024-03-20T08:02:44.042 INFO:tasks.rados.rados.0.smithi118.stdout:443: done (0 left) 2024-03-20T08:02:44.042 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 43 2024-03-20T08:02:44.042 INFO:tasks.rados.rados.0.smithi118.stdout:444: seq_num 162 ranges {2785280=319488} 2024-03-20T08:02:44.058 INFO:tasks.rados.rados.0.smithi118.stdout:444: writing smithi11881747-49 from 2785280 to 3104768 tid 1 2024-03-20T08:02:44.059 INFO:tasks.rados.rados.0.smithi118.stdout:445: snap_remove snap 37 2024-03-20T08:02:44.367 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-mon[27391]: osdmap e147: 8 total, 8 up, 8 in 2024-03-20T08:02:44.367 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-mon[27391]: pgmap v12: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:44.368 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-mon[38981]: osdmap e147: 8 total, 8 up, 8 in 2024-03-20T08:02:44.368 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-mon[38981]: pgmap v12: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:44.368 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:44.258+0000 7f3bdfc18200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T08:02:44.368 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:44.368+0000 7f3bdfc18200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T08:02:44.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:44 smithi196 ceph-mon[31033]: osdmap e147: 8 total, 8 up, 8 in 2024-03-20T08:02:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:44 smithi196 ceph-mon[31033]: pgmap v12: 121 pgs: 9 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 110 active+clean; 110 MiB data, 412 MiB used, 715 GiB / 715 GiB avail 2024-03-20T08:02:44.629 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:44.615+0000 7f3bdfc18200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T08:02:44.629 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:02:44] ENGINE Bus STARTING 2024-03-20T08:02:44.907 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: CherryPy Checker: 2024-03-20T08:02:44.907 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: The Application mounted at '' has an empty config. 2024-03-20T08:02:44.907 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: 2024-03-20T08:02:44.907 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:02:44] ENGINE Serving on http://:::9283 2024-03-20T08:02:44.907 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: [20/Mar/2024:08:02:44] ENGINE Bus STARTED 2024-03-20T08:02:45.038 INFO:tasks.rados.rados.0.smithi118.stdout:445: done (1 left) 2024-03-20T08:02:45.038 INFO:tasks.rados.rados.0.smithi118.stdout:446: copy_from oid 46 from oid 11 current snap is 43 2024-03-20T08:02:45.039 INFO:tasks.rados.rados.0.smithi118.stdout:447: setattr oid 30 current snap is 43 2024-03-20T08:02:45.039 INFO:tasks.rados.rados.0.smithi118.stdout:444: finishing write tid 1 to smithi11881747-49 2024-03-20T08:02:45.039 INFO:tasks.rados.rados.0.smithi118.stdout:444: finishing write tid 2 to smithi11881747-49 2024-03-20T08:02:45.039 INFO:tasks.rados.rados.0.smithi118.stdout:444: finishing write tid 3 to smithi11881747-49 2024-03-20T08:02:45.039 INFO:tasks.rados.rados.0.smithi118.stdout:444: oid 49 updating version 0 to 191 2024-03-20T08:02:45.039 INFO:tasks.rados.rados.0.smithi118.stdout:444: oid 49 version 191 is already newer than 190 2024-03-20T08:02:45.039 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 191 (ObjNum 162 snap 43 seq_num 162) dirty exists 2024-03-20T08:02:45.039 INFO:tasks.rados.rados.0.smithi118.stdout:444: left oid 49 (ObjNum 162 snap 43 seq_num 162) 2024-03-20T08:02:45.043 INFO:tasks.rados.rados.0.smithi118.stdout:444: done (2 left) 2024-03-20T08:02:45.044 INFO:tasks.rados.rados.0.smithi118.stdout:448: rollback oid 29 current snap is 43 2024-03-20T08:02:45.044 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 29 2024-03-20T08:02:45.044 INFO:tasks.rados.rados.0.smithi118.stdout:449: setattr oid 40 current snap is 43 2024-03-20T08:02:45.044 INFO:tasks.rados.rados.0.smithi118.stdout:450: snap_create 2024-03-20T08:02:45.048 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 146 (ObjNum 33 snap 0 seq_num 322394128) dirty exists 2024-03-20T08:02:45.049 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 37 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-03-20T08:02:45.051 INFO:tasks.rados.rados.0.smithi118.stdout:448: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:02:45.051 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 0 (ObjNum 97 snap 0 seq_num 1) dirty dne 2024-03-20T08:02:45.057 INFO:tasks.rados.rados.0.smithi118.stdout:446: finishing copy_from racing read to smithi11881747-46 2024-03-20T08:02:45.058 INFO:tasks.rados.rados.0.smithi118.stdout:446: finishing copy_from to smithi11881747-46 2024-03-20T08:02:45.058 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 51 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:02:45.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[38981]: osdmap e148: 8 total, 8 up, 8 in 2024-03-20T08:02:45.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[38981]: Standby manager daemon y started 2024-03-20T08:02:45.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-20T08:02:45.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:02:45.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-20T08:02:45.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:02:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[27391]: osdmap e148: 8 total, 8 up, 8 in 2024-03-20T08:02:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[27391]: Standby manager daemon y started 2024-03-20T08:02:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-20T08:02:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:02:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-20T08:02:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:45 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:02:45.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:45 smithi196 ceph-mon[31033]: osdmap e148: 8 total, 8 up, 8 in 2024-03-20T08:02:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:45 smithi196 ceph-mon[31033]: Standby manager daemon y started 2024-03-20T08:02:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:45 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-20T08:02:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:45 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:02:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:45 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-20T08:02:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:45 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.118:0/3057633433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:02:46.040 INFO:tasks.rados.rados.0.smithi118.stdout:446: done (4 left) 2024-03-20T08:02:46.040 INFO:tasks.rados.rados.0.smithi118.stdout:447: done (3 left) 2024-03-20T08:02:46.040 INFO:tasks.rados.rados.0.smithi118.stdout:448: done (2 left) 2024-03-20T08:02:46.041 INFO:tasks.rados.rados.0.smithi118.stdout:449: done (1 left) 2024-03-20T08:02:46.041 INFO:tasks.rados.rados.0.smithi118.stdout:450: done (0 left) 2024-03-20T08:02:46.041 INFO:tasks.rados.rados.0.smithi118.stdout:451: snap_create 2024-03-20T08:02:46.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:46 smithi196 ceph-mon[31033]: mgrmap e27: x(active, since 7s), standbys: y 2024-03-20T08:02:46.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:46 smithi196 ceph-mon[31033]: osdmap e149: 8 total, 8 up, 8 in 2024-03-20T08:02:46.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:46 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:02:46.351 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:46 smithi196 ceph-mon[31033]: pgmap v15: 121 pgs: 121 active+clean; 116 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-03-20T08:02:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:46 smithi118 ceph-mon[27391]: mgrmap e27: x(active, since 7s), standbys: y 2024-03-20T08:02:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:46 smithi118 ceph-mon[27391]: osdmap e149: 8 total, 8 up, 8 in 2024-03-20T08:02:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:46 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:02:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:46 smithi118 ceph-mon[27391]: pgmap v15: 121 pgs: 121 active+clean; 116 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-03-20T08:02:46.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:46 smithi118 ceph-mon[38981]: mgrmap e27: x(active, since 7s), standbys: y 2024-03-20T08:02:46.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:46 smithi118 ceph-mon[38981]: osdmap e149: 8 total, 8 up, 8 in 2024-03-20T08:02:46.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:46 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:02:46.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:46 smithi118 ceph-mon[38981]: pgmap v15: 121 pgs: 121 active+clean; 116 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-03-20T08:02:47.042 INFO:tasks.rados.rados.0.smithi118.stdout:451: done (0 left) 2024-03-20T08:02:47.042 INFO:tasks.rados.rados.0.smithi118.stdout:452: setattr oid 4 current snap is 45 2024-03-20T08:02:47.043 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 45 2024-03-20T08:02:47.044 INFO:tasks.rados.rados.0.smithi118.stdout:453: seq_num 163 ranges {0=335872} 2024-03-20T08:02:47.046 INFO:tasks.rados.rados.0.smithi118.stdout:453: writing smithi11881747-14 from 0 to 335872 tid 1 2024-03-20T08:02:47.046 INFO:tasks.rados.rados.0.smithi118.stdout:454: read oid 16 snap 23 2024-03-20T08:02:47.046 INFO:tasks.rados.rados.0.smithi118.stdout:454: expect (ObjNum 81 snap 14 seq_num 81) 2024-03-20T08:02:47.046 INFO:tasks.rados.rados.0.smithi118.stdout:455: rmattr oid 10 current snap is 45 2024-03-20T08:02:47.046 INFO:tasks.rados.rados.0.smithi118.stdout:455: done (3 left) 2024-03-20T08:02:47.046 INFO:tasks.rados.rados.0.smithi118.stdout:456: copy_from oid 41 from oid 17 current snap is 45 2024-03-20T08:02:47.046 INFO:tasks.rados.rados.0.smithi118.stdout:457: delete oid 32 current snap is 45 2024-03-20T08:02:47.050 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 39 (ObjNum 138 snap 40 seq_num 138) dirty exists 2024-03-20T08:02:47.050 INFO:tasks.rados.rados.0.smithi118.stdout:453: finishing write tid 1 to smithi11881747-14 2024-03-20T08:02:47.050 INFO:tasks.rados.rados.0.smithi118.stdout:453: finishing write tid 2 to smithi11881747-14 2024-03-20T08:02:47.051 INFO:tasks.rados.rados.0.smithi118.stdout:452: done (4 left) 2024-03-20T08:02:47.051 INFO:tasks.rados.rados.0.smithi118.stdout:457: done (3 left) 2024-03-20T08:02:47.051 INFO:tasks.rados.rados.0.smithi118.stdout:458: rmattr oid 28 current snap is 45 2024-03-20T08:02:47.052 INFO:tasks.rados.rados.0.smithi118.stdout:459: rollback oid 31 current snap is 45 2024-03-20T08:02:47.052 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 27 2024-03-20T08:02:47.052 INFO:tasks.rados.rados.0.smithi118.stdout:460: copy_from oid 19 from oid 46 current snap is 45 2024-03-20T08:02:47.052 INFO:tasks.rados.rados.0.smithi118.stdout:461: read oid 45 snap -1 2024-03-20T08:02:47.052 INFO:tasks.rados.rados.0.smithi118.stdout:461: expect (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:02:47.052 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 45 2024-03-20T08:02:47.052 INFO:tasks.rados.rados.0.smithi118.stdout:462: seq_num 165 ranges {2129920=368640} 2024-03-20T08:02:47.066 INFO:tasks.rados.rados.0.smithi118.stdout:462: writing smithi11881747-20 from 2129920 to 2498560 tid 1 2024-03-20T08:02:47.066 INFO:tasks.rados.rados.0.smithi118.stdout:463: rollback oid 33 current snap is 45 2024-03-20T08:02:47.066 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 43 2024-03-20T08:02:47.066 INFO:tasks.rados.rados.0.smithi118.stdout:453: finishing write tid 3 to smithi11881747-14 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:453: oid 14 updating version 0 to 108 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:453: oid 14 updating version 108 to 109 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 109 (ObjNum 163 snap 45 seq_num 163) dirty exists 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:453: left oid 14 (ObjNum 163 snap 45 seq_num 163) 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:453: done (8 left) 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:464: read oid 29 snap -1 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:464: expect deleted 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:465: rollback oid 25 current snap is 45 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 25 to 31 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:459: finishing rollback tid 0 to smithi11881747-31 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 45 2024-03-20T08:02:47.067 INFO:tasks.rados.rados.0.smithi118.stdout:466: seq_num 166 ranges {1277952=385024} 2024-03-20T08:02:47.076 INFO:tasks.rados.rados.0.smithi118.stdout:466: writing smithi11881747-15 from 1277952 to 1662976 tid 1 2024-03-20T08:02:47.076 INFO:tasks.rados.rados.0.smithi118.stdout:467: rollback oid 42 current snap is 45 2024-03-20T08:02:47.076 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 31 2024-03-20T08:02:47.076 INFO:tasks.rados.rados.0.smithi118.stdout:459: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:02:47.077 INFO:tasks.rados.rados.0.smithi118.stdout:459: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:02:47.077 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 217 (ObjNum 87 snap 17 seq_num 87) dirty exists 2024-03-20T08:02:47.077 INFO:tasks.rados.rados.0.smithi118.stdout:461: expect (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:02:47.161 INFO:tasks.rados.rados.0.smithi118.stdout:454: expect (ObjNum 81 snap 14 seq_num 81) 2024-03-20T08:02:47.268 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 235 (ObjNum 159 snap 43 seq_num 159) dirty exists 2024-03-20T08:02:47.268 INFO:tasks.rados.rados.0.smithi118.stdout:465: finishing rollback tid 1 to smithi11881747-25 2024-03-20T08:02:47.268 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 0 (ObjNum 33 snap 0 seq_num 1486959406) dirty dne 2024-03-20T08:02:47.268 INFO:tasks.rados.rados.0.smithi118.stdout:460: finishing copy_from racing read to smithi11881747-19 2024-03-20T08:02:47.268 INFO:tasks.rados.rados.0.smithi118.stdout:454: done (11 left) 2024-03-20T08:02:47.268 INFO:tasks.rados.rados.0.smithi118.stdout:458: done (10 left) 2024-03-20T08:02:47.269 INFO:tasks.rados.rados.0.smithi118.stdout:459: done (9 left) 2024-03-20T08:02:47.269 INFO:tasks.rados.rados.0.smithi118.stdout:461: done (8 left) 2024-03-20T08:02:47.269 INFO:tasks.rados.rados.0.smithi118.stdout:464: done (7 left) 2024-03-20T08:02:47.269 INFO:tasks.rados.rados.0.smithi118.stdout:465: done (6 left) 2024-03-20T08:02:47.269 INFO:tasks.rados.rados.0.smithi118.stdout:468: snap_remove snap 39 2024-03-20T08:02:47.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:47 smithi196 ceph-mon[31033]: osdmap e150: 8 total, 8 up, 8 in 2024-03-20T08:02:47.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:47 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:47.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:47 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:47 smithi118 ceph-mon[27391]: osdmap e150: 8 total, 8 up, 8 in 2024-03-20T08:02:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:47 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:47 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:47.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:47 smithi118 ceph-mon[38981]: osdmap e150: 8 total, 8 up, 8 in 2024-03-20T08:02:47.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:47 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:47.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:47 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.051 INFO:tasks.rados.rados.0.smithi118.stdout:468: done (6 left) 2024-03-20T08:02:48.051 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 45 2024-03-20T08:02:48.051 INFO:tasks.rados.rados.0.smithi118.stdout:469: seq_num 167 ranges {491520=401408} 2024-03-20T08:02:48.057 INFO:tasks.rados.rados.0.smithi118.stdout:469: writing smithi11881747-30 from 491520 to 892928 tid 1 2024-03-20T08:02:48.057 INFO:tasks.rados.rados.0.smithi118.stdout:470: delete oid 21 current snap is 45 2024-03-20T08:02:48.057 INFO:tasks.rados.rados.0.smithi118.stdout:463: finishing rollback tid 0 to smithi11881747-33 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:462: finishing write tid 1 to smithi11881747-20 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:463: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:463: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 94 (ObjNum 89 snap 18 seq_num 89) dirty exists 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:462: finishing write tid 2 to smithi11881747-20 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:462: finishing write tid 3 to smithi11881747-20 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:462: oid 20 updating version 0 to 200 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:462: oid 20 version 200 is already newer than 199 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 200 (ObjNum 165 snap 45 seq_num 165) dirty exists 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:462: left oid 20 (ObjNum 165 snap 45 seq_num 165) 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:460: finishing copy_from to smithi11881747-19 2024-03-20T08:02:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 51 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:467: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:467: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:467: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 177 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:466: finishing write tid 1 to smithi11881747-15 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:466: finishing write tid 2 to smithi11881747-15 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:456: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:466: finishing write tid 3 to smithi11881747-15 2024-03-20T08:02:48.059 INFO:tasks.rados.rados.0.smithi118.stdout:466: oid 15 updating version 0 to 61 2024-03-20T08:02:48.060 INFO:tasks.rados.rados.0.smithi118.stdout:466: oid 15 updating version 61 to 62 2024-03-20T08:02:48.060 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 62 (ObjNum 166 snap 45 seq_num 166) dirty exists 2024-03-20T08:02:48.060 INFO:tasks.rados.rados.0.smithi118.stdout:466: left oid 15 (ObjNum 166 snap 45 seq_num 166) 2024-03-20T08:02:48.060 INFO:tasks.rados.rados.0.smithi118.stdout:456: finishing copy_from to smithi11881747-41 2024-03-20T08:02:48.060 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 42 (ObjNum 125 snap 33 seq_num 125) dirty exists 2024-03-20T08:02:48.061 INFO:tasks.rados.rados.0.smithi118.stdout:456: done (7 left) 2024-03-20T08:02:48.061 INFO:tasks.rados.rados.0.smithi118.stdout:460: done (6 left) 2024-03-20T08:02:48.061 INFO:tasks.rados.rados.0.smithi118.stdout:462: done (5 left) 2024-03-20T08:02:48.061 INFO:tasks.rados.rados.0.smithi118.stdout:463: done (4 left) 2024-03-20T08:02:48.061 INFO:tasks.rados.rados.0.smithi118.stdout:466: done (3 left) 2024-03-20T08:02:48.061 INFO:tasks.rados.rados.0.smithi118.stdout:467: done (2 left) 2024-03-20T08:02:48.061 INFO:tasks.rados.rados.0.smithi118.stdout:470: done (1 left) 2024-03-20T08:02:48.062 INFO:tasks.rados.rados.0.smithi118.stdout:471: snap_create 2024-03-20T08:02:48.064 INFO:tasks.rados.rados.0.smithi118.stdout:469: finishing write tid 1 to smithi11881747-30 2024-03-20T08:02:48.065 INFO:tasks.rados.rados.0.smithi118.stdout:469: finishing write tid 2 to smithi11881747-30 2024-03-20T08:02:48.066 INFO:tasks.rados.rados.0.smithi118.stdout:469: finishing write tid 3 to smithi11881747-30 2024-03-20T08:02:48.066 INFO:tasks.rados.rados.0.smithi118.stdout:469: oid 30 updating version 0 to 44 2024-03-20T08:02:48.066 INFO:tasks.rados.rados.0.smithi118.stdout:469: oid 30 version 44 is already newer than 43 2024-03-20T08:02:48.066 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 44 (ObjNum 167 snap 45 seq_num 167) dirty exists 2024-03-20T08:02:48.066 INFO:tasks.rados.rados.0.smithi118.stdout:469: left oid 30 (ObjNum 167 snap 45 seq_num 167) 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: osdmap e151: 8 total, 8 up, 8 in 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi118 to 2480M 2024-03-20T08:02:48.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:48 smithi196 ceph-mon[31033]: pgmap v18: 121 pgs: 121 active+clean; 116 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: osdmap e151: 8 total, 8 up, 8 in 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi118 to 2480M 2024-03-20T08:02:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[27391]: pgmap v18: 121 pgs: 121 active+clean; 116 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-03-20T08:02:48.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: osdmap e151: 8 total, 8 up, 8 in 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi118 to 2480M 2024-03-20T08:02:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:48 smithi118 ceph-mon[38981]: pgmap v18: 121 pgs: 121 active+clean; 116 MiB data, 426 MiB used, 715 GiB / 715 GiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-03-20T08:02:49.055 INFO:tasks.rados.rados.0.smithi118.stdout:469: done (1 left) 2024-03-20T08:02:49.056 INFO:tasks.rados.rados.0.smithi118.stdout:471: done (0 left) 2024-03-20T08:02:49.056 INFO:tasks.rados.rados.0.smithi118.stdout:472: snap_create 2024-03-20T08:02:49.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:49 smithi196 ceph-mon[31033]: osdmap e152: 8 total, 8 up, 8 in 2024-03-20T08:02:49.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:49 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:49.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:49 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:49.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:49 smithi118 ceph-mon[27391]: osdmap e152: 8 total, 8 up, 8 in 2024-03-20T08:02:49.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:49 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:49.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:49 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:49.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:49 smithi118 ceph-mon[38981]: osdmap e152: 8 total, 8 up, 8 in 2024-03-20T08:02:49.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:49 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:49.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:49 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:50.060 INFO:tasks.rados.rados.0.smithi118.stdout:472: done (0 left) 2024-03-20T08:02:50.060 INFO:tasks.rados.rados.0.smithi118.stdout:473: snap_remove snap 38 2024-03-20T08:02:50.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:50 smithi196 ceph-mon[31033]: osdmap e153: 8 total, 8 up, 8 in 2024-03-20T08:02:50.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:50 smithi196 ceph-mon[31033]: pgmap v21: 121 pgs: 121 active+clean; 128 MiB data, 444 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 25 op/s 2024-03-20T08:02:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:50 smithi118 ceph-mon[27391]: osdmap e153: 8 total, 8 up, 8 in 2024-03-20T08:02:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:50 smithi118 ceph-mon[27391]: pgmap v21: 121 pgs: 121 active+clean; 128 MiB data, 444 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 25 op/s 2024-03-20T08:02:50.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:50 smithi118 ceph-mon[38981]: osdmap e153: 8 total, 8 up, 8 in 2024-03-20T08:02:50.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:50 smithi118 ceph-mon[38981]: pgmap v21: 121 pgs: 121 active+clean; 128 MiB data, 444 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 25 op/s 2024-03-20T08:02:51.019 INFO:tasks.rados.rados.0.smithi118.stdout:473: done (0 left) 2024-03-20T08:02:51.019 INFO:tasks.rados.rados.0.smithi118.stdout:474: read oid 21 snap -1 2024-03-20T08:02:51.019 INFO:tasks.rados.rados.0.smithi118.stdout:474: expect deleted 2024-03-20T08:02:51.019 INFO:tasks.rados.rados.0.smithi118.stdout:475: snap_create 2024-03-20T08:02:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:51 smithi196 ceph-mon[31033]: osdmap e154: 8 total, 8 up, 8 in 2024-03-20T08:02:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:51 smithi196 ceph-mon[31033]: osdmap e155: 8 total, 8 up, 8 in 2024-03-20T08:02:51.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:51 smithi118 ceph-mon[27391]: osdmap e154: 8 total, 8 up, 8 in 2024-03-20T08:02:51.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:51 smithi118 ceph-mon[27391]: osdmap e155: 8 total, 8 up, 8 in 2024-03-20T08:02:51.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:51 smithi118 ceph-mon[38981]: osdmap e154: 8 total, 8 up, 8 in 2024-03-20T08:02:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:51 smithi118 ceph-mon[38981]: osdmap e155: 8 total, 8 up, 8 in 2024-03-20T08:02:51.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:02:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[27646]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:02:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-03-20T08:02:52.019 INFO:tasks.rados.rados.0.smithi118.stdout:474: done (1 left) 2024-03-20T08:02:52.020 INFO:tasks.rados.rados.0.smithi118.stdout:475: done (0 left) 2024-03-20T08:02:52.020 INFO:tasks.rados.rados.0.smithi118.stdout:476: copy_from oid 24 from oid 25 current snap is 48 2024-03-20T08:02:52.020 INFO:tasks.rados.rados.0.smithi118.stdout:477: snap_remove snap 33 2024-03-20T08:02:52.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: pgmap v24: 121 pgs: 121 active+clean; 128 MiB data, 444 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 25 op/s 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:02:52.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:52 smithi196 ceph-mon[31033]: osdmap e156: 8 total, 8 up, 8 in 2024-03-20T08:02:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: pgmap v24: 121 pgs: 121 active+clean; 128 MiB data, 444 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 25 op/s 2024-03-20T08:02:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[27391]: osdmap e156: 8 total, 8 up, 8 in 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: pgmap v24: 121 pgs: 121 active+clean; 128 MiB data, 444 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 25 op/s 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:02:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:52 smithi118 ceph-mon[38981]: osdmap e156: 8 total, 8 up, 8 in 2024-03-20T08:02:53.022 INFO:tasks.rados.rados.0.smithi118.stdout:477: done (1 left) 2024-03-20T08:02:53.022 INFO:tasks.rados.rados.0.smithi118.stdout:478: rollback oid 16 current snap is 48 2024-03-20T08:02:53.022 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 43 2024-03-20T08:02:53.022 INFO:tasks.rados.rados.0.smithi118.stdout:476: finishing copy_from to smithi11881747-24 2024-03-20T08:02:53.022 INFO:tasks.rados.rados.0.smithi118.stdout:476: got expected ENOENT (src dne) 2024-03-20T08:02:53.022 INFO:tasks.rados.rados.0.smithi118.stdout:476: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:02:53.022 INFO:tasks.rados.rados.0.smithi118.stdout:476: done (1 left) 2024-03-20T08:02:53.022 INFO:tasks.rados.rados.0.smithi118.stdout:479: read oid 47 snap 34 2024-03-20T08:02:53.023 INFO:tasks.rados.rados.0.smithi118.stdout:479: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:53.023 INFO:tasks.rados.rados.0.smithi118.stdout:480: rollback oid 17 current snap is 48 2024-03-20T08:02:53.023 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 46 2024-03-20T08:02:53.023 INFO:tasks.rados.rados.0.smithi118.stdout:481: snap_create 2024-03-20T08:02:53.027 INFO:tasks.rados.rados.0.smithi118.stdout:480: finishing rollback tid 0 to smithi11881747-17 2024-03-20T08:02:53.028 INFO:tasks.rados.rados.0.smithi118.stdout:480: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:02:53.029 INFO:tasks.rados.rados.0.smithi118.stdout:480: finishing rollback tid 2 to smithi11881747-17 2024-03-20T08:02:53.029 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 46 (ObjNum 125 snap 33 seq_num 125) dirty exists 2024-03-20T08:02:53.033 INFO:tasks.rados.rados.0.smithi118.stdout:478: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:02:53.033 INFO:tasks.rados.rados.0.smithi118.stdout:478: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:02:53.033 INFO:tasks.rados.rados.0.smithi118.stdout:478: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:02:53.033 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 263 (ObjNum 110 snap 27 seq_num 110) dirty exists 2024-03-20T08:02:53.034 INFO:tasks.rados.rados.0.smithi118.stdout:479: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Adjusting osd_memory_target on smithi196 to 2480M 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:53 smithi196 ceph-mon[31033]: osdmap e157: 8 total, 8 up, 8 in 2024-03-20T08:02:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Adjusting osd_memory_target on smithi196 to 2480M 2024-03-20T08:02:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:02:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:02:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:02:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:02:53.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[38981]: osdmap e157: 8 total, 8 up, 8 in 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Adjusting osd_memory_target on smithi196 to 2480M 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:02:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:53 smithi118 ceph-mon[27391]: osdmap e157: 8 total, 8 up, 8 in 2024-03-20T08:02:54.023 INFO:tasks.rados.rados.0.smithi118.stdout:478: done (3 left) 2024-03-20T08:02:54.023 INFO:tasks.rados.rados.0.smithi118.stdout:479: done (2 left) 2024-03-20T08:02:54.023 INFO:tasks.rados.rados.0.smithi118.stdout:480: done (1 left) 2024-03-20T08:02:54.023 INFO:tasks.rados.rados.0.smithi118.stdout:481: done (0 left) 2024-03-20T08:02:54.023 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 49 2024-03-20T08:02:54.023 INFO:tasks.rados.rados.0.smithi118.stdout:482: seq_num 169 ranges {2801664=434176} 2024-03-20T08:02:54.040 INFO:tasks.rados.rados.0.smithi118.stdout:482: writing smithi11881747-33 from 2801664 to 3235840 tid 1 2024-03-20T08:02:54.041 INFO:tasks.rados.rados.0.smithi118.stdout:483: read oid 27 snap -1 2024-03-20T08:02:54.041 INFO:tasks.rados.rados.0.smithi118.stdout:483: expect deleted 2024-03-20T08:02:54.041 INFO:tasks.rados.rados.0.smithi118.stdout:484: copy_from oid 30 from oid 25 current snap is 49 2024-03-20T08:02:54.041 INFO:tasks.rados.rados.0.smithi118.stdout:485: read oid 31 snap -1 2024-03-20T08:02:54.041 INFO:tasks.rados.rados.0.smithi118.stdout:485: expect (ObjNum 87 snap 17 seq_num 87) 2024-03-20T08:02:54.041 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 49 2024-03-20T08:02:54.041 INFO:tasks.rados.rados.0.smithi118.stdout:486: seq_num 170 ranges {0=450560} 2024-03-20T08:02:54.043 INFO:tasks.rados.rados.0.smithi118.stdout:486: writing smithi11881747-6 from 0 to 450560 tid 1 2024-03-20T08:02:54.044 INFO:tasks.rados.rados.0.smithi118.stdout:487: read oid 41 snap 36 2024-03-20T08:02:54.044 INFO:tasks.rados.rados.0.smithi118.stdout:487: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:54.044 INFO:tasks.rados.rados.0.smithi118.stdout:488: read oid 22 snap 45 2024-03-20T08:02:54.044 INFO:tasks.rados.rados.0.smithi118.stdout:488: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:02:54.044 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 49 2024-03-20T08:02:54.044 INFO:tasks.rados.rados.0.smithi118.stdout:484: finishing copy_from to smithi11881747-30 2024-03-20T08:02:54.044 INFO:tasks.rados.rados.0.smithi118.stdout:484: got expected ENOENT (src dne) 2024-03-20T08:02:54.044 INFO:tasks.rados.rados.0.smithi118.stdout:489: seq_num 171 ranges {1662976=466944} 2024-03-20T08:02:54.056 INFO:tasks.rados.rados.0.smithi118.stdout:489: writing smithi11881747-15 from 1662976 to 2129920 tid 1 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:483: done (7 left) 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:490: read oid 20 snap -1 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:490: expect (ObjNum 165 snap 45 seq_num 165) 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:491: rollback oid 37 current snap is 49 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 44 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:484: finishing copy_from racing read to smithi11881747-30 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:484: done (8 left) 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:492: read oid 26 snap -1 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:492: expect (ObjNum 148 snap 42 seq_num 148) 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 49 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:482: finishing write tid 1 to smithi11881747-33 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:482: finishing write tid 2 to smithi11881747-33 2024-03-20T08:02:54.057 INFO:tasks.rados.rados.0.smithi118.stdout:487: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:02:54.079 INFO:tasks.rados.rados.0.smithi118.stdout:486: finishing write tid 1 to smithi11881747-6 2024-03-20T08:02:54.079 INFO:tasks.rados.rados.0.smithi118.stdout:486: finishing write tid 2 to smithi11881747-6 2024-03-20T08:02:54.079 INFO:tasks.rados.rados.0.smithi118.stdout:486: finishing write tid 3 to smithi11881747-6 2024-03-20T08:02:54.079 INFO:tasks.rados.rados.0.smithi118.stdout:486: oid 6 updating version 0 to 121 2024-03-20T08:02:54.079 INFO:tasks.rados.rados.0.smithi118.stdout:486: oid 6 version 121 is already newer than 120 2024-03-20T08:02:54.079 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 121 (ObjNum 170 snap 49 seq_num 170) dirty exists 2024-03-20T08:02:54.080 INFO:tasks.rados.rados.0.smithi118.stdout:486: left oid 6 (ObjNum 170 snap 49 seq_num 170) 2024-03-20T08:02:54.080 INFO:tasks.rados.rados.0.smithi118.stdout:482: finishing write tid 3 to smithi11881747-33 2024-03-20T08:02:54.080 INFO:tasks.rados.rados.0.smithi118.stdout:482: oid 33 updating version 0 to 108 2024-03-20T08:02:54.080 INFO:tasks.rados.rados.0.smithi118.stdout:482: oid 33 updating version 108 to 109 2024-03-20T08:02:54.080 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 109 (ObjNum 169 snap 49 seq_num 169) dirty exists 2024-03-20T08:02:54.080 INFO:tasks.rados.rados.0.smithi118.stdout:482: left oid 33 (ObjNum 169 snap 49 seq_num 169) 2024-03-20T08:02:54.080 INFO:tasks.rados.rados.0.smithi118.stdout:493: seq_num 172 ranges {2056192=483328} 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:493: writing smithi11881747-8 from 2056192 to 2539520 tid 1 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:482: done (9 left) 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:486: done (8 left) 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:487: done (7 left) 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:494: read oid 5 snap 29 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:494: expect (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:495: snap_create 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:491: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 128 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:02:54.093 INFO:tasks.rados.rados.0.smithi118.stdout:492: expect (ObjNum 148 snap 42 seq_num 148) 2024-03-20T08:02:54.113 INFO:tasks.rados.rados.0.smithi118.stdout:490: expect (ObjNum 165 snap 45 seq_num 165) 2024-03-20T08:02:54.236 INFO:tasks.rados.rados.0.smithi118.stdout:485: expect (ObjNum 87 snap 17 seq_num 87) 2024-03-20T08:02:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:54 smithi196 ceph-mon[31033]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:02:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:54 smithi196 ceph-mon[31033]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:02:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:54 smithi196 ceph-mon[31033]: pgmap v27: 121 pgs: 121 active+clean; 128 MiB data, 462 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T08:02:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:54 smithi196 ceph-mon[31033]: osdmap e158: 8 total, 8 up, 8 in 2024-03-20T08:02:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:54 smithi118 ceph-mon[27391]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:02:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:54 smithi118 ceph-mon[27391]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:02:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:54 smithi118 ceph-mon[27391]: pgmap v27: 121 pgs: 121 active+clean; 128 MiB data, 462 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T08:02:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:54 smithi118 ceph-mon[27391]: osdmap e158: 8 total, 8 up, 8 in 2024-03-20T08:02:54.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:54 smithi118 ceph-mon[38981]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:02:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:54 smithi118 ceph-mon[38981]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:02:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:54 smithi118 ceph-mon[38981]: pgmap v27: 121 pgs: 121 active+clean; 128 MiB data, 462 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-20T08:02:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:54 smithi118 ceph-mon[38981]: osdmap e158: 8 total, 8 up, 8 in 2024-03-20T08:02:54.534 INFO:tasks.rados.rados.0.smithi118.stdout:489: finishing write tid 1 to smithi11881747-15 2024-03-20T08:02:54.534 INFO:tasks.rados.rados.0.smithi118.stdout:489: finishing write tid 2 to smithi11881747-15 2024-03-20T08:02:54.534 INFO:tasks.rados.rados.0.smithi118.stdout:489: finishing write tid 3 to smithi11881747-15 2024-03-20T08:02:54.534 INFO:tasks.rados.rados.0.smithi118.stdout:489: oid 15 updating version 0 to 71 2024-03-20T08:02:54.534 INFO:tasks.rados.rados.0.smithi118.stdout:489: oid 15 version 71 is already newer than 70 2024-03-20T08:02:54.534 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 71 (ObjNum 171 snap 49 seq_num 171) dirty exists 2024-03-20T08:02:54.534 INFO:tasks.rados.rados.0.smithi118.stdout:489: left oid 15 (ObjNum 171 snap 49 seq_num 171) 2024-03-20T08:02:54.534 INFO:tasks.rados.rados.0.smithi118.stdout:488: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:02:54.641 INFO:tasks.rados.rados.0.smithi118.stdout:494: expect (ObjNum 4 snap 0 seq_num 4) 2024-03-20T08:02:54.642 INFO:tasks.rados.rados.0.smithi118.stdout:493: finishing write tid 1 to smithi11881747-8 2024-03-20T08:02:54.642 INFO:tasks.rados.rados.0.smithi118.stdout:493: finishing write tid 2 to smithi11881747-8 2024-03-20T08:02:54.643 INFO:tasks.rados.rados.0.smithi118.stdout:493: finishing write tid 3 to smithi11881747-8 2024-03-20T08:02:54.643 INFO:tasks.rados.rados.0.smithi118.stdout:493: oid 8 updating version 0 to 121 2024-03-20T08:02:54.643 INFO:tasks.rados.rados.0.smithi118.stdout:493: oid 8 version 121 is already newer than 120 2024-03-20T08:02:54.643 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 121 (ObjNum 172 snap 49 seq_num 172) dirty exists 2024-03-20T08:02:54.643 INFO:tasks.rados.rados.0.smithi118.stdout:493: left oid 8 (ObjNum 172 snap 49 seq_num 172) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:485: done (8 left) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:488: done (7 left) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:489: done (6 left) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:490: done (5 left) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:491: done (4 left) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:492: done (3 left) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:493: done (2 left) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:494: done (1 left) 2024-03-20T08:02:55.074 INFO:tasks.rados.rados.0.smithi118.stdout:495: done (0 left) 2024-03-20T08:02:55.075 INFO:tasks.rados.rados.0.smithi118.stdout:496: delete oid 40 current snap is 50 2024-03-20T08:02:55.078 INFO:tasks.rados.rados.0.smithi118.stdout:496: done (0 left) 2024-03-20T08:02:55.078 INFO:tasks.rados.rados.0.smithi118.stdout:497: rmattr oid 4 current snap is 50 2024-03-20T08:02:55.078 INFO:tasks.rados.rados.0.smithi118.stdout:498: snap_create 2024-03-20T08:02:55.080 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 50 (ObjNum 138 snap 40 seq_num 138) dirty exists 2024-03-20T08:02:56.079 INFO:tasks.rados.rados.0.smithi118.stdout:497: done (1 left) 2024-03-20T08:02:56.079 INFO:tasks.rados.rados.0.smithi118.stdout:498: done (0 left) 2024-03-20T08:02:56.079 INFO:tasks.rados.rados.0.smithi118.stdout:499: delete oid 37 current snap is 51 2024-03-20T08:02:56.082 INFO:tasks.rados.rados.0.smithi118.stdout:499: done (0 left) 2024-03-20T08:02:56.083 INFO:tasks.rados.rados.0.smithi118.stdout:500: read oid 28 snap 14 2024-03-20T08:02:56.083 INFO:tasks.rados.rados.0.smithi118.stdout:500: expect (ObjNum 9 snap 0 seq_num 9) 2024-03-20T08:02:56.083 INFO:tasks.rados.rados.0.smithi118.stdout:501: read oid 6 snap -1 2024-03-20T08:02:56.083 INFO:tasks.rados.rados.0.smithi118.stdout:501: expect (ObjNum 170 snap 49 seq_num 170) 2024-03-20T08:02:56.083 INFO:tasks.rados.rados.0.smithi118.stdout:502: read oid 24 snap 42 2024-03-20T08:02:56.083 INFO:tasks.rados.rados.0.smithi118.stdout:502: expect deleted 2024-03-20T08:02:56.083 INFO:tasks.rados.rados.0.smithi118.stdout:503: snap_create 2024-03-20T08:02:56.084 INFO:tasks.rados.rados.0.smithi118.stdout:500: expect (ObjNum 9 snap 0 seq_num 9) 2024-03-20T08:02:56.092 INFO:tasks.rados.rados.0.smithi118.stdout:501: expect (ObjNum 170 snap 49 seq_num 170) 2024-03-20T08:02:56.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:56 smithi196 ceph-mon[31033]: osdmap e159: 8 total, 8 up, 8 in 2024-03-20T08:02:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:56 smithi196 ceph-mon[31033]: pgmap v30: 121 pgs: 121 active+clean; 126 MiB data, 476 MiB used, 715 GiB / 715 GiB avail; 570 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:02:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:56 smithi118 ceph-mon[27391]: osdmap e159: 8 total, 8 up, 8 in 2024-03-20T08:02:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:56 smithi118 ceph-mon[27391]: pgmap v30: 121 pgs: 121 active+clean; 126 MiB data, 476 MiB used, 715 GiB / 715 GiB avail; 570 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:02:56.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:56 smithi118 ceph-mon[38981]: osdmap e159: 8 total, 8 up, 8 in 2024-03-20T08:02:56.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:56 smithi118 ceph-mon[38981]: pgmap v30: 121 pgs: 121 active+clean; 126 MiB data, 476 MiB used, 715 GiB / 715 GiB avail; 570 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:02:56.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:56 smithi196 systemd[1]: Stopping Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:02:57.080 INFO:tasks.rados.rados.0.smithi118.stdout:500: done (3 left) 2024-03-20T08:02:57.080 INFO:tasks.rados.rados.0.smithi118.stdout:501: done (2 left) 2024-03-20T08:02:57.080 INFO:tasks.rados.rados.0.smithi118.stdout:502: done (1 left) 2024-03-20T08:02:57.080 INFO:tasks.rados.rados.0.smithi118.stdout:503: done (0 left) 2024-03-20T08:02:57.081 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 52 2024-03-20T08:02:57.081 INFO:tasks.rados.rados.0.smithi118.stdout:504: seq_num 175 ranges {0=540672} 2024-03-20T08:02:57.083 INFO:tasks.rados.rados.0.smithi118.stdout:504: writing smithi11881747-3 from 0 to 540672 tid 1 2024-03-20T08:02:57.083 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 52 2024-03-20T08:02:57.083 INFO:tasks.rados.rados.0.smithi118.stdout:505: seq_num 176 ranges {3104768=557056} 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:505: writing smithi11881747-49 from 3104768 to 3661824 tid 1 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:506: read oid 32 snap 49 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:506: expect deleted 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:507: rollback oid 50 current snap is 52 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 50 to 51 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:504: finishing write tid 1 to smithi11881747-3 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:504: finishing write tid 2 to smithi11881747-3 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:504: finishing write tid 3 to smithi11881747-3 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:504: oid 3 updating version 0 to 111 2024-03-20T08:02:57.103 INFO:tasks.rados.rados.0.smithi118.stdout:504: oid 3 updating version 111 to 112 2024-03-20T08:02:57.104 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 112 (ObjNum 175 snap 52 seq_num 175) dirty exists 2024-03-20T08:02:57.104 INFO:tasks.rados.rados.0.smithi118.stdout:504: left oid 3 (ObjNum 175 snap 52 seq_num 175) 2024-03-20T08:02:57.104 INFO:tasks.rados.rados.0.smithi118.stdout:504: done (3 left) 2024-03-20T08:02:57.104 INFO:tasks.rados.rados.0.smithi118.stdout:508: snap_create 2024-03-20T08:02:57.105 INFO:tasks.rados.rados.0.smithi118.stdout:507: finishing rollback tid 0 to smithi11881747-50 2024-03-20T08:02:57.106 INFO:tasks.rados.rados.0.smithi118.stdout:507: finishing rollback tid 1 to smithi11881747-50 2024-03-20T08:02:57.106 INFO:tasks.rados.rados.0.smithi118.stdout:507: finishing rollback tid 2 to smithi11881747-50 2024-03-20T08:02:57.106 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 123 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:02:57.108 INFO:tasks.rados.rados.0.smithi118.stdout:505: finishing write tid 1 to smithi11881747-49 2024-03-20T08:02:57.108 INFO:tasks.rados.rados.0.smithi118.stdout:505: finishing write tid 2 to smithi11881747-49 2024-03-20T08:02:57.109 INFO:tasks.rados.rados.0.smithi118.stdout:505: finishing write tid 3 to smithi11881747-49 2024-03-20T08:02:57.109 INFO:tasks.rados.rados.0.smithi118.stdout:505: oid 49 updating version 0 to 228 2024-03-20T08:02:57.109 INFO:tasks.rados.rados.0.smithi118.stdout:505: oid 49 updating version 228 to 229 2024-03-20T08:02:57.109 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 229 (ObjNum 176 snap 52 seq_num 176) dirty exists 2024-03-20T08:02:57.110 INFO:tasks.rados.rados.0.smithi118.stdout:505: left oid 49 (ObjNum 176 snap 52 seq_num 176) 2024-03-20T08:02:57.324 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-mon[31033]: osdmap e160: 8 total, 8 up, 8 in 2024-03-20T08:02:57.324 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-20T08:02:57.324 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-20T08:02:57.324 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-20T08:02:57.324 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-20T08:02:57.324 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-20T08:02:57.325 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-20T08:02:57.325 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-20T08:02:57.325 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-20T08:02:57.325 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.011Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-20T08:02:57.325 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.012Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-20T08:02:57.325 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.012Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-20T08:02:57.325 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[64211]: ts=2024-03-20T08:02:57.012Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-20T08:02:57.325 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 podman[75117]: 2024-03-20 08:02:57.021935359 +0000 UTC m=+0.122082350 container died d713b2506adb9b0900416a40006ead7bde111e8e12cb0f461edc076a98a699b6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:02:57.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:57 smithi118 ceph-mon[38981]: osdmap e160: 8 total, 8 up, 8 in 2024-03-20T08:02:57.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:57 smithi118 ceph-mon[27391]: osdmap e160: 8 total, 8 up, 8 in 2024-03-20T08:02:57.638 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 podman[75117]: 2024-03-20 08:02:57.525320261 +0000 UTC m=+0.625467294 container cleanup d713b2506adb9b0900416a40006ead7bde111e8e12cb0f461edc076a98a699b6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:02:57.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 bash[75117]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a 2024-03-20T08:02:58.087 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 podman[75136]: 2024-03-20 08:02:57.760136054 +0000 UTC m=+0.739179108 container remove d713b2506adb9b0900416a40006ead7bde111e8e12cb0f461edc076a98a699b6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:02:58.087 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-20T08:02:58.087 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 systemd[1]: Stopped Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:02:58.087 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service: Consumed 1.511s CPU time. 2024-03-20T08:02:58.087 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:57 smithi196 systemd[1]: Starting Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:505: done (3 left) 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:506: done (2 left) 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:507: done (1 left) 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:508: done (0 left) 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:509: read oid 22 snap -1 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:509: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:510: rollback oid 47 current snap is 53 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 32 2024-03-20T08:02:58.090 INFO:tasks.rados.rados.0.smithi118.stdout:511: snap_remove snap 14 2024-03-20T08:02:58.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-mon[31033]: osdmap e161: 8 total, 8 up, 8 in 2024-03-20T08:02:58.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-mon[31033]: pgmap v33: 121 pgs: 121 active+clean; 126 MiB data, 476 MiB used, 715 GiB / 715 GiB avail; 570 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:02:58.370 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 podman[75240]: 2024-03-20 08:02:58.012710107 +0000 UTC m=+0.019522488 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-20T08:02:58.371 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 podman[75240]: 2024-03-20 08:02:58.13625272 +0000 UTC m=+0.143065087 container create dc4c97c24deef5271e3f940ca6de2569b9cbd2ce9fdb81b24e4bbe00b8bf6a3f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:02:58.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:58 smithi118 ceph-mon[27391]: osdmap e161: 8 total, 8 up, 8 in 2024-03-20T08:02:58.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:58 smithi118 ceph-mon[27391]: pgmap v33: 121 pgs: 121 active+clean; 126 MiB data, 476 MiB used, 715 GiB / 715 GiB avail; 570 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:02:58.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:58 smithi118 ceph-mon[38981]: osdmap e161: 8 total, 8 up, 8 in 2024-03-20T08:02:58.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:58 smithi118 ceph-mon[38981]: pgmap v33: 121 pgs: 121 active+clean; 126 MiB data, 476 MiB used, 715 GiB / 715 GiB avail; 570 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:02:58.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 podman[75240]: 2024-03-20 08:02:58.37213334 +0000 UTC m=+0.378945710 container init dc4c97c24deef5271e3f940ca6de2569b9cbd2ce9fdb81b24e4bbe00b8bf6a3f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:02:58.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 podman[75240]: 2024-03-20 08:02:58.377387545 +0000 UTC m=+0.384199914 container start dc4c97c24deef5271e3f940ca6de2569b9cbd2ce9fdb81b24e4bbe00b8bf6a3f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.400Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.400Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.400Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi196 (none))" 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.400Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.400Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.401Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.402Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.403Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.403Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.405Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.405Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=134.128µs 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.405Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-20T08:02:58.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.406Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=2 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.414Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=2 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.414Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=2 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.415Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=22.719µs wal_replay_duration=9.309752ms wbl_replay_duration=104ns total_replay_duration=9.486652ms 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.416Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.416Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.416Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.437Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.489695ms db_storage=850ns remote_storage=1.101µs web_handler=486ns query_engine=665ns scrape=424.062µs scrape_sd=82.189µs notify=15.466µs notify_sd=55.375µs rules=19.332697ms tracing=6.873µs 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.437Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:02:58.437Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 bash[75240]: dc4c97c24deef5271e3f940ca6de2569b9cbd2ce9fdb81b24e4bbe00b8bf6a3f 2024-03-20T08:02:58.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:02:58 smithi196 systemd[1]: Started Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:02:59.094 INFO:tasks.rados.rados.0.smithi118.stdout:511: done (2 left) 2024-03-20T08:02:59.094 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 53 2024-03-20T08:02:59.094 INFO:tasks.rados.rados.0.smithi118.stdout:512: seq_num 177 ranges {450560=573440} 2024-03-20T08:02:59.099 INFO:tasks.rados.rados.0.smithi118.stdout:512: writing smithi11881747-6 from 450560 to 1024000 tid 1 2024-03-20T08:02:59.099 INFO:tasks.rados.rados.0.smithi118.stdout:513: copy_from oid 41 from oid 46 current snap is 53 2024-03-20T08:02:59.099 INFO:tasks.rados.rados.0.smithi118.stdout:510: finishing rollback tid 0 to smithi11881747-47 2024-03-20T08:02:59.099 INFO:tasks.rados.rados.0.smithi118.stdout:510: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:02:59.099 INFO:tasks.rados.rados.0.smithi118.stdout:510: finishing rollback tid 2 to smithi11881747-47 2024-03-20T08:02:59.099 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 273 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:02:59.099 INFO:tasks.rados.rados.0.smithi118.stdout:509: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:512: finishing write tid 1 to smithi11881747-6 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:512: finishing write tid 2 to smithi11881747-6 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:512: finishing write tid 3 to smithi11881747-6 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:512: oid 6 updating version 0 to 127 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:512: oid 6 updating version 127 to 128 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 128 (ObjNum 177 snap 53 seq_num 177) dirty exists 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:512: left oid 6 (ObjNum 177 snap 53 seq_num 177) 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:513: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:509: done (3 left) 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:510: done (2 left) 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:512: done (1 left) 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:514: rollback oid 8 current snap is 53 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 8 to 49 2024-03-20T08:02:59.217 INFO:tasks.rados.rados.0.smithi118.stdout:513: finishing copy_from to smithi11881747-41 2024-03-20T08:02:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 51 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:02:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:513: done (1 left) 2024-03-20T08:02:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:515: snap_remove snap 45 2024-03-20T08:02:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:59 smithi196 ceph-mon[31033]: osdmap e162: 8 total, 8 up, 8 in 2024-03-20T08:02:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:59 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:59 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:59 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:02:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:59 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:02:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:59 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:59 smithi196 ceph-mon[31033]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-20T08:02:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:02:59 smithi196 ceph-mon[31033]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-20T08:02:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[27391]: osdmap e162: 8 total, 8 up, 8 in 2024-03-20T08:02:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[27391]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[27391]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[38981]: osdmap e162: 8 total, 8 up, 8 in 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[38981]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-20T08:02:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:02:59 smithi118 ceph-mon[38981]: Upgrade: Need to upgrade myself (mgr.x) 2024-03-20T08:03:00.102 INFO:tasks.rados.rados.0.smithi118.stdout:514: finishing rollback tid 0 to smithi11881747-8 2024-03-20T08:03:00.102 INFO:tasks.rados.rados.0.smithi118.stdout:514: finishing rollback tid 1 to smithi11881747-8 2024-03-20T08:03:00.102 INFO:tasks.rados.rados.0.smithi118.stdout:514: finishing rollback tid 2 to smithi11881747-8 2024-03-20T08:03:00.102 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 129 (ObjNum 172 snap 49 seq_num 172) dirty exists 2024-03-20T08:03:00.102 INFO:tasks.rados.rados.0.smithi118.stdout:514: done (1 left) 2024-03-20T08:03:00.102 INFO:tasks.rados.rados.0.smithi118.stdout:515: done (0 left) 2024-03-20T08:03:00.103 INFO:tasks.rados.rados.0.smithi118.stdout:516: snap_remove snap 51 2024-03-20T08:03:00.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:00 smithi196 ceph-mon[31033]: osdmap e163: 8 total, 8 up, 8 in 2024-03-20T08:03:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:00 smithi196 ceph-mon[31033]: pgmap v36: 121 pgs: 121 active+clean; 129 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 716 KiB/s wr, 16 op/s 2024-03-20T08:03:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:00 smithi118 ceph-mon[27391]: osdmap e163: 8 total, 8 up, 8 in 2024-03-20T08:03:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:00 smithi118 ceph-mon[27391]: pgmap v36: 121 pgs: 121 active+clean; 129 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 716 KiB/s wr, 16 op/s 2024-03-20T08:03:00.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:00 smithi118 ceph-mon[38981]: osdmap e163: 8 total, 8 up, 8 in 2024-03-20T08:03:00.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:00 smithi118 ceph-mon[38981]: pgmap v36: 121 pgs: 121 active+clean; 129 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 716 KiB/s wr, 16 op/s 2024-03-20T08:03:00.847 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:03:01.021 INFO:tasks.rados.rados.0.smithi118.stdout:516: done (0 left) 2024-03-20T08:03:01.021 INFO:tasks.rados.rados.0.smithi118.stdout:517: snap_create 2024-03-20T08:03:01.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: osdmap e164: 8 total, 8 up, 8 in 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: Upgrade: Updating mgr.y 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: Deploying daemon mgr.y on smithi118 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:01.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:01 smithi196 ceph-mon[31033]: osdmap e165: 8 total, 8 up, 8 in 2024-03-20T08:03:01.179 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:03:01.179 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (2m) 14s ago 3m 28.5M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (3m) 13s ago 3m 89.3M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (4m) 14s ago 4m 13.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (4m) 13s ago 4m 18.0M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (38s) 13s ago 8m 492M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 838b6245d898 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:9283,8765 running (10m) 14s ago 10m 375M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 64dace5519ed 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (10m) 14s ago 10m 52.7M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 89e74e5abc84 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (8m) 13s ago 8m 44.9M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 fd26544c725b 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (8m) 14s ago 8m 48.9M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 8193e23658db 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (3m) 14s ago 3m 14.8M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (2m) 13s ago 2m 15.5M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (7m) 14s ago 7m 93.4M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 481bba84ab31 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (7m) 14s ago 7m 87.7M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (7m) 14s ago 7m 78.2M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (6m) 14s ago 6m 67.6M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (6m) 13s ago 6m 89.9M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:03:01.180 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (5m) 13s ago 5m 79.6M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:03:01.181 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (5m) 13s ago 5m 66.7M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:03:01.181 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (5m) 13s ago 5m 60.3M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:03:01.181 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 starting - - - - 2024-03-20T08:03:01.311 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: osdmap e164: 8 total, 8 up, 8 in 2024-03-20T08:03:01.311 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: Upgrade: Updating mgr.y 2024-03-20T08:03:01.311 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:01.311 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:01.311 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:01.311 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: Deploying daemon mgr.y on smithi118 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[27391]: osdmap e165: 8 total, 8 up, 8 in 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: osdmap e164: 8 total, 8 up, 8 in 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: Upgrade: Updating mgr.y 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:01.312 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: Deploying daemon mgr.y on smithi118 2024-03-20T08:03:01.313 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:01.313 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:01 smithi118 ceph-mon[38981]: osdmap e165: 8 total, 8 up, 8 in 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 1, 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 1 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:03:01.574 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:03:01.575 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:03:01.575 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:03:01.575 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:03:01.575 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 14, 2024-03-20T08:03:01.575 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 1 2024-03-20T08:03:01.575 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:03:01.575 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:03:02.022 INFO:tasks.rados.rados.0.smithi118.stdout:517: done (0 left) 2024-03-20T08:03:02.022 INFO:tasks.rados.rados.0.smithi118.stdout:518: read oid 47 snap -1 2024-03-20T08:03:02.023 INFO:tasks.rados.rados.0.smithi118.stdout:518: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:03:02.023 INFO:tasks.rados.rados.0.smithi118.stdout:519: delete oid 33 current snap is 54 2024-03-20T08:03:02.025 INFO:tasks.rados.rados.0.smithi118.stdout:519: done (1 left) 2024-03-20T08:03:02.025 INFO:tasks.rados.rados.0.smithi118.stdout:520: rollback oid 8 current snap is 54 2024-03-20T08:03:02.025 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 8 to 40 2024-03-20T08:03:02.025 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 54 2024-03-20T08:03:02.025 INFO:tasks.rados.rados.0.smithi118.stdout:521: seq_num 179 ranges {352256=606208} 2024-03-20T08:03:02.031 INFO:tasks.rados.rados.0.smithi118.stdout:521: writing smithi11881747-12 from 352256 to 958464 tid 1 2024-03-20T08:03:02.031 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 54 2024-03-20T08:03:02.031 INFO:tasks.rados.rados.0.smithi118.stdout:522: seq_num 180 ranges {5849088=622592} 2024-03-20T08:03:02.069 INFO:tasks.rados.rados.0.smithi118.stdout:522: writing smithi11881747-31 from 5849088 to 6471680 tid 1 2024-03-20T08:03:02.069 INFO:tasks.rados.rados.0.smithi118.stdout:523: rollback oid 41 current snap is 54 2024-03-20T08:03:02.069 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 41 to 42 2024-03-20T08:03:02.069 INFO:tasks.rados.rados.0.smithi118.stdout:518: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:03:02.096 INFO:tasks.rados.rados.0.smithi118.stdout:520: finishing rollback tid 0 to smithi11881747-8 2024-03-20T08:03:02.096 INFO:tasks.rados.rados.0.smithi118.stdout:520: finishing rollback tid 1 to smithi11881747-8 2024-03-20T08:03:02.096 INFO:tasks.rados.rados.0.smithi118.stdout:518: done (4 left) 2024-03-20T08:03:02.096 INFO:tasks.rados.rados.0.smithi118.stdout:524: read oid 18 snap -1 2024-03-20T08:03:02.096 INFO:tasks.rados.rados.0.smithi118.stdout:524: expect deleted 2024-03-20T08:03:02.096 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 54 2024-03-20T08:03:02.097 INFO:tasks.rados.rados.0.smithi118.stdout:520: finishing rollback tid 2 to smithi11881747-8 2024-03-20T08:03:02.097 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 137 (ObjNum 122 snap 32 seq_num 122) dirty exists 2024-03-20T08:03:02.097 INFO:tasks.rados.rados.0.smithi118.stdout:521: finishing write tid 1 to smithi11881747-12 2024-03-20T08:03:02.097 INFO:tasks.rados.rados.0.smithi118.stdout:521: finishing write tid 2 to smithi11881747-12 2024-03-20T08:03:02.097 INFO:tasks.rados.rados.0.smithi118.stdout:525: seq_num 181 ranges {679936=638976} 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:525: writing smithi11881747-36 from 679936 to 1318912 tid 1 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:520: done (5 left) 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:526: snap_create 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:521: finishing write tid 3 to smithi11881747-12 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:521: oid 12 updating version 0 to 51 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:521: oid 12 updating version 51 to 52 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 52 (ObjNum 179 snap 54 seq_num 179) dirty exists 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:521: left oid 12 (ObjNum 179 snap 54 seq_num 179) 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:523: finishing rollback tid 0 to smithi11881747-41 2024-03-20T08:03:02.104 INFO:tasks.rados.rados.0.smithi118.stdout:523: finishing rollback tid 1 to smithi11881747-41 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:523: finishing rollback tid 2 to smithi11881747-41 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 299 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:522: finishing write tid 1 to smithi11881747-31 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:522: finishing write tid 2 to smithi11881747-31 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:522: finishing write tid 3 to smithi11881747-31 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:522: oid 31 updating version 0 to 234 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:522: oid 31 version 234 is already newer than 233 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 234 (ObjNum 180 snap 54 seq_num 180) dirty exists 2024-03-20T08:03:02.105 INFO:tasks.rados.rados.0.smithi118.stdout:522: left oid 31 (ObjNum 180 snap 54 seq_num 180) 2024-03-20T08:03:02.107 INFO:tasks.rados.rados.0.smithi118.stdout:525: finishing write tid 1 to smithi11881747-36 2024-03-20T08:03:02.108 INFO:tasks.rados.rados.0.smithi118.stdout:525: finishing write tid 2 to smithi11881747-36 2024-03-20T08:03:02.109 INFO:tasks.rados.rados.0.smithi118.stdout:525: finishing write tid 3 to smithi11881747-36 2024-03-20T08:03:02.109 INFO:tasks.rados.rados.0.smithi118.stdout:525: oid 36 updating version 0 to 80 2024-03-20T08:03:02.109 INFO:tasks.rados.rados.0.smithi118.stdout:525: oid 36 updating version 80 to 81 2024-03-20T08:03:02.109 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 81 (ObjNum 181 snap 54 seq_num 181) dirty exists 2024-03-20T08:03:02.109 INFO:tasks.rados.rados.0.smithi118.stdout:525: left oid 36 (ObjNum 181 snap 54 seq_num 181) 2024-03-20T08:03:02.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:02 smithi118 ceph-mon[38981]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:02.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:02 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3028723745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:02.242 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:02 smithi118 ceph-mon[38981]: pgmap v39: 121 pgs: 121 active+clean; 129 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 716 KiB/s wr, 16 op/s 2024-03-20T08:03:02.243 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:02 smithi118 ceph-mon[38981]: osdmap e166: 8 total, 8 up, 8 in 2024-03-20T08:03:02.243 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:02 smithi118 ceph-mon[27391]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:02.243 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:02 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/3028723745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:02.243 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:02 smithi118 ceph-mon[27391]: pgmap v39: 121 pgs: 121 active+clean; 129 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 716 KiB/s wr, 16 op/s 2024-03-20T08:03:02.243 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:02 smithi118 ceph-mon[27391]: osdmap e166: 8 total, 8 up, 8 in 2024-03-20T08:03:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:02 smithi196 ceph-mon[31033]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:02 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3028723745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:02 smithi196 ceph-mon[31033]: pgmap v39: 121 pgs: 121 active+clean; 129 MiB data, 499 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 716 KiB/s wr, 16 op/s 2024-03-20T08:03:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:02 smithi196 ceph-mon[31033]: osdmap e166: 8 total, 8 up, 8 in 2024-03-20T08:03:02.520 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:02 smithi118 systemd[1]: Stopping Ceph mgr.y for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:03:02.823 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:02 smithi118 podman[84361]: 2024-03-20 08:03:02.521215597 +0000 UTC m=+0.138715239 container died 64dace5519ed3ce907358ac484af4ed5a9035b244e9ecf041a247c392b5a8c3a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=reef-78dbc53, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-20T08:03:03.107 INFO:tasks.rados.rados.0.smithi118.stdout:521: done (5 left) 2024-03-20T08:03:03.107 INFO:tasks.rados.rados.0.smithi118.stdout:522: done (4 left) 2024-03-20T08:03:03.107 INFO:tasks.rados.rados.0.smithi118.stdout:523: done (3 left) 2024-03-20T08:03:03.107 INFO:tasks.rados.rados.0.smithi118.stdout:524: done (2 left) 2024-03-20T08:03:03.108 INFO:tasks.rados.rados.0.smithi118.stdout:525: done (1 left) 2024-03-20T08:03:03.108 INFO:tasks.rados.rados.0.smithi118.stdout:526: done (0 left) 2024-03-20T08:03:03.108 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 55 2024-03-20T08:03:03.108 INFO:tasks.rados.rados.0.smithi118.stdout:527: seq_num 182 ranges {3137536=655360} 2024-03-20T08:03:03.127 INFO:tasks.rados.rados.0.smithi118.stdout:527: writing smithi11881747-23 from 3137536 to 3792896 tid 1 2024-03-20T08:03:03.128 INFO:tasks.rados.rados.0.smithi118.stdout:528: copy_from oid 26 from oid 22 current snap is 55 2024-03-20T08:03:03.128 INFO:tasks.rados.rados.0.smithi118.stdout:529: snap_create 2024-03-20T08:03:03.131 INFO:tasks.rados.rados.0.smithi118.stdout:527: finishing write tid 1 to smithi11881747-23 2024-03-20T08:03:03.131 INFO:tasks.rados.rados.0.smithi118.stdout:527: finishing write tid 2 to smithi11881747-23 2024-03-20T08:03:03.133 INFO:tasks.rados.rados.0.smithi118.stdout:527: finishing write tid 3 to smithi11881747-23 2024-03-20T08:03:03.133 INFO:tasks.rados.rados.0.smithi118.stdout:527: oid 23 updating version 0 to 301 2024-03-20T08:03:03.133 INFO:tasks.rados.rados.0.smithi118.stdout:527: oid 23 updating version 301 to 302 2024-03-20T08:03:03.133 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 302 (ObjNum 182 snap 55 seq_num 182) dirty exists 2024-03-20T08:03:03.133 INFO:tasks.rados.rados.0.smithi118.stdout:527: left oid 23 (ObjNum 182 snap 55 seq_num 182) 2024-03-20T08:03:03.144 INFO:tasks.rados.rados.0.smithi118.stdout:528: finishing copy_from racing read to smithi11881747-26 2024-03-20T08:03:03.147 INFO:tasks.rados.rados.0.smithi118.stdout:528: finishing copy_from to smithi11881747-26 2024-03-20T08:03:03.147 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:03:03.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:02 smithi118 podman[84361]: 2024-03-20 08:03:02.898203859 +0000 UTC m=+0.515703513 container cleanup 64dace5519ed3ce907358ac484af4ed5a9035b244e9ecf041a247c392b5a8c3a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-20T08:03:03.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:02 smithi118 bash[84361]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y 2024-03-20T08:03:03.467 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:03 smithi118 podman[84381]: 2024-03-20 08:03:03.216369209 +0000 UTC m=+0.693907732 container remove 64dace5519ed3ce907358ac484af4ed5a9035b244e9ecf041a247c392b5a8c3a (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=reef-78dbc53, org.label-schema.build-date=20240319) 2024-03-20T08:03:03.467 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:03 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:03:03.882 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:03:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:03:03.511Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:03:03.882 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:03 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-03-20T08:03:03.883 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:03 smithi118 systemd[1]: Stopped Ceph mgr.y for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:03:03.883 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:03 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y.service: Consumed 1min 3.439s CPU time. 2024-03-20T08:03:03.883 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:03 smithi118 systemd[1]: Starting Ceph mgr.y for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:03:04.112 INFO:tasks.rados.rados.0.smithi118.stdout:527: done (2 left) 2024-03-20T08:03:04.113 INFO:tasks.rados.rados.0.smithi118.stdout:528: done (1 left) 2024-03-20T08:03:04.113 INFO:tasks.rados.rados.0.smithi118.stdout:529: done (0 left) 2024-03-20T08:03:04.113 INFO:tasks.rados.rados.0.smithi118.stdout:530: snap_create 2024-03-20T08:03:04.152 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:04 smithi118 ceph-mon[27391]: osdmap e167: 8 total, 8 up, 8 in 2024-03-20T08:03:04.152 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:04 smithi118 ceph-mon[27391]: pgmap v42: 121 pgs: 121 active+clean; 129 MiB data, 503 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 296 KiB/s wr, 1 op/s 2024-03-20T08:03:04.152 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:04 smithi118 ceph-mon[38981]: osdmap e167: 8 total, 8 up, 8 in 2024-03-20T08:03:04.152 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:04 smithi118 ceph-mon[38981]: pgmap v42: 121 pgs: 121 active+clean; 129 MiB data, 503 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 296 KiB/s wr, 1 op/s 2024-03-20T08:03:04.153 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:03 smithi118 podman[84518]: 2024-03-20 08:03:03.788053575 +0000 UTC m=+0.019060750 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:03:04.153 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:03 smithi118 podman[84518]: 2024-03-20 08:03:03.892578864 +0000 UTC m=+0.123586028 container create 4dac06b91452454138244ea59c0207d6be22f4b7ece73c48b89d10ca2b044ef8 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:03:04.153 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:04 smithi118 podman[84518]: 2024-03-20 08:03:04.153863722 +0000 UTC m=+0.384870903 container init 4dac06b91452454138244ea59c0207d6be22f4b7ece73c48b89d10ca2b044ef8 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-20T08:03:04.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:04 smithi196 ceph-mon[31033]: osdmap e167: 8 total, 8 up, 8 in 2024-03-20T08:03:04.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:04 smithi196 ceph-mon[31033]: pgmap v42: 121 pgs: 121 active+clean; 129 MiB data, 503 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 296 KiB/s wr, 1 op/s 2024-03-20T08:03:04.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:04 smithi118 podman[84518]: 2024-03-20 08:03:04.159131977 +0000 UTC m=+0.390139143 container start 4dac06b91452454138244ea59c0207d6be22f4b7ece73c48b89d10ca2b044ef8 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-20T08:03:04.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:04 smithi118 ceph-mgr[84555]: -- 172.21.15.118:0/2638697910 <== mon.0 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560ab1a305a0 con 0x560ab1a04800 2024-03-20T08:03:04.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:04 smithi118 bash[84518]: 4dac06b91452454138244ea59c0207d6be22f4b7ece73c48b89d10ca2b044ef8 2024-03-20T08:03:04.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:04 smithi118 systemd[1]: Started Ceph mgr.y for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:03:05.113 INFO:tasks.rados.rados.0.smithi118.stdout:530: done (0 left) 2024-03-20T08:03:05.113 INFO:tasks.rados.rados.0.smithi118.stdout:531: delete oid 4 current snap is 57 2024-03-20T08:03:05.114 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:04.724+0000 7f1f24720200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T08:03:05.114 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:04.824+0000 7f1f24720200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T08:03:05.119 INFO:tasks.rados.rados.0.smithi118.stdout:531: done (0 left) 2024-03-20T08:03:05.119 INFO:tasks.rados.rados.0.smithi118.stdout:532: delete oid 15 current snap is 57 2024-03-20T08:03:05.123 INFO:tasks.rados.rados.0.smithi118.stdout:532: done (0 left) 2024-03-20T08:03:05.123 INFO:tasks.rados.rados.0.smithi118.stdout:533: snap_create 2024-03-20T08:03:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:05 smithi196 ceph-mon[31033]: osdmap e168: 8 total, 8 up, 8 in 2024-03-20T08:03:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:05 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:05 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:05 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-mon[27391]: osdmap e168: 8 total, 8 up, 8 in 2024-03-20T08:03:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:05.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-mon[38981]: osdmap e168: 8 total, 8 up, 8 in 2024-03-20T08:03:05.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:05.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:05.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:05.759 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:05.512+0000 7f1f24720200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T08:03:05.759 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:05.604+0000 7f1f24720200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T08:03:06.016 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:05.759+0000 7f1f24720200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T08:03:06.016 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:05.858+0000 7f1f24720200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T08:03:06.016 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:05.940+0000 7f1f24720200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T08:03:06.122 INFO:tasks.rados.rados.0.smithi118.stdout:533: done (0 left) 2024-03-20T08:03:06.122 INFO:tasks.rados.rados.0.smithi118.stdout:534: setattr oid 19 current snap is 58 2024-03-20T08:03:06.122 INFO:tasks.rados.rados.0.smithi118.stdout:535: rmattr oid 42 current snap is 58 2024-03-20T08:03:06.123 INFO:tasks.rados.rados.0.smithi118.stdout:535: done (1 left) 2024-03-20T08:03:06.123 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 58 2024-03-20T08:03:06.123 INFO:tasks.rados.rados.0.smithi118.stdout:536: seq_num 185 ranges {638976=704512} 2024-03-20T08:03:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:536: writing smithi11881747-39 from 638976 to 1343488 tid 1 2024-03-20T08:03:06.132 INFO:tasks.rados.rados.0.smithi118.stdout:537: read oid 47 snap -1 2024-03-20T08:03:06.132 INFO:tasks.rados.rados.0.smithi118.stdout:537: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:03:06.132 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 58 2024-03-20T08:03:06.132 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 79 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:03:06.132 INFO:tasks.rados.rados.0.smithi118.stdout:538: seq_num 186 ranges {679936=720896} 2024-03-20T08:03:06.141 INFO:tasks.rados.rados.0.smithi118.stdout:538: writing smithi11881747-41 from 679936 to 1400832 tid 1 2024-03-20T08:03:06.141 INFO:tasks.rados.rados.0.smithi118.stdout:534: done (3 left) 2024-03-20T08:03:06.141 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 58 2024-03-20T08:03:06.141 INFO:tasks.rados.rados.0.smithi118.stdout:539: seq_num 187 ranges {3792896=737280} 2024-03-20T08:03:06.166 INFO:tasks.rados.rados.0.smithi118.stdout:539: writing smithi11881747-23 from 3792896 to 4530176 tid 1 2024-03-20T08:03:06.166 INFO:tasks.rados.rados.0.smithi118.stdout:540: read oid 30 snap -1 2024-03-20T08:03:06.166 INFO:tasks.rados.rados.0.smithi118.stdout:540: expect (ObjNum 167 snap 45 seq_num 167) 2024-03-20T08:03:06.166 INFO:tasks.rados.rados.0.smithi118.stdout:541: snap_create 2024-03-20T08:03:06.166 INFO:tasks.rados.rados.0.smithi118.stdout:536: finishing write tid 1 to smithi11881747-39 2024-03-20T08:03:06.166 INFO:tasks.rados.rados.0.smithi118.stdout:537: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:03:06.203 INFO:tasks.rados.rados.0.smithi118.stdout:536: finishing write tid 2 to smithi11881747-39 2024-03-20T08:03:06.203 INFO:tasks.rados.rados.0.smithi118.stdout:536: finishing write tid 3 to smithi11881747-39 2024-03-20T08:03:06.203 INFO:tasks.rados.rados.0.smithi118.stdout:536: oid 39 updating version 0 to 246 2024-03-20T08:03:06.203 INFO:tasks.rados.rados.0.smithi118.stdout:536: oid 39 version 246 is already newer than 245 2024-03-20T08:03:06.203 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 246 (ObjNum 185 snap 58 seq_num 185) dirty exists 2024-03-20T08:03:06.203 INFO:tasks.rados.rados.0.smithi118.stdout:536: left oid 39 (ObjNum 185 snap 58 seq_num 185) 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:538: finishing write tid 1 to smithi11881747-41 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:538: finishing write tid 2 to smithi11881747-41 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:538: finishing write tid 3 to smithi11881747-41 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:538: oid 41 updating version 0 to 304 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:538: oid 41 updating version 304 to 305 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 305 (ObjNum 186 snap 58 seq_num 186) dirty exists 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:538: left oid 41 (ObjNum 186 snap 58 seq_num 186) 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:539: finishing write tid 1 to smithi11881747-23 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:539: finishing write tid 2 to smithi11881747-23 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:539: finishing write tid 3 to smithi11881747-23 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:539: oid 23 updating version 0 to 307 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:539: oid 23 updating version 307 to 308 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 308 (ObjNum 187 snap 58 seq_num 187) dirty exists 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:539: left oid 23 (ObjNum 187 snap 58 seq_num 187) 2024-03-20T08:03:06.204 INFO:tasks.rados.rados.0.smithi118.stdout:540: expect (ObjNum 167 snap 45 seq_num 167) 2024-03-20T08:03:06.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-mon[27391]: osdmap e169: 8 total, 8 up, 8 in 2024-03-20T08:03:06.267 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-mon[27391]: pgmap v45: 121 pgs: 121 active+clean; 131 MiB data, 504 MiB used, 715 GiB / 715 GiB avail; 571 KiB/s rd, 2.2 MiB/s wr, 7 op/s 2024-03-20T08:03:06.268 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:06.231+0000 7f1f24720200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T08:03:06.268 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-mon[38981]: osdmap e169: 8 total, 8 up, 8 in 2024-03-20T08:03:06.268 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-mon[38981]: pgmap v45: 121 pgs: 121 active+clean; 131 MiB data, 504 MiB used, 715 GiB / 715 GiB avail; 571 KiB/s rd, 2.2 MiB/s wr, 7 op/s 2024-03-20T08:03:06.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:06 smithi196 ceph-mon[31033]: osdmap e169: 8 total, 8 up, 8 in 2024-03-20T08:03:06.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:06 smithi196 ceph-mon[31033]: pgmap v45: 121 pgs: 121 active+clean; 131 MiB data, 504 MiB used, 715 GiB / 715 GiB avail; 571 KiB/s rd, 2.2 MiB/s wr, 7 op/s 2024-03-20T08:03:06.560 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:06.348+0000 7f1f24720200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:06.560 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:06.440+0000 7f1f24720200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T08:03:06.561 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:06.560+0000 7f1f24720200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T08:03:06.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:06.790+0000 7f1f24720200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T08:03:07.124 INFO:tasks.rados.rados.0.smithi118.stdout:536: done (5 left) 2024-03-20T08:03:07.124 INFO:tasks.rados.rados.0.smithi118.stdout:537: done (4 left) 2024-03-20T08:03:07.124 INFO:tasks.rados.rados.0.smithi118.stdout:538: done (3 left) 2024-03-20T08:03:07.124 INFO:tasks.rados.rados.0.smithi118.stdout:539: done (2 left) 2024-03-20T08:03:07.124 INFO:tasks.rados.rados.0.smithi118.stdout:540: done (1 left) 2024-03-20T08:03:07.125 INFO:tasks.rados.rados.0.smithi118.stdout:541: done (0 left) 2024-03-20T08:03:07.125 INFO:tasks.rados.rados.0.smithi118.stdout:542: rollback oid 42 current snap is 59 2024-03-20T08:03:07.125 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 23 2024-03-20T08:03:07.125 INFO:tasks.rados.rados.0.smithi118.stdout:543: copy_from oid 3 from oid 1 current snap is 59 2024-03-20T08:03:07.125 INFO:tasks.rados.rados.0.smithi118.stdout:544: snap_remove snap 46 2024-03-20T08:03:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:07 smithi196 ceph-mon[31033]: osdmap e170: 8 total, 8 up, 8 in 2024-03-20T08:03:07.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:07 smithi118 ceph-mon[27391]: osdmap e170: 8 total, 8 up, 8 in 2024-03-20T08:03:07.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:07 smithi118 ceph-mon[38981]: osdmap e170: 8 total, 8 up, 8 in 2024-03-20T08:03:07.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:03:07] "GET /metrics HTTP/1.1" 200 36069 "" "Prometheus/2.43.0" 2024-03-20T08:03:08.129 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:07.835+0000 7f1f24720200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T08:03:08.129 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:07.975+0000 7f1f24720200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:544: done (2 left) 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:545: snap_create 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:543: finishing copy_from to smithi11881747-3 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:543: got expected ENOENT (src dne) 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:543: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:542: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:542: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:542: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:03:08.133 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 207 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-03-20T08:03:08.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:08 smithi196 ceph-mon[31033]: osdmap e171: 8 total, 8 up, 8 in 2024-03-20T08:03:08.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:08 smithi196 ceph-mon[31033]: pgmap v48: 121 pgs: 121 active+clean; 131 MiB data, 504 MiB used, 715 GiB / 715 GiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-03-20T08:03:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:08 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:08.419 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-mon[27391]: osdmap e171: 8 total, 8 up, 8 in 2024-03-20T08:03:08.420 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-mon[27391]: pgmap v48: 121 pgs: 121 active+clean; 131 MiB data, 504 MiB used, 715 GiB / 715 GiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-03-20T08:03:08.420 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:08.420 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:08.322+0000 7f1f24720200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T08:03:08.421 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-mon[38981]: osdmap e171: 8 total, 8 up, 8 in 2024-03-20T08:03:08.421 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-mon[38981]: pgmap v48: 121 pgs: 121 active+clean; 131 MiB data, 504 MiB used, 715 GiB / 715 GiB avail; 571 KiB/s rd, 1.9 MiB/s wr, 5 op/s 2024-03-20T08:03:08.421 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:08.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:08.419+0000 7f1f24720200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T08:03:08.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:08.630+0000 7f1f24720200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:09.134 INFO:tasks.rados.rados.0.smithi118.stdout:542: done (2 left) 2024-03-20T08:03:09.134 INFO:tasks.rados.rados.0.smithi118.stdout:543: done (1 left) 2024-03-20T08:03:09.134 INFO:tasks.rados.rados.0.smithi118.stdout:545: done (0 left) 2024-03-20T08:03:09.134 INFO:tasks.rados.rados.0.smithi118.stdout:546: read oid 46 snap -1 2024-03-20T08:03:09.134 INFO:tasks.rados.rados.0.smithi118.stdout:546: expect (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:03:09.134 INFO:tasks.rados.rados.0.smithi118.stdout:547: delete oid 26 current snap is 60 2024-03-20T08:03:09.135 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:08.837+0000 7f1f24720200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T08:03:09.136 INFO:tasks.rados.rados.0.smithi118.stdout:547: done (1 left) 2024-03-20T08:03:09.136 INFO:tasks.rados.rados.0.smithi118.stdout:548: delete oid 47 current snap is 60 2024-03-20T08:03:09.138 INFO:tasks.rados.rados.0.smithi118.stdout:548: done (1 left) 2024-03-20T08:03:09.138 INFO:tasks.rados.rados.0.smithi118.stdout:549: read oid 6 snap -1 2024-03-20T08:03:09.138 INFO:tasks.rados.rados.0.smithi118.stdout:549: expect (ObjNum 177 snap 53 seq_num 177) 2024-03-20T08:03:09.139 INFO:tasks.rados.rados.0.smithi118.stdout:550: rollback oid 31 current snap is 60 2024-03-20T08:03:09.139 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 42 2024-03-20T08:03:09.139 INFO:tasks.rados.rados.0.smithi118.stdout:551: delete oid 50 current snap is 60 2024-03-20T08:03:09.143 INFO:tasks.rados.rados.0.smithi118.stdout:551: done (3 left) 2024-03-20T08:03:09.143 INFO:tasks.rados.rados.0.smithi118.stdout:552: read oid 29 snap -1 2024-03-20T08:03:09.143 INFO:tasks.rados.rados.0.smithi118.stdout:552: expect deleted 2024-03-20T08:03:09.143 INFO:tasks.rados.rados.0.smithi118.stdout:553: snap_create 2024-03-20T08:03:09.144 INFO:tasks.rados.rados.0.smithi118.stdout:550: finishing rollback tid 0 to smithi11881747-31 2024-03-20T08:03:09.144 INFO:tasks.rados.rados.0.smithi118.stdout:550: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:03:09.144 INFO:tasks.rados.rados.0.smithi118.stdout:550: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:03:09.144 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 240 (ObjNum 123 snap 33 seq_num 123) dirty exists 2024-03-20T08:03:09.152 INFO:tasks.rados.rados.0.smithi118.stdout:546: expect (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:03:09.234 INFO:tasks.rados.rados.0.smithi118.stdout:549: expect (ObjNum 177 snap 53 seq_num 177) 2024-03-20T08:03:09.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:09 smithi196 ceph-mon[31033]: osdmap e172: 8 total, 8 up, 8 in 2024-03-20T08:03:09.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:09 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:09 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:09 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:09 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:09.171+0000 7f1f24720200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[27391]: osdmap e172: 8 total, 8 up, 8 in 2024-03-20T08:03:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[38981]: osdmap e172: 8 total, 8 up, 8 in 2024-03-20T08:03:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:09.698 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:09.444+0000 7f1f24720200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:09.698 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:09.698+0000 7f1f24720200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T08:03:10.143 INFO:tasks.rados.rados.0.smithi118.stdout:546: done (4 left) 2024-03-20T08:03:10.143 INFO:tasks.rados.rados.0.smithi118.stdout:549: done (3 left) 2024-03-20T08:03:10.144 INFO:tasks.rados.rados.0.smithi118.stdout:550: done (2 left) 2024-03-20T08:03:10.144 INFO:tasks.rados.rados.0.smithi118.stdout:552: done (1 left) 2024-03-20T08:03:10.144 INFO:tasks.rados.rados.0.smithi118.stdout:553: done (0 left) 2024-03-20T08:03:10.144 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 61 2024-03-20T08:03:10.144 INFO:tasks.rados.rados.0.smithi118.stdout:554: seq_num 191 ranges {1343488=802816} 2024-03-20T08:03:10.155 INFO:tasks.rados.rados.0.smithi118.stdout:554: writing smithi11881747-39 from 1343488 to 2146304 tid 1 2024-03-20T08:03:10.155 INFO:tasks.rados.rados.0.smithi118.stdout:555: read oid 48 snap -1 2024-03-20T08:03:10.155 INFO:tasks.rados.rados.0.smithi118.stdout:555: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:03:10.155 INFO:tasks.rados.rados.0.smithi118.stdout:556: read oid 26 snap 27 2024-03-20T08:03:10.155 INFO:tasks.rados.rados.0.smithi118.stdout:556: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:03:10.156 INFO:tasks.rados.rados.0.smithi118.stdout:557: rollback oid 14 current snap is 61 2024-03-20T08:03:10.156 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 49 2024-03-20T08:03:10.156 INFO:tasks.rados.rados.0.smithi118.stdout:558: snap_create 2024-03-20T08:03:10.159 INFO:tasks.rados.rados.0.smithi118.stdout:557: finishing rollback tid 0 to smithi11881747-14 2024-03-20T08:03:10.160 INFO:tasks.rados.rados.0.smithi118.stdout:557: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:03:10.160 INFO:tasks.rados.rados.0.smithi118.stdout:554: finishing write tid 1 to smithi11881747-39 2024-03-20T08:03:10.160 INFO:tasks.rados.rados.0.smithi118.stdout:557: finishing rollback tid 2 to smithi11881747-14 2024-03-20T08:03:10.160 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 143 (ObjNum 163 snap 45 seq_num 163) dirty exists 2024-03-20T08:03:10.161 INFO:tasks.rados.rados.0.smithi118.stdout:554: finishing write tid 2 to smithi11881747-39 2024-03-20T08:03:10.162 INFO:tasks.rados.rados.0.smithi118.stdout:554: finishing write tid 3 to smithi11881747-39 2024-03-20T08:03:10.162 INFO:tasks.rados.rados.0.smithi118.stdout:554: oid 39 updating version 0 to 253 2024-03-20T08:03:10.162 INFO:tasks.rados.rados.0.smithi118.stdout:554: oid 39 version 253 is already newer than 252 2024-03-20T08:03:10.162 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 253 (ObjNum 191 snap 61 seq_num 191) dirty exists 2024-03-20T08:03:10.162 INFO:tasks.rados.rados.0.smithi118.stdout:554: left oid 39 (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:10.162 INFO:tasks.rados.rados.0.smithi118.stdout:556: expect (ObjNum 25 snap 0 seq_num 25) 2024-03-20T08:03:10.173 INFO:tasks.rados.rados.0.smithi118.stdout:555: expect (ObjNum 46 snap 0 seq_num 46) 2024-03-20T08:03:10.371 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-mon[27391]: osdmap e173: 8 total, 8 up, 8 in 2024-03-20T08:03:10.371 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:10.371 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:10.371 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-mon[27391]: pgmap v51: 121 pgs: 121 active+clean; 133 MiB data, 508 MiB used, 715 GiB / 715 GiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 10 op/s 2024-03-20T08:03:10.371 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-mon[38981]: osdmap e173: 8 total, 8 up, 8 in 2024-03-20T08:03:10.372 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:10.372 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:10.372 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-mon[38981]: pgmap v51: 121 pgs: 121 active+clean; 133 MiB data, 508 MiB used, 715 GiB / 715 GiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 10 op/s 2024-03-20T08:03:10.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:10 smithi196 ceph-mon[31033]: osdmap e173: 8 total, 8 up, 8 in 2024-03-20T08:03:10.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:10 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:10.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:10 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:10.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:10 smithi196 ceph-mon[31033]: pgmap v51: 121 pgs: 121 active+clean; 133 MiB data, 508 MiB used, 715 GiB / 715 GiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 10 op/s 2024-03-20T08:03:10.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:10.572+0000 7f1f24720200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T08:03:11.024 INFO:tasks.rados.rados.0.smithi118.stdout:554: done (4 left) 2024-03-20T08:03:11.024 INFO:tasks.rados.rados.0.smithi118.stdout:555: done (3 left) 2024-03-20T08:03:11.024 INFO:tasks.rados.rados.0.smithi118.stdout:556: done (2 left) 2024-03-20T08:03:11.024 INFO:tasks.rados.rados.0.smithi118.stdout:557: done (1 left) 2024-03-20T08:03:11.024 INFO:tasks.rados.rados.0.smithi118.stdout:558: done (0 left) 2024-03-20T08:03:11.024 INFO:tasks.rados.rados.0.smithi118.stdout:559: copy_from oid 16 from oid 10 current snap is 62 2024-03-20T08:03:11.024 INFO:tasks.rados.rados.0.smithi118.stdout:560: snap_remove snap 56 2024-03-20T08:03:11.407 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-mon[27391]: osdmap e174: 8 total, 8 up, 8 in 2024-03-20T08:03:11.408 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-mon[27391]: osdmap e175: 8 total, 8 up, 8 in 2024-03-20T08:03:11.408 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-mon[38981]: osdmap e174: 8 total, 8 up, 8 in 2024-03-20T08:03:11.408 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-mon[38981]: osdmap e175: 8 total, 8 up, 8 in 2024-03-20T08:03:11.408 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:11.156+0000 7f1f24720200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T08:03:11.408 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:11.258+0000 7f1f24720200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T08:03:11.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:11 smithi196 ceph-mon[31033]: osdmap e174: 8 total, 8 up, 8 in 2024-03-20T08:03:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:11 smithi196 ceph-mon[31033]: osdmap e175: 8 total, 8 up, 8 in 2024-03-20T08:03:11.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:11.489+0000 7f1f24720200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T08:03:11.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:11] ENGINE Bus STARTING 2024-03-20T08:03:11.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: CherryPy Checker: 2024-03-20T08:03:11.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: The Application mounted at '' has an empty config. 2024-03-20T08:03:11.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:11.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:11] ENGINE Serving on http://:::9283 2024-03-20T08:03:11.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:11] ENGINE Bus STARTED 2024-03-20T08:03:12.031 INFO:tasks.rados.rados.0.smithi118.stdout:560: done (1 left) 2024-03-20T08:03:12.031 INFO:tasks.rados.rados.0.smithi118.stdout:561: read oid 35 snap -1 2024-03-20T08:03:12.031 INFO:tasks.rados.rados.0.smithi118.stdout:561: expect deleted 2024-03-20T08:03:12.031 INFO:tasks.rados.rados.0.smithi118.stdout:562: snap_create 2024-03-20T08:03:12.031 INFO:tasks.rados.rados.0.smithi118.stdout:559: finishing copy_from racing read to smithi11881747-16 2024-03-20T08:03:12.031 INFO:tasks.rados.rados.0.smithi118.stdout:559: finishing copy_from to smithi11881747-16 2024-03-20T08:03:12.031 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 188 (ObjNum 146 snap 41 seq_num 146) dirty exists 2024-03-20T08:03:12.396 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[27391]: Standby manager daemon y restarted 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[27391]: Standby manager daemon y started 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[27391]: pgmap v54: 121 pgs: 121 active+clean; 133 MiB data, 508 MiB used, 715 GiB / 715 GiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 10 op/s 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[27391]: osdmap e176: 8 total, 8 up, 8 in 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[38981]: Standby manager daemon y restarted 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[38981]: Standby manager daemon y started 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-20T08:03:12.397 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:12.398 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-20T08:03:12.398 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:12.398 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[38981]: pgmap v54: 121 pgs: 121 active+clean; 133 MiB data, 508 MiB used, 715 GiB / 715 GiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 10 op/s 2024-03-20T08:03:12.398 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:12 smithi118 ceph-mon[38981]: osdmap e176: 8 total, 8 up, 8 in 2024-03-20T08:03:12.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:12 smithi196 ceph-mon[31033]: Standby manager daemon y restarted 2024-03-20T08:03:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:12 smithi196 ceph-mon[31033]: Standby manager daemon y started 2024-03-20T08:03:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:12 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-20T08:03:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:12 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:12 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-20T08:03:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:12 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:12 smithi196 ceph-mon[31033]: pgmap v54: 121 pgs: 121 active+clean; 133 MiB data, 508 MiB used, 715 GiB / 715 GiB avail; 1.2 MiB/s rd, 528 KiB/s wr, 10 op/s 2024-03-20T08:03:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:12 smithi196 ceph-mon[31033]: osdmap e176: 8 total, 8 up, 8 in 2024-03-20T08:03:13.031 INFO:tasks.rados.rados.0.smithi118.stdout:559: done (2 left) 2024-03-20T08:03:13.032 INFO:tasks.rados.rados.0.smithi118.stdout:561: done (1 left) 2024-03-20T08:03:13.032 INFO:tasks.rados.rados.0.smithi118.stdout:562: done (0 left) 2024-03-20T08:03:13.032 INFO:tasks.rados.rados.0.smithi118.stdout:563: rmattr oid 33 current snap is 63 2024-03-20T08:03:13.032 INFO:tasks.rados.rados.0.smithi118.stdout:563: done (0 left) 2024-03-20T08:03:13.032 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 63 2024-03-20T08:03:13.032 INFO:tasks.rados.rados.0.smithi118.stdout:564: seq_num 192 ranges {0=417792,417792=401408} 2024-03-20T08:03:13.033 INFO:tasks.rados.rados.0.smithi118.stdout:564: writing smithi11881747-32 from 0 to 417792 tid 1 2024-03-20T08:03:13.036 INFO:tasks.rados.rados.0.smithi118.stdout:564: writing smithi11881747-32 from 417792 to 819200 tid 2 2024-03-20T08:03:13.036 INFO:tasks.rados.rados.0.smithi118.stdout:565: copy_from oid 11 from oid 45 current snap is 63 2024-03-20T08:03:13.036 INFO:tasks.rados.rados.0.smithi118.stdout:566: read oid 12 snap -1 2024-03-20T08:03:13.037 INFO:tasks.rados.rados.0.smithi118.stdout:566: expect (ObjNum 179 snap 54 seq_num 179) 2024-03-20T08:03:13.037 INFO:tasks.rados.rados.0.smithi118.stdout:567: rollback oid 13 current snap is 63 2024-03-20T08:03:13.037 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 13 to 49 2024-03-20T08:03:13.037 INFO:tasks.rados.rados.0.smithi118.stdout:568: setattr oid 30 current snap is 63 2024-03-20T08:03:13.037 INFO:tasks.rados.rados.0.smithi118.stdout:569: rmattr oid 41 current snap is 63 2024-03-20T08:03:13.037 INFO:tasks.rados.rados.0.smithi118.stdout:570: read oid 31 snap -1 2024-03-20T08:03:13.038 INFO:tasks.rados.rados.0.smithi118.stdout:570: expect (ObjNum 123 snap 33 seq_num 123) 2024-03-20T08:03:13.038 INFO:tasks.rados.rados.0.smithi118.stdout:571: rollback oid 26 current snap is 63 2024-03-20T08:03:13.038 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 32 2024-03-20T08:03:13.038 INFO:tasks.rados.rados.0.smithi118.stdout:572: copy_from oid 23 from oid 1 current snap is 63 2024-03-20T08:03:13.039 INFO:tasks.rados.rados.0.smithi118.stdout:573: delete oid 25 current snap is 63 2024-03-20T08:03:13.042 INFO:tasks.rados.rados.0.smithi118.stdout:567: finishing rollback tid 0 to smithi11881747-13 2024-03-20T08:03:13.045 INFO:tasks.rados.rados.0.smithi118.stdout:564: finishing write tid 1 to smithi11881747-32 2024-03-20T08:03:13.045 INFO:tasks.rados.rados.0.smithi118.stdout:564: finishing write tid 2 to smithi11881747-32 2024-03-20T08:03:13.045 INFO:tasks.rados.rados.0.smithi118.stdout:571: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:03:13.045 INFO:tasks.rados.rados.0.smithi118.stdout:564: finishing write tid 3 to smithi11881747-32 2024-03-20T08:03:13.045 INFO:tasks.rados.rados.0.smithi118.stdout:571: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:03:13.045 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 151 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-03-20T08:03:13.045 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 330 (ObjNum 186 snap 58 seq_num 186) dirty exists 2024-03-20T08:03:13.046 INFO:tasks.rados.rados.0.smithi118.stdout:566: expect (ObjNum 179 snap 54 seq_num 179) 2024-03-20T08:03:13.080 INFO:tasks.rados.rados.0.smithi118.stdout:564: finishing write tid 4 to smithi11881747-32 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:564: oid 32 updating version 0 to 146 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:564: oid 32 updating version 146 to 147 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:564: oid 32 updating version 147 to 148 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 148 (ObjNum 192 snap 63 seq_num 192) dirty exists 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:564: left oid 32 (ObjNum 192 snap 63 seq_num 192) 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 62 (ObjNum 167 snap 45 seq_num 167) dirty exists 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:565: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:567: finishing rollback tid 1 to smithi11881747-13 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:567: finishing rollback tid 2 to smithi11881747-13 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 215 (ObjNum 126 snap 34 seq_num 126) dirty exists 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:565: finishing copy_from to smithi11881747-11 2024-03-20T08:03:13.081 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 51 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:572: finishing copy_from to smithi11881747-23 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:572: got expected ENOENT (src dne) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:572: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:564: done (9 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:565: done (8 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:566: done (7 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:567: done (6 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:568: done (5 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:569: done (4 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:571: done (3 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:572: done (2 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:573: done (1 left) 2024-03-20T08:03:13.082 INFO:tasks.rados.rados.0.smithi118.stdout:574: rollback oid 23 current snap is 63 2024-03-20T08:03:13.083 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 23 to 27 2024-03-20T08:03:13.083 INFO:tasks.rados.rados.0.smithi118.stdout:575: snap_remove snap 42 2024-03-20T08:03:13.338 INFO:tasks.rados.rados.0.smithi118.stdout:575: done (2 left) 2024-03-20T08:03:13.339 INFO:tasks.rados.rados.0.smithi118.stdout:576: read oid 49 snap -1 2024-03-20T08:03:13.339 INFO:tasks.rados.rados.0.smithi118.stdout:576: expect (ObjNum 176 snap 52 seq_num 176) 2024-03-20T08:03:13.339 INFO:tasks.rados.rados.0.smithi118.stdout:577: delete oid 26 current snap is 63 2024-03-20T08:03:13.339 INFO:tasks.rados.rados.0.smithi118.stdout:574: finishing rollback tid 0 to smithi11881747-23 2024-03-20T08:03:13.339 INFO:tasks.rados.rados.0.smithi118.stdout:574: finishing rollback tid 1 to smithi11881747-23 2024-03-20T08:03:13.339 INFO:tasks.rados.rados.0.smithi118.stdout:574: finishing rollback tid 2 to smithi11881747-23 2024-03-20T08:03:13.339 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 334 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-03-20T08:03:13.339 INFO:tasks.rados.rados.0.smithi118.stdout:570: expect (ObjNum 123 snap 33 seq_num 123) 2024-03-20T08:03:13.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-mon[27391]: mgrmap e28: x(active, since 34s), standbys: y 2024-03-20T08:03:13.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-mon[27391]: osdmap e177: 8 total, 8 up, 8 in 2024-03-20T08:03:13.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:13] ENGINE Bus STOPPING 2024-03-20T08:03:13.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:03:13.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:13] ENGINE Bus STOPPED 2024-03-20T08:03:13.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-mon[38981]: mgrmap e28: x(active, since 34s), standbys: y 2024-03-20T08:03:13.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-mon[38981]: osdmap e177: 8 total, 8 up, 8 in 2024-03-20T08:03:13.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:13 smithi196 ceph-mon[31033]: mgrmap e28: x(active, since 34s), standbys: y 2024-03-20T08:03:13.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:13 smithi196 ceph-mon[31033]: osdmap e177: 8 total, 8 up, 8 in 2024-03-20T08:03:13.504 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: ignoring --setuser ceph since I am not root 2024-03-20T08:03:13.504 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: ignoring --setgroup ceph since I am not root 2024-03-20T08:03:13.504 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:13 smithi196 ceph-mgr[66322]: -- 172.21.15.196:0/1087888869 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555bd2abc5a0 con 0x555bd2a90800 2024-03-20T08:03:13.840 INFO:tasks.rados.rados.0.smithi118.stdout:576: expect (ObjNum 176 snap 52 seq_num 176) 2024-03-20T08:03:13.888 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:13.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:03:13.922 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:03:13.510Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:03:13.922 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:03:13.511Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:13] ENGINE Bus STARTING 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.492+0000 7f1e823ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.492+0000 7f1e823ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.492+0000 7f1e823ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.492+0000 7f1e823ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.492+0000 7f1e823ac700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.493+0000 7f1e853b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.493+0000 7f1e853b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.493+0000 7f1e853b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.493+0000 7f1e853b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:13.493+0000 7f1e853b2700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:13] ENGINE Serving on http://:::9283 2024-03-20T08:03:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:13] ENGINE Bus STARTED 2024-03-20T08:03:14.046 INFO:tasks.rados.rados.0.smithi118.stdout:570: done (3 left) 2024-03-20T08:03:14.046 INFO:tasks.rados.rados.0.smithi118.stdout:574: done (2 left) 2024-03-20T08:03:14.046 INFO:tasks.rados.rados.0.smithi118.stdout:576: done (1 left) 2024-03-20T08:03:14.046 INFO:tasks.rados.rados.0.smithi118.stdout:577: done (0 left) 2024-03-20T08:03:14.046 INFO:tasks.rados.rados.0.smithi118.stdout:578: rollback oid 44 current snap is 63 2024-03-20T08:03:14.046 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 44 to 44 2024-03-20T08:03:14.047 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 63 2024-03-20T08:03:14.047 INFO:tasks.rados.rados.0.smithi118.stdout:579: seq_num 195 ranges {892928=475136,1368064=401408} 2024-03-20T08:03:14.053 INFO:tasks.rados.rados.0.smithi118.stdout:579: writing smithi11881747-30 from 892928 to 1368064 tid 1 2024-03-20T08:03:14.055 INFO:tasks.rados.rados.0.smithi118.stdout:579: writing smithi11881747-30 from 1368064 to 1769472 tid 2 2024-03-20T08:03:14.056 INFO:tasks.rados.rados.0.smithi118.stdout:580: rollback oid 40 current snap is 63 2024-03-20T08:03:14.056 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 36 2024-03-20T08:03:14.056 INFO:tasks.rados.rados.0.smithi118.stdout:578: finishing rollback tid 0 to smithi11881747-44 2024-03-20T08:03:14.056 INFO:tasks.rados.rados.0.smithi118.stdout:578: finishing rollback tid 1 to smithi11881747-44 2024-03-20T08:03:14.056 INFO:tasks.rados.rados.0.smithi118.stdout:578: finishing rollback tid 2 to smithi11881747-44 2024-03-20T08:03:14.056 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 162 (ObjNum 156 snap 43 seq_num 156) dirty exists 2024-03-20T08:03:14.056 INFO:tasks.rados.rados.0.smithi118.stdout:578: done (2 left) 2024-03-20T08:03:14.056 INFO:tasks.rados.rados.0.smithi118.stdout:581: snap_remove snap 32 2024-03-20T08:03:14.343 INFO:tasks.rados.rados.0.smithi118.stdout:581: done (2 left) 2024-03-20T08:03:14.343 INFO:tasks.rados.rados.0.smithi118.stdout:582: snap_create 2024-03-20T08:03:14.343 INFO:tasks.rados.rados.0.smithi118.stdout:580: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:03:14.343 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 151 (ObjNum 129 snap 0 seq_num 321821856) dirty dne 2024-03-20T08:03:14.343 INFO:tasks.rados.rados.0.smithi118.stdout:579: finishing write tid 1 to smithi11881747-30 2024-03-20T08:03:14.344 INFO:tasks.rados.rados.0.smithi118.stdout:579: finishing write tid 2 to smithi11881747-30 2024-03-20T08:03:14.344 INFO:tasks.rados.rados.0.smithi118.stdout:579: finishing write tid 3 to smithi11881747-30 2024-03-20T08:03:14.344 INFO:tasks.rados.rados.0.smithi118.stdout:579: finishing write tid 4 to smithi11881747-30 2024-03-20T08:03:14.344 INFO:tasks.rados.rados.0.smithi118.stdout:579: oid 30 updating version 0 to 68 2024-03-20T08:03:14.344 INFO:tasks.rados.rados.0.smithi118.stdout:579: oid 30 updating version 68 to 69 2024-03-20T08:03:14.344 INFO:tasks.rados.rados.0.smithi118.stdout:579: oid 30 version 69 is already newer than 67 2024-03-20T08:03:14.344 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 69 (ObjNum 195 snap 63 seq_num 195) dirty exists 2024-03-20T08:03:14.344 INFO:tasks.rados.rados.0.smithi118.stdout:579: left oid 30 (ObjNum 195 snap 63 seq_num 195) 2024-03-20T08:03:14.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:13.929+0000 7ff959da5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T08:03:14.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:14.026+0000 7ff959da5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: osdmap e178: 8 total, 8 up, 8 in 2024-03-20T08:03:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.24805 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: mgrmap e29: y(active, starting, since 0.00740097s) 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:03:14.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: Manager daemon y is now available 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:14.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:14.422 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:03:14.047Z caller=notify.go:743 level=info component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify success" attempts=2 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: osdmap e178: 8 total, 8 up, 8 in 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.24805 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: mgrmap e29: y(active, starting, since 0.00740097s) 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:03:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:03:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: Manager daemon y is now available 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 172.21.15.196:0/4009833772' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: osdmap e178: 8 total, 8 up, 8 in 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.24805 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: mgrmap e29: y(active, starting, since 0.00740097s) 2024-03-20T08:03:14.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:03:14.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:03:14.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:03:14.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:03:14.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:03:14.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:03:14.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:03:14.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:03:14.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:03:14.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: Manager daemon y is now available 2024-03-20T08:03:14.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:14.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:14.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:14.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:14.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:14.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:14 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:14.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:14.738+0000 7ff959da5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T08:03:14.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:14.832+0000 7ff959da5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T08:03:15.346 INFO:tasks.rados.rados.0.smithi118.stdout:579: done (2 left) 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:580: done (1 left) 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:582: done (0 left) 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:583: rmattr oid 40 current snap is 64 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:583: done (0 left) 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:584: read oid 10 snap 52 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:584: expect (ObjNum 146 snap 41 seq_num 146) 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:585: read oid 50 snap -1 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:585: expect deleted 2024-03-20T08:03:15.347 INFO:tasks.rados.rados.0.smithi118.stdout:586: setattr oid 7 current snap is 64 2024-03-20T08:03:15.349 INFO:tasks.rados.rados.0.smithi118.stdout:585: done (2 left) 2024-03-20T08:03:15.349 INFO:tasks.rados.rados.0.smithi118.stdout:587: read oid 47 snap 60 2024-03-20T08:03:15.349 INFO:tasks.rados.rados.0.smithi118.stdout:587: expect deleted 2024-03-20T08:03:15.349 INFO:tasks.rados.rados.0.smithi118.stdout:588: copy_from oid 26 from oid 13 current snap is 64 2024-03-20T08:03:15.349 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 64 2024-03-20T08:03:15.349 INFO:tasks.rados.rados.0.smithi118.stdout:589: seq_num 196 ranges {2547712=491520,3039232=401408} 2024-03-20T08:03:15.365 INFO:tasks.rados.rados.0.smithi118.stdout:589: writing smithi11881747-45 from 2547712 to 3039232 tid 1 2024-03-20T08:03:15.367 INFO:tasks.rados.rados.0.smithi118.stdout:589: writing smithi11881747-45 from 3039232 to 3440640 tid 2 2024-03-20T08:03:15.368 INFO:tasks.rados.rados.0.smithi118.stdout:590: rollback oid 33 current snap is 64 2024-03-20T08:03:15.368 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 58 2024-03-20T08:03:15.368 INFO:tasks.rados.rados.0.smithi118.stdout:584: expect (ObjNum 146 snap 41 seq_num 146) 2024-03-20T08:03:15.369 INFO:tasks.rados.rados.0.smithi118.stdout:584: done (5 left) 2024-03-20T08:03:15.369 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 64 2024-03-20T08:03:15.369 INFO:tasks.rados.rados.0.smithi118.stdout:591: seq_num 197 ranges {0=507904,507904=401408} 2024-03-20T08:03:15.371 INFO:tasks.rados.rados.0.smithi118.stdout:591: writing smithi11881747-25 from 0 to 507904 tid 1 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:591: writing smithi11881747-25 from 507904 to 909312 tid 2 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:592: rollback oid 38 current snap is 64 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 38 to 47 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:593: rollback oid 42 current snap is 64 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 60 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:594: read oid 17 snap 58 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:594: expect (ObjNum 125 snap 33 seq_num 125) 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:595: copy_from oid 18 from oid 14 current snap is 64 2024-03-20T08:03:15.374 INFO:tasks.rados.rados.0.smithi118.stdout:596: snap_remove snap 36 2024-03-20T08:03:15.388 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:14.976+0000 7ff959da5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T08:03:15.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:15.076+0000 7ff959da5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T08:03:15.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:15.173+0000 7ff959da5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T08:03:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-mon[31033]: [20/Mar/2024:08:03:13] ENGINE Bus STARTING 2024-03-20T08:03:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-mon[31033]: [20/Mar/2024:08:03:13] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T08:03:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-mon[31033]: [20/Mar/2024:08:03:14] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T08:03:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-mon[31033]: [20/Mar/2024:08:03:14] ENGINE Bus STARTED 2024-03-20T08:03:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-mon[31033]: mgrmap e30: y(active, since 1.00827s) 2024-03-20T08:03:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-mon[31033]: osdmap e179: 8 total, 8 up, 8 in 2024-03-20T08:03:15.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[27391]: [20/Mar/2024:08:03:13] ENGINE Bus STARTING 2024-03-20T08:03:15.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[27391]: [20/Mar/2024:08:03:13] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T08:03:15.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[27391]: [20/Mar/2024:08:03:14] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T08:03:15.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[27391]: [20/Mar/2024:08:03:14] ENGINE Bus STARTED 2024-03-20T08:03:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[27391]: mgrmap e30: y(active, since 1.00827s) 2024-03-20T08:03:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[27391]: osdmap e179: 8 total, 8 up, 8 in 2024-03-20T08:03:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[38981]: [20/Mar/2024:08:03:13] ENGINE Bus STARTING 2024-03-20T08:03:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[38981]: [20/Mar/2024:08:03:13] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T08:03:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[38981]: [20/Mar/2024:08:03:14] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T08:03:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[38981]: [20/Mar/2024:08:03:14] ENGINE Bus STARTED 2024-03-20T08:03:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[38981]: mgrmap e30: y(active, since 1.00827s) 2024-03-20T08:03:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:15 smithi118 ceph-mon[38981]: osdmap e179: 8 total, 8 up, 8 in 2024-03-20T08:03:15.761 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:15.460+0000 7ff959da5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T08:03:15.762 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:15.578+0000 7ff959da5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:15.762 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:15.666+0000 7ff959da5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T08:03:16.026 INFO:tasks.rados.rados.0.smithi118.stdout:596: done (10 left) 2024-03-20T08:03:16.027 INFO:tasks.rados.rados.0.smithi118.stdout:597: snap_remove snap 48 2024-03-20T08:03:16.138 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:15.762+0000 7ff959da5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T08:03:16.139 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:15.952+0000 7ff959da5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T08:03:16.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:16 smithi196 ceph-mon[31033]: pgmap v4: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 32 GiB/s rd, 15 GiB/s wr, 281.56k op/s; 179 MiB/s, 740 objects/s recovering 2024-03-20T08:03:16.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:16 smithi196 ceph-mon[31033]: osdmap e180: 8 total, 8 up, 8 in 2024-03-20T08:03:16.468 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:16 smithi196 ceph-mon[31033]: osdmap e181: 8 total, 8 up, 8 in 2024-03-20T08:03:16.634 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:16 smithi118 ceph-mon[27391]: pgmap v4: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 32 GiB/s rd, 15 GiB/s wr, 281.56k op/s; 179 MiB/s, 740 objects/s recovering 2024-03-20T08:03:16.635 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:16 smithi118 ceph-mon[27391]: osdmap e180: 8 total, 8 up, 8 in 2024-03-20T08:03:16.635 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:16 smithi118 ceph-mon[27391]: osdmap e181: 8 total, 8 up, 8 in 2024-03-20T08:03:16.635 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:16 smithi118 ceph-mon[38981]: pgmap v4: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 32 GiB/s rd, 15 GiB/s wr, 281.56k op/s; 179 MiB/s, 740 objects/s recovering 2024-03-20T08:03:16.635 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:16 smithi118 ceph-mon[38981]: osdmap e180: 8 total, 8 up, 8 in 2024-03-20T08:03:16.635 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:16 smithi118 ceph-mon[38981]: osdmap e181: 8 total, 8 up, 8 in 2024-03-20T08:03:17.026 INFO:tasks.rados.rados.0.smithi118.stdout:597: done (10 left) 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:598: read oid 44 snap -1 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:598: expect (ObjNum 156 snap 43 seq_num 156) 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:599: copy_from oid 20 from oid 10 current snap is 64 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:600: snap_create 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:592: finishing rollback tid 0 to smithi11881747-38 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:592: finishing rollback tid 1 to smithi11881747-38 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:592: finishing rollback tid 2 to smithi11881747-38 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 187 (ObjNum 128 snap 34 seq_num 128) dirty exists 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:591: finishing write tid 1 to smithi11881747-25 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:591: finishing write tid 2 to smithi11881747-25 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:590: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 124 (ObjNum 33 snap 0 seq_num 291742968) dirty dne 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:591: finishing write tid 3 to smithi11881747-25 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:589: finishing write tid 1 to smithi11881747-45 2024-03-20T08:03:17.027 INFO:tasks.rados.rados.0.smithi118.stdout:589: finishing write tid 2 to smithi11881747-45 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:589: finishing write tid 3 to smithi11881747-45 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:595: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:589: finishing write tid 4 to smithi11881747-45 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:589: oid 45 updating version 0 to 170 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:589: oid 45 updating version 170 to 171 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:589: oid 45 version 171 is already newer than 169 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 171 (ObjNum 196 snap 64 seq_num 196) dirty exists 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:589: left oid 45 (ObjNum 196 snap 64 seq_num 196) 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:591: finishing write tid 4 to smithi11881747-25 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:591: oid 25 updating version 0 to 236 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:591: oid 25 updating version 236 to 237 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:591: oid 25 version 237 is already newer than 235 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 237 (ObjNum 197 snap 64 seq_num 197) dirty exists 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:591: left oid 25 (ObjNum 197 snap 64 seq_num 197) 2024-03-20T08:03:17.028 INFO:tasks.rados.rados.0.smithi118.stdout:593: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:593: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:593: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 241 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:595: finishing copy_from to smithi11881747-18 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 143 (ObjNum 163 snap 45 seq_num 163) dirty exists 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:588: finishing copy_from racing read to smithi11881747-26 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 60 (ObjNum 145 snap 41 seq_num 145) dirty exists 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:588: finishing copy_from to smithi11881747-26 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 215 (ObjNum 126 snap 34 seq_num 126) dirty exists 2024-03-20T08:03:17.029 INFO:tasks.rados.rados.0.smithi118.stdout:594: expect (ObjNum 125 snap 33 seq_num 125) 2024-03-20T08:03:17.245 INFO:tasks.rados.rados.0.smithi118.stdout:598: expect (ObjNum 156 snap 43 seq_num 156) 2024-03-20T08:03:17.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:17 smithi196 ceph-mon[31033]: pgmap v6: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 311 MiB/s rd, 147 MiB/s wr, 2.66k op/s; 1.7 MiB/s, 7 objects/s recovering 2024-03-20T08:03:17.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:17 smithi196 ceph-mon[31033]: mgrmap e31: y(active, since 3s) 2024-03-20T08:03:17.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:17 smithi196 ceph-mon[31033]: osdmap e182: 8 total, 8 up, 8 in 2024-03-20T08:03:17.248 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:16.993+0000 7ff959da5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T08:03:17.248 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:17.136+0000 7ff959da5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T08:03:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:599: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:03:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:599: finishing copy_from to smithi11881747-20 2024-03-20T08:03:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 188 (ObjNum 146 snap 41 seq_num 146) dirty exists 2024-03-20T08:03:17.499 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:17 smithi118 ceph-mon[27391]: pgmap v6: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 311 MiB/s rd, 147 MiB/s wr, 2.66k op/s; 1.7 MiB/s, 7 objects/s recovering 2024-03-20T08:03:17.499 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:17 smithi118 ceph-mon[27391]: mgrmap e31: y(active, since 3s) 2024-03-20T08:03:17.499 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:17 smithi118 ceph-mon[27391]: osdmap e182: 8 total, 8 up, 8 in 2024-03-20T08:03:17.499 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:17 smithi118 ceph-mon[38981]: pgmap v6: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 311 MiB/s rd, 147 MiB/s wr, 2.66k op/s; 1.7 MiB/s, 7 objects/s recovering 2024-03-20T08:03:17.499 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:17 smithi118 ceph-mon[38981]: mgrmap e31: y(active, since 3s) 2024-03-20T08:03:17.499 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:17 smithi118 ceph-mon[38981]: osdmap e182: 8 total, 8 up, 8 in 2024-03-20T08:03:17.809 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:17.501+0000 7ff959da5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T08:03:17.809 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:17.593+0000 7ff959da5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T08:03:17.809 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:17.809+0000 7ff959da5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:586: done (12 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:587: done (11 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:588: done (10 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:589: done (9 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:590: done (8 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:591: done (7 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:592: done (6 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:593: done (5 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:594: done (4 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:595: done (3 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:598: done (2 left) 2024-03-20T08:03:18.027 INFO:tasks.rados.rados.0.smithi118.stdout:599: done (1 left) 2024-03-20T08:03:18.028 INFO:tasks.rados.rados.0.smithi118.stdout:600: done (0 left) 2024-03-20T08:03:18.028 INFO:tasks.rados.rados.0.smithi118.stdout:601: setattr oid 21 current snap is 65 2024-03-20T08:03:18.029 INFO:tasks.rados.rados.0.smithi118.stdout:602: rmattr oid 25 current snap is 65 2024-03-20T08:03:18.029 INFO:tasks.rados.rados.0.smithi118.stdout:602: done (1 left) 2024-03-20T08:03:18.029 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 65 2024-03-20T08:03:18.029 INFO:tasks.rados.rados.0.smithi118.stdout:603: seq_num 198 ranges {1769472=524288,2293760=401408} 2024-03-20T08:03:18.042 INFO:tasks.rados.rados.0.smithi118.stdout:603: writing smithi11881747-30 from 1769472 to 2293760 tid 1 2024-03-20T08:03:18.046 INFO:tasks.rados.rados.0.smithi118.stdout:603: writing smithi11881747-30 from 2293760 to 2695168 tid 2 2024-03-20T08:03:18.046 INFO:tasks.rados.rados.0.smithi118.stdout:604: snap_create 2024-03-20T08:03:18.046 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 194 (ObjNum 129 snap 0 seq_num 322155976) dirty exists 2024-03-20T08:03:18.049 INFO:tasks.rados.rados.0.smithi118.stdout:603: finishing write tid 1 to smithi11881747-30 2024-03-20T08:03:18.050 INFO:tasks.rados.rados.0.smithi118.stdout:603: finishing write tid 2 to smithi11881747-30 2024-03-20T08:03:18.050 INFO:tasks.rados.rados.0.smithi118.stdout:603: finishing write tid 3 to smithi11881747-30 2024-03-20T08:03:18.051 INFO:tasks.rados.rados.0.smithi118.stdout:603: finishing write tid 4 to smithi11881747-30 2024-03-20T08:03:18.051 INFO:tasks.rados.rados.0.smithi118.stdout:603: oid 30 updating version 0 to 81 2024-03-20T08:03:18.051 INFO:tasks.rados.rados.0.smithi118.stdout:603: oid 30 version 81 is already newer than 80 2024-03-20T08:03:18.051 INFO:tasks.rados.rados.0.smithi118.stdout:603: oid 30 version 81 is already newer than 79 2024-03-20T08:03:18.051 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 81 (ObjNum 198 snap 65 seq_num 198) dirty exists 2024-03-20T08:03:18.051 INFO:tasks.rados.rados.0.smithi118.stdout:603: left oid 30 (ObjNum 198 snap 65 seq_num 198) 2024-03-20T08:03:18.093 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:18.006+0000 7ff959da5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T08:03:18.388 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:18.306+0000 7ff959da5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:18.830 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:18.579+0000 7ff959da5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:19.029 INFO:tasks.rados.rados.0.smithi118.stdout:601: done (2 left) 2024-03-20T08:03:19.029 INFO:tasks.rados.rados.0.smithi118.stdout:603: done (1 left) 2024-03-20T08:03:19.030 INFO:tasks.rados.rados.0.smithi118.stdout:604: done (0 left) 2024-03-20T08:03:19.030 INFO:tasks.rados.rados.0.smithi118.stdout:605: snap_remove snap 31 2024-03-20T08:03:19.138 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:18.831+0000 7ff959da5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T08:03:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-mon[31033]: pgmap v9: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 104 MiB/s rd, 49 MiB/s wr, 887 op/s; 576 KiB/s, 2 objects/s recovering 2024-03-20T08:03:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-mon[31033]: osdmap e183: 8 total, 8 up, 8 in 2024-03-20T08:03:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:18 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[27391]: pgmap v9: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 104 MiB/s rd, 49 MiB/s wr, 887 op/s; 576 KiB/s, 2 objects/s recovering 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[27391]: osdmap e183: 8 total, 8 up, 8 in 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[38981]: pgmap v9: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 140 MiB data, 537 MiB used, 715 GiB / 715 GiB avail; 104 MiB/s rd, 49 MiB/s wr, 887 op/s; 576 KiB/s, 2 objects/s recovering 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[38981]: osdmap e183: 8 total, 8 up, 8 in 2024-03-20T08:03:19.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:19.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:18 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.033 INFO:tasks.rados.rados.0.smithi118.stdout:605: done (0 left) 2024-03-20T08:03:20.033 INFO:tasks.rados.rados.0.smithi118.stdout:606: snap_remove snap 52 2024-03-20T08:03:20.138 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:19.690+0000 7ff959da5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T08:03:20.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:19 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:19 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:19 smithi196 ceph-mon[31033]: osdmap e184: 8 total, 8 up, 8 in 2024-03-20T08:03:20.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:19 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:19 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[38981]: osdmap e184: 8 total, 8 up, 8 in 2024-03-20T08:03:20.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[27391]: osdmap e184: 8 total, 8 up, 8 in 2024-03-20T08:03:20.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:19 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:20.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:20.276+0000 7ff959da5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T08:03:20.595 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:20.373+0000 7ff959da5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T08:03:20.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:20.596+0000 7ff959da5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T08:03:20.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:03:20] ENGINE Bus STARTING 2024-03-20T08:03:20.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: CherryPy Checker: 2024-03-20T08:03:20.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: The Application mounted at '' has an empty config. 2024-03-20T08:03:20.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: 2024-03-20T08:03:20.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:03:20] ENGINE Serving on http://:::9283 2024-03-20T08:03:20.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[66299]: [20/Mar/2024:08:03:20] ENGINE Bus STARTED 2024-03-20T08:03:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:606: done (0 left) 2024-03-20T08:03:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:607: rollback oid 12 current snap is 66 2024-03-20T08:03:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 12 to 63 2024-03-20T08:03:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:608: read oid 21 snap -1 2024-03-20T08:03:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:608: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:03:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:609: rmattr oid 8 current snap is 66 2024-03-20T08:03:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 66 2024-03-20T08:03:21.045 INFO:tasks.rados.rados.0.smithi118.stdout:610: seq_num 199 ranges {778240=540672,1318912=401408} 2024-03-20T08:03:21.048 INFO:tasks.rados.rados.0.smithi118.stdout:610: writing smithi11881747-48 from 778240 to 1318912 tid 1 2024-03-20T08:03:21.052 INFO:tasks.rados.rados.0.smithi118.stdout:610: writing smithi11881747-48 from 1318912 to 1720320 tid 2 2024-03-20T08:03:21.052 INFO:tasks.rados.rados.0.smithi118.stdout:611: rollback oid 25 current snap is 66 2024-03-20T08:03:21.052 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 25 to 29 2024-03-20T08:03:21.052 INFO:tasks.rados.rados.0.smithi118.stdout:608: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:03:21.056 INFO:tasks.rados.rados.0.smithi118.stdout:607: finishing rollback tid 0 to smithi11881747-12 2024-03-20T08:03:21.056 INFO:tasks.rados.rados.0.smithi118.stdout:607: finishing rollback tid 1 to smithi11881747-12 2024-03-20T08:03:21.056 INFO:tasks.rados.rados.0.smithi118.stdout:608: done (4 left) 2024-03-20T08:03:21.056 INFO:tasks.rados.rados.0.smithi118.stdout:612: read oid 20 snap -1 2024-03-20T08:03:21.056 INFO:tasks.rados.rados.0.smithi118.stdout:612: expect (ObjNum 146 snap 41 seq_num 146) 2024-03-20T08:03:21.056 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 66 2024-03-20T08:03:21.057 INFO:tasks.rados.rados.0.smithi118.stdout:607: finishing rollback tid 2 to smithi11881747-12 2024-03-20T08:03:21.057 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 76 (ObjNum 179 snap 54 seq_num 179) dirty exists 2024-03-20T08:03:21.057 INFO:tasks.rados.rados.0.smithi118.stdout:610: finishing write tid 1 to smithi11881747-48 2024-03-20T08:03:21.057 INFO:tasks.rados.rados.0.smithi118.stdout:610: finishing write tid 2 to smithi11881747-48 2024-03-20T08:03:21.057 INFO:tasks.rados.rados.0.smithi118.stdout:613: seq_num 200 ranges {425984=557056,983040=401408} 2024-03-20T08:03:21.062 INFO:tasks.rados.rados.0.smithi118.stdout:613: writing smithi11881747-28 from 425984 to 983040 tid 1 2024-03-20T08:03:21.064 INFO:tasks.rados.rados.0.smithi118.stdout:613: writing smithi11881747-28 from 983040 to 1384448 tid 2 2024-03-20T08:03:21.064 INFO:tasks.rados.rados.0.smithi118.stdout:607: done (5 left) 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:614: snap_create 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 178 (ObjNum 122 snap 32 seq_num 122) dirty exists 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:610: finishing write tid 3 to smithi11881747-48 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:611: finishing rollback tid 0 to smithi11881747-25 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:611: finishing rollback tid 1 to smithi11881747-25 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 268 (ObjNum 33 snap 0 seq_num 322557168) dirty dne 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:610: finishing write tid 4 to smithi11881747-48 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:610: oid 48 updating version 0 to 204 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:610: oid 48 version 204 is already newer than 202 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:610: oid 48 version 204 is already newer than 203 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 204 (ObjNum 199 snap 66 seq_num 199) dirty exists 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:610: left oid 48 (ObjNum 199 snap 66 seq_num 199) 2024-03-20T08:03:21.065 INFO:tasks.rados.rados.0.smithi118.stdout:612: expect (ObjNum 146 snap 41 seq_num 146) 2024-03-20T08:03:21.067 INFO:tasks.rados.rados.0.smithi118.stdout:613: finishing write tid 1 to smithi11881747-28 2024-03-20T08:03:21.067 INFO:tasks.rados.rados.0.smithi118.stdout:613: finishing write tid 2 to smithi11881747-28 2024-03-20T08:03:21.067 INFO:tasks.rados.rados.0.smithi118.stdout:613: finishing write tid 3 to smithi11881747-28 2024-03-20T08:03:21.068 INFO:tasks.rados.rados.0.smithi118.stdout:613: finishing write tid 4 to smithi11881747-28 2024-03-20T08:03:21.068 INFO:tasks.rados.rados.0.smithi118.stdout:613: oid 28 updating version 0 to 392 2024-03-20T08:03:21.068 INFO:tasks.rados.rados.0.smithi118.stdout:613: oid 28 updating version 392 to 393 2024-03-20T08:03:21.068 INFO:tasks.rados.rados.0.smithi118.stdout:613: oid 28 updating version 393 to 394 2024-03-20T08:03:21.068 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 394 (ObjNum 200 snap 66 seq_num 200) dirty exists 2024-03-20T08:03:21.068 INFO:tasks.rados.rados.0.smithi118.stdout:613: left oid 28 (ObjNum 200 snap 66 seq_num 200) 2024-03-20T08:03:21.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[27391]: pgmap v12: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 141 MiB data, 547 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 222 KiB/s wr, 1 op/s 2024-03-20T08:03:21.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[27391]: osdmap e185: 8 total, 8 up, 8 in 2024-03-20T08:03:21.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[27391]: Standby manager daemon x started 2024-03-20T08:03:21.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:21.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:21.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:21.285 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[38981]: pgmap v12: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 141 MiB data, 547 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 222 KiB/s wr, 1 op/s 2024-03-20T08:03:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[38981]: osdmap e185: 8 total, 8 up, 8 in 2024-03-20T08:03:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[38981]: Standby manager daemon x started 2024-03-20T08:03:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:21.286 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:21 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:21 smithi196 ceph-mon[31033]: pgmap v12: 121 pgs: 6 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 112 active+clean; 141 MiB data, 547 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 222 KiB/s wr, 1 op/s 2024-03-20T08:03:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:21 smithi196 ceph-mon[31033]: osdmap e185: 8 total, 8 up, 8 in 2024-03-20T08:03:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:21 smithi196 ceph-mon[31033]: Standby manager daemon x started 2024-03-20T08:03:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:21 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:21 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:21 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:21 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4086048992' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:22.042 INFO:tasks.rados.rados.0.smithi118.stdout:609: done (5 left) 2024-03-20T08:03:22.042 INFO:tasks.rados.rados.0.smithi118.stdout:610: done (4 left) 2024-03-20T08:03:22.042 INFO:tasks.rados.rados.0.smithi118.stdout:611: done (3 left) 2024-03-20T08:03:22.042 INFO:tasks.rados.rados.0.smithi118.stdout:612: done (2 left) 2024-03-20T08:03:22.042 INFO:tasks.rados.rados.0.smithi118.stdout:613: done (1 left) 2024-03-20T08:03:22.042 INFO:tasks.rados.rados.0.smithi118.stdout:614: done (0 left) 2024-03-20T08:03:22.042 INFO:tasks.rados.rados.0.smithi118.stdout:615: delete oid 2 current snap is 67 2024-03-20T08:03:22.045 INFO:tasks.rados.rados.0.smithi118.stdout:615: done (0 left) 2024-03-20T08:03:22.045 INFO:tasks.rados.rados.0.smithi118.stdout:616: delete oid 5 current snap is 67 2024-03-20T08:03:22.047 INFO:tasks.rados.rados.0.smithi118.stdout:616: done (0 left) 2024-03-20T08:03:22.047 INFO:tasks.rados.rados.0.smithi118.stdout:617: read oid 43 snap -1 2024-03-20T08:03:22.048 INFO:tasks.rados.rados.0.smithi118.stdout:617: expect (ObjNum 72 snap 10 seq_num 72) 2024-03-20T08:03:22.048 INFO:tasks.rados.rados.0.smithi118.stdout:618: read oid 28 snap 29 2024-03-20T08:03:22.048 INFO:tasks.rados.rados.0.smithi118.stdout:618: expect (ObjNum 9 snap 0 seq_num 9) 2024-03-20T08:03:22.048 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 67 2024-03-20T08:03:22.048 INFO:tasks.rados.rados.0.smithi118.stdout:619: seq_num 203 ranges {2547712=606208,3153920=401408} 2024-03-20T08:03:22.065 INFO:tasks.rados.rados.0.smithi118.stdout:619: writing smithi11881747-11 from 2547712 to 3153920 tid 1 2024-03-20T08:03:22.068 INFO:tasks.rados.rados.0.smithi118.stdout:619: writing smithi11881747-11 from 3153920 to 3555328 tid 2 2024-03-20T08:03:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:620: read oid 39 snap -1 2024-03-20T08:03:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:620: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:621: rollback oid 18 current snap is 67 2024-03-20T08:03:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 18 to 49 2024-03-20T08:03:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:618: expect (ObjNum 9 snap 0 seq_num 9) 2024-03-20T08:03:22.076 INFO:tasks.rados.rados.0.smithi118.stdout:617: expect (ObjNum 72 snap 10 seq_num 72) 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: finishing write tid 1 to smithi11881747-11 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: finishing write tid 2 to smithi11881747-11 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: finishing write tid 3 to smithi11881747-11 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: finishing write tid 4 to smithi11881747-11 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: oid 11 updating version 0 to 275 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: oid 11 updating version 275 to 276 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: oid 11 version 276 is already newer than 274 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 276 (ObjNum 203 snap 67 seq_num 203) dirty exists 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: left oid 11 (ObjNum 203 snap 67 seq_num 203) 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:617: done (4 left) 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:618: done (3 left) 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:619: done (2 left) 2024-03-20T08:03:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:622: setattr oid 40 current snap is 67 2024-03-20T08:03:22.141 INFO:tasks.rados.rados.0.smithi118.stdout:621: finishing rollback tid 0 to smithi11881747-18 2024-03-20T08:03:22.141 INFO:tasks.rados.rados.0.smithi118.stdout:621: finishing rollback tid 1 to smithi11881747-18 2024-03-20T08:03:22.141 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 305 (ObjNum 65 snap 0 seq_num 321739248) dirty dne 2024-03-20T08:03:22.141 INFO:tasks.rados.rados.0.smithi118.stdout:620: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:22.226 INFO:tasks.rados.rados.0.smithi118.stdout:620: done (2 left) 2024-03-20T08:03:22.226 INFO:tasks.rados.rados.0.smithi118.stdout:621: done (1 left) 2024-03-20T08:03:22.227 INFO:tasks.rados.rados.0.smithi118.stdout:623: delete oid 27 current snap is 67 2024-03-20T08:03:22.228 INFO:tasks.rados.rados.0.smithi118.stdout:623: done (1 left) 2024-03-20T08:03:22.228 INFO:tasks.rados.rados.0.smithi118.stdout:624: rollback oid 30 current snap is 67 2024-03-20T08:03:22.228 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 58 2024-03-20T08:03:22.228 INFO:tasks.rados.rados.0.smithi118.stdout:625: setattr oid 39 current snap is 67 2024-03-20T08:03:22.228 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 186 (ObjNum 65 snap 0 seq_num 1616467824) dirty exists 2024-03-20T08:03:22.230 INFO:tasks.rados.rados.0.smithi118.stdout:622: done (2 left) 2024-03-20T08:03:22.230 INFO:tasks.rados.rados.0.smithi118.stdout:626: setattr oid 26 current snap is 67 2024-03-20T08:03:22.232 INFO:tasks.rados.rados.0.smithi118.stdout:627: snap_remove snap 27 2024-03-20T08:03:22.331 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:22 smithi118 ceph-mon[38981]: osdmap e186: 8 total, 8 up, 8 in 2024-03-20T08:03:22.331 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:22 smithi118 ceph-mon[38981]: mgrmap e32: y(active, since 7s), standbys: x 2024-03-20T08:03:22.331 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:22 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:03:22.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:22 smithi118 ceph-mon[27391]: osdmap e186: 8 total, 8 up, 8 in 2024-03-20T08:03:22.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:22 smithi118 ceph-mon[27391]: mgrmap e32: y(active, since 7s), standbys: x 2024-03-20T08:03:22.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:22 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:03:22.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:22 smithi196 ceph-mon[31033]: osdmap e186: 8 total, 8 up, 8 in 2024-03-20T08:03:22.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:22 smithi196 ceph-mon[31033]: mgrmap e32: y(active, since 7s), standbys: x 2024-03-20T08:03:22.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:22 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:03:23.054 INFO:tasks.rados.rados.0.smithi118.stdout:627: done (3 left) 2024-03-20T08:03:23.054 INFO:tasks.rados.rados.0.smithi118.stdout:628: delete oid 35 current snap is 67 2024-03-20T08:03:23.054 INFO:tasks.rados.rados.0.smithi118.stdout:624: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:03:23.054 INFO:tasks.rados.rados.0.smithi118.stdout:624: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:03:23.054 INFO:tasks.rados.rados.0.smithi118.stdout:624: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:03:23.054 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 91 (ObjNum 167 snap 45 seq_num 167) dirty exists 2024-03-20T08:03:23.054 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 308 (ObjNum 191 snap 61 seq_num 191) dirty exists 2024-03-20T08:03:23.054 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 216 (ObjNum 126 snap 34 seq_num 126) dirty exists 2024-03-20T08:03:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:624: done (3 left) 2024-03-20T08:03:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:625: done (2 left) 2024-03-20T08:03:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:626: done (1 left) 2024-03-20T08:03:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:628: done (0 left) 2024-03-20T08:03:23.065 INFO:tasks.rados.rados.0.smithi118.stdout:629: read oid 42 snap -1 2024-03-20T08:03:23.065 INFO:tasks.rados.rados.0.smithi118.stdout:629: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:03:23.065 INFO:tasks.rados.rados.0.smithi118.stdout:630: read oid 24 snap -1 2024-03-20T08:03:23.065 INFO:tasks.rados.rados.0.smithi118.stdout:630: expect deleted 2024-03-20T08:03:23.065 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 67 2024-03-20T08:03:23.065 INFO:tasks.rados.rados.0.smithi118.stdout:631: seq_num 206 ranges {0=655360,655360=401408} 2024-03-20T08:03:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:631: writing smithi11881747-21 from 0 to 655360 tid 1 2024-03-20T08:03:23.070 INFO:tasks.rados.rados.0.smithi118.stdout:631: writing smithi11881747-21 from 655360 to 1056768 tid 2 2024-03-20T08:03:23.071 INFO:tasks.rados.rados.0.smithi118.stdout:632: snap_remove snap 53 2024-03-20T08:03:23.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:23 smithi196 ceph-mon[31033]: pgmap v15: 121 pgs: 121 active+clean; 143 MiB data, 551 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-20T08:03:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:23 smithi196 ceph-mon[31033]: osdmap e187: 8 total, 8 up, 8 in 2024-03-20T08:03:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:23 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:23 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:23 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:23 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[38981]: pgmap v15: 121 pgs: 121 active+clean; 143 MiB data, 551 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-20T08:03:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[38981]: osdmap e187: 8 total, 8 up, 8 in 2024-03-20T08:03:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[27391]: pgmap v15: 121 pgs: 121 active+clean; 143 MiB data, 551 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-20T08:03:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[27391]: osdmap e187: 8 total, 8 up, 8 in 2024-03-20T08:03:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:23.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:23 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:23.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:23.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:03:24.055 INFO:tasks.rados.rados.0.smithi118.stdout:632: done (3 left) 2024-03-20T08:03:24.056 INFO:tasks.rados.rados.0.smithi118.stdout:633: snap_remove snap 40 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: osdmap e188: 8 total, 8 up, 8 in 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:24 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:03:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:03:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:03:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:03:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: osdmap e188: 8 total, 8 up, 8 in 2024-03-20T08:03:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: osdmap e188: 8 total, 8 up, 8 in 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:24.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:24 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:25.060 INFO:tasks.rados.rados.0.smithi118.stdout:633: done (3 left) 2024-03-20T08:03:25.061 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 67 2024-03-20T08:03:25.061 INFO:tasks.rados.rados.0.smithi118.stdout:634: seq_num 207 ranges {7921664=671744,8593408=401408} 2024-03-20T08:03:25.107 INFO:tasks.rados.rados.0.smithi118.stdout:634: writing smithi11881747-31 from 7921664 to 8593408 tid 1 2024-03-20T08:03:25.109 INFO:tasks.rados.rados.0.smithi118.stdout:634: writing smithi11881747-31 from 8593408 to 8994816 tid 2 2024-03-20T08:03:25.110 INFO:tasks.rados.rados.0.smithi118.stdout:635: read oid 9 snap -1 2024-03-20T08:03:25.110 INFO:tasks.rados.rados.0.smithi118.stdout:635: expect (ObjNum 153 snap 43 seq_num 153) 2024-03-20T08:03:25.110 INFO:tasks.rados.rados.0.smithi118.stdout:636: rollback oid 40 current snap is 67 2024-03-20T08:03:25.110 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 61 2024-03-20T08:03:25.110 INFO:tasks.rados.rados.0.smithi118.stdout:629: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:631: finishing write tid 1 to smithi11881747-21 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:631: finishing write tid 2 to smithi11881747-21 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:631: finishing write tid 3 to smithi11881747-21 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:631: finishing write tid 4 to smithi11881747-21 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:631: oid 21 updating version 0 to 216 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:631: oid 21 version 216 is already newer than 214 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:631: oid 21 version 216 is already newer than 215 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 216 (ObjNum 206 snap 67 seq_num 206) dirty exists 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:631: left oid 21 (ObjNum 206 snap 67 seq_num 206) 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:634: finishing write tid 1 to smithi11881747-31 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:634: finishing write tid 2 to smithi11881747-31 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:636: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:03:25.130 INFO:tasks.rados.rados.0.smithi118.stdout:634: finishing write tid 3 to smithi11881747-31 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:629: done (5 left) 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:630: done (4 left) 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:631: done (3 left) 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:637: delete oid 11 current snap is 67 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:636: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 198 (ObjNum 65 snap 0 seq_num 1616467824) dirty dne 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:634: finishing write tid 4 to smithi11881747-31 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:634: oid 31 updating version 0 to 274 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:634: oid 31 version 274 is already newer than 272 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:634: oid 31 version 274 is already newer than 273 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 274 (ObjNum 207 snap 67 seq_num 207) dirty exists 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:634: left oid 31 (ObjNum 207 snap 67 seq_num 207) 2024-03-20T08:03:25.131 INFO:tasks.rados.rados.0.smithi118.stdout:635: expect (ObjNum 153 snap 43 seq_num 153) 2024-03-20T08:03:25.226 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:25 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:03:25.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:25 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:03:25.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:25 smithi196 ceph-mon[31033]: pgmap v18: 121 pgs: 121 active+clean; 143 MiB data, 551 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.0 MiB/s wr, 45 op/s 2024-03-20T08:03:25.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:25 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:03:25.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:25 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:03:25.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:25 smithi196 ceph-mon[31033]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:03:25.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:25 smithi196 ceph-mon[31033]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:03:25.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:25 smithi196 ceph-mon[31033]: osdmap e189: 8 total, 8 up, 8 in 2024-03-20T08:03:25.271 INFO:tasks.rados.rados.0.smithi118.stdout:634: done (3 left) 2024-03-20T08:03:25.272 INFO:tasks.rados.rados.0.smithi118.stdout:635: done (2 left) 2024-03-20T08:03:25.272 INFO:tasks.rados.rados.0.smithi118.stdout:636: done (1 left) 2024-03-20T08:03:25.272 INFO:tasks.rados.rados.0.smithi118.stdout:637: done (0 left) 2024-03-20T08:03:25.272 INFO:tasks.rados.rados.0.smithi118.stdout:638: read oid 39 snap -1 2024-03-20T08:03:25.272 INFO:tasks.rados.rados.0.smithi118.stdout:638: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:25.272 INFO:tasks.rados.rados.0.smithi118.stdout:639: snap_remove snap 61 2024-03-20T08:03:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[27391]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:03:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[27391]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:03:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[27391]: pgmap v18: 121 pgs: 121 active+clean; 143 MiB data, 551 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.0 MiB/s wr, 45 op/s 2024-03-20T08:03:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[27391]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:03:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[27391]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:03:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[27391]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:03:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[27391]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[27391]: osdmap e189: 8 total, 8 up, 8 in 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[38981]: pgmap v18: 121 pgs: 121 active+clean; 143 MiB data, 551 MiB used, 715 GiB / 715 GiB avail; 3.9 MiB/s rd, 1.0 MiB/s wr, 45 op/s 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[38981]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[38981]: Reconfiguring daemon prometheus.a on smithi196 2024-03-20T08:03:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:25 smithi118 ceph-mon[38981]: osdmap e189: 8 total, 8 up, 8 in 2024-03-20T08:03:26.027 INFO:tasks.rados.rados.0.smithi118.stdout:638: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:26.120 INFO:tasks.rados.rados.0.smithi118.stdout:638: done (1 left) 2024-03-20T08:03:26.120 INFO:tasks.rados.rados.0.smithi118.stdout:639: done (0 left) 2024-03-20T08:03:26.120 INFO:tasks.rados.rados.0.smithi118.stdout:640: rmattr oid 31 current snap is 67 2024-03-20T08:03:26.120 INFO:tasks.rados.rados.0.smithi118.stdout:640: done (0 left) 2024-03-20T08:03:26.120 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 67 2024-03-20T08:03:26.120 INFO:tasks.rados.rados.0.smithi118.stdout:641: seq_num 209 ranges {0=704512,704512=401408} 2024-03-20T08:03:26.123 INFO:tasks.rados.rados.0.smithi118.stdout:641: writing smithi11881747-29 from 0 to 704512 tid 1 2024-03-20T08:03:26.126 INFO:tasks.rados.rados.0.smithi118.stdout:641: writing smithi11881747-29 from 704512 to 1105920 tid 2 2024-03-20T08:03:26.126 INFO:tasks.rados.rados.0.smithi118.stdout:642: read oid 40 snap -1 2024-03-20T08:03:26.126 INFO:tasks.rados.rados.0.smithi118.stdout:642: expect deleted 2024-03-20T08:03:26.126 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 67 2024-03-20T08:03:26.126 INFO:tasks.rados.rados.0.smithi118.stdout:643: seq_num 210 ranges {958464=720896,1679360=401408} 2024-03-20T08:03:26.136 INFO:tasks.rados.rados.0.smithi118.stdout:643: writing smithi11881747-12 from 958464 to 1679360 tid 1 2024-03-20T08:03:26.139 INFO:tasks.rados.rados.0.smithi118.stdout:643: writing smithi11881747-12 from 1679360 to 2080768 tid 2 2024-03-20T08:03:26.139 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 67 2024-03-20T08:03:26.139 INFO:tasks.rados.rados.0.smithi118.stdout:644: seq_num 211 ranges {49152=737280,786432=401408} 2024-03-20T08:03:26.143 INFO:tasks.rados.rados.0.smithi118.stdout:644: writing smithi11881747-16 from 49152 to 786432 tid 1 2024-03-20T08:03:26.146 INFO:tasks.rados.rados.0.smithi118.stdout:644: writing smithi11881747-16 from 786432 to 1187840 tid 2 2024-03-20T08:03:26.146 INFO:tasks.rados.rados.0.smithi118.stdout:645: read oid 36 snap 29 2024-03-20T08:03:26.146 INFO:tasks.rados.rados.0.smithi118.stdout:645: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:03:26.146 INFO:tasks.rados.rados.0.smithi118.stdout:646: snap_remove snap 55 2024-03-20T08:03:26.233 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:26 smithi196 ceph-mon[31033]: osdmap e190: 8 total, 8 up, 8 in 2024-03-20T08:03:26.233 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:26 smithi196 ceph-mon[31033]: osdmap e191: 8 total, 8 up, 8 in 2024-03-20T08:03:26.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:26 smithi118 ceph-mon[38981]: osdmap e190: 8 total, 8 up, 8 in 2024-03-20T08:03:26.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:26 smithi118 ceph-mon[38981]: osdmap e191: 8 total, 8 up, 8 in 2024-03-20T08:03:26.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:26 smithi118 ceph-mon[27391]: osdmap e190: 8 total, 8 up, 8 in 2024-03-20T08:03:26.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:26 smithi118 ceph-mon[27391]: osdmap e191: 8 total, 8 up, 8 in 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:641: finishing write tid 1 to smithi11881747-29 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:641: finishing write tid 2 to smithi11881747-29 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:641: finishing write tid 3 to smithi11881747-29 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:641: finishing write tid 4 to smithi11881747-29 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:641: oid 29 updating version 0 to 427 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:641: oid 29 version 427 is already newer than 426 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:641: oid 29 version 427 is already newer than 425 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 427 (ObjNum 209 snap 67 seq_num 209) dirty exists 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:641: left oid 29 (ObjNum 209 snap 67 seq_num 209) 2024-03-20T08:03:27.072 INFO:tasks.rados.rados.0.smithi118.stdout:643: finishing write tid 1 to smithi11881747-12 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:643: finishing write tid 2 to smithi11881747-12 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:643: finishing write tid 3 to smithi11881747-12 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:643: finishing write tid 4 to smithi11881747-12 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:643: oid 12 updating version 0 to 92 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:643: oid 12 version 92 is already newer than 91 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:643: oid 12 version 92 is already newer than 90 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 92 (ObjNum 210 snap 67 seq_num 210) dirty exists 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:643: left oid 12 (ObjNum 210 snap 67 seq_num 210) 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:644: finishing write tid 1 to smithi11881747-16 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:644: finishing write tid 2 to smithi11881747-16 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:644: finishing write tid 3 to smithi11881747-16 2024-03-20T08:03:27.073 INFO:tasks.rados.rados.0.smithi118.stdout:645: expect (ObjNum 40 snap 0 seq_num 40) 2024-03-20T08:03:27.096 INFO:tasks.rados.rados.0.smithi118.stdout:644: finishing write tid 4 to smithi11881747-16 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:644: oid 16 updating version 0 to 437 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:644: oid 16 updating version 437 to 439 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:644: oid 16 version 439 is already newer than 438 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 439 (ObjNum 211 snap 67 seq_num 211) dirty exists 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:644: left oid 16 (ObjNum 211 snap 67 seq_num 211) 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:641: done (5 left) 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:642: done (4 left) 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:643: done (3 left) 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:644: done (2 left) 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:645: done (1 left) 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:646: done (0 left) 2024-03-20T08:03:27.097 INFO:tasks.rados.rados.0.smithi118.stdout:647: snap_remove snap 50 2024-03-20T08:03:27.281 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-mon[31033]: pgmap v21: 121 pgs: 121 active+clean; 143 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 3.6 MiB/s rd, 968 KiB/s wr, 28 op/s 2024-03-20T08:03:27.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:27 smithi118 ceph-mon[38981]: pgmap v21: 121 pgs: 121 active+clean; 143 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 3.6 MiB/s rd, 968 KiB/s wr, 28 op/s 2024-03-20T08:03:27.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:27 smithi118 ceph-mon[27391]: pgmap v21: 121 pgs: 121 active+clean; 143 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 3.6 MiB/s rd, 968 KiB/s wr, 28 op/s 2024-03-20T08:03:27.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 systemd[1]: Stopping Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:03:27.969 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.689Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-20T08:03:27.970 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-20T08:03:27.971 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[75255]: ts=2024-03-20T08:03:27.690Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-20T08:03:27.971 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:27 smithi196 podman[77969]: 2024-03-20 08:03:27.700097386 +0000 UTC m=+0.133572903 container died dc4c97c24deef5271e3f940ca6de2569b9cbd2ce9fdb81b24e4bbe00b8bf6a3f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:03:28.075 INFO:tasks.rados.rados.0.smithi118.stdout:647: done (0 left) 2024-03-20T08:03:28.075 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 67 2024-03-20T08:03:28.075 INFO:tasks.rados.rados.0.smithi118.stdout:648: seq_num 212 ranges {2547712=753664,3301376=401408} 2024-03-20T08:03:28.094 INFO:tasks.rados.rados.0.smithi118.stdout:648: writing smithi11881747-46 from 2547712 to 3301376 tid 1 2024-03-20T08:03:28.096 INFO:tasks.rados.rados.0.smithi118.stdout:648: writing smithi11881747-46 from 3301376 to 3702784 tid 2 2024-03-20T08:03:28.096 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 67 2024-03-20T08:03:28.096 INFO:tasks.rados.rados.0.smithi118.stdout:649: seq_num 213 ranges {778240=770048,1548288=401408} 2024-03-20T08:03:28.104 INFO:tasks.rados.rados.0.smithi118.stdout:649: writing smithi11881747-23 from 778240 to 1548288 tid 1 2024-03-20T08:03:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:649: writing smithi11881747-23 from 1548288 to 1949696 tid 2 2024-03-20T08:03:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:650: snap_create 2024-03-20T08:03:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:648: finishing write tid 1 to smithi11881747-46 2024-03-20T08:03:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:648: finishing write tid 2 to smithi11881747-46 2024-03-20T08:03:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:648: finishing write tid 3 to smithi11881747-46 2024-03-20T08:03:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:648: finishing write tid 4 to smithi11881747-46 2024-03-20T08:03:28.107 INFO:tasks.rados.rados.0.smithi118.stdout:648: oid 46 updating version 0 to 172 2024-03-20T08:03:28.109 INFO:tasks.rados.rados.0.smithi118.stdout:648: oid 46 version 172 is already newer than 170 2024-03-20T08:03:28.110 INFO:tasks.rados.rados.0.smithi118.stdout:648: oid 46 version 172 is already newer than 171 2024-03-20T08:03:28.110 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 172 (ObjNum 212 snap 67 seq_num 212) dirty exists 2024-03-20T08:03:28.110 INFO:tasks.rados.rados.0.smithi118.stdout:648: left oid 46 (ObjNum 212 snap 67 seq_num 212) 2024-03-20T08:03:28.112 INFO:tasks.rados.rados.0.smithi118.stdout:649: finishing write tid 1 to smithi11881747-23 2024-03-20T08:03:28.112 INFO:tasks.rados.rados.0.smithi118.stdout:649: finishing write tid 2 to smithi11881747-23 2024-03-20T08:03:28.112 INFO:tasks.rados.rados.0.smithi118.stdout:649: finishing write tid 3 to smithi11881747-23 2024-03-20T08:03:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:649: finishing write tid 4 to smithi11881747-23 2024-03-20T08:03:28.116 INFO:tasks.rados.rados.0.smithi118.stdout:649: oid 23 updating version 0 to 452 2024-03-20T08:03:28.116 INFO:tasks.rados.rados.0.smithi118.stdout:649: oid 23 version 452 is already newer than 451 2024-03-20T08:03:28.116 INFO:tasks.rados.rados.0.smithi118.stdout:649: oid 23 updating version 452 to 453 2024-03-20T08:03:28.116 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 453 (ObjNum 213 snap 67 seq_num 213) dirty exists 2024-03-20T08:03:28.116 INFO:tasks.rados.rados.0.smithi118.stdout:649: left oid 23 (ObjNum 213 snap 67 seq_num 213) 2024-03-20T08:03:28.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-mon[31033]: osdmap e192: 8 total, 8 up, 8 in 2024-03-20T08:03:28.337 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 podman[77969]: 2024-03-20 08:03:28.078328902 +0000 UTC m=+0.511804418 container cleanup dc4c97c24deef5271e3f940ca6de2569b9cbd2ce9fdb81b24e4bbe00b8bf6a3f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:03:28.337 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 bash[77969]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a 2024-03-20T08:03:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:28 smithi118 ceph-mon[27391]: osdmap e192: 8 total, 8 up, 8 in 2024-03-20T08:03:28.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:28 smithi118 ceph-mon[38981]: osdmap e192: 8 total, 8 up, 8 in 2024-03-20T08:03:28.638 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 podman[77988]: 2024-03-20 08:03:28.338370202 +0000 UTC m=+0.639475901 container remove dc4c97c24deef5271e3f940ca6de2569b9cbd2ce9fdb81b24e4bbe00b8bf6a3f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:03:28.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-20T08:03:28.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 systemd[1]: Stopped Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:03:28.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service: Consumed 1.155s CPU time. 2024-03-20T08:03:28.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 systemd[1]: Starting Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:03:28.981 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 podman[78089]: 2024-03-20 08:03:28.625593032 +0000 UTC m=+0.018928826 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-20T08:03:28.981 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 podman[78089]: 2024-03-20 08:03:28.739459046 +0000 UTC m=+0.132794841 container create 1fb280ede3dcb0830e57079bc46ab0ed065bf9dd1a3e466e3fb391bd7ac9ee21 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:03:28.981 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 podman[78089]: 2024-03-20 08:03:28.941839958 +0000 UTC m=+0.335175759 container init 1fb280ede3dcb0830e57079bc46ab0ed065bf9dd1a3e466e3fb391bd7ac9ee21 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:03:28.981 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 podman[78089]: 2024-03-20 08:03:28.947408758 +0000 UTC m=+0.340744556 container start 1fb280ede3dcb0830e57079bc46ab0ed065bf9dd1a3e466e3fb391bd7ac9ee21 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:03:29.081 INFO:tasks.rados.rados.0.smithi118.stdout:648: done (2 left) 2024-03-20T08:03:29.081 INFO:tasks.rados.rados.0.smithi118.stdout:649: done (1 left) 2024-03-20T08:03:29.081 INFO:tasks.rados.rados.0.smithi118.stdout:650: done (0 left) 2024-03-20T08:03:29.081 INFO:tasks.rados.rados.0.smithi118.stdout:651: rollback oid 13 current snap is 68 2024-03-20T08:03:29.081 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 13 to 59 2024-03-20T08:03:29.081 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 68 2024-03-20T08:03:29.081 INFO:tasks.rados.rados.0.smithi118.stdout:652: seq_num 214 ranges {0=794624,794624=401408} 2024-03-20T08:03:29.086 INFO:tasks.rados.rados.0.smithi118.stdout:652: writing smithi11881747-35 from 0 to 794624 tid 1 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:652: writing smithi11881747-35 from 794624 to 1196032 tid 2 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:653: read oid 1 snap -1 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:653: expect deleted 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:654: copy_from oid 17 from oid 38 current snap is 68 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:651: finishing rollback tid 0 to smithi11881747-13 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:651: finishing rollback tid 1 to smithi11881747-13 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:651: finishing rollback tid 2 to smithi11881747-13 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 331 (ObjNum 126 snap 34 seq_num 126) dirty exists 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:651: done (3 left) 2024-03-20T08:03:29.089 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 68 2024-03-20T08:03:29.090 INFO:tasks.rados.rados.0.smithi118.stdout:655: seq_num 215 ranges {1024000=409600,1433600=532480,1966080=270336} 2024-03-20T08:03:29.096 INFO:tasks.rados.rados.0.smithi118.stdout:655: writing smithi11881747-6 from 1024000 to 1433600 tid 1 2024-03-20T08:03:29.099 INFO:tasks.rados.rados.0.smithi118.stdout:655: writing smithi11881747-6 from 1433600 to 1966080 tid 2 2024-03-20T08:03:29.101 INFO:tasks.rados.rados.0.smithi118.stdout:655: writing smithi11881747-6 from 1966080 to 2236416 tid 3 2024-03-20T08:03:29.101 INFO:tasks.rados.rados.0.smithi118.stdout:656: setattr oid 27 current snap is 68 2024-03-20T08:03:29.101 INFO:tasks.rados.rados.0.smithi118.stdout:652: finishing write tid 1 to smithi11881747-35 2024-03-20T08:03:29.101 INFO:tasks.rados.rados.0.smithi118.stdout:652: finishing write tid 2 to smithi11881747-35 2024-03-20T08:03:29.101 INFO:tasks.rados.rados.0.smithi118.stdout:652: finishing write tid 3 to smithi11881747-35 2024-03-20T08:03:29.102 INFO:tasks.rados.rados.0.smithi118.stdout:652: finishing write tid 4 to smithi11881747-35 2024-03-20T08:03:29.102 INFO:tasks.rados.rados.0.smithi118.stdout:652: oid 35 updating version 0 to 174 2024-03-20T08:03:29.102 INFO:tasks.rados.rados.0.smithi118.stdout:652: oid 35 updating version 174 to 175 2024-03-20T08:03:29.102 INFO:tasks.rados.rados.0.smithi118.stdout:652: oid 35 version 175 is already newer than 173 2024-03-20T08:03:29.102 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 175 (ObjNum 214 snap 68 seq_num 214) dirty exists 2024-03-20T08:03:29.102 INFO:tasks.rados.rados.0.smithi118.stdout:652: left oid 35 (ObjNum 214 snap 68 seq_num 214) 2024-03-20T08:03:29.102 INFO:tasks.rados.rados.0.smithi118.stdout:655: finishing write tid 1 to smithi11881747-6 2024-03-20T08:03:29.103 INFO:tasks.rados.rados.0.smithi118.stdout:652: done (4 left) 2024-03-20T08:03:29.103 INFO:tasks.rados.rados.0.smithi118.stdout:653: done (3 left) 2024-03-20T08:03:29.103 INFO:tasks.rados.rados.0.smithi118.stdout:657: setattr oid 2 current snap is 68 2024-03-20T08:03:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:655: finishing write tid 2 to smithi11881747-6 2024-03-20T08:03:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:655: finishing write tid 3 to smithi11881747-6 2024-03-20T08:03:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:655: finishing write tid 4 to smithi11881747-6 2024-03-20T08:03:29.105 INFO:tasks.rados.rados.0.smithi118.stdout:654: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:03:29.105 INFO:tasks.rados.rados.0.smithi118.stdout:658: snap_create 2024-03-20T08:03:29.106 INFO:tasks.rados.rados.0.smithi118.stdout:655: finishing write tid 5 to smithi11881747-6 2024-03-20T08:03:29.106 INFO:tasks.rados.rados.0.smithi118.stdout:655: oid 6 updating version 0 to 177 2024-03-20T08:03:29.106 INFO:tasks.rados.rados.0.smithi118.stdout:655: oid 6 updating version 177 to 178 2024-03-20T08:03:29.106 INFO:tasks.rados.rados.0.smithi118.stdout:655: oid 6 updating version 178 to 179 2024-03-20T08:03:29.106 INFO:tasks.rados.rados.0.smithi118.stdout:655: oid 6 updating version 179 to 180 2024-03-20T08:03:29.106 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 180 (ObjNum 215 snap 68 seq_num 215) dirty exists 2024-03-20T08:03:29.106 INFO:tasks.rados.rados.0.smithi118.stdout:655: left oid 6 (ObjNum 215 snap 68 seq_num 215) 2024-03-20T08:03:29.107 INFO:tasks.rados.rados.0.smithi118.stdout:654: finishing copy_from to smithi11881747-17 2024-03-20T08:03:29.107 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 187 (ObjNum 128 snap 34 seq_num 128) dirty exists 2024-03-20T08:03:29.108 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 181 (ObjNum 65 snap 0 seq_num 1601066321) dirty exists 2024-03-20T08:03:29.108 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 345 (ObjNum 33 snap 0 seq_num 324603440) dirty exists 2024-03-20T08:03:29.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-mon[31033]: pgmap v24: 121 pgs: 121 active+clean; 143 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 3.6 MiB/s rd, 968 KiB/s wr, 28 op/s 2024-03-20T08:03:29.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-mon[31033]: osdmap e193: 8 total, 8 up, 8 in 2024-03-20T08:03:29.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:29.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.983Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-20T08:03:29.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.983Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-20T08:03:29.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.983Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi196 (none))" 2024-03-20T08:03:29.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.983Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.983Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.985Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.986Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.986Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.986Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.989Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.989Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=192.171µs 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.989Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.991Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=3 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:28.996Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=3 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.003Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=3 2024-03-20T08:03:29.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.003Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=3 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.003Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=45.768µs wal_replay_duration=14.029339ms wbl_replay_duration=173ns total_replay_duration=14.290584ms 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.005Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.005Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.005Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.029Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=23.8694ms db_storage=778ns remote_storage=1.098µs web_handler=407ns query_engine=715ns scrape=450.107µs scrape_sd=95.043µs notify=24.681µs notify_sd=22.351µs rules=22.961953ms tracing=7.112µs 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.029Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:29.029Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 bash[78089]: 1fb280ede3dcb0830e57079bc46ab0ed065bf9dd1a3e466e3fb391bd7ac9ee21 2024-03-20T08:03:29.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:29 smithi196 systemd[1]: Started Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:03:29.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:29 smithi118 ceph-mon[38981]: pgmap v24: 121 pgs: 121 active+clean; 143 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 3.6 MiB/s rd, 968 KiB/s wr, 28 op/s 2024-03-20T08:03:29.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:29 smithi118 ceph-mon[38981]: osdmap e193: 8 total, 8 up, 8 in 2024-03-20T08:03:29.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:29 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:29 smithi118 ceph-mon[27391]: pgmap v24: 121 pgs: 121 active+clean; 143 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 3.6 MiB/s rd, 968 KiB/s wr, 28 op/s 2024-03-20T08:03:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:29 smithi118 ceph-mon[27391]: osdmap e193: 8 total, 8 up, 8 in 2024-03-20T08:03:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:29 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:30.086 INFO:tasks.rados.rados.0.smithi118.stdout:654: done (4 left) 2024-03-20T08:03:30.086 INFO:tasks.rados.rados.0.smithi118.stdout:655: done (3 left) 2024-03-20T08:03:30.087 INFO:tasks.rados.rados.0.smithi118.stdout:656: done (2 left) 2024-03-20T08:03:30.087 INFO:tasks.rados.rados.0.smithi118.stdout:657: done (1 left) 2024-03-20T08:03:30.087 INFO:tasks.rados.rados.0.smithi118.stdout:658: done (0 left) 2024-03-20T08:03:30.087 INFO:tasks.rados.rados.0.smithi118.stdout:659: rollback oid 32 current snap is 69 2024-03-20T08:03:30.087 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 32 to 41 2024-03-20T08:03:30.087 INFO:tasks.rados.rados.0.smithi118.stdout:660: snap_remove snap 68 2024-03-20T08:03:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:30 smithi196 ceph-mon[31033]: osdmap e194: 8 total, 8 up, 8 in 2024-03-20T08:03:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:30 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:30 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:30 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:03:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:30 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[27391]: osdmap e194: 8 total, 8 up, 8 in 2024-03-20T08:03:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:03:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[38981]: osdmap e194: 8 total, 8 up, 8 in 2024-03-20T08:03:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:03:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:30 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:31.030 INFO:tasks.rados.rados.0.smithi118.stdout:660: done (1 left) 2024-03-20T08:03:31.030 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 69 2024-03-20T08:03:31.030 INFO:tasks.rados.rados.0.smithi118.stdout:659: finishing rollback tid 0 to smithi11881747-32 2024-03-20T08:03:31.030 INFO:tasks.rados.rados.0.smithi118.stdout:659: finishing rollback tid 1 to smithi11881747-32 2024-03-20T08:03:31.030 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 205 (ObjNum 65 snap 0 seq_num 1268268464) dirty dne 2024-03-20T08:03:31.030 INFO:tasks.rados.rados.0.smithi118.stdout:661: seq_num 216 ranges {0=425984,425984=409600,835584=393216} 2024-03-20T08:03:31.032 INFO:tasks.rados.rados.0.smithi118.stdout:661: writing smithi11881747-24 from 0 to 425984 tid 1 2024-03-20T08:03:31.034 INFO:tasks.rados.rados.0.smithi118.stdout:661: writing smithi11881747-24 from 425984 to 835584 tid 2 2024-03-20T08:03:31.037 INFO:tasks.rados.rados.0.smithi118.stdout:661: writing smithi11881747-24 from 835584 to 1228800 tid 3 2024-03-20T08:03:31.037 INFO:tasks.rados.rados.0.smithi118.stdout:659: done (1 left) 2024-03-20T08:03:31.037 INFO:tasks.rados.rados.0.smithi118.stdout:662: copy_from oid 35 from oid 27 current snap is 69 2024-03-20T08:03:31.037 INFO:tasks.rados.rados.0.smithi118.stdout:661: finishing write tid 1 to smithi11881747-24 2024-03-20T08:03:31.037 INFO:tasks.rados.rados.0.smithi118.stdout:663: snap_create 2024-03-20T08:03:31.039 INFO:tasks.rados.rados.0.smithi118.stdout:661: finishing write tid 2 to smithi11881747-24 2024-03-20T08:03:31.039 INFO:tasks.rados.rados.0.smithi118.stdout:661: finishing write tid 3 to smithi11881747-24 2024-03-20T08:03:31.040 INFO:tasks.rados.rados.0.smithi118.stdout:661: finishing write tid 4 to smithi11881747-24 2024-03-20T08:03:31.042 INFO:tasks.rados.rados.0.smithi118.stdout:661: finishing write tid 5 to smithi11881747-24 2024-03-20T08:03:31.042 INFO:tasks.rados.rados.0.smithi118.stdout:661: oid 24 updating version 0 to 227 2024-03-20T08:03:31.042 INFO:tasks.rados.rados.0.smithi118.stdout:661: oid 24 version 227 is already newer than 226 2024-03-20T08:03:31.042 INFO:tasks.rados.rados.0.smithi118.stdout:661: oid 24 updating version 227 to 229 2024-03-20T08:03:31.042 INFO:tasks.rados.rados.0.smithi118.stdout:661: oid 24 version 229 is already newer than 228 2024-03-20T08:03:31.042 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 229 (ObjNum 216 snap 69 seq_num 216) dirty exists 2024-03-20T08:03:31.042 INFO:tasks.rados.rados.0.smithi118.stdout:661: left oid 24 (ObjNum 216 snap 69 seq_num 216) 2024-03-20T08:03:31.042 INFO:tasks.rados.rados.0.smithi118.stdout:662: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:03:31.043 INFO:tasks.rados.rados.0.smithi118.stdout:662: finishing copy_from to smithi11881747-35 2024-03-20T08:03:31.043 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 345 (ObjNum 97 snap 0 seq_num 1268681013) dirty exists 2024-03-20T08:03:31.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:31 smithi196 ceph-mon[31033]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:03:31.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:31 smithi196 ceph-mon[31033]: pgmap v27: 121 pgs: 121 active+clean; 148 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 11 op/s 2024-03-20T08:03:31.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:31 smithi196 ceph-mon[31033]: osdmap e195: 8 total, 8 up, 8 in 2024-03-20T08:03:31.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:31 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:31.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:31 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:31.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:31 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:03:31.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:31 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:31.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:31 smithi196 ceph-mon[31033]: osdmap e196: 8 total, 8 up, 8 in 2024-03-20T08:03:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[38981]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:03:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[38981]: pgmap v27: 121 pgs: 121 active+clean; 148 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 11 op/s 2024-03-20T08:03:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[38981]: osdmap e195: 8 total, 8 up, 8 in 2024-03-20T08:03:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:03:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[38981]: osdmap e196: 8 total, 8 up, 8 in 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[27391]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[27391]: pgmap v27: 121 pgs: 121 active+clean; 148 MiB data, 590 MiB used, 715 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 11 op/s 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[27391]: osdmap e195: 8 total, 8 up, 8 in 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:31 smithi118 ceph-mon[27391]: osdmap e196: 8 total, 8 up, 8 in 2024-03-20T08:03:31.932 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:661: done (2 left) 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:662: done (1 left) 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:663: done (0 left) 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:664: read oid 39 snap -1 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:664: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:665: rollback oid 37 current snap is 70 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 58 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:666: copy_from oid 12 from oid 22 current snap is 70 2024-03-20T08:03:32.038 INFO:tasks.rados.rados.0.smithi118.stdout:667: snap_remove snap 59 2024-03-20T08:03:32.254 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:03:32.255 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (3m) 14s ago 4m 28.5M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:03:32.255 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (3m) 13s ago 3m 89.4M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:03:32.255 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (5m) 14s ago 5m 13.5M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:03:32.255 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (5m) 13s ago 5m 18.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:03:32.255 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (69s) 13s ago 8m 74.8M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 838b6245d898 2024-03-20T08:03:32.255 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (28s) 14s ago 10m 481M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:03:32.258 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (10m) 14s ago 10m 53.7M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 89e74e5abc84 2024-03-20T08:03:32.258 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (9m) 13s ago 9m 43.7M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 fd26544c725b 2024-03-20T08:03:32.258 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (9m) 14s ago 9m 47.4M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 8193e23658db 2024-03-20T08:03:32.258 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (3m) 14s ago 3m 14.6M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (3m) 13s ago 3m 15.6M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (8m) 14s ago 8m 107M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 481bba84ab31 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (8m) 14s ago 8m 106M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (7m) 14s ago 7m 92.3M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (7m) 14s ago 7m 81.9M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (6m) 13s ago 6m 117M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (6m) 13s ago 6m 91.4M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (5m) 13s ago 5m 74.4M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (5m) 13s ago 5m 75.9M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:03:32.259 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 starting - - - - 2024-03-20T08:03:32.259 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:32 smithi118 ceph-mon[38981]: Upgrade: Updating mgr.x 2024-03-20T08:03:32.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:32 smithi118 ceph-mon[38981]: Deploying daemon mgr.x on smithi196 2024-03-20T08:03:32.260 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:32 smithi118 ceph-mon[38981]: osdmap e197: 8 total, 8 up, 8 in 2024-03-20T08:03:32.260 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:32 smithi118 ceph-mon[27391]: Upgrade: Updating mgr.x 2024-03-20T08:03:32.260 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:32 smithi118 ceph-mon[27391]: Deploying daemon mgr.x on smithi196 2024-03-20T08:03:32.260 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:32 smithi118 ceph-mon[27391]: osdmap e197: 8 total, 8 up, 8 in 2024-03-20T08:03:32.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:32 smithi196 ceph-mon[31033]: Upgrade: Updating mgr.x 2024-03-20T08:03:32.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:32 smithi196 ceph-mon[31033]: Deploying daemon mgr.x on smithi196 2024-03-20T08:03:32.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:32 smithi196 ceph-mon[31033]: osdmap e197: 8 total, 8 up, 8 in 2024-03-20T08:03:32.648 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:03:32.648 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:03:32.648 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3 2024-03-20T08:03:32.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:03:32.648 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:03:32.648 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:03:32.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 13, 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:03:32.649 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:03:32.993 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:32 smithi196 systemd[1]: Stopping Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:03:33.039 INFO:tasks.rados.rados.0.smithi118.stdout:667: done (3 left) 2024-03-20T08:03:33.040 INFO:tasks.rados.rados.0.smithi118.stdout:668: delete oid 48 current snap is 70 2024-03-20T08:03:33.042 INFO:tasks.rados.rados.0.smithi118.stdout:665: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:03:33.042 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 128 (ObjNum 65 snap 0 seq_num 1601066321) dirty dne 2024-03-20T08:03:33.042 INFO:tasks.rados.rados.0.smithi118.stdout:664: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:666: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:03:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:666: finishing copy_from to smithi11881747-12 2024-03-20T08:03:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:03:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:664: done (3 left) 2024-03-20T08:03:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:665: done (2 left) 2024-03-20T08:03:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:666: done (1 left) 2024-03-20T08:03:33.134 INFO:tasks.rados.rados.0.smithi118.stdout:668: done (0 left) 2024-03-20T08:03:33.134 INFO:tasks.rados.rados.0.smithi118.stdout:669: snap_create 2024-03-20T08:03:33.276 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:32 smithi196 podman[78510]: 2024-03-20 08:03:32.993856405 +0000 UTC m=+0.135372531 container died 838b6245d898211f970d446b8339ab52a6c2517e9c9ee51a5fdc8741e0bedb8d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-20T08:03:33.276 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:33 smithi196 ceph-mon[31033]: pgmap v30: 121 pgs: 121 active+clean; 153 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 22 op/s 2024-03-20T08:03:33.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:33 smithi196 ceph-mon[31033]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:33.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:33 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/1370926298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:33.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:33 smithi196 ceph-mon[31033]: osdmap e198: 8 total, 8 up, 8 in 2024-03-20T08:03:33.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:33 smithi118 ceph-mon[38981]: pgmap v30: 121 pgs: 121 active+clean; 153 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 22 op/s 2024-03-20T08:03:33.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:33 smithi118 ceph-mon[38981]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:33.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:33 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/1370926298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:33.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:33 smithi118 ceph-mon[38981]: osdmap e198: 8 total, 8 up, 8 in 2024-03-20T08:03:33.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:33 smithi118 ceph-mon[27391]: pgmap v30: 121 pgs: 121 active+clean; 153 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 4.2 MiB/s rd, 2.8 MiB/s wr, 22 op/s 2024-03-20T08:03:33.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:33 smithi118 ceph-mon[27391]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:33.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:33 smithi118 ceph-mon[27391]: from='client.? 172.21.15.118:0/1370926298' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:33.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:33 smithi118 ceph-mon[27391]: osdmap e198: 8 total, 8 up, 8 in 2024-03-20T08:03:33.610 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:33.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:03:33.610 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:33 smithi196 podman[78510]: 2024-03-20 08:03:33.351777233 +0000 UTC m=+0.493293358 container cleanup 838b6245d898211f970d446b8339ab52a6c2517e9c9ee51a5fdc8741e0bedb8d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-20T08:03:33.610 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:33 smithi196 bash[78510]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x 2024-03-20T08:03:33.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:33 smithi196 podman[78529]: 2024-03-20 08:03:33.611483492 +0000 UTC m=+0.620411916 container remove 838b6245d898211f970d446b8339ab52a6c2517e9c9ee51a5fdc8741e0bedb8d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:03:33.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:33 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:03:33.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:33 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-03-20T08:03:33.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:33 smithi196 systemd[1]: Stopped Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:03:33.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:33 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x.service: Consumed 25.055s CPU time. 2024-03-20T08:03:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:669: done (0 left) 2024-03-20T08:03:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 71 2024-03-20T08:03:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:670: seq_num 218 ranges {2121728=458752,2580480=581632,3162112=221184} 2024-03-20T08:03:34.112 INFO:tasks.rados.rados.0.smithi118.stdout:670: writing smithi11881747-26 from 2121728 to 2580480 tid 1 2024-03-20T08:03:34.115 INFO:tasks.rados.rados.0.smithi118.stdout:670: writing smithi11881747-26 from 2580480 to 3162112 tid 2 2024-03-20T08:03:34.117 INFO:tasks.rados.rados.0.smithi118.stdout:670: writing smithi11881747-26 from 3162112 to 3383296 tid 3 2024-03-20T08:03:34.117 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 71 2024-03-20T08:03:34.117 INFO:tasks.rados.rados.0.smithi118.stdout:670: finishing write tid 1 to smithi11881747-26 2024-03-20T08:03:34.117 INFO:tasks.rados.rados.0.smithi118.stdout:671: seq_num 219 ranges {2236416=475136,2711552=466944,3178496=335872} 2024-03-20T08:03:34.131 INFO:tasks.rados.rados.0.smithi118.stdout:671: writing smithi11881747-6 from 2236416 to 2711552 tid 1 2024-03-20T08:03:34.134 INFO:tasks.rados.rados.0.smithi118.stdout:671: writing smithi11881747-6 from 2711552 to 3178496 tid 2 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:671: writing smithi11881747-6 from 3178496 to 3514368 tid 3 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:672: copy_from oid 41 from oid 39 current snap is 71 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:670: finishing write tid 2 to smithi11881747-26 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:670: finishing write tid 3 to smithi11881747-26 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:670: finishing write tid 4 to smithi11881747-26 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:670: finishing write tid 5 to smithi11881747-26 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:670: oid 26 updating version 0 to 240 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:670: oid 26 version 240 is already newer than 238 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:670: oid 26 version 240 is already newer than 237 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:670: oid 26 version 240 is already newer than 239 2024-03-20T08:03:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 240 (ObjNum 218 snap 71 seq_num 218) dirty exists 2024-03-20T08:03:34.138 INFO:tasks.rados.rados.0.smithi118.stdout:670: left oid 26 (ObjNum 218 snap 71 seq_num 218) 2024-03-20T08:03:34.138 INFO:tasks.rados.rados.0.smithi118.stdout:670: done (2 left) 2024-03-20T08:03:34.138 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 71 2024-03-20T08:03:34.138 INFO:tasks.rados.rados.0.smithi118.stdout:673: seq_num 220 ranges {2547712=491520,3039232=753664,3792896=49152} 2024-03-20T08:03:34.153 INFO:tasks.rados.rados.0.smithi118.stdout:673: writing smithi11881747-19 from 2547712 to 3039232 tid 1 2024-03-20T08:03:34.157 INFO:tasks.rados.rados.0.smithi118.stdout:673: writing smithi11881747-19 from 3039232 to 3792896 tid 2 2024-03-20T08:03:34.158 INFO:tasks.rados.rados.0.smithi118.stdout:673: writing smithi11881747-19 from 3792896 to 3842048 tid 3 2024-03-20T08:03:34.158 INFO:tasks.rados.rados.0.smithi118.stdout:674: delete oid 46 current snap is 71 2024-03-20T08:03:34.158 INFO:tasks.rados.rados.0.smithi118.stdout:671: finishing write tid 1 to smithi11881747-6 2024-03-20T08:03:34.158 INFO:tasks.rados.rados.0.smithi118.stdout:671: finishing write tid 2 to smithi11881747-6 2024-03-20T08:03:34.158 INFO:tasks.rados.rados.0.smithi118.stdout:671: finishing write tid 3 to smithi11881747-6 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:671: finishing write tid 4 to smithi11881747-6 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:671: finishing write tid 5 to smithi11881747-6 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:671: oid 6 updating version 0 to 349 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:671: oid 6 version 349 is already newer than 347 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:671: oid 6 version 349 is already newer than 348 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:671: oid 6 version 349 is already newer than 346 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 349 (ObjNum 219 snap 71 seq_num 219) dirty exists 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:671: left oid 6 (ObjNum 219 snap 71 seq_num 219) 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:672: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:672: finishing copy_from to smithi11881747-41 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 308 (ObjNum 191 snap 61 seq_num 191) dirty exists 2024-03-20T08:03:34.159 INFO:tasks.rados.rados.0.smithi118.stdout:673: finishing write tid 1 to smithi11881747-19 2024-03-20T08:03:34.160 INFO:tasks.rados.rados.0.smithi118.stdout:671: done (3 left) 2024-03-20T08:03:34.160 INFO:tasks.rados.rados.0.smithi118.stdout:672: done (2 left) 2024-03-20T08:03:34.160 INFO:tasks.rados.rados.0.smithi118.stdout:674: done (1 left) 2024-03-20T08:03:34.160 INFO:tasks.rados.rados.0.smithi118.stdout:675: snap_remove snap 41 2024-03-20T08:03:34.318 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:34 smithi196 ceph-mon[31033]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:34.318 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:33 smithi196 systemd[1]: Starting Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:03:34.318 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:34 smithi196 podman[78667]: 2024-03-20 08:03:34.066935944 +0000 UTC m=+0.017729733 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:03:34.318 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:34 smithi196 podman[78667]: 2024-03-20 08:03:34.196021874 +0000 UTC m=+0.146815658 container create 043bae454c20ff29934ed7c84a96cbc983734e54231cce96c5b2d60355e191ce (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:03:34.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:34 smithi118 ceph-mon[27391]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:34.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:34 smithi118 ceph-mon[38981]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:03:34.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:34 smithi196 podman[78667]: 2024-03-20 08:03:34.432595552 +0000 UTC m=+0.383389337 container init 043bae454c20ff29934ed7c84a96cbc983734e54231cce96c5b2d60355e191ce (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-20T08:03:34.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:34 smithi196 podman[78667]: 2024-03-20 08:03:34.438530585 +0000 UTC m=+0.389324378 container start 043bae454c20ff29934ed7c84a96cbc983734e54231cce96c5b2d60355e191ce (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-20T08:03:34.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:34 smithi196 ceph-mgr[78704]: -- 172.21.15.196:0/3451954684 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561259d25a0 con 0x5561259a7000 2024-03-20T08:03:34.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:34 smithi196 bash[78667]: 043bae454c20ff29934ed7c84a96cbc983734e54231cce96c5b2d60355e191ce 2024-03-20T08:03:34.889 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:34 smithi196 systemd[1]: Started Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:03:35.102 INFO:tasks.rados.rados.0.smithi118.stdout:675: done (1 left) 2024-03-20T08:03:35.102 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 71 2024-03-20T08:03:35.102 INFO:tasks.rados.rados.0.smithi118.stdout:676: seq_num 222 ranges {1318912=524288,1843200=565248,2408448=237568} 2024-03-20T08:03:35.113 INFO:tasks.rados.rados.0.smithi118.stdout:676: writing smithi11881747-36 from 1318912 to 1843200 tid 1 2024-03-20T08:03:35.117 INFO:tasks.rados.rados.0.smithi118.stdout:676: writing smithi11881747-36 from 1843200 to 2408448 tid 2 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:676: writing smithi11881747-36 from 2408448 to 2646016 tid 3 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:677: read oid 17 snap -1 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:677: expect (ObjNum 128 snap 34 seq_num 128) 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:678: snap_create 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:673: finishing write tid 2 to smithi11881747-19 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:673: finishing write tid 3 to smithi11881747-19 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:673: finishing write tid 4 to smithi11881747-19 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:673: finishing write tid 5 to smithi11881747-19 2024-03-20T08:03:35.119 INFO:tasks.rados.rados.0.smithi118.stdout:673: oid 19 updating version 0 to 123 2024-03-20T08:03:35.120 INFO:tasks.rados.rados.0.smithi118.stdout:673: oid 19 updating version 123 to 124 2024-03-20T08:03:35.120 INFO:tasks.rados.rados.0.smithi118.stdout:673: oid 19 version 124 is already newer than 121 2024-03-20T08:03:35.120 INFO:tasks.rados.rados.0.smithi118.stdout:673: oid 19 version 124 is already newer than 122 2024-03-20T08:03:35.120 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 124 (ObjNum 220 snap 71 seq_num 220) dirty exists 2024-03-20T08:03:35.120 INFO:tasks.rados.rados.0.smithi118.stdout:673: left oid 19 (ObjNum 220 snap 71 seq_num 220) 2024-03-20T08:03:35.121 INFO:tasks.rados.rados.0.smithi118.stdout:676: finishing write tid 1 to smithi11881747-36 2024-03-20T08:03:35.122 INFO:tasks.rados.rados.0.smithi118.stdout:676: finishing write tid 2 to smithi11881747-36 2024-03-20T08:03:35.123 INFO:tasks.rados.rados.0.smithi118.stdout:676: finishing write tid 3 to smithi11881747-36 2024-03-20T08:03:35.124 INFO:tasks.rados.rados.0.smithi118.stdout:676: finishing write tid 4 to smithi11881747-36 2024-03-20T08:03:35.126 INFO:tasks.rados.rados.0.smithi118.stdout:676: finishing write tid 5 to smithi11881747-36 2024-03-20T08:03:35.126 INFO:tasks.rados.rados.0.smithi118.stdout:676: oid 36 updating version 0 to 117 2024-03-20T08:03:35.126 INFO:tasks.rados.rados.0.smithi118.stdout:676: oid 36 version 117 is already newer than 116 2024-03-20T08:03:35.127 INFO:tasks.rados.rados.0.smithi118.stdout:676: oid 36 version 117 is already newer than 115 2024-03-20T08:03:35.127 INFO:tasks.rados.rados.0.smithi118.stdout:676: oid 36 updating version 117 to 118 2024-03-20T08:03:35.127 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 118 (ObjNum 222 snap 71 seq_num 222) dirty exists 2024-03-20T08:03:35.127 INFO:tasks.rados.rados.0.smithi118.stdout:676: left oid 36 (ObjNum 222 snap 71 seq_num 222) 2024-03-20T08:03:35.135 INFO:tasks.rados.rados.0.smithi118.stdout:677: expect (ObjNum 128 snap 34 seq_num 128) 2024-03-20T08:03:35.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:35.013+0000 7f79b816d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T08:03:35.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:35.137+0000 7f79b816d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T08:03:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-mon[31033]: pgmap v33: 121 pgs: 121 active+clean; 153 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-20T08:03:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-mon[31033]: osdmap e199: 8 total, 8 up, 8 in 2024-03-20T08:03:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[27391]: pgmap v33: 121 pgs: 121 active+clean; 153 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-20T08:03:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[27391]: osdmap e199: 8 total, 8 up, 8 in 2024-03-20T08:03:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:35.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[38981]: pgmap v33: 121 pgs: 121 active+clean; 153 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-03-20T08:03:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[38981]: osdmap e199: 8 total, 8 up, 8 in 2024-03-20T08:03:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:35 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:36.072 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:35.818+0000 7f79b816d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T08:03:36.072 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:35.923+0000 7f79b816d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T08:03:36.106 INFO:tasks.rados.rados.0.smithi118.stdout:673: done (3 left) 2024-03-20T08:03:36.106 INFO:tasks.rados.rados.0.smithi118.stdout:676: done (2 left) 2024-03-20T08:03:36.106 INFO:tasks.rados.rados.0.smithi118.stdout:677: done (1 left) 2024-03-20T08:03:36.106 INFO:tasks.rados.rados.0.smithi118.stdout:678: done (0 left) 2024-03-20T08:03:36.106 INFO:tasks.rados.rados.0.smithi118.stdout:679: setattr oid 33 current snap is 72 2024-03-20T08:03:36.108 INFO:tasks.rados.rados.0.smithi118.stdout:680: read oid 6 snap -1 2024-03-20T08:03:36.108 INFO:tasks.rados.rados.0.smithi118.stdout:680: expect (ObjNum 219 snap 71 seq_num 219) 2024-03-20T08:03:36.108 INFO:tasks.rados.rados.0.smithi118.stdout:681: delete oid 27 current snap is 72 2024-03-20T08:03:36.111 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 170 (ObjNum 129 snap 0 seq_num 322096400) dirty exists 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:679: done (2 left) 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:681: done (1 left) 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:682: rollback oid 34 current snap is 72 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 34 to 44 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:683: read oid 9 snap -1 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:683: expect (ObjNum 153 snap 43 seq_num 153) 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:684: read oid 18 snap 54 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:684: expect deleted 2024-03-20T08:03:36.114 INFO:tasks.rados.rados.0.smithi118.stdout:685: snap_create 2024-03-20T08:03:36.119 INFO:tasks.rados.rados.0.smithi118.stdout:682: finishing rollback tid 0 to smithi11881747-34 2024-03-20T08:03:36.119 INFO:tasks.rados.rados.0.smithi118.stdout:682: finishing rollback tid 1 to smithi11881747-34 2024-03-20T08:03:36.119 INFO:tasks.rados.rados.0.smithi118.stdout:682: finishing rollback tid 2 to smithi11881747-34 2024-03-20T08:03:36.119 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 122 (ObjNum 152 snap 43 seq_num 152) dirty exists 2024-03-20T08:03:36.125 INFO:tasks.rados.rados.0.smithi118.stdout:683: expect (ObjNum 153 snap 43 seq_num 153) 2024-03-20T08:03:36.274 INFO:tasks.rados.rados.0.smithi118.stdout:680: expect (ObjNum 219 snap 71 seq_num 219) 2024-03-20T08:03:36.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-mon[31033]: osdmap e200: 8 total, 8 up, 8 in 2024-03-20T08:03:36.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:36.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:36.332 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:36.072+0000 7f79b816d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T08:03:36.332 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:36.179+0000 7f79b816d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T08:03:36.332 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:36.266+0000 7f79b816d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T08:03:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:36 smithi118 ceph-mon[27391]: osdmap e200: 8 total, 8 up, 8 in 2024-03-20T08:03:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:36 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:36 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:36.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:36 smithi118 ceph-mon[38981]: osdmap e200: 8 total, 8 up, 8 in 2024-03-20T08:03:36.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:36 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:36.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:36 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:36.638 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:36.540+0000 7f79b816d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T08:03:37.064 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:36.663+0000 7f79b816d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:37.064 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:36.765+0000 7f79b816d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T08:03:37.064 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:36.877+0000 7f79b816d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T08:03:37.064 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:37.065+0000 7f79b816d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T08:03:37.111 INFO:tasks.rados.rados.0.smithi118.stdout:680: done (4 left) 2024-03-20T08:03:37.111 INFO:tasks.rados.rados.0.smithi118.stdout:682: done (3 left) 2024-03-20T08:03:37.111 INFO:tasks.rados.rados.0.smithi118.stdout:683: done (2 left) 2024-03-20T08:03:37.111 INFO:tasks.rados.rados.0.smithi118.stdout:684: done (1 left) 2024-03-20T08:03:37.112 INFO:tasks.rados.rados.0.smithi118.stdout:685: done (0 left) 2024-03-20T08:03:37.112 INFO:tasks.rados.rados.0.smithi118.stdout:686: snap_remove snap 44 2024-03-20T08:03:37.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:37 smithi196 ceph-mon[31033]: pgmap v36: 121 pgs: 121 active+clean; 162 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-03-20T08:03:37.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:37 smithi196 ceph-mon[31033]: osdmap e201: 8 total, 8 up, 8 in 2024-03-20T08:03:37.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:37 smithi118 ceph-mon[27391]: pgmap v36: 121 pgs: 121 active+clean; 162 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-03-20T08:03:37.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:37 smithi118 ceph-mon[27391]: osdmap e201: 8 total, 8 up, 8 in 2024-03-20T08:03:37.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:37 smithi118 ceph-mon[38981]: pgmap v36: 121 pgs: 121 active+clean; 162 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-03-20T08:03:37.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:37 smithi118 ceph-mon[38981]: osdmap e201: 8 total, 8 up, 8 in 2024-03-20T08:03:38.117 INFO:tasks.rados.rados.0.smithi118.stdout:686: done (0 left) 2024-03-20T08:03:38.117 INFO:tasks.rados.rados.0.smithi118.stdout:687: snap_remove snap 63 2024-03-20T08:03:38.296 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:38 smithi196 ceph-mon[31033]: osdmap e202: 8 total, 8 up, 8 in 2024-03-20T08:03:38.296 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:38.079+0000 7f79b816d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T08:03:38.296 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:38.231+0000 7f79b816d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T08:03:38.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:38 smithi118 ceph-mon[27391]: osdmap e202: 8 total, 8 up, 8 in 2024-03-20T08:03:38.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:38 smithi118 ceph-mon[38981]: osdmap e202: 8 total, 8 up, 8 in 2024-03-20T08:03:38.850 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:38.580+0000 7f79b816d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T08:03:38.900 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:38.676+0000 7f79b816d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T08:03:39.114 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:38.880+0000 7f79b816d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:39.114 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:39.070+0000 7f79b816d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T08:03:39.122 INFO:tasks.rados.rados.0.smithi118.stdout:687: done (0 left) 2024-03-20T08:03:39.122 INFO:tasks.rados.rados.0.smithi118.stdout:688: copy_from oid 48 from oid 5 current snap is 73 2024-03-20T08:03:39.122 INFO:tasks.rados.rados.0.smithi118.stdout:689: setattr oid 20 current snap is 73 2024-03-20T08:03:39.123 INFO:tasks.rados.rados.0.smithi118.stdout:690: read oid 3 snap -1 2024-03-20T08:03:39.123 INFO:tasks.rados.rados.0.smithi118.stdout:690: expect (ObjNum 175 snap 52 seq_num 175) 2024-03-20T08:03:39.123 INFO:tasks.rados.rados.0.smithi118.stdout:691: snap_remove snap 70 2024-03-20T08:03:39.376 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:39 smithi196 ceph-mon[31033]: pgmap v39: 121 pgs: 121 active+clean; 162 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-03-20T08:03:39.376 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:39 smithi196 ceph-mon[31033]: osdmap e203: 8 total, 8 up, 8 in 2024-03-20T08:03:39.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:39 smithi118 ceph-mon[27391]: pgmap v39: 121 pgs: 121 active+clean; 162 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-03-20T08:03:39.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:39 smithi118 ceph-mon[27391]: osdmap e203: 8 total, 8 up, 8 in 2024-03-20T08:03:39.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:39 smithi118 ceph-mon[38981]: pgmap v39: 121 pgs: 121 active+clean; 162 MiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 8 op/s 2024-03-20T08:03:39.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:39 smithi118 ceph-mon[38981]: osdmap e203: 8 total, 8 up, 8 in 2024-03-20T08:03:39.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:39.377+0000 7f79b816d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:39.939 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:39.653+0000 7f79b816d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:39.939 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:39.940+0000 7f79b816d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T08:03:40.128 INFO:tasks.rados.rados.0.smithi118.stdout:691: done (3 left) 2024-03-20T08:03:40.128 INFO:tasks.rados.rados.0.smithi118.stdout:692: rmattr oid 41 current snap is 73 2024-03-20T08:03:40.128 INFO:tasks.rados.rados.0.smithi118.stdout:693: delete oid 26 current snap is 73 2024-03-20T08:03:40.128 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 371 (ObjNum 146 snap 41 seq_num 146) dirty exists 2024-03-20T08:03:40.128 INFO:tasks.rados.rados.0.smithi118.stdout:688: finishing copy_from to smithi11881747-48 2024-03-20T08:03:40.128 INFO:tasks.rados.rados.0.smithi118.stdout:688: got expected ENOENT (src dne) 2024-03-20T08:03:40.128 INFO:tasks.rados.rados.0.smithi118.stdout:688: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:03:40.128 INFO:tasks.rados.rados.0.smithi118.stdout:690: expect (ObjNum 175 snap 52 seq_num 175) 2024-03-20T08:03:40.149 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 507 (ObjNum 191 snap 61 seq_num 191) dirty exists 2024-03-20T08:03:40.149 INFO:tasks.rados.rados.0.smithi118.stdout:688: done (4 left) 2024-03-20T08:03:40.149 INFO:tasks.rados.rados.0.smithi118.stdout:689: done (3 left) 2024-03-20T08:03:40.149 INFO:tasks.rados.rados.0.smithi118.stdout:690: done (2 left) 2024-03-20T08:03:40.149 INFO:tasks.rados.rados.0.smithi118.stdout:692: done (1 left) 2024-03-20T08:03:40.149 INFO:tasks.rados.rados.0.smithi118.stdout:693: done (0 left) 2024-03-20T08:03:40.149 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 73 2024-03-20T08:03:40.149 INFO:tasks.rados.rados.0.smithi118.stdout:694: seq_num 225 ranges {0=573440,573440=622592,1196032=180224} 2024-03-20T08:03:40.152 INFO:tasks.rados.rados.0.smithi118.stdout:694: writing smithi11881747-4 from 0 to 573440 tid 1 2024-03-20T08:03:40.155 INFO:tasks.rados.rados.0.smithi118.stdout:694: writing smithi11881747-4 from 573440 to 1196032 tid 2 2024-03-20T08:03:40.157 INFO:tasks.rados.rados.0.smithi118.stdout:694: writing smithi11881747-4 from 1196032 to 1376256 tid 3 2024-03-20T08:03:40.157 INFO:tasks.rados.rados.0.smithi118.stdout:695: read oid 11 snap -1 2024-03-20T08:03:40.157 INFO:tasks.rados.rados.0.smithi118.stdout:695: expect deleted 2024-03-20T08:03:40.157 INFO:tasks.rados.rados.0.smithi118.stdout:696: read oid 42 snap 49 2024-03-20T08:03:40.157 INFO:tasks.rados.rados.0.smithi118.stdout:696: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:03:40.157 INFO:tasks.rados.rados.0.smithi118.stdout:697: setattr oid 18 current snap is 73 2024-03-20T08:03:40.157 INFO:tasks.rados.rados.0.smithi118.stdout:694: finishing write tid 1 to smithi11881747-4 2024-03-20T08:03:40.158 INFO:tasks.rados.rados.0.smithi118.stdout:695: done (3 left) 2024-03-20T08:03:40.158 INFO:tasks.rados.rados.0.smithi118.stdout:698: snap_remove snap 54 2024-03-20T08:03:40.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:40 smithi196 ceph-mon[31033]: osdmap e204: 8 total, 8 up, 8 in 2024-03-20T08:03:40.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:40 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:40.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:40 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:40.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:40 smithi118 ceph-mon[27391]: osdmap e204: 8 total, 8 up, 8 in 2024-03-20T08:03:40.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:40 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:40.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:40 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:40.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:40 smithi118 ceph-mon[38981]: osdmap e204: 8 total, 8 up, 8 in 2024-03-20T08:03:40.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:40 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:40.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:40 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:41.034 INFO:tasks.rados.rados.0.smithi118.stdout:694: finishing write tid 2 to smithi11881747-4 2024-03-20T08:03:41.035 INFO:tasks.rados.rados.0.smithi118.stdout:694: finishing write tid 3 to smithi11881747-4 2024-03-20T08:03:41.035 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 376 (ObjNum 129 snap 0 seq_num 321817056) dirty exists 2024-03-20T08:03:41.035 INFO:tasks.rados.rados.0.smithi118.stdout:694: finishing write tid 4 to smithi11881747-4 2024-03-20T08:03:41.035 INFO:tasks.rados.rados.0.smithi118.stdout:696: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:03:41.060 INFO:tasks.rados.rados.0.smithi118.stdout:694: finishing write tid 5 to smithi11881747-4 2024-03-20T08:03:41.060 INFO:tasks.rados.rados.0.smithi118.stdout:694: oid 4 updating version 0 to 126 2024-03-20T08:03:41.060 INFO:tasks.rados.rados.0.smithi118.stdout:694: oid 4 version 126 is already newer than 124 2024-03-20T08:03:41.060 INFO:tasks.rados.rados.0.smithi118.stdout:694: oid 4 version 126 is already newer than 125 2024-03-20T08:03:41.060 INFO:tasks.rados.rados.0.smithi118.stdout:694: oid 4 updating version 126 to 127 2024-03-20T08:03:41.061 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 127 (ObjNum 225 snap 73 seq_num 225) dirty exists 2024-03-20T08:03:41.061 INFO:tasks.rados.rados.0.smithi118.stdout:694: left oid 4 (ObjNum 225 snap 73 seq_num 225) 2024-03-20T08:03:41.061 INFO:tasks.rados.rados.0.smithi118.stdout:694: done (3 left) 2024-03-20T08:03:41.061 INFO:tasks.rados.rados.0.smithi118.stdout:696: done (2 left) 2024-03-20T08:03:41.061 INFO:tasks.rados.rados.0.smithi118.stdout:697: done (1 left) 2024-03-20T08:03:41.061 INFO:tasks.rados.rados.0.smithi118.stdout:698: done (0 left) 2024-03-20T08:03:41.061 INFO:tasks.rados.rados.0.smithi118.stdout:699: delete oid 43 current snap is 73 2024-03-20T08:03:41.062 INFO:tasks.rados.rados.0.smithi118.stdout:699: done (0 left) 2024-03-20T08:03:41.062 INFO:tasks.rados.rados.0.smithi118.stdout:700: snap_create 2024-03-20T08:03:41.123 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:40.815+0000 7f79b816d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T08:03:41.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-mon[31033]: pgmap v42: 121 pgs: 121 active+clean; 163 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 6.6 MiB/s rd, 324 KiB/s wr, 4 op/s 2024-03-20T08:03:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-mon[31033]: osdmap e205: 8 total, 8 up, 8 in 2024-03-20T08:03:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-mon[31033]: osdmap e206: 8 total, 8 up, 8 in 2024-03-20T08:03:41.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[27391]: pgmap v42: 121 pgs: 121 active+clean; 163 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 6.6 MiB/s rd, 324 KiB/s wr, 4 op/s 2024-03-20T08:03:41.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[27391]: osdmap e205: 8 total, 8 up, 8 in 2024-03-20T08:03:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:41.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[27391]: osdmap e206: 8 total, 8 up, 8 in 2024-03-20T08:03:41.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[38981]: pgmap v42: 121 pgs: 121 active+clean; 163 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 6.6 MiB/s rd, 324 KiB/s wr, 4 op/s 2024-03-20T08:03:41.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[38981]: osdmap e205: 8 total, 8 up, 8 in 2024-03-20T08:03:41.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:41.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:41.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:41 smithi118 ceph-mon[38981]: osdmap e206: 8 total, 8 up, 8 in 2024-03-20T08:03:41.751 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:41.427+0000 7f79b816d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T08:03:41.751 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:41.528+0000 7f79b816d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T08:03:41.751 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:41.752+0000 7f79b816d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T08:03:42.043 INFO:tasks.rados.rados.0.smithi118.stdout:700: done (0 left) 2024-03-20T08:03:42.043 INFO:tasks.rados.rados.0.smithi118.stdout:701: setattr oid 43 current snap is 74 2024-03-20T08:03:42.044 INFO:tasks.rados.rados.0.smithi118.stdout:702: snap_remove snap 72 2024-03-20T08:03:42.130 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: [20/Mar/2024:08:03:41] ENGINE Bus STARTING 2024-03-20T08:03:42.130 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: CherryPy Checker: 2024-03-20T08:03:42.130 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: The Application mounted at '' has an empty config. 2024-03-20T08:03:42.130 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:42.130 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: [20/Mar/2024:08:03:41] ENGINE Serving on http://:::9283 2024-03-20T08:03:42.130 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: [20/Mar/2024:08:03:41] ENGINE Bus STARTED 2024-03-20T08:03:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:42 smithi196 ceph-mon[31033]: Standby manager daemon x restarted 2024-03-20T08:03:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:42 smithi196 ceph-mon[31033]: Standby manager daemon x started 2024-03-20T08:03:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:42 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:42 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:42 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:42 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:42 smithi196 ceph-mon[31033]: osdmap e207: 8 total, 8 up, 8 in 2024-03-20T08:03:42.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[38981]: Standby manager daemon x restarted 2024-03-20T08:03:42.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[38981]: Standby manager daemon x started 2024-03-20T08:03:42.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:42.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:42.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[38981]: osdmap e207: 8 total, 8 up, 8 in 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[27391]: Standby manager daemon x restarted 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[27391]: Standby manager daemon x started 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[27391]: from='mgr.? 172.21.15.196:0/4117234808' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:42.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:42 smithi118 ceph-mon[27391]: osdmap e207: 8 total, 8 up, 8 in 2024-03-20T08:03:43.045 INFO:tasks.rados.rados.0.smithi118.stdout:702: done (1 left) 2024-03-20T08:03:43.045 INFO:tasks.rados.rados.0.smithi118.stdout:703: read oid 21 snap -1 2024-03-20T08:03:43.045 INFO:tasks.rados.rados.0.smithi118.stdout:703: expect (ObjNum 206 snap 67 seq_num 206) 2024-03-20T08:03:43.045 INFO:tasks.rados.rados.0.smithi118.stdout:704: rollback oid 5 current snap is 74 2024-03-20T08:03:43.046 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 23 2024-03-20T08:03:43.046 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 293 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-03-20T08:03:43.046 INFO:tasks.rados.rados.0.smithi118.stdout:701: done (2 left) 2024-03-20T08:03:43.046 INFO:tasks.rados.rados.0.smithi118.stdout:705: read oid 50 snap -1 2024-03-20T08:03:43.046 INFO:tasks.rados.rados.0.smithi118.stdout:705: expect deleted 2024-03-20T08:03:43.046 INFO:tasks.rados.rados.0.smithi118.stdout:706: snap_remove snap 67 2024-03-20T08:03:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:43 smithi196 ceph-mon[31033]: pgmap v45: 121 pgs: 121 active+clean; 163 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 7.0 MiB/s rd, 324 KiB/s wr, 15 op/s 2024-03-20T08:03:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:43 smithi196 ceph-mon[31033]: mgrmap e33: y(active, since 28s), standbys: x 2024-03-20T08:03:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:43 smithi196 ceph-mon[31033]: osdmap e208: 8 total, 8 up, 8 in 2024-03-20T08:03:43.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:43 smithi118 ceph-mon[38981]: pgmap v45: 121 pgs: 121 active+clean; 163 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 7.0 MiB/s rd, 324 KiB/s wr, 15 op/s 2024-03-20T08:03:43.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:43 smithi118 ceph-mon[38981]: mgrmap e33: y(active, since 28s), standbys: x 2024-03-20T08:03:43.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:43 smithi118 ceph-mon[38981]: osdmap e208: 8 total, 8 up, 8 in 2024-03-20T08:03:43.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:43 smithi118 ceph-mon[27391]: pgmap v45: 121 pgs: 121 active+clean; 163 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 7.0 MiB/s rd, 324 KiB/s wr, 15 op/s 2024-03-20T08:03:43.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:43 smithi118 ceph-mon[27391]: mgrmap e33: y(active, since 28s), standbys: x 2024-03-20T08:03:43.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:43 smithi118 ceph-mon[27391]: osdmap e208: 8 total, 8 up, 8 in 2024-03-20T08:03:43.694 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:43.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:03:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:03:43] "GET /metrics HTTP/1.1" 200 36070 "" "Prometheus/2.43.0" 2024-03-20T08:03:44.049 INFO:tasks.rados.rados.0.smithi118.stdout:706: done (3 left) 2024-03-20T08:03:44.050 INFO:tasks.rados.rados.0.smithi118.stdout:707: copy_from oid 4 from oid 31 current snap is 74 2024-03-20T08:03:44.050 INFO:tasks.rados.rados.0.smithi118.stdout:708: setattr oid 41 current snap is 74 2024-03-20T08:03:44.050 INFO:tasks.rados.rados.0.smithi118.stdout:704: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:03:44.050 INFO:tasks.rados.rados.0.smithi118.stdout:704: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:03:44.050 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 375 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-03-20T08:03:44.050 INFO:tasks.rados.rados.0.smithi118.stdout:703: expect (ObjNum 206 snap 67 seq_num 206) 2024-03-20T08:03:44.084 INFO:tasks.rados.rados.0.smithi118.stdout:703: done (4 left) 2024-03-20T08:03:44.084 INFO:tasks.rados.rados.0.smithi118.stdout:704: done (3 left) 2024-03-20T08:03:44.084 INFO:tasks.rados.rados.0.smithi118.stdout:705: done (2 left) 2024-03-20T08:03:44.084 INFO:tasks.rados.rados.0.smithi118.stdout:709: read oid 24 snap 62 2024-03-20T08:03:44.084 INFO:tasks.rados.rados.0.smithi118.stdout:709: expect deleted 2024-03-20T08:03:44.084 INFO:tasks.rados.rados.0.smithi118.stdout:710: snap_create 2024-03-20T08:03:44.086 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 523 (ObjNum 191 snap 61 seq_num 191) dirty exists 2024-03-20T08:03:44.088 INFO:tasks.rados.rados.0.smithi118.stdout:707: finishing copy_from racing read to smithi11881747-4 2024-03-20T08:03:44.088 INFO:tasks.rados.rados.0.smithi118.stdout:707: finishing copy_from to smithi11881747-4 2024-03-20T08:03:44.089 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 274 (ObjNum 207 snap 67 seq_num 207) dirty exists 2024-03-20T08:03:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:44 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:44 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:44 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:44 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:44 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:44 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:44 smithi196 ceph-mon[31033]: osdmap e209: 8 total, 8 up, 8 in 2024-03-20T08:03:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[38981]: osdmap e209: 8 total, 8 up, 8 in 2024-03-20T08:03:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:03:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:44 smithi118 ceph-mon[27391]: osdmap e209: 8 total, 8 up, 8 in 2024-03-20T08:03:45.047 INFO:tasks.rados.rados.0.smithi118.stdout:707: done (3 left) 2024-03-20T08:03:45.047 INFO:tasks.rados.rados.0.smithi118.stdout:708: done (2 left) 2024-03-20T08:03:45.047 INFO:tasks.rados.rados.0.smithi118.stdout:709: done (1 left) 2024-03-20T08:03:45.047 INFO:tasks.rados.rados.0.smithi118.stdout:710: done (0 left) 2024-03-20T08:03:45.047 INFO:tasks.rados.rados.0.smithi118.stdout:711: setattr oid 5 current snap is 75 2024-03-20T08:03:45.048 INFO:tasks.rados.rados.0.smithi118.stdout:712: snap_create 2024-03-20T08:03:45.052 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 379 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-03-20T08:03:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: pgmap v48: 121 pgs: 121 active+clean; 163 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 420 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T08:03:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: Upgrade: Setting container_image for all mgr 2024-03-20T08:03:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-20T08:03:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-20T08:03:45.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-20T08:03:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-20T08:03:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-20T08:03:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-03-20T08:03:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:03:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-20T08:03:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: Upgrade: It appears safe to stop mon.a 2024-03-20T08:03:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:45 smithi196 ceph-mon[31033]: osdmap e210: 8 total, 8 up, 8 in 2024-03-20T08:03:45.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: pgmap v48: 121 pgs: 121 active+clean; 163 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 420 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: Upgrade: Setting container_image for all mgr 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-20T08:03:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: Upgrade: It appears safe to stop mon.a 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[38981]: osdmap e210: 8 total, 8 up, 8 in 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: pgmap v48: 121 pgs: 121 active+clean; 163 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 420 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: Upgrade: Setting container_image for all mgr 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-03-20T08:03:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-20T08:03:45.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-03-20T08:03:45.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-03-20T08:03:45.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:03:45.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-20T08:03:45.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: Upgrade: It appears safe to stop mon.a 2024-03-20T08:03:45.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:45 smithi118 ceph-mon[27391]: osdmap e210: 8 total, 8 up, 8 in 2024-03-20T08:03:46.048 INFO:tasks.rados.rados.0.smithi118.stdout:711: done (1 left) 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:712: done (0 left) 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:713: read oid 2 snap 58 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:713: expect deleted 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:714: read oid 33 snap -1 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:714: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:715: rollback oid 14 current snap is 76 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 65 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:716: read oid 45 snap -1 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:716: expect (ObjNum 196 snap 64 seq_num 196) 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:717: rollback oid 4 current snap is 76 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 4 to 23 2024-03-20T08:03:46.049 INFO:tasks.rados.rados.0.smithi118.stdout:718: setattr oid 23 current snap is 76 2024-03-20T08:03:46.050 INFO:tasks.rados.rados.0.smithi118.stdout:719: delete oid 25 current snap is 76 2024-03-20T08:03:46.051 INFO:tasks.rados.rados.0.smithi118.stdout:714: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:03:46.056 INFO:tasks.rados.rados.0.smithi118.stdout:717: finishing rollback tid 0 to smithi11881747-4 2024-03-20T08:03:46.057 INFO:tasks.rados.rados.0.smithi118.stdout:717: finishing rollback tid 1 to smithi11881747-4 2024-03-20T08:03:46.057 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 276 (ObjNum 65 snap 0 seq_num 1616467824) dirty dne 2024-03-20T08:03:46.057 INFO:tasks.rados.rados.0.smithi118.stdout:715: finishing rollback tid 0 to smithi11881747-14 2024-03-20T08:03:46.057 INFO:tasks.rados.rados.0.smithi118.stdout:715: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:03:46.057 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 525 (ObjNum 213 snap 67 seq_num 213) dirty exists 2024-03-20T08:03:46.057 INFO:tasks.rados.rados.0.smithi118.stdout:715: finishing rollback tid 2 to smithi11881747-14 2024-03-20T08:03:46.057 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 231 (ObjNum 163 snap 45 seq_num 163) dirty exists 2024-03-20T08:03:46.058 INFO:tasks.rados.rados.0.smithi118.stdout:713: done (6 left) 2024-03-20T08:03:46.058 INFO:tasks.rados.rados.0.smithi118.stdout:714: done (5 left) 2024-03-20T08:03:46.058 INFO:tasks.rados.rados.0.smithi118.stdout:715: done (4 left) 2024-03-20T08:03:46.058 INFO:tasks.rados.rados.0.smithi118.stdout:717: done (3 left) 2024-03-20T08:03:46.059 INFO:tasks.rados.rados.0.smithi118.stdout:718: done (2 left) 2024-03-20T08:03:46.059 INFO:tasks.rados.rados.0.smithi118.stdout:719: done (1 left) 2024-03-20T08:03:46.059 INFO:tasks.rados.rados.0.smithi118.stdout:720: snap_create 2024-03-20T08:03:46.062 INFO:tasks.rados.rados.0.smithi118.stdout:716: expect (ObjNum 196 snap 64 seq_num 196) 2024-03-20T08:03:47.049 INFO:tasks.rados.rados.0.smithi118.stdout:716: done (1 left) 2024-03-20T08:03:47.049 INFO:tasks.rados.rados.0.smithi118.stdout:720: done (0 left) 2024-03-20T08:03:47.049 INFO:tasks.rados.rados.0.smithi118.stdout:721: delete oid 9 current snap is 77 2024-03-20T08:03:47.052 INFO:tasks.rados.rados.0.smithi118.stdout:721: done (0 left) 2024-03-20T08:03:47.053 INFO:tasks.rados.rados.0.smithi118.stdout:722: snap_create 2024-03-20T08:03:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:46 smithi196 ceph-mon[31033]: pgmap v51: 121 pgs: 121 active+clean; 173 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:03:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:46 smithi196 ceph-mon[31033]: Upgrade: Updating mon.a 2024-03-20T08:03:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:46 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:46 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:03:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:46 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:03:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:46 smithi196 ceph-mon[31033]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:46 smithi196 ceph-mon[31033]: Deploying daemon mon.a on smithi118 2024-03-20T08:03:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:46 smithi196 ceph-mon[31033]: osdmap e211: 8 total, 8 up, 8 in 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[27391]: pgmap v51: 121 pgs: 121 active+clean; 173 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[27391]: Upgrade: Updating mon.a 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[27391]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[27391]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[27391]: Deploying daemon mon.a on smithi118 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[27391]: osdmap e211: 8 total, 8 up, 8 in 2024-03-20T08:03:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[38981]: pgmap v51: 121 pgs: 121 active+clean; 173 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:03:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[38981]: Upgrade: Updating mon.a 2024-03-20T08:03:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='mgr.y' 2024-03-20T08:03:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:03:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:03:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[38981]: from='mgr.15147 172.21.15.118:0/2564299107' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:03:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[38981]: Deploying daemon mon.a on smithi118 2024-03-20T08:03:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:46 smithi118 ceph-mon[38981]: osdmap e211: 8 total, 8 up, 8 in 2024-03-20T08:03:48.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:47 smithi118 systemd[1]: Stopping Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:03:48.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[27368]: 2024-03-20T08:03:47.962+0000 7f8e0a3a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:03:48.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[27368]: 2024-03-20T08:03:47.962+0000 7f8e0a3a5700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-20T08:03:48.485 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:48 smithi118 podman[92927]: 2024-03-20 08:03:48.1775031 +0000 UTC m=+0.319565441 container died 89e74e5abc841d5afb9c39a870540e6a5a79c4a0e8ad433c679d700f58ac2f54 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, org.label-schema.license=GPLv2, RELEASE=reef-78dbc53, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True) 2024-03-20T08:03:48.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:48 smithi118 podman[92927]: 2024-03-20 08:03:48.586046053 +0000 UTC m=+0.728108391 container cleanup 89e74e5abc841d5afb9c39a870540e6a5a79c4a0e8ad433c679d700f58ac2f54 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=reef-78dbc53, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-20T08:03:48.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:48 smithi118 bash[92927]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a 2024-03-20T08:03:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:48 smithi118 podman[92946]: 2024-03-20 08:03:48.778731857 +0000 UTC m=+0.591916775 container remove 89e74e5abc841d5afb9c39a870540e6a5a79c4a0e8ad433c679d700f58ac2f54 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, RELEASE=reef-78dbc53, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-20T08:03:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:48 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-20T08:03:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 systemd[1]: Stopped Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:03:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a.service: Consumed 13.460s CPU time. 2024-03-20T08:03:49.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 systemd[1]: Starting Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:03:49.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 podman[93083]: 2024-03-20 08:03:49.314836181 +0000 UTC m=+0.017904226 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:03:49.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 podman[93083]: 2024-03-20 08:03:49.446695397 +0000 UTC m=+0.149763468 container create 51c68c4d30af2a5c840ae69fed5980424145aa24b213b815a304fac875da3d8a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-20T08:03:50.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 podman[93083]: 2024-03-20 08:03:49.69085346 +0000 UTC m=+0.393921505 container init 51c68c4d30af2a5c840ae69fed5980424145aa24b213b815a304fac875da3d8a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-20T08:03:50.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 podman[93083]: 2024-03-20 08:03:49.696260877 +0000 UTC m=+0.399328908 container start 51c68c4d30af2a5c840ae69fed5980424145aa24b213b815a304fac875da3d8a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-20T08:03:50.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: set uid:gid to 167:167 (ceph:ceph) 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev), process ceph-mon, pid 7 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: pidfile_write: ignore empty --pid-file 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: load: jerasure load: lrc 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: RocksDB version: 7.9.2 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Git sha 0 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Compile date 2024-03-18 19:18:37 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: DB SUMMARY 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: DB Session ID: GF6VSW61A3VZ115UBCY0 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: CURRENT file: CURRENT 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: IDENTITY file: IDENTITY 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1267 Bytes 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000029.sst 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000027.log size: 3815821 ; 2024-03-20T08:03:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.error_if_exists: 0 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.create_if_missing: 0 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.paranoid_checks: 1 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.env: 0x55fb851f2b20 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.fs: PosixFileSystem 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.info_log: 0x55fb87171920 2024-03-20T08:03:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_file_opening_threads: 16 2024-03-20T08:03:50.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.statistics: (nil) 2024-03-20T08:03:50.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.use_fsync: 0 2024-03-20T08:03:50.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_log_file_size: 0 2024-03-20T08:03:50.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-20T08:03:50.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-20T08:03:50.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.keep_log_file_num: 1000 2024-03-20T08:03:50.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.recycle_log_file_num: 0 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.allow_fallocate: 1 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.allow_mmap_reads: 0 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.allow_mmap_writes: 0 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.use_direct_reads: 0 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.create_missing_column_families: 0 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.db_log_dir: 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.wal_dir: 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-20T08:03:50.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.advise_random_on_open: 1 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.db_write_buffer_size: 0 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.write_buffer_manager: 0x55fb87144820 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.rate_limiter: (nil) 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-20T08:03:50.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.wal_recovery_mode: 2 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.enable_thread_tracking: 0 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.enable_pipelined_write: 0 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.unordered_write: 0 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.row_cache: None 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.wal_filter: None 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-20T08:03:50.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.allow_ingest_behind: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.two_write_queues: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.manual_wal_flush: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.wal_compression: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.atomic_flush: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.log_readahead_size: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.best_efforts_recovery: 0 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-20T08:03:50.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.allow_data_in_errors: 0 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.db_host_id: __hostname__ 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_background_jobs: 2 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_background_compactions: -1 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_subcompactions: 1 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_total_wal_size: 0 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-20T08:03:50.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_open_files: -1 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bytes_per_sync: 0 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_readahead_size: 0 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_background_flushes: -1 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Compression algorithms supported: 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: kZSTD supported: 0 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: kXpressCompression supported: 0 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: kLZ4HCCompression supported: 1 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: kZlibCompression supported: 1 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: kSnappyCompression supported: 1 2024-03-20T08:03:50.182 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: kLZ4Compression supported: 1 2024-03-20T08:03:50.183 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: kBZip2Compression supported: 0 2024-03-20T08:03:50.183 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-20T08:03:50.183 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-20T08:03:50.183 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 2024-03-20T08:03:50.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-20T08:03:50.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-20T08:03:50.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.merge_operator: 2024-03-20T08:03:50.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_filter: None 2024-03-20T08:03:50.184 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_filter_factory: None 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.sst_partitioner_factory: None 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fb87171a20) 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: cache_index_and_filter_blocks: 1 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: pin_top_level_index_and_filter: 1 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_type: 0 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: data_block_index_type: 0 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_shortening: 1 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: checksum: 4 2024-03-20T08:03:50.185 INFO:journalctl@ceph.mon.a.smithi118.stdout: no_block_cache: 0 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache: 0x55fb87177090 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_name: BinnedLRUCache 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_options: 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: capacity : 536870912 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: num_shard_bits : 4 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: strict_capacity_limit : 0 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: high_pri_pool_ratio: 0.000 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_cache_compressed: (nil) 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: persistent_cache: (nil) 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_size: 4096 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_size_deviation: 10 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_restart_interval: 16 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: index_block_restart_interval: 1 2024-03-20T08:03:50.186 INFO:journalctl@ceph.mon.a.smithi118.stdout: metadata_block_size: 4096 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: partition_filters: 0 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: use_delta_encoding: 1 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: filter_policy: bloomfilter 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: whole_key_filtering: 1 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: verify_compression: 0 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: read_amp_bytes_per_bit: 0 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: format_version: 5 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: enable_index_compression: 1 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: block_align: 0 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: max_auto_readahead_size: 262144 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: prepopulate_block_cache: 0 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: initial_auto_readahead_size: 8192 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout: num_file_reads_for_auto_readahead: 2 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.write_buffer_size: 33554432 2024-03-20T08:03:50.187 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_write_buffer_number: 2 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression: NoCompression 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression: Disabled 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.prefix_extractor: nullptr 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.num_levels: 7 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-20T08:03:50.188 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.level: 32767 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.strategy: 0 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.enabled: false 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-20T08:03:50.189 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.target_file_size_base: 67108864 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-20T08:03:50.190 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.arena_block_size: 1048576 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.disable_auto_compactions: 0 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-20T08:03:50.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.table_properties_collectors: 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.inplace_update_support: 0 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.bloom_locality: 0 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.max_successive_merges: 0 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.paranoid_file_checks: 0 2024-03-20T08:03:50.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.force_consistency_checks: 1 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.report_bg_io_stats: 0 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.ttl: 2592000 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.enable_blob_files: false 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.min_blob_size: 0 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.blob_file_size: 268435456 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.blob_file_starting_level: 0 2024-03-20T08:03:50.193 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 31, last_sequence is 12946, log_number is 27,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 27 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 27 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 337d34b6-ad27-444c-b877-fa393755793d 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921829725356, "job": 1, "event": "recovery_started", "wal_files": [27]} 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #27 mode 2 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921829736496, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 32, "file_size": 3539935, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12947, "largest_seqno": 14888, "table_properties": {"data_size": 3530089, "index_size": 6714, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2117, "raw_key_size": 20224, "raw_average_key_size": 24, "raw_value_size": 3511245, "raw_average_value_size": 4225, "num_data_blocks": 296, "num_entries": 831, "num_filter_entries": 831, "num_deletions": 28, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710921829, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "337d34b6-ad27-444c-b877-fa393755793d", "db_session_id": "GF6VSW61A3VZ115UBCY0", "orig_file_number": 32, "seqno_to_time_mapping": "N/A"}} 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921829736574, "job": 1, "event": "recovery_finished"} 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/version_set.cc:5047] Creating manifest 34 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000027.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-20T08:03:50.194 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fb8726a000 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: DB pointer 0x55fb87254000 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** DB Stats ** 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** Compaction Stats [default] ** 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T08:03:50.195 INFO:journalctl@ceph.mon.a.smithi118.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: L0 1/0 3.38 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 596.9 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: L6 1/0 7.27 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: Sum 2/0 10.65 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 596.9 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 596.9 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** Compaction Stats [default] ** 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 596.9 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-03-20T08:03:50.196 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: Cumulative compaction: 0.00 GB write, 248.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: Interval compaction: 0.00 GB write, 248.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: Block cache BinnedLRUCache@0x55fb87177090#7 capacity: 512.00 MB usage: 10.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.28 KB,0.000435114%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: starting mon.a rank 0 at public addrs [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] at bind addrs [v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: mon.a@-1(???) e3 preinit fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: mon.a@-1(???).mds e6 new map 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: mon.a@-1(???).mds e6 print_map 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: e6 2024-03-20T08:03:50.197 INFO:journalctl@ceph.mon.a.smithi118.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: legacy client fscid: 1 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: Filesystem 'cephfs' (1) 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: fs_name cephfs 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: epoch 6 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: created 2024-03-20T07:58:26.102802+0000 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: modified 2024-03-20T07:58:28.113441+0000 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: tableserver 0 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: root 0 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: session_timeout 60 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: session_autoclose 300 2024-03-20T08:03:50.198 INFO:journalctl@ceph.mon.a.smithi118.stdout: max_file_size 1099511627776 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: max_xattr_size 65536 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: required_client_features {} 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: last_failure 0 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: last_failure_osd_epoch 0 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: max_mds 1 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: in 0 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: up {0=24343} 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: failed 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: damaged 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: stopped 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: data_pools [3] 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: metadata_pool 2 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: inline_data disabled 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: balancer 2024-03-20T08:03:50.199 INFO:journalctl@ceph.mon.a.smithi118.stdout: bal_rank_mask -1 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout: standby_count_wanted 0 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout: qdb_cluster leader: 0 members: 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout: [mds.b{0:24343} state up:active seq 3 addr [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] compat {c=[1],r=[1],i=[7ff]}] 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout: Standby daemons: 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout: 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout: [mds.a{-1:14430} state up:standby seq 1 addr [v2:172.21.15.118:6834/3215808514,v1:172.21.15.118:6835/3215808514] compat {c=[1],r=[1],i=[7ff]}] 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: mon.a@-1(???).osd e212 crush map has features 3314933069573799936, adjusting msgr requires 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: mon.a@-1(???).osd e212 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: mon.a@-1(???).osd e212 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: mon.a@-1(???).osd e212 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 ceph-mon[93121]: mon.a@-1(???).paxosservice(auth 1..18) refresh upgraded, format 0 -> 3 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 bash[93083]: 51c68c4d30af2a5c840ae69fed5980424145aa24b213b815a304fac875da3d8a 2024-03-20T08:03:50.200 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:49 smithi118 systemd[1]: Started Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:03:51.229 INFO:tasks.rados.rados.0.smithi118.stdout:722: done (0 left) 2024-03-20T08:03:51.229 INFO:tasks.rados.rados.0.smithi118.stdout:723: rmattr oid 13 current snap is 78 2024-03-20T08:03:51.229 INFO:tasks.rados.rados.0.smithi118.stdout:723: done (0 left) 2024-03-20T08:03:51.229 INFO:tasks.rados.rados.0.smithi118.stdout:724: snap_create 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: ignoring --setuser ceph since I am not root 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: ignoring --setgroup ceph since I am not root 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mgr[78704]: -- 172.21.15.196:0/1078544857 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ce0c8005a0 con 0x55ce0c7d4800 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: pgmap v54: 121 pgs: 121 active+clean; 173 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: pgmap v55: 121 pgs: 121 active+clean; 173 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 980 KiB/s rd, 1.9 MiB/s wr, 20 op/s 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: mon.a calling monitor election 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: osdmap e212: 8 total, 8 up, 8 in 2024-03-20T08:03:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: mgrmap e33: y(active, since 37s), standbys: x 2024-03-20T08:03:51.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: overall HEALTH_OK 2024-03-20T08:03:51.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: from='mgr.15147 ' entity='' 2024-03-20T08:03:51.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-mon[31033]: mgrmap e34: y(active, since 37s), standbys: x 2024-03-20T08:03:51.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: pgmap v54: 121 pgs: 121 active+clean; 173 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:03:51.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: pgmap v55: 121 pgs: 121 active+clean; 173 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 980 KiB/s rd, 1.9 MiB/s wr, 20 op/s 2024-03-20T08:03:51.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: mon.a calling monitor election 2024-03-20T08:03:51.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:03:51.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:03:51.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: osdmap e212: 8 total, 8 up, 8 in 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: mgrmap e33: y(active, since 37s), standbys: x 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: overall HEALTH_OK 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: from='mgr.15147 ' entity='' 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[93121]: mgrmap e34: y(active, since 37s), standbys: x 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ignoring --setuser ceph since I am not root 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ignoring --setgroup ceph since I am not root 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mgr[84555]: -- 172.21.15.118:0/831802653 <== mon.0 v2:172.21.15.118:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56390b56c5a0 con 0x56390b540800 2024-03-20T08:03:51.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: pgmap v54: 121 pgs: 121 active+clean; 173 MiB data, 788 MiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: pgmap v55: 121 pgs: 121 active+clean; 173 MiB data, 797 MiB used, 714 GiB / 715 GiB avail; 980 KiB/s rd, 1.9 MiB/s wr, 20 op/s 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: mon.a calling monitor election 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: osdmap e212: 8 total, 8 up, 8 in 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: mgrmap e33: y(active, since 37s), standbys: x 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: overall HEALTH_OK 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: from='mgr.15147 ' entity='' 2024-03-20T08:03:51.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-mon[38981]: mgrmap e34: y(active, since 37s), standbys: x 2024-03-20T08:03:52.138 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:51.751+0000 7f6fa4c22200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T08:03:52.139 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:51.844+0000 7f6fa4c22200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T08:03:52.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:51.776+0000 7ff9f9017200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-20T08:03:52.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:51.868+0000 7ff9f9017200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-20T08:03:52.229 INFO:tasks.rados.rados.0.smithi118.stdout:724: done (0 left) 2024-03-20T08:03:52.229 INFO:tasks.rados.rados.0.smithi118.stdout:725: copy_from oid 43 from oid 39 current snap is 79 2024-03-20T08:03:52.229 INFO:tasks.rados.rados.0.smithi118.stdout:726: delete oid 49 current snap is 79 2024-03-20T08:03:52.234 INFO:tasks.rados.rados.0.smithi118.stdout:726: done (1 left) 2024-03-20T08:03:52.235 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 79 2024-03-20T08:03:52.235 INFO:tasks.rados.rados.0.smithi118.stdout:727: seq_num 230 ranges {1949696=655360,2605056=491520,3096576=311296} 2024-03-20T08:03:52.248 INFO:tasks.rados.rados.0.smithi118.stdout:727: writing smithi11881747-23 from 1949696 to 2605056 tid 1 2024-03-20T08:03:52.251 INFO:tasks.rados.rados.0.smithi118.stdout:727: writing smithi11881747-23 from 2605056 to 3096576 tid 2 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:727: writing smithi11881747-23 from 3096576 to 3407872 tid 3 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:728: rollback oid 3 current snap is 79 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 3 to 47 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:725: finishing copy_from racing read to smithi11881747-43 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:725: finishing copy_from to smithi11881747-43 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 308 (ObjNum 191 snap 61 seq_num 191) dirty exists 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:727: finishing write tid 1 to smithi11881747-23 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:725: done (2 left) 2024-03-20T08:03:52.254 INFO:tasks.rados.rados.0.smithi118.stdout:729: snap_create 2024-03-20T08:03:52.256 INFO:tasks.rados.rados.0.smithi118.stdout:727: finishing write tid 2 to smithi11881747-23 2024-03-20T08:03:52.256 INFO:tasks.rados.rados.0.smithi118.stdout:727: finishing write tid 3 to smithi11881747-23 2024-03-20T08:03:52.257 INFO:tasks.rados.rados.0.smithi118.stdout:727: finishing write tid 4 to smithi11881747-23 2024-03-20T08:03:52.257 INFO:tasks.rados.rados.0.smithi118.stdout:728: finishing rollback tid 0 to smithi11881747-3 2024-03-20T08:03:52.257 INFO:tasks.rados.rados.0.smithi118.stdout:728: finishing rollback tid 1 to smithi11881747-3 2024-03-20T08:03:52.258 INFO:tasks.rados.rados.0.smithi118.stdout:728: finishing rollback tid 2 to smithi11881747-3 2024-03-20T08:03:52.258 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 178 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:03:52.258 INFO:tasks.rados.rados.0.smithi118.stdout:727: finishing write tid 5 to smithi11881747-23 2024-03-20T08:03:52.259 INFO:tasks.rados.rados.0.smithi118.stdout:727: oid 23 updating version 0 to 527 2024-03-20T08:03:52.259 INFO:tasks.rados.rados.0.smithi118.stdout:727: oid 23 updating version 527 to 528 2024-03-20T08:03:52.259 INFO:tasks.rados.rados.0.smithi118.stdout:727: oid 23 updating version 528 to 529 2024-03-20T08:03:52.259 INFO:tasks.rados.rados.0.smithi118.stdout:727: oid 23 updating version 529 to 530 2024-03-20T08:03:52.259 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 530 (ObjNum 230 snap 79 seq_num 230) dirty exists 2024-03-20T08:03:52.259 INFO:tasks.rados.rados.0.smithi118.stdout:727: left oid 23 (ObjNum 230 snap 79 seq_num 230) 2024-03-20T08:03:52.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:52 smithi196 ceph-mon[31033]: osdmap e213: 8 total, 8 up, 8 in 2024-03-20T08:03:52.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:52 smithi118 ceph-mon[93121]: osdmap e213: 8 total, 8 up, 8 in 2024-03-20T08:03:52.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:52 smithi118 ceph-mon[38981]: osdmap e213: 8 total, 8 up, 8 in 2024-03-20T08:03:52.820 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:52.490+0000 7f6fa4c22200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T08:03:52.820 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:52.583+0000 7f6fa4c22200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T08:03:52.820 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:52.724+0000 7f6fa4c22200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T08:03:52.820 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:52.820+0000 7f6fa4c22200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T08:03:52.843 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:52.513+0000 7ff9f9017200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-20T08:03:52.843 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:52.606+0000 7ff9f9017200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-20T08:03:52.843 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:52.748+0000 7ff9f9017200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-20T08:03:52.843 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:52.843+0000 7ff9f9017200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-20T08:03:53.138 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:52.902+0000 7f6fa4c22200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T08:03:53.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:52.926+0000 7ff9f9017200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-20T08:03:53.240 INFO:tasks.rados.rados.0.smithi118.stdout:727: done (2 left) 2024-03-20T08:03:53.240 INFO:tasks.rados.rados.0.smithi118.stdout:728: done (1 left) 2024-03-20T08:03:53.240 INFO:tasks.rados.rados.0.smithi118.stdout:729: done (0 left) 2024-03-20T08:03:53.240 INFO:tasks.rados.rados.0.smithi118.stdout:730: read oid 42 snap -1 2024-03-20T08:03:53.240 INFO:tasks.rados.rados.0.smithi118.stdout:730: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:03:53.240 INFO:tasks.rados.rados.0.smithi118.stdout:731: read oid 24 snap -1 2024-03-20T08:03:53.240 INFO:tasks.rados.rados.0.smithi118.stdout:731: expect (ObjNum 216 snap 69 seq_num 216) 2024-03-20T08:03:53.240 INFO:tasks.rados.rados.0.smithi118.stdout:732: snap_create 2024-03-20T08:03:53.242 INFO:tasks.rados.rados.0.smithi118.stdout:730: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:03:53.261 INFO:tasks.rados.rados.0.smithi118.stdout:731: expect (ObjNum 216 snap 69 seq_num 216) 2024-03-20T08:03:53.463 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:53 smithi196 ceph-mon[31033]: osdmap e214: 8 total, 8 up, 8 in 2024-03-20T08:03:53.463 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:53.172+0000 7f6fa4c22200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T08:03:53.463 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:53.285+0000 7f6fa4c22200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:53.463 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:53.368+0000 7f6fa4c22200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T08:03:53.463 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:53.462+0000 7f6fa4c22200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T08:03:53.496 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:53 smithi118 ceph-mon[93121]: osdmap e214: 8 total, 8 up, 8 in 2024-03-20T08:03:53.497 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:53 smithi118 ceph-mon[38981]: osdmap e214: 8 total, 8 up, 8 in 2024-03-20T08:03:53.497 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:53.199+0000 7ff9f9017200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-20T08:03:53.497 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:53.314+0000 7ff9f9017200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:53.497 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:53.401+0000 7ff9f9017200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-20T08:03:53.497 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:53.496+0000 7ff9f9017200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-20T08:03:53.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:53.642+0000 7f6fa4c22200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T08:03:53.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:03:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:03:53.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:03:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:53.674+0000 7ff9f9017200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-20T08:03:54.241 INFO:tasks.rados.rados.0.smithi118.stdout:730: done (2 left) 2024-03-20T08:03:54.241 INFO:tasks.rados.rados.0.smithi118.stdout:731: done (1 left) 2024-03-20T08:03:54.241 INFO:tasks.rados.rados.0.smithi118.stdout:732: done (0 left) 2024-03-20T08:03:54.241 INFO:tasks.rados.rados.0.smithi118.stdout:733: delete oid 10 current snap is 81 2024-03-20T08:03:54.247 INFO:tasks.rados.rados.0.smithi118.stdout:733: done (0 left) 2024-03-20T08:03:54.247 INFO:tasks.rados.rados.0.smithi118.stdout:734: snap_create 2024-03-20T08:03:54.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:54 smithi196 ceph-mon[31033]: osdmap e215: 8 total, 8 up, 8 in 2024-03-20T08:03:54.637 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:54 smithi118 ceph-mon[93121]: osdmap e215: 8 total, 8 up, 8 in 2024-03-20T08:03:54.637 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:54 smithi118 ceph-mon[38981]: osdmap e215: 8 total, 8 up, 8 in 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:734: done (0 left) 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:735: read oid 17 snap -1 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:735: expect (ObjNum 128 snap 34 seq_num 128) 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:736: rollback oid 6 current snap is 82 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 43 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:737: read oid 27 snap 73 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:737: expect deleted 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:738: copy_from oid 18 from oid 22 current snap is 82 2024-03-20T08:03:54.748 INFO:tasks.rados.rados.0.smithi118.stdout:739: rmattr oid 14 current snap is 82 2024-03-20T08:03:54.749 INFO:tasks.rados.rados.0.smithi118.stdout:739: done (4 left) 2024-03-20T08:03:54.749 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 82 2024-03-20T08:03:54.749 INFO:tasks.rados.rados.0.smithi118.stdout:740: seq_num 232 ranges {0=688128,688128=663552,1351680=139264} 2024-03-20T08:03:54.752 INFO:tasks.rados.rados.0.smithi118.stdout:740: writing smithi11881747-11 from 0 to 688128 tid 1 2024-03-20T08:03:54.757 INFO:tasks.rados.rados.0.smithi118.stdout:740: writing smithi11881747-11 from 688128 to 1351680 tid 2 2024-03-20T08:03:54.758 INFO:tasks.rados.rados.0.smithi118.stdout:740: writing smithi11881747-11 from 1351680 to 1490944 tid 3 2024-03-20T08:03:54.758 INFO:tasks.rados.rados.0.smithi118.stdout:741: setattr oid 25 current snap is 82 2024-03-20T08:03:54.758 INFO:tasks.rados.rados.0.smithi118.stdout:736: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:03:54.760 INFO:tasks.rados.rados.0.smithi118.stdout:736: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:03:54.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 352 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-03-20T08:03:54.761 INFO:tasks.rados.rados.0.smithi118.stdout:736: done (5 left) 2024-03-20T08:03:54.761 INFO:tasks.rados.rados.0.smithi118.stdout:737: done (4 left) 2024-03-20T08:03:54.761 INFO:tasks.rados.rados.0.smithi118.stdout:742: read oid 3 snap -1 2024-03-20T08:03:54.761 INFO:tasks.rados.rados.0.smithi118.stdout:742: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:03:54.761 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 82 2024-03-20T08:03:54.761 INFO:tasks.rados.rados.0.smithi118.stdout:743: seq_num 233 ranges {0=712704,712704=548864,1261568=253952} 2024-03-20T08:03:54.766 INFO:tasks.rados.rados.0.smithi118.stdout:743: writing smithi11881747-6 from 0 to 712704 tid 1 2024-03-20T08:03:54.770 INFO:tasks.rados.rados.0.smithi118.stdout:743: writing smithi11881747-6 from 712704 to 1261568 tid 2 2024-03-20T08:03:54.772 INFO:tasks.rados.rados.0.smithi118.stdout:743: writing smithi11881747-6 from 1261568 to 1515520 tid 3 2024-03-20T08:03:54.772 INFO:tasks.rados.rados.0.smithi118.stdout:744: delete oid 1 current snap is 82 2024-03-20T08:03:54.772 INFO:tasks.rados.rados.0.smithi118.stdout:740: finishing write tid 1 to smithi11881747-11 2024-03-20T08:03:54.772 INFO:tasks.rados.rados.0.smithi118.stdout:742: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: finishing write tid 2 to smithi11881747-11 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: finishing write tid 3 to smithi11881747-11 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: finishing write tid 4 to smithi11881747-11 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: finishing write tid 5 to smithi11881747-11 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: oid 11 updating version 0 to 386 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: oid 11 version 386 is already newer than 383 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: oid 11 version 386 is already newer than 385 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: oid 11 version 386 is already newer than 384 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 386 (ObjNum 232 snap 82 seq_num 232) dirty exists 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:740: left oid 11 (ObjNum 232 snap 82 seq_num 232) 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 387 (ObjNum 33 snap 0 seq_num 322468352) dirty exists 2024-03-20T08:03:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:735: expect (ObjNum 128 snap 34 seq_num 128) 2024-03-20T08:03:54.860 INFO:tasks.rados.rados.0.smithi118.stdout:743: finishing write tid 1 to smithi11881747-6 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:738: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:743: finishing write tid 2 to smithi11881747-6 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:735: done (6 left) 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:740: done (5 left) 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:741: done (4 left) 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:742: done (3 left) 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:744: done (2 left) 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:745: read oid 46 snap -1 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:745: expect deleted 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:746: read oid 1 snap 65 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:746: expect deleted 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 82 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:743: finishing write tid 3 to smithi11881747-6 2024-03-20T08:03:54.861 INFO:tasks.rados.rados.0.smithi118.stdout:743: finishing write tid 4 to smithi11881747-6 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:738: finishing copy_from to smithi11881747-18 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:743: finishing write tid 5 to smithi11881747-6 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:743: oid 6 updating version 0 to 354 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:743: oid 6 updating version 354 to 355 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:743: oid 6 updating version 355 to 356 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:743: oid 6 version 356 is already newer than 353 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 356 (ObjNum 233 snap 82 seq_num 233) dirty exists 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:743: left oid 6 (ObjNum 233 snap 82 seq_num 233) 2024-03-20T08:03:54.862 INFO:tasks.rados.rados.0.smithi118.stdout:747: seq_num 235 ranges {1105920=745472,1851392=712704,2564096=90112} 2024-03-20T08:03:54.870 INFO:tasks.rados.rados.0.smithi118.stdout:747: writing smithi11881747-29 from 1105920 to 1851392 tid 1 2024-03-20T08:03:54.874 INFO:tasks.rados.rados.0.smithi118.stdout:747: writing smithi11881747-29 from 1851392 to 2564096 tid 2 2024-03-20T08:03:54.875 INFO:tasks.rados.rados.0.smithi118.stdout:747: writing smithi11881747-29 from 2564096 to 2654208 tid 3 2024-03-20T08:03:54.875 INFO:tasks.rados.rados.0.smithi118.stdout:738: done (4 left) 2024-03-20T08:03:54.875 INFO:tasks.rados.rados.0.smithi118.stdout:743: done (3 left) 2024-03-20T08:03:54.875 INFO:tasks.rados.rados.0.smithi118.stdout:748: rmattr oid 30 current snap is 82 2024-03-20T08:03:54.875 INFO:tasks.rados.rados.0.smithi118.stdout:747: finishing write tid 1 to smithi11881747-29 2024-03-20T08:03:54.875 INFO:tasks.rados.rados.0.smithi118.stdout:745: done (3 left) 2024-03-20T08:03:54.875 INFO:tasks.rados.rados.0.smithi118.stdout:746: done (2 left) 2024-03-20T08:03:54.875 INFO:tasks.rados.rados.0.smithi118.stdout:749: snap_remove snap 71 2024-03-20T08:03:54.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:54.617+0000 7f6fa4c22200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T08:03:54.888 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:54.750+0000 7f6fa4c22200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T08:03:54.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:54.637+0000 7ff9f9017200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-20T08:03:54.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:54.784+0000 7ff9f9017200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-20T08:03:55.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:55 smithi196 ceph-mon[31033]: osdmap e216: 8 total, 8 up, 8 in 2024-03-20T08:03:55.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:55 smithi196 ceph-mon[31033]: osdmap e217: 8 total, 8 up, 8 in 2024-03-20T08:03:55.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:55.104+0000 7f6fa4c22200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T08:03:55.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:55.195+0000 7f6fa4c22200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T08:03:55.417 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-mon[93121]: osdmap e216: 8 total, 8 up, 8 in 2024-03-20T08:03:55.417 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-mon[93121]: osdmap e217: 8 total, 8 up, 8 in 2024-03-20T08:03:55.418 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-mon[38981]: osdmap e216: 8 total, 8 up, 8 in 2024-03-20T08:03:55.418 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-mon[38981]: osdmap e217: 8 total, 8 up, 8 in 2024-03-20T08:03:55.418 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:55.125+0000 7ff9f9017200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-20T08:03:55.418 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:55.216+0000 7ff9f9017200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-20T08:03:55.418 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:55.417+0000 7ff9f9017200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:55.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:55.606+0000 7ff9f9017200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T08:03:55.748 INFO:tasks.rados.rados.0.smithi118.stdout:749: done (2 left) 2024-03-20T08:03:55.748 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 82 2024-03-20T08:03:55.748 INFO:tasks.rados.rados.0.smithi118.stdout:750: seq_num 236 ranges {0=761856,761856=647168,1409024=155648} 2024-03-20T08:03:55.753 INFO:tasks.rados.rados.0.smithi118.stdout:750: writing smithi11881747-4 from 0 to 761856 tid 1 2024-03-20T08:03:55.757 INFO:tasks.rados.rados.0.smithi118.stdout:750: writing smithi11881747-4 from 761856 to 1409024 tid 2 2024-03-20T08:03:55.759 INFO:tasks.rados.rados.0.smithi118.stdout:750: writing smithi11881747-4 from 1409024 to 1564672 tid 3 2024-03-20T08:03:55.759 INFO:tasks.rados.rados.0.smithi118.stdout:751: rollback oid 6 current snap is 82 2024-03-20T08:03:55.759 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 81 2024-03-20T08:03:55.759 INFO:tasks.rados.rados.0.smithi118.stdout:747: finishing write tid 2 to smithi11881747-29 2024-03-20T08:03:55.759 INFO:tasks.rados.rados.0.smithi118.stdout:747: finishing write tid 3 to smithi11881747-29 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:747: finishing write tid 4 to smithi11881747-29 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 277 (ObjNum 167 snap 45 seq_num 167) dirty exists 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:747: finishing write tid 5 to smithi11881747-29 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:747: oid 29 updating version 0 to 533 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:747: oid 29 updating version 533 to 534 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:747: oid 29 version 534 is already newer than 532 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:747: oid 29 updating version 534 to 535 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 535 (ObjNum 235 snap 82 seq_num 235) dirty exists 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:747: left oid 29 (ObjNum 235 snap 82 seq_num 235) 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:750: finishing write tid 1 to smithi11881747-4 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:747: done (3 left) 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:748: done (2 left) 2024-03-20T08:03:55.760 INFO:tasks.rados.rados.0.smithi118.stdout:752: snap_create 2024-03-20T08:03:55.762 INFO:tasks.rados.rados.0.smithi118.stdout:751: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:03:55.762 INFO:tasks.rados.rados.0.smithi118.stdout:751: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:03:55.762 INFO:tasks.rados.rados.0.smithi118.stdout:751: finishing rollback tid 2 to smithi11881747-6 2024-03-20T08:03:55.762 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 359 (ObjNum 219 snap 71 seq_num 219) dirty exists 2024-03-20T08:03:55.762 INFO:tasks.rados.rados.0.smithi118.stdout:750: finishing write tid 2 to smithi11881747-4 2024-03-20T08:03:55.763 INFO:tasks.rados.rados.0.smithi118.stdout:750: finishing write tid 3 to smithi11881747-4 2024-03-20T08:03:55.763 INFO:tasks.rados.rados.0.smithi118.stdout:750: finishing write tid 4 to smithi11881747-4 2024-03-20T08:03:55.765 INFO:tasks.rados.rados.0.smithi118.stdout:750: finishing write tid 5 to smithi11881747-4 2024-03-20T08:03:55.765 INFO:tasks.rados.rados.0.smithi118.stdout:750: oid 4 updating version 0 to 279 2024-03-20T08:03:55.765 INFO:tasks.rados.rados.0.smithi118.stdout:750: oid 4 updating version 279 to 281 2024-03-20T08:03:55.765 INFO:tasks.rados.rados.0.smithi118.stdout:750: oid 4 version 281 is already newer than 280 2024-03-20T08:03:55.765 INFO:tasks.rados.rados.0.smithi118.stdout:750: oid 4 version 281 is already newer than 278 2024-03-20T08:03:55.766 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 281 (ObjNum 236 snap 82 seq_num 236) dirty exists 2024-03-20T08:03:55.766 INFO:tasks.rados.rados.0.smithi118.stdout:750: left oid 4 (ObjNum 236 snap 82 seq_num 236) 2024-03-20T08:03:55.879 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:55.394+0000 7f6fa4c22200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:55.879 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:55.583+0000 7f6fa4c22200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-20T08:03:56.139 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:55.880+0000 7f6fa4c22200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:56.163 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:55.903+0000 7ff9f9017200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-20T08:03:56.163 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:56.164+0000 7ff9f9017200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:56.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:56.405+0000 7ff9f9017200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T08:03:56.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:56.141+0000 7f6fa4c22200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-20T08:03:56.639 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:56.383+0000 7f6fa4c22200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:750: done (2 left) 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:751: done (1 left) 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:752: done (0 left) 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:753: copy_from oid 48 from oid 7 current snap is 83 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:754: read oid 2 snap -1 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:754: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:755: read oid 50 snap -1 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:755: expect deleted 2024-03-20T08:03:56.753 INFO:tasks.rados.rados.0.smithi118.stdout:756: snap_create 2024-03-20T08:03:56.755 INFO:tasks.rados.rados.0.smithi118.stdout:754: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:03:56.758 INFO:tasks.rados.rados.0.smithi118.stdout:753: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:03:56.759 INFO:tasks.rados.rados.0.smithi118.stdout:753: finishing copy_from to smithi11881747-48 2024-03-20T08:03:56.759 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 60 (ObjNum 145 snap 41 seq_num 145) dirty exists 2024-03-20T08:03:57.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:56 smithi196 ceph-mon[31033]: osdmap e218: 8 total, 8 up, 8 in 2024-03-20T08:03:57.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:56 smithi118 ceph-mon[93121]: osdmap e218: 8 total, 8 up, 8 in 2024-03-20T08:03:57.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:56 smithi118 ceph-mon[38981]: osdmap e218: 8 total, 8 up, 8 in 2024-03-20T08:03:57.413 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:57.205+0000 7f6fa4c22200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T08:03:57.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:57.219+0000 7ff9f9017200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-20T08:03:57.753 INFO:tasks.rados.rados.0.smithi118.stdout:753: done (3 left) 2024-03-20T08:03:57.754 INFO:tasks.rados.rados.0.smithi118.stdout:754: done (2 left) 2024-03-20T08:03:57.754 INFO:tasks.rados.rados.0.smithi118.stdout:755: done (1 left) 2024-03-20T08:03:57.754 INFO:tasks.rados.rados.0.smithi118.stdout:756: done (0 left) 2024-03-20T08:03:57.754 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 84 2024-03-20T08:03:57.754 INFO:tasks.rados.rados.0.smithi118.stdout:757: seq_num 237 ranges {0=778240,778240=532480,1310720=270336} 2024-03-20T08:03:57.757 INFO:tasks.rados.rados.0.smithi118.stdout:757: writing smithi11881747-35 from 0 to 778240 tid 1 2024-03-20T08:03:57.760 INFO:tasks.rados.rados.0.smithi118.stdout:757: writing smithi11881747-35 from 778240 to 1310720 tid 2 2024-03-20T08:03:57.762 INFO:tasks.rados.rados.0.smithi118.stdout:757: writing smithi11881747-35 from 1310720 to 1581056 tid 3 2024-03-20T08:03:57.762 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 40 current snap is 84 2024-03-20T08:03:57.762 INFO:tasks.rados.rados.0.smithi118.stdout:758: seq_num 238 ranges {0=794624,794624=417792,1212416=385024} 2024-03-20T08:03:57.766 INFO:tasks.rados.rados.0.smithi118.stdout:758: writing smithi11881747-40 from 0 to 794624 tid 1 2024-03-20T08:03:57.769 INFO:tasks.rados.rados.0.smithi118.stdout:758: writing smithi11881747-40 from 794624 to 1212416 tid 2 2024-03-20T08:03:57.773 INFO:tasks.rados.rados.0.smithi118.stdout:758: writing smithi11881747-40 from 1212416 to 1597440 tid 3 2024-03-20T08:03:57.773 INFO:tasks.rados.rados.0.smithi118.stdout:759: read oid 22 snap -1 2024-03-20T08:03:57.773 INFO:tasks.rados.rados.0.smithi118.stdout:759: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:03:57.773 INFO:tasks.rados.rados.0.smithi118.stdout:760: read oid 8 snap -1 2024-03-20T08:03:57.773 INFO:tasks.rados.rados.0.smithi118.stdout:760: expect (ObjNum 122 snap 32 seq_num 122) 2024-03-20T08:03:57.773 INFO:tasks.rados.rados.0.smithi118.stdout:761: setattr oid 3 current snap is 84 2024-03-20T08:03:57.773 INFO:tasks.rados.rados.0.smithi118.stdout:757: finishing write tid 1 to smithi11881747-35 2024-03-20T08:03:57.773 INFO:tasks.rados.rados.0.smithi118.stdout:757: finishing write tid 2 to smithi11881747-35 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:757: finishing write tid 3 to smithi11881747-35 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:757: finishing write tid 4 to smithi11881747-35 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:757: finishing write tid 5 to smithi11881747-35 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:757: oid 35 updating version 0 to 361 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:757: oid 35 updating version 361 to 363 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:757: oid 35 version 363 is already newer than 362 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:757: oid 35 version 363 is already newer than 360 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 363 (ObjNum 237 snap 84 seq_num 237) dirty exists 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:757: left oid 35 (ObjNum 237 snap 84 seq_num 237) 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:758: finishing write tid 1 to smithi11881747-40 2024-03-20T08:03:57.774 INFO:tasks.rados.rados.0.smithi118.stdout:758: finishing write tid 2 to smithi11881747-40 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:757: done (4 left) 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:762: read oid 39 snap 74 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:762: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:763: read oid 49 snap 69 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:763: expect (ObjNum 176 snap 52 seq_num 176) 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:764: rmattr oid 24 current snap is 84 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:764: done (6 left) 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 84 2024-03-20T08:03:57.776 INFO:tasks.rados.rados.0.smithi118.stdout:765: seq_num 239 ranges {1384448=409600,1794048=704512,2498560=499712} 2024-03-20T08:03:57.785 INFO:tasks.rados.rados.0.smithi118.stdout:765: writing smithi11881747-28 from 1384448 to 1794048 tid 1 2024-03-20T08:03:57.789 INFO:tasks.rados.rados.0.smithi118.stdout:765: writing smithi11881747-28 from 1794048 to 2498560 tid 2 2024-03-20T08:03:57.793 INFO:tasks.rados.rados.0.smithi118.stdout:765: writing smithi11881747-28 from 2498560 to 2998272 tid 3 2024-03-20T08:03:57.793 INFO:tasks.rados.rados.0.smithi118.stdout:766: read oid 12 snap -1 2024-03-20T08:03:57.793 INFO:tasks.rados.rados.0.smithi118.stdout:766: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:03:57.793 INFO:tasks.rados.rados.0.smithi118.stdout:767: read oid 50 snap -1 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:767: expect deleted 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 84 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:758: finishing write tid 3 to smithi11881747-40 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:758: finishing write tid 4 to smithi11881747-40 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:758: finishing write tid 5 to smithi11881747-40 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:758: oid 40 updating version 0 to 219 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:758: oid 40 updating version 219 to 221 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:758: oid 40 version 221 is already newer than 220 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:758: oid 40 version 221 is already newer than 218 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 221 (ObjNum 238 snap 84 seq_num 238) dirty exists 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:758: left oid 40 (ObjNum 238 snap 84 seq_num 238) 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 182 (ObjNum 145 snap 0 seq_num 324637944) dirty exists 2024-03-20T08:03:57.794 INFO:tasks.rados.rados.0.smithi118.stdout:760: expect (ObjNum 122 snap 32 seq_num 122) 2024-03-20T08:03:57.852 INFO:tasks.rados.rados.0.smithi118.stdout:759: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:03:57.964 INFO:tasks.rados.rados.0.smithi118.stdout:762: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:03:58.056 INFO:tasks.rados.rados.0.smithi118.stdout:765: finishing write tid 1 to smithi11881747-28 2024-03-20T08:03:58.056 INFO:tasks.rados.rados.0.smithi118.stdout:768: seq_num 240 ranges {2146304=425984,2572288=589824,3162112=483328,3645440=131072} 2024-03-20T08:03:58.069 INFO:tasks.rados.rados.0.smithi118.stdout:768: writing smithi11881747-41 from 2146304 to 2572288 tid 1 2024-03-20T08:03:58.072 INFO:tasks.rados.rados.0.smithi118.stdout:768: writing smithi11881747-41 from 2572288 to 3162112 tid 2 2024-03-20T08:03:58.075 INFO:tasks.rados.rados.0.smithi118.stdout:768: writing smithi11881747-41 from 3162112 to 3645440 tid 3 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:768: writing smithi11881747-41 from 3645440 to 3776512 tid 4 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:758: done (9 left) 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:759: done (8 left) 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:760: done (7 left) 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:761: done (6 left) 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:762: done (5 left) 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:769: read oid 11 snap 66 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:769: expect (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:03:58.077 INFO:tasks.rados.rados.0.smithi118.stdout:770: snap_remove snap 43 2024-03-20T08:03:58.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:57 smithi196 ceph-mon[31033]: osdmap e219: 8 total, 8 up, 8 in 2024-03-20T08:03:58.139 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:57.821+0000 7f6fa4c22200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T08:03:58.139 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:57.927+0000 7f6fa4c22200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T08:03:58.150 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:57 smithi118 ceph-mon[93121]: osdmap e219: 8 total, 8 up, 8 in 2024-03-20T08:03:58.151 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:57 smithi118 ceph-mon[38981]: osdmap e219: 8 total, 8 up, 8 in 2024-03-20T08:03:58.151 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:57.815+0000 7ff9f9017200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-20T08:03:58.152 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:57.920+0000 7ff9f9017200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-20T08:03:58.174 INFO:tasks.rados.rados.0.smithi118.stdout:770: done (6 left) 2024-03-20T08:03:58.174 INFO:tasks.rados.rados.0.smithi118.stdout:771: snap_remove snap 80 2024-03-20T08:03:58.403 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:58.152+0000 7f6fa4c22200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T08:03:58.403 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: [20/Mar/2024:08:03:58] ENGINE Bus STARTING 2024-03-20T08:03:58.403 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: CherryPy Checker: 2024-03-20T08:03:58.404 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: The Application mounted at '' has an empty config. 2024-03-20T08:03:58.404 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: 2024-03-20T08:03:58.404 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: [20/Mar/2024:08:03:58] ENGINE Serving on http://:::9283 2024-03-20T08:03:58.404 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x[78681]: [20/Mar/2024:08:03:58] ENGINE Bus STARTED 2024-03-20T08:03:58.405 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.151+0000 7ff9f9017200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-20T08:03:58.405 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:58] ENGINE Bus STARTING 2024-03-20T08:03:58.405 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff95abeb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.405 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff955be1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.405 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff955be1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff955be1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff955be1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff955be1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff95abeb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff95abeb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff95abeb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.324+0000 7ff95abeb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: CherryPy Checker: 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: The Application mounted at '' has an empty config. 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:58] ENGINE Serving on http://:::9283 2024-03-20T08:03:58.406 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:03:58] ENGINE Bus STARTED 2024-03-20T08:03:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: osdmap e220: 8 total, 8 up, 8 in 2024-03-20T08:03:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: Active manager daemon y restarted 2024-03-20T08:03:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: Activating manager daemon y 2024-03-20T08:03:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: osdmap e221: 8 total, 8 up, 8 in 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: Standby manager daemon x restarted 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: Standby manager daemon x started 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: mgrmap e35: y(active, starting, since 0.0095277s), standbys: x 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:03:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: Manager daemon y is now available 2024-03-20T08:03:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:59.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:59.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:59.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:59.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:59.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:59.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:03:58 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: osdmap e220: 8 total, 8 up, 8 in 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: Active manager daemon y restarted 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: Activating manager daemon y 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: osdmap e221: 8 total, 8 up, 8 in 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: Standby manager daemon x restarted 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: Standby manager daemon x started 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: mgrmap e35: y(active, starting, since 0.0095277s), standbys: x 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:03:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:03:59.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: Manager daemon y is now available 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: osdmap e220: 8 total, 8 up, 8 in 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: Active manager daemon y restarted 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: Activating manager daemon y 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: osdmap e221: 8 total, 8 up, 8 in 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: Standby manager daemon x restarted 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: Standby manager daemon x started 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: mgrmap e35: y(active, starting, since 0.0095277s), standbys: x 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:03:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.? 172.21.15.196:0/853097391' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:03:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: Manager daemon y is now available 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:59.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:03:58 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-20T08:03:59.178 INFO:tasks.rados.rados.0.smithi118.stdout:771: done (6 left) 2024-03-20T08:03:59.179 INFO:tasks.rados.rados.0.smithi118.stdout:772: copy_from oid 14 from oid 29 current snap is 84 2024-03-20T08:03:59.179 INFO:tasks.rados.rados.0.smithi118.stdout:773: setattr oid 24 current snap is 84 2024-03-20T08:03:59.179 INFO:tasks.rados.rados.0.smithi118.stdout:765: finishing write tid 2 to smithi11881747-28 2024-03-20T08:03:59.179 INFO:tasks.rados.rados.0.smithi118.stdout:765: finishing write tid 3 to smithi11881747-28 2024-03-20T08:03:59.179 INFO:tasks.rados.rados.0.smithi118.stdout:765: finishing write tid 4 to smithi11881747-28 2024-03-20T08:03:59.179 INFO:tasks.rados.rados.0.smithi118.stdout:763: expect (ObjNum 176 snap 52 seq_num 176) 2024-03-20T08:03:59.390 INFO:tasks.rados.rados.0.smithi118.stdout:765: finishing write tid 5 to smithi11881747-28 2024-03-20T08:03:59.390 INFO:tasks.rados.rados.0.smithi118.stdout:765: oid 28 updating version 0 to 544 2024-03-20T08:03:59.390 INFO:tasks.rados.rados.0.smithi118.stdout:765: oid 28 updating version 544 to 545 2024-03-20T08:03:59.390 INFO:tasks.rados.rados.0.smithi118.stdout:765: oid 28 updating version 545 to 546 2024-03-20T08:03:59.390 INFO:tasks.rados.rados.0.smithi118.stdout:765: oid 28 version 546 is already newer than 543 2024-03-20T08:03:59.391 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 546 (ObjNum 239 snap 84 seq_num 239) dirty exists 2024-03-20T08:03:59.391 INFO:tasks.rados.rados.0.smithi118.stdout:765: left oid 28 (ObjNum 239 snap 84 seq_num 239) 2024-03-20T08:03:59.391 INFO:tasks.rados.rados.0.smithi118.stdout:766: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:03:59.489 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:03:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:03:59.193+0000 7ff9af1ba700 -1 mgr.server handle_report got status from non-daemon mon.a 2024-03-20T08:03:59.510 INFO:tasks.rados.rados.0.smithi118.stdout:768: finishing write tid 1 to smithi11881747-41 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: finishing write tid 2 to smithi11881747-41 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: finishing write tid 3 to smithi11881747-41 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: finishing write tid 4 to smithi11881747-41 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: finishing write tid 5 to smithi11881747-41 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: finishing write tid 6 to smithi11881747-41 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: oid 41 updating version 0 to 548 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: oid 41 updating version 548 to 552 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: oid 41 version 552 is already newer than 549 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: oid 41 version 552 is already newer than 551 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: oid 41 version 552 is already newer than 550 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 552 (ObjNum 240 snap 84 seq_num 240) dirty exists 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:768: left oid 41 (ObjNum 240 snap 84 seq_num 240) 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:763: done (7 left) 2024-03-20T08:03:59.511 INFO:tasks.rados.rados.0.smithi118.stdout:765: done (6 left) 2024-03-20T08:03:59.512 INFO:tasks.rados.rados.0.smithi118.stdout:766: done (5 left) 2024-03-20T08:03:59.512 INFO:tasks.rados.rados.0.smithi118.stdout:767: done (4 left) 2024-03-20T08:03:59.512 INFO:tasks.rados.rados.0.smithi118.stdout:768: done (3 left) 2024-03-20T08:03:59.512 INFO:tasks.rados.rados.0.smithi118.stdout:774: setattr oid 35 current snap is 84 2024-03-20T08:03:59.512 INFO:tasks.rados.rados.0.smithi118.stdout:769: expect (ObjNum 102 snap 23 seq_num 102) 2024-03-20T08:03:59.594 INFO:tasks.rados.rados.0.smithi118.stdout:772: finishing copy_from racing read to smithi11881747-14 2024-03-20T08:03:59.594 INFO:tasks.rados.rados.0.smithi118.stdout:772: finishing copy_from to smithi11881747-14 2024-03-20T08:03:59.594 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 535 (ObjNum 235 snap 82 seq_num 235) dirty exists 2024-03-20T08:03:59.594 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 276 (ObjNum 216 snap 69 seq_num 216) dirty exists 2024-03-20T08:03:59.594 INFO:tasks.rados.rados.0.smithi118.stdout:769: done (3 left) 2024-03-20T08:03:59.594 INFO:tasks.rados.rados.0.smithi118.stdout:772: done (2 left) 2024-03-20T08:03:59.594 INFO:tasks.rados.rados.0.smithi118.stdout:773: done (1 left) 2024-03-20T08:03:59.594 INFO:tasks.rados.rados.0.smithi118.stdout:775: snap_create 2024-03-20T08:03:59.596 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 364 (ObjNum 237 snap 84 seq_num 237) dirty exists 2024-03-20T08:04:00.183 INFO:tasks.rados.rados.0.smithi118.stdout:774: done (1 left) 2024-03-20T08:04:00.183 INFO:tasks.rados.rados.0.smithi118.stdout:775: done (0 left) 2024-03-20T08:04:00.183 INFO:tasks.rados.rados.0.smithi118.stdout:776: rollback oid 27 current snap is 85 2024-03-20T08:04:00.183 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 29 2024-03-20T08:04:00.183 INFO:tasks.rados.rados.0.smithi118.stdout:777: snap_remove snap 60 2024-03-20T08:04:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:00 smithi118 ceph-mon[93121]: osdmap e222: 8 total, 8 up, 8 in 2024-03-20T08:04:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:00 smithi118 ceph-mon[93121]: mgrmap e36: y(active, since 1.01644s), standbys: x 2024-03-20T08:04:00.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:00 smithi118 ceph-mon[93121]: pgmap v4: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:00 smithi118 ceph-mon[38981]: osdmap e222: 8 total, 8 up, 8 in 2024-03-20T08:04:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:00 smithi118 ceph-mon[38981]: mgrmap e36: y(active, since 1.01644s), standbys: x 2024-03-20T08:04:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:00 smithi118 ceph-mon[38981]: pgmap v4: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:00.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:00 smithi196 ceph-mon[31033]: osdmap e222: 8 total, 8 up, 8 in 2024-03-20T08:04:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:00 smithi196 ceph-mon[31033]: mgrmap e36: y(active, since 1.01644s), standbys: x 2024-03-20T08:04:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:00 smithi196 ceph-mon[31033]: pgmap v4: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:01.188 INFO:tasks.rados.rados.0.smithi118.stdout:777: done (1 left) 2024-03-20T08:04:01.188 INFO:tasks.rados.rados.0.smithi118.stdout:778: copy_from oid 11 from oid 41 current snap is 85 2024-03-20T08:04:01.188 INFO:tasks.rados.rados.0.smithi118.stdout:779: copy_from oid 24 from oid 48 current snap is 85 2024-03-20T08:04:01.188 INFO:tasks.rados.rados.0.smithi118.stdout:780: delete oid 43 current snap is 85 2024-03-20T08:04:01.188 INFO:tasks.rados.rados.0.smithi118.stdout:776: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:04:01.188 INFO:tasks.rados.rados.0.smithi118.stdout:776: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:04:01.188 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 404 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-03-20T08:04:01.198 INFO:tasks.rados.rados.0.smithi118.stdout:776: done (3 left) 2024-03-20T08:04:01.199 INFO:tasks.rados.rados.0.smithi118.stdout:780: done (2 left) 2024-03-20T08:04:01.199 INFO:tasks.rados.rados.0.smithi118.stdout:781: snap_remove snap 76 2024-03-20T08:04:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:01 smithi196 ceph-mon[31033]: [20/Mar/2024:08:03:59] ENGINE Bus STARTING 2024-03-20T08:04:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:01 smithi196 ceph-mon[31033]: [20/Mar/2024:08:03:59] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T08:04:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:01 smithi196 ceph-mon[31033]: [20/Mar/2024:08:03:59] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T08:04:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:01 smithi196 ceph-mon[31033]: [20/Mar/2024:08:03:59] ENGINE Bus STARTED 2024-03-20T08:04:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:01 smithi196 ceph-mon[31033]: pgmap v6: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:01 smithi196 ceph-mon[31033]: osdmap e223: 8 total, 8 up, 8 in 2024-03-20T08:04:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:01 smithi196 ceph-mon[31033]: mgrmap e37: y(active, since 3s), standbys: x 2024-03-20T08:04:01.457 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[38981]: [20/Mar/2024:08:03:59] ENGINE Bus STARTING 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[38981]: [20/Mar/2024:08:03:59] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[38981]: [20/Mar/2024:08:03:59] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[38981]: [20/Mar/2024:08:03:59] ENGINE Bus STARTED 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[38981]: pgmap v6: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[38981]: osdmap e223: 8 total, 8 up, 8 in 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[38981]: mgrmap e37: y(active, since 3s), standbys: x 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[93121]: [20/Mar/2024:08:03:59] ENGINE Bus STARTING 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[93121]: [20/Mar/2024:08:03:59] ENGINE Serving on http://172.21.15.118:8765 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[93121]: [20/Mar/2024:08:03:59] ENGINE Serving on https://172.21.15.118:7150 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[93121]: [20/Mar/2024:08:03:59] ENGINE Bus STARTED 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[93121]: pgmap v6: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:01.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[93121]: osdmap e223: 8 total, 8 up, 8 in 2024-03-20T08:04:01.459 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:01 smithi118 ceph-mon[93121]: mgrmap e37: y(active, since 3s), standbys: x 2024-03-20T08:04:02.196 INFO:tasks.rados.rados.0.smithi118.stdout:781: done (2 left) 2024-03-20T08:04:02.196 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 85 2024-03-20T08:04:02.196 INFO:tasks.rados.rados.0.smithi118.stdout:779: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:04:02.196 INFO:tasks.rados.rados.0.smithi118.stdout:779: finishing copy_from to smithi11881747-24 2024-03-20T08:04:02.197 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 60 (ObjNum 145 snap 41 seq_num 145) dirty exists 2024-03-20T08:04:02.197 INFO:tasks.rados.rados.0.smithi118.stdout:778: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:04:02.197 INFO:tasks.rados.rados.0.smithi118.stdout:778: finishing copy_from to smithi11881747-11 2024-03-20T08:04:02.197 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 552 (ObjNum 240 snap 84 seq_num 240) dirty exists 2024-03-20T08:04:02.197 INFO:tasks.rados.rados.0.smithi118.stdout:782: seq_num 242 ranges {3391488=458752,3850240=753664,4603904=450560} 2024-03-20T08:04:02.217 INFO:tasks.rados.rados.0.smithi118.stdout:782: writing smithi11881747-12 from 3391488 to 3850240 tid 1 2024-03-20T08:04:02.222 INFO:tasks.rados.rados.0.smithi118.stdout:782: writing smithi11881747-12 from 3850240 to 4603904 tid 2 2024-03-20T08:04:02.225 INFO:tasks.rados.rados.0.smithi118.stdout:782: writing smithi11881747-12 from 4603904 to 5054464 tid 3 2024-03-20T08:04:02.225 INFO:tasks.rados.rados.0.smithi118.stdout:778: done (2 left) 2024-03-20T08:04:02.225 INFO:tasks.rados.rados.0.smithi118.stdout:779: done (1 left) 2024-03-20T08:04:02.225 INFO:tasks.rados.rados.0.smithi118.stdout:783: delete oid 3 current snap is 85 2024-03-20T08:04:02.225 INFO:tasks.rados.rados.0.smithi118.stdout:782: finishing write tid 1 to smithi11881747-12 2024-03-20T08:04:02.226 INFO:tasks.rados.rados.0.smithi118.stdout:782: finishing write tid 2 to smithi11881747-12 2024-03-20T08:04:02.228 INFO:tasks.rados.rados.0.smithi118.stdout:783: done (1 left) 2024-03-20T08:04:02.228 INFO:tasks.rados.rados.0.smithi118.stdout:784: snap_remove snap 34 2024-03-20T08:04:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:02 smithi118 ceph-mon[93121]: osdmap e224: 8 total, 8 up, 8 in 2024-03-20T08:04:02.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:02 smithi118 ceph-mon[38981]: osdmap e224: 8 total, 8 up, 8 in 2024-03-20T08:04:02.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:02 smithi196 ceph-mon[31033]: osdmap e224: 8 total, 8 up, 8 in 2024-03-20T08:04:03.003 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:04:03.201 INFO:tasks.rados.rados.0.smithi118.stdout:784: done (1 left) 2024-03-20T08:04:03.201 INFO:tasks.rados.rados.0.smithi118.stdout:785: read oid 9 snap -1 2024-03-20T08:04:03.201 INFO:tasks.rados.rados.0.smithi118.stdout:785: expect deleted 2024-03-20T08:04:03.201 INFO:tasks.rados.rados.0.smithi118.stdout:786: snap_remove snap 49 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (3m) 0s ago 4m 28.5M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (4m) 0s ago 4m 89.5M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (5m) 0s ago 5m 14.1M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (5m) 0s ago 5m 18.8M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (29s) 0s ago 9m 447M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (59s) 0s ago 11m 474M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (14s) 0s ago 11m 30.9M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (9m) 0s ago 9m 50.7M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 fd26544c725b 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (9m) 0s ago 9m 55.0M 2048M 18.2.2-904-g78dbc53a 3b5d1eaa4228 8193e23658db 2024-03-20T08:04:03.355 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (4m) 0s ago 4m 14.4M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (3m) 0s ago 3m 15.5M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (8m) 0s ago 8m 135M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 481bba84ab31 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (8m) 0s ago 8m 134M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (8m) 0s ago 8m 114M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (7m) 0s ago 7m 116M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (7m) 0s ago 7m 157M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (6m) 0s ago 6m 114M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (6m) 0s ago 6m 93.1M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (6m) 0s ago 6m 98.2M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:04:03.356 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (34s) 0s ago 5m 44.8M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[93121]: pgmap v8: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[93121]: osdmap e225: 8 total, 8 up, 8 in 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[93121]: mgrmap e38: y(active, since 4s), standbys: x 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[38981]: pgmap v8: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[38981]: osdmap e225: 8 total, 8 up, 8 in 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[38981]: mgrmap e38: y(active, since 4s), standbys: x 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.357 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.358 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.358 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:03 smithi196 ceph-mon[31033]: pgmap v8: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:03.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:03 smithi196 ceph-mon[31033]: osdmap e225: 8 total, 8 up, 8 in 2024-03-20T08:04:03.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:03 smithi196 ceph-mon[31033]: mgrmap e38: y(active, since 4s), standbys: x 2024-03-20T08:04:03.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:03 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:03 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:03 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:03 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:03.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:04:03] "GET /metrics HTTP/1.1" 200 33354 "" "Prometheus/2.43.0" 2024-03-20T08:04:03.772 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:04:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:04:03.772 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2, 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 1 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:04:03.773 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:04:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:04:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 12, 2024-03-20T08:04:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:04:03.774 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:04:03.774 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:04:03.888 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:03.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:04.205 INFO:tasks.rados.rados.0.smithi118.stdout:786: done (2 left) 2024-03-20T08:04:04.205 INFO:tasks.rados.rados.0.smithi118.stdout:787: read oid 47 snap -1 2024-03-20T08:04:04.205 INFO:tasks.rados.rados.0.smithi118.stdout:787: expect deleted 2024-03-20T08:04:04.205 INFO:tasks.rados.rados.0.smithi118.stdout:788: rollback oid 33 current snap is 85 2024-03-20T08:04:04.206 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 73 2024-03-20T08:04:04.206 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 85 2024-03-20T08:04:04.206 INFO:tasks.rados.rados.0.smithi118.stdout:789: seq_num 244 ranges {1581056=491520,2072576=573440,2646016=458752,3104768=172032} 2024-03-20T08:04:04.217 INFO:tasks.rados.rados.0.smithi118.stdout:789: writing smithi11881747-35 from 1581056 to 2072576 tid 1 2024-03-20T08:04:04.221 INFO:tasks.rados.rados.0.smithi118.stdout:789: writing smithi11881747-35 from 2072576 to 2646016 tid 2 2024-03-20T08:04:04.224 INFO:tasks.rados.rados.0.smithi118.stdout:789: writing smithi11881747-35 from 2646016 to 3104768 tid 3 2024-03-20T08:04:04.226 INFO:tasks.rados.rados.0.smithi118.stdout:789: writing smithi11881747-35 from 3104768 to 3276800 tid 4 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:790: read oid 25 snap -1 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:790: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:791: delete oid 30 current snap is 85 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:782: finishing write tid 3 to smithi11881747-12 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:782: finishing write tid 4 to smithi11881747-12 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:782: finishing write tid 5 to smithi11881747-12 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:782: oid 12 updating version 0 to 134 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:782: oid 12 version 134 is already newer than 132 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:782: oid 12 version 134 is already newer than 133 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:782: oid 12 updating version 134 to 135 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 135 (ObjNum 242 snap 85 seq_num 242) dirty exists 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:782: left oid 12 (ObjNum 242 snap 85 seq_num 242) 2024-03-20T08:04:04.227 INFO:tasks.rados.rados.0.smithi118.stdout:788: finishing rollback tid 0 to smithi11881747-33 2024-03-20T08:04:04.228 INFO:tasks.rados.rados.0.smithi118.stdout:788: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:04:04.228 INFO:tasks.rados.rados.0.smithi118.stdout:788: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:04:04.228 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 206 (ObjNum 129 snap 0 seq_num 322007440) dirty exists 2024-03-20T08:04:04.228 INFO:tasks.rados.rados.0.smithi118.stdout:790: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:04:04.231 INFO:tasks.rados.rados.0.smithi118.stdout:789: finishing write tid 1 to smithi11881747-35 2024-03-20T08:04:04.231 INFO:tasks.rados.rados.0.smithi118.stdout:789: finishing write tid 2 to smithi11881747-35 2024-03-20T08:04:04.234 INFO:tasks.rados.rados.0.smithi118.stdout:789: finishing write tid 3 to smithi11881747-35 2024-03-20T08:04:04.234 INFO:tasks.rados.rados.0.smithi118.stdout:782: done (6 left) 2024-03-20T08:04:04.234 INFO:tasks.rados.rados.0.smithi118.stdout:785: done (5 left) 2024-03-20T08:04:04.235 INFO:tasks.rados.rados.0.smithi118.stdout:787: done (4 left) 2024-03-20T08:04:04.235 INFO:tasks.rados.rados.0.smithi118.stdout:788: done (3 left) 2024-03-20T08:04:04.235 INFO:tasks.rados.rados.0.smithi118.stdout:790: done (2 left) 2024-03-20T08:04:04.235 INFO:tasks.rados.rados.0.smithi118.stdout:791: done (1 left) 2024-03-20T08:04:04.235 INFO:tasks.rados.rados.0.smithi118.stdout:792: setattr oid 48 current snap is 85 2024-03-20T08:04:04.235 INFO:tasks.rados.rados.0.smithi118.stdout:789: finishing write tid 4 to smithi11881747-35 2024-03-20T08:04:04.235 INFO:tasks.rados.rados.0.smithi118.stdout:789: finishing write tid 5 to smithi11881747-35 2024-03-20T08:04:04.235 INFO:tasks.rados.rados.0.smithi118.stdout:789: finishing write tid 6 to smithi11881747-35 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:789: oid 35 updating version 0 to 367 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:789: oid 35 version 367 is already newer than 366 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:789: oid 35 updating version 367 to 369 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:789: oid 35 version 369 is already newer than 365 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:789: oid 35 version 369 is already newer than 368 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 369 (ObjNum 244 snap 85 seq_num 244) dirty exists 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:789: left oid 35 (ObjNum 244 snap 85 seq_num 244) 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:789: done (1 left) 2024-03-20T08:04:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:793: rmattr oid 3 current snap is 85 2024-03-20T08:04:04.237 INFO:tasks.rados.rados.0.smithi118.stdout:793: done (1 left) 2024-03-20T08:04:04.237 INFO:tasks.rados.rados.0.smithi118.stdout:794: setattr oid 31 current snap is 85 2024-03-20T08:04:04.238 INFO:tasks.rados.rados.0.smithi118.stdout:795: delete oid 43 current snap is 85 2024-03-20T08:04:04.239 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 289 (ObjNum 145 snap 41 seq_num 145) dirty exists 2024-03-20T08:04:04.241 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 340 (ObjNum 207 snap 67 seq_num 207) dirty exists 2024-03-20T08:04:04.242 INFO:tasks.rados.rados.0.smithi118.stdout:792: done (2 left) 2024-03-20T08:04:04.242 INFO:tasks.rados.rados.0.smithi118.stdout:794: done (1 left) 2024-03-20T08:04:04.242 INFO:tasks.rados.rados.0.smithi118.stdout:795: done (0 left) 2024-03-20T08:04:04.242 INFO:tasks.rados.rados.0.smithi118.stdout:796: copy_from oid 8 from oid 47 current snap is 85 2024-03-20T08:04:04.242 INFO:tasks.rados.rados.0.smithi118.stdout:797: read oid 25 snap -1 2024-03-20T08:04:04.242 INFO:tasks.rados.rados.0.smithi118.stdout:797: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:04:04.243 INFO:tasks.rados.rados.0.smithi118.stdout:798: snap_create 2024-03-20T08:04:04.243 INFO:tasks.rados.rados.0.smithi118.stdout:797: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:04:04.244 INFO:tasks.rados.rados.0.smithi118.stdout:796: finishing copy_from to smithi11881747-8 2024-03-20T08:04:04.244 INFO:tasks.rados.rados.0.smithi118.stdout:796: got expected ENOENT (src dne) 2024-03-20T08:04:04.244 INFO:tasks.rados.rados.0.smithi118.stdout:796: finishing copy_from racing read to smithi11881747-8 2024-03-20T08:04:04.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:04 smithi196 ceph-mon[31033]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:04 smithi196 ceph-mon[31033]: osdmap e226: 8 total, 8 up, 8 in 2024-03-20T08:04:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:04 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:04 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:04 smithi196 ceph-mon[31033]: from='client.? 172.21.15.118:0/3601610409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:04 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:04 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[93121]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[93121]: osdmap e226: 8 total, 8 up, 8 in 2024-03-20T08:04:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3601610409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[38981]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[38981]: osdmap e226: 8 total, 8 up, 8 in 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[38981]: from='client.? 172.21.15.118:0/3601610409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:04 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:04.750 INFO:tasks.rados.rados.0.smithi118.stdout:796: done (2 left) 2024-03-20T08:04:04.750 INFO:tasks.rados.rados.0.smithi118.stdout:797: done (1 left) 2024-03-20T08:04:04.750 INFO:tasks.rados.rados.0.smithi118.stdout:798: done (0 left) 2024-03-20T08:04:04.750 INFO:tasks.rados.rados.0.smithi118.stdout:799: rmattr oid 35 current snap is 86 2024-03-20T08:04:04.750 INFO:tasks.rados.rados.0.smithi118.stdout:800: read oid 16 snap -1 2024-03-20T08:04:04.750 INFO:tasks.rados.rados.0.smithi118.stdout:800: expect (ObjNum 211 snap 67 seq_num 211) 2024-03-20T08:04:04.750 INFO:tasks.rados.rados.0.smithi118.stdout:801: setattr oid 12 current snap is 86 2024-03-20T08:04:04.751 INFO:tasks.rados.rados.0.smithi118.stdout:802: rollback oid 43 current snap is 86 2024-03-20T08:04:04.751 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 43 to 57 2024-03-20T08:04:04.751 INFO:tasks.rados.rados.0.smithi118.stdout:803: snap_remove snap 23 2024-03-20T08:04:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:05 smithi196 ceph-mon[31033]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:05 smithi196 ceph-mon[31033]: pgmap v11: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:05 smithi196 ceph-mon[31033]: osdmap e227: 8 total, 8 up, 8 in 2024-03-20T08:04:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:05 smithi196 ceph-mon[31033]: osdmap e228: 8 total, 8 up, 8 in 2024-03-20T08:04:05.574 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:05 smithi118 ceph-mon[93121]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:05.574 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:05 smithi118 ceph-mon[93121]: pgmap v11: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:05.574 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:05 smithi118 ceph-mon[93121]: osdmap e227: 8 total, 8 up, 8 in 2024-03-20T08:04:05.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:05 smithi118 ceph-mon[93121]: osdmap e228: 8 total, 8 up, 8 in 2024-03-20T08:04:05.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:05 smithi118 ceph-mon[38981]: from='client.25169 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:05.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:05 smithi118 ceph-mon[38981]: pgmap v11: 121 pgs: 1 active+clean+snaptrim, 120 active+clean; 190 MiB data, 841 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:05.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:05 smithi118 ceph-mon[38981]: osdmap e227: 8 total, 8 up, 8 in 2024-03-20T08:04:05.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:05 smithi118 ceph-mon[38981]: osdmap e228: 8 total, 8 up, 8 in 2024-03-20T08:04:05.751 INFO:tasks.rados.rados.0.smithi118.stdout:800: expect (ObjNum 211 snap 67 seq_num 211) 2024-03-20T08:04:05.792 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 143 (ObjNum 242 snap 85 seq_num 242) dirty exists 2024-03-20T08:04:05.792 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 370 (ObjNum 244 snap 85 seq_num 244) dirty exists 2024-03-20T08:04:05.792 INFO:tasks.rados.rados.0.smithi118.stdout:802: finishing rollback tid 1 to smithi11881747-43 2024-03-20T08:04:05.792 INFO:tasks.rados.rados.0.smithi118.stdout:802: finishing rollback tid 2 to smithi11881747-43 2024-03-20T08:04:05.792 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 343 (ObjNum 72 snap 10 seq_num 72) dirty exists 2024-03-20T08:04:05.792 INFO:tasks.rados.rados.0.smithi118.stdout:799: done (4 left) 2024-03-20T08:04:05.793 INFO:tasks.rados.rados.0.smithi118.stdout:800: done (3 left) 2024-03-20T08:04:05.793 INFO:tasks.rados.rados.0.smithi118.stdout:801: done (2 left) 2024-03-20T08:04:05.793 INFO:tasks.rados.rados.0.smithi118.stdout:802: done (1 left) 2024-03-20T08:04:05.793 INFO:tasks.rados.rados.0.smithi118.stdout:803: done (0 left) 2024-03-20T08:04:05.793 INFO:tasks.rados.rados.0.smithi118.stdout:804: snap_create 2024-03-20T08:04:06.754 INFO:tasks.rados.rados.0.smithi118.stdout:804: done (0 left) 2024-03-20T08:04:06.754 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 87 2024-03-20T08:04:06.754 INFO:tasks.rados.rados.0.smithi118.stdout:805: seq_num 247 ranges {0=540672,540672=622592,1163264=581632} 2024-03-20T08:04:06.757 INFO:tasks.rados.rados.0.smithi118.stdout:805: writing smithi11881747-1 from 0 to 540672 tid 1 2024-03-20T08:04:06.760 INFO:tasks.rados.rados.0.smithi118.stdout:805: writing smithi11881747-1 from 540672 to 1163264 tid 2 2024-03-20T08:04:06.764 INFO:tasks.rados.rados.0.smithi118.stdout:805: writing smithi11881747-1 from 1163264 to 1744896 tid 3 2024-03-20T08:04:06.764 INFO:tasks.rados.rados.0.smithi118.stdout:806: copy_from oid 48 from oid 18 current snap is 87 2024-03-20T08:04:06.764 INFO:tasks.rados.rados.0.smithi118.stdout:805: finishing write tid 1 to smithi11881747-1 2024-03-20T08:04:06.764 INFO:tasks.rados.rados.0.smithi118.stdout:807: read oid 33 snap -1 2024-03-20T08:04:06.764 INFO:tasks.rados.rados.0.smithi118.stdout:807: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:04:06.764 INFO:tasks.rados.rados.0.smithi118.stdout:808: copy_from oid 32 from oid 37 current snap is 87 2024-03-20T08:04:06.764 INFO:tasks.rados.rados.0.smithi118.stdout:809: delete oid 20 current snap is 87 2024-03-20T08:04:06.764 INFO:tasks.rados.rados.0.smithi118.stdout:805: finishing write tid 2 to smithi11881747-1 2024-03-20T08:04:06.767 INFO:tasks.rados.rados.0.smithi118.stdout:805: finishing write tid 3 to smithi11881747-1 2024-03-20T08:04:06.767 INFO:tasks.rados.rados.0.smithi118.stdout:805: finishing write tid 4 to smithi11881747-1 2024-03-20T08:04:06.767 INFO:tasks.rados.rados.0.smithi118.stdout:807: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:808: finishing copy_from to smithi11881747-32 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:808: got expected ENOENT (src dne) 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:808: finishing copy_from racing read to smithi11881747-32 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:805: finishing write tid 5 to smithi11881747-1 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:805: oid 1 updating version 0 to 352 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:805: oid 1 updating version 352 to 353 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:805: oid 1 updating version 353 to 354 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:805: oid 1 updating version 354 to 355 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 355 (ObjNum 247 snap 87 seq_num 247) dirty exists 2024-03-20T08:04:06.774 INFO:tasks.rados.rados.0.smithi118.stdout:805: left oid 1 (ObjNum 247 snap 87 seq_num 247) 2024-03-20T08:04:06.775 INFO:tasks.rados.rados.0.smithi118.stdout:805: done (4 left) 2024-03-20T08:04:06.775 INFO:tasks.rados.rados.0.smithi118.stdout:807: done (3 left) 2024-03-20T08:04:06.775 INFO:tasks.rados.rados.0.smithi118.stdout:808: done (2 left) 2024-03-20T08:04:06.775 INFO:tasks.rados.rados.0.smithi118.stdout:809: done (1 left) 2024-03-20T08:04:06.775 INFO:tasks.rados.rados.0.smithi118.stdout:810: snap_create 2024-03-20T08:04:06.781 INFO:tasks.rados.rados.0.smithi118.stdout:806: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:04:06.784 INFO:tasks.rados.rados.0.smithi118.stdout:806: finishing copy_from to smithi11881747-48 2024-03-20T08:04:06.784 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:04:07.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:06 smithi196 ceph-mon[31033]: osdmap e229: 8 total, 8 up, 8 in 2024-03-20T08:04:07.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:06 smithi196 ceph-mon[31033]: pgmap v15: 121 pgs: 121 active+clean; 195 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.9 MiB/s wr, 42 op/s 2024-03-20T08:04:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:06 smithi118 ceph-mon[93121]: osdmap e229: 8 total, 8 up, 8 in 2024-03-20T08:04:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:06 smithi118 ceph-mon[93121]: pgmap v15: 121 pgs: 121 active+clean; 195 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.9 MiB/s wr, 42 op/s 2024-03-20T08:04:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:06 smithi118 ceph-mon[38981]: osdmap e229: 8 total, 8 up, 8 in 2024-03-20T08:04:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:06 smithi118 ceph-mon[38981]: pgmap v15: 121 pgs: 121 active+clean; 195 MiB data, 852 MiB used, 714 GiB / 715 GiB avail; 52 KiB/s rd, 1.9 MiB/s wr, 42 op/s 2024-03-20T08:04:07.759 INFO:tasks.rados.rados.0.smithi118.stdout:806: done (1 left) 2024-03-20T08:04:07.759 INFO:tasks.rados.rados.0.smithi118.stdout:810: done (0 left) 2024-03-20T08:04:07.759 INFO:tasks.rados.rados.0.smithi118.stdout:811: read oid 28 snap -1 2024-03-20T08:04:07.759 INFO:tasks.rados.rados.0.smithi118.stdout:811: expect (ObjNum 239 snap 84 seq_num 239) 2024-03-20T08:04:07.759 INFO:tasks.rados.rados.0.smithi118.stdout:812: delete oid 10 current snap is 88 2024-03-20T08:04:07.762 INFO:tasks.rados.rados.0.smithi118.stdout:812: done (1 left) 2024-03-20T08:04:07.762 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 88 2024-03-20T08:04:07.762 INFO:tasks.rados.rados.0.smithi118.stdout:813: seq_num 250 ranges {425984=589824,1015808=679936,1695744=409600,2105344=114688} 2024-03-20T08:04:07.768 INFO:tasks.rados.rados.0.smithi118.stdout:813: writing smithi11881747-27 from 425984 to 1015808 tid 1 2024-03-20T08:04:07.772 INFO:tasks.rados.rados.0.smithi118.stdout:813: writing smithi11881747-27 from 1015808 to 1695744 tid 2 2024-03-20T08:04:07.775 INFO:tasks.rados.rados.0.smithi118.stdout:813: writing smithi11881747-27 from 1695744 to 2105344 tid 3 2024-03-20T08:04:07.776 INFO:tasks.rados.rados.0.smithi118.stdout:813: writing smithi11881747-27 from 2105344 to 2220032 tid 4 2024-03-20T08:04:07.776 INFO:tasks.rados.rados.0.smithi118.stdout:814: snap_create 2024-03-20T08:04:07.777 INFO:tasks.rados.rados.0.smithi118.stdout:811: expect (ObjNum 239 snap 84 seq_num 239) 2024-03-20T08:04:07.861 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:07 smithi196 ceph-mon[31033]: osdmap e230: 8 total, 8 up, 8 in 2024-03-20T08:04:07.861 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:07 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:07.861 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:07 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:07.861 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:07 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:07.861 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:07 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: finishing write tid 1 to smithi11881747-27 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: finishing write tid 2 to smithi11881747-27 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: finishing write tid 3 to smithi11881747-27 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: finishing write tid 4 to smithi11881747-27 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: finishing write tid 5 to smithi11881747-27 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: finishing write tid 6 to smithi11881747-27 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: oid 27 updating version 0 to 445 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: oid 27 updating version 445 to 447 2024-03-20T08:04:07.907 INFO:tasks.rados.rados.0.smithi118.stdout:813: oid 27 version 447 is already newer than 446 2024-03-20T08:04:07.908 INFO:tasks.rados.rados.0.smithi118.stdout:813: oid 27 updating version 447 to 448 2024-03-20T08:04:07.908 INFO:tasks.rados.rados.0.smithi118.stdout:813: oid 27 version 448 is already newer than 444 2024-03-20T08:04:07.908 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 448 (ObjNum 250 snap 88 seq_num 250) dirty exists 2024-03-20T08:04:07.908 INFO:tasks.rados.rados.0.smithi118.stdout:813: left oid 27 (ObjNum 250 snap 88 seq_num 250) 2024-03-20T08:04:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[93121]: osdmap e230: 8 total, 8 up, 8 in 2024-03-20T08:04:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:08.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[38981]: osdmap e230: 8 total, 8 up, 8 in 2024-03-20T08:04:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:07 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:08.764 INFO:tasks.rados.rados.0.smithi118.stdout:811: done (2 left) 2024-03-20T08:04:08.764 INFO:tasks.rados.rados.0.smithi118.stdout:813: done (1 left) 2024-03-20T08:04:08.764 INFO:tasks.rados.rados.0.smithi118.stdout:814: done (0 left) 2024-03-20T08:04:08.764 INFO:tasks.rados.rados.0.smithi118.stdout:815: read oid 10 snap -1 2024-03-20T08:04:08.764 INFO:tasks.rados.rados.0.smithi118.stdout:815: expect deleted 2024-03-20T08:04:08.764 INFO:tasks.rados.rados.0.smithi118.stdout:816: read oid 21 snap 57 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:816: expect deleted 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:817: read oid 26 snap -1 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:817: expect deleted 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:818: rmattr oid 49 current snap is 89 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:818: done (3 left) 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:819: rmattr oid 30 current snap is 89 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:819: done (3 left) 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 89 2024-03-20T08:04:08.765 INFO:tasks.rados.rados.0.smithi118.stdout:820: seq_num 251 ranges {2056192=606208,2662400=614400,3276800=589824} 2024-03-20T08:04:08.777 INFO:tasks.rados.rados.0.smithi118.stdout:820: writing smithi11881747-8 from 2056192 to 2662400 tid 1 2024-03-20T08:04:08.781 INFO:tasks.rados.rados.0.smithi118.stdout:820: writing smithi11881747-8 from 2662400 to 3276800 tid 2 2024-03-20T08:04:08.785 INFO:tasks.rados.rados.0.smithi118.stdout:820: writing smithi11881747-8 from 3276800 to 3866624 tid 3 2024-03-20T08:04:08.786 INFO:tasks.rados.rados.0.smithi118.stdout:821: rollback oid 19 current snap is 89 2024-03-20T08:04:08.786 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 19 to 73 2024-03-20T08:04:08.786 INFO:tasks.rados.rados.0.smithi118.stdout:820: finishing write tid 1 to smithi11881747-8 2024-03-20T08:04:08.786 INFO:tasks.rados.rados.0.smithi118.stdout:820: finishing write tid 2 to smithi11881747-8 2024-03-20T08:04:08.786 INFO:tasks.rados.rados.0.smithi118.stdout:815: done (4 left) 2024-03-20T08:04:08.786 INFO:tasks.rados.rados.0.smithi118.stdout:816: done (3 left) 2024-03-20T08:04:08.786 INFO:tasks.rados.rados.0.smithi118.stdout:817: done (2 left) 2024-03-20T08:04:08.786 INFO:tasks.rados.rados.0.smithi118.stdout:822: delete oid 34 current snap is 89 2024-03-20T08:04:08.788 INFO:tasks.rados.rados.0.smithi118.stdout:822: done (2 left) 2024-03-20T08:04:08.788 INFO:tasks.rados.rados.0.smithi118.stdout:823: rmattr oid 50 current snap is 89 2024-03-20T08:04:08.788 INFO:tasks.rados.rados.0.smithi118.stdout:823: done (2 left) 2024-03-20T08:04:08.788 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 89 2024-03-20T08:04:08.789 INFO:tasks.rados.rados.0.smithi118.stdout:824: seq_num 253 ranges {0=647168,647168=786432,1433600=417792} 2024-03-20T08:04:08.792 INFO:tasks.rados.rados.0.smithi118.stdout:824: writing smithi11881747-25 from 0 to 647168 tid 1 2024-03-20T08:04:08.797 INFO:tasks.rados.rados.0.smithi118.stdout:824: writing smithi11881747-25 from 647168 to 1433600 tid 2 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:824: writing smithi11881747-25 from 1433600 to 1851392 tid 3 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:825: rollback oid 32 current snap is 89 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 32 to 73 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:820: finishing write tid 3 to smithi11881747-8 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:826: read oid 46 snap -1 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:826: expect deleted 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 89 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:821: finishing rollback tid 0 to smithi11881747-19 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:820: finishing write tid 4 to smithi11881747-8 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:821: finishing rollback tid 1 to smithi11881747-19 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:821: finishing rollback tid 2 to smithi11881747-19 2024-03-20T08:04:08.801 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 158 (ObjNum 220 snap 71 seq_num 220) dirty exists 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:820: finishing write tid 5 to smithi11881747-8 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:820: oid 8 updating version 0 to 537 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:820: oid 8 updating version 537 to 539 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:820: oid 8 version 539 is already newer than 536 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:820: oid 8 version 539 is already newer than 538 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 539 (ObjNum 251 snap 89 seq_num 251) dirty exists 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:820: left oid 8 (ObjNum 251 snap 89 seq_num 251) 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:824: finishing write tid 1 to smithi11881747-25 2024-03-20T08:04:08.802 INFO:tasks.rados.rados.0.smithi118.stdout:827: seq_num 254 ranges {0=663552,663552=663552,1327104=540672} 2024-03-20T08:04:08.805 INFO:tasks.rados.rados.0.smithi118.stdout:827: writing smithi11881747-10 from 0 to 663552 tid 1 2024-03-20T08:04:08.808 INFO:tasks.rados.rados.0.smithi118.stdout:827: writing smithi11881747-10 from 663552 to 1327104 tid 2 2024-03-20T08:04:08.811 INFO:tasks.rados.rados.0.smithi118.stdout:827: writing smithi11881747-10 from 1327104 to 1867776 tid 3 2024-03-20T08:04:08.811 INFO:tasks.rados.rados.0.smithi118.stdout:820: done (5 left) 2024-03-20T08:04:08.811 INFO:tasks.rados.rados.0.smithi118.stdout:821: done (4 left) 2024-03-20T08:04:08.811 INFO:tasks.rados.rados.0.smithi118.stdout:828: setattr oid 4 current snap is 89 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: finishing write tid 2 to smithi11881747-25 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:825: finishing rollback tid 1 to smithi11881747-32 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 205 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: finishing write tid 3 to smithi11881747-25 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: finishing write tid 4 to smithi11881747-25 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: finishing write tid 5 to smithi11881747-25 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: oid 25 updating version 0 to 553 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: oid 25 updating version 553 to 554 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: oid 25 updating version 554 to 555 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: oid 25 updating version 555 to 556 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 556 (ObjNum 253 snap 89 seq_num 253) dirty exists 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:824: left oid 25 (ObjNum 253 snap 89 seq_num 253) 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:827: finishing write tid 1 to smithi11881747-10 2024-03-20T08:04:08.812 INFO:tasks.rados.rados.0.smithi118.stdout:827: finishing write tid 2 to smithi11881747-10 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:827: finishing write tid 3 to smithi11881747-10 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:824: done (4 left) 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:825: done (3 left) 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:826: done (2 left) 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:829: copy_from oid 41 from oid 48 current snap is 89 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:830: read oid 12 snap 57 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:830: expect (ObjNum 179 snap 54 seq_num 179) 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:831: rmattr oid 23 current snap is 89 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:827: finishing write tid 4 to smithi11881747-10 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 89 2024-03-20T08:04:08.814 INFO:tasks.rados.rados.0.smithi118.stdout:832: seq_num 255 ranges {3391488=679936,4071424=548864,4620288=598016,5218304=57344} 2024-03-20T08:04:08.836 INFO:tasks.rados.rados.0.smithi118.stdout:832: writing smithi11881747-22 from 3391488 to 4071424 tid 1 2024-03-20T08:04:08.839 INFO:tasks.rados.rados.0.smithi118.stdout:832: writing smithi11881747-22 from 4071424 to 4620288 tid 2 2024-03-20T08:04:08.842 INFO:tasks.rados.rados.0.smithi118.stdout:832: writing smithi11881747-22 from 4620288 to 5218304 tid 3 2024-03-20T08:04:08.843 INFO:tasks.rados.rados.0.smithi118.stdout:832: writing smithi11881747-22 from 5218304 to 5275648 tid 4 2024-03-20T08:04:08.843 INFO:tasks.rados.rados.0.smithi118.stdout:833: rmattr oid 43 current snap is 89 2024-03-20T08:04:08.843 INFO:tasks.rados.rados.0.smithi118.stdout:833: done (6 left) 2024-03-20T08:04:08.843 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 89 2024-03-20T08:04:08.843 INFO:tasks.rados.rados.0.smithi118.stdout:827: finishing write tid 5 to smithi11881747-10 2024-03-20T08:04:08.843 INFO:tasks.rados.rados.0.smithi118.stdout:827: oid 10 updating version 0 to 359 2024-03-20T08:04:08.843 INFO:tasks.rados.rados.0.smithi118.stdout:827: oid 10 updating version 359 to 360 2024-03-20T08:04:08.844 INFO:tasks.rados.rados.0.smithi118.stdout:827: oid 10 version 360 is already newer than 358 2024-03-20T08:04:08.844 INFO:tasks.rados.rados.0.smithi118.stdout:827: oid 10 version 360 is already newer than 357 2024-03-20T08:04:08.844 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 360 (ObjNum 254 snap 89 seq_num 254) dirty exists 2024-03-20T08:04:08.844 INFO:tasks.rados.rados.0.smithi118.stdout:827: left oid 10 (ObjNum 254 snap 89 seq_num 254) 2024-03-20T08:04:08.844 INFO:tasks.rados.rados.0.smithi118.stdout:834: seq_num 256 ranges {3514368=696320,4210688=434176,4644864=770048} 2024-03-20T08:04:08.865 INFO:tasks.rados.rados.0.smithi118.stdout:834: writing smithi11881747-6 from 3514368 to 4210688 tid 1 2024-03-20T08:04:08.868 INFO:tasks.rados.rados.0.smithi118.stdout:834: writing smithi11881747-6 from 4210688 to 4644864 tid 2 2024-03-20T08:04:08.872 INFO:tasks.rados.rados.0.smithi118.stdout:834: writing smithi11881747-6 from 4644864 to 5414912 tid 3 2024-03-20T08:04:08.873 INFO:tasks.rados.rados.0.smithi118.stdout:827: done (6 left) 2024-03-20T08:04:08.873 INFO:tasks.rados.rados.0.smithi118.stdout:835: read oid 14 snap 69 2024-03-20T08:04:08.873 INFO:tasks.rados.rados.0.smithi118.stdout:835: expect (ObjNum 163 snap 45 seq_num 163) 2024-03-20T08:04:08.873 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 89 2024-03-20T08:04:08.873 INFO:tasks.rados.rados.0.smithi118.stdout:830: expect (ObjNum 179 snap 54 seq_num 179) 2024-03-20T08:04:08.905 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 614 (ObjNum 230 snap 79 seq_num 230) dirty exists 2024-03-20T08:04:08.906 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 283 (ObjNum 236 snap 82 seq_num 236) dirty exists 2024-03-20T08:04:08.906 INFO:tasks.rados.rados.0.smithi118.stdout:829: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:04:08.906 INFO:tasks.rados.rados.0.smithi118.stdout:829: finishing copy_from to smithi11881747-41 2024-03-20T08:04:08.906 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:04:08.906 INFO:tasks.rados.rados.0.smithi118.stdout:832: finishing write tid 1 to smithi11881747-22 2024-03-20T08:04:08.906 INFO:tasks.rados.rados.0.smithi118.stdout:836: seq_num 257 ranges {1187840=712704,1900544=720896,2621440=483328} 2024-03-20T08:04:08.915 INFO:tasks.rados.rados.0.smithi118.stdout:836: writing smithi11881747-16 from 1187840 to 1900544 tid 1 2024-03-20T08:04:08.920 INFO:tasks.rados.rados.0.smithi118.stdout:836: writing smithi11881747-16 from 1900544 to 2621440 tid 2 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:836: writing smithi11881747-16 from 2621440 to 3104768 tid 3 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:828: done (7 left) 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:829: done (6 left) 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:830: done (5 left) 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:831: done (4 left) 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:837: read oid 30 snap 74 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:837: expect (ObjNum 167 snap 45 seq_num 167) 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 89 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:832: finishing write tid 2 to smithi11881747-22 2024-03-20T08:04:08.924 INFO:tasks.rados.rados.0.smithi118.stdout:838: seq_num 258 ranges {0=729088,729088=606208,1335296=598016} 2024-03-20T08:04:08.927 INFO:tasks.rados.rados.0.smithi118.stdout:838: writing smithi11881747-2 from 0 to 729088 tid 1 2024-03-20T08:04:08.931 INFO:tasks.rados.rados.0.smithi118.stdout:838: writing smithi11881747-2 from 729088 to 1335296 tid 2 2024-03-20T08:04:08.935 INFO:tasks.rados.rados.0.smithi118.stdout:838: writing smithi11881747-2 from 1335296 to 1933312 tid 3 2024-03-20T08:04:08.935 INFO:tasks.rados.rados.0.smithi118.stdout:839: delete oid 35 current snap is 89 2024-03-20T08:04:08.935 INFO:tasks.rados.rados.0.smithi118.stdout:832: finishing write tid 3 to smithi11881747-22 2024-03-20T08:04:08.935 INFO:tasks.rados.rados.0.smithi118.stdout:832: finishing write tid 4 to smithi11881747-22 2024-03-20T08:04:08.935 INFO:tasks.rados.rados.0.smithi118.stdout:832: finishing write tid 5 to smithi11881747-22 2024-03-20T08:04:08.935 INFO:tasks.rados.rados.0.smithi118.stdout:832: finishing write tid 6 to smithi11881747-22 2024-03-20T08:04:08.935 INFO:tasks.rados.rados.0.smithi118.stdout:832: oid 22 updating version 0 to 303 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:832: oid 22 version 303 is already newer than 301 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:832: oid 22 updating version 303 to 305 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:832: oid 22 version 305 is already newer than 302 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:832: oid 22 version 305 is already newer than 304 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 305 (ObjNum 255 snap 89 seq_num 255) dirty exists 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:832: left oid 22 (ObjNum 255 snap 89 seq_num 255) 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:834: finishing write tid 1 to smithi11881747-6 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:834: finishing write tid 2 to smithi11881747-6 2024-03-20T08:04:08.936 INFO:tasks.rados.rados.0.smithi118.stdout:835: expect (ObjNum 163 snap 45 seq_num 163) 2024-03-20T08:04:08.945 INFO:tasks.rados.rados.0.smithi118.stdout:834: finishing write tid 3 to smithi11881747-6 2024-03-20T08:04:08.945 INFO:tasks.rados.rados.0.smithi118.stdout:834: finishing write tid 4 to smithi11881747-6 2024-03-20T08:04:08.945 INFO:tasks.rados.rados.0.smithi118.stdout:834: finishing write tid 5 to smithi11881747-6 2024-03-20T08:04:08.945 INFO:tasks.rados.rados.0.smithi118.stdout:834: oid 6 updating version 0 to 374 2024-03-20T08:04:08.945 INFO:tasks.rados.rados.0.smithi118.stdout:834: oid 6 version 374 is already newer than 373 2024-03-20T08:04:08.945 INFO:tasks.rados.rados.0.smithi118.stdout:834: oid 6 version 374 is already newer than 371 2024-03-20T08:04:08.945 INFO:tasks.rados.rados.0.smithi118.stdout:834: oid 6 version 374 is already newer than 372 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 374 (ObjNum 256 snap 89 seq_num 256) dirty exists 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:834: left oid 6 (ObjNum 256 snap 89 seq_num 256) 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:836: finishing write tid 1 to smithi11881747-16 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:836: finishing write tid 2 to smithi11881747-16 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:832: done (6 left) 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:834: done (5 left) 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:835: done (4 left) 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:839: done (3 left) 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:840: rollback oid 19 current snap is 89 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 19 to 69 2024-03-20T08:04:08.946 INFO:tasks.rados.rados.0.smithi118.stdout:836: finishing write tid 3 to smithi11881747-16 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:836: finishing write tid 4 to smithi11881747-16 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:836: finishing write tid 5 to smithi11881747-16 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:836: oid 16 updating version 0 to 620 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:836: oid 16 updating version 620 to 621 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:836: oid 16 version 621 is already newer than 619 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:836: oid 16 version 621 is already newer than 618 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 621 (ObjNum 257 snap 89 seq_num 257) dirty exists 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:836: left oid 16 (ObjNum 257 snap 89 seq_num 257) 2024-03-20T08:04:08.947 INFO:tasks.rados.rados.0.smithi118.stdout:837: expect (ObjNum 167 snap 45 seq_num 167) 2024-03-20T08:04:08.983 INFO:tasks.rados.rados.0.smithi118.stdout:838: finishing write tid 1 to smithi11881747-2 2024-03-20T08:04:08.983 INFO:tasks.rados.rados.0.smithi118.stdout:838: finishing write tid 2 to smithi11881747-2 2024-03-20T08:04:08.983 INFO:tasks.rados.rados.0.smithi118.stdout:838: finishing write tid 3 to smithi11881747-2 2024-03-20T08:04:08.983 INFO:tasks.rados.rados.0.smithi118.stdout:838: finishing write tid 4 to smithi11881747-2 2024-03-20T08:04:08.983 INFO:tasks.rados.rados.0.smithi118.stdout:838: finishing write tid 5 to smithi11881747-2 2024-03-20T08:04:08.983 INFO:tasks.rados.rados.0.smithi118.stdout:838: oid 2 updating version 0 to 376 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:838: oid 2 updating version 376 to 377 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:838: oid 2 updating version 377 to 378 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:838: oid 2 version 378 is already newer than 375 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 378 (ObjNum 258 snap 89 seq_num 258) dirty exists 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:838: left oid 2 (ObjNum 258 snap 89 seq_num 258) 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:840: finishing rollback tid 0 to smithi11881747-19 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:836: done (3 left) 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:837: done (2 left) 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:838: done (1 left) 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:841: snap_remove snap 83 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:840: finishing rollback tid 1 to smithi11881747-19 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:840: finishing rollback tid 2 to smithi11881747-19 2024-03-20T08:04:08.984 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 161 (ObjNum 102 snap 23 seq_num 102) dirty exists 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: osdmap e231: 8 total, 8 up, 8 in 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: pgmap v18: 121 pgs: 121 active+clean; 195 MiB data, 852 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:09.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:09.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:04:09.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:08 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-20T08:04:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:04:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:04:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: osdmap e231: 8 total, 8 up, 8 in 2024-03-20T08:04:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:04:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:04:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: pgmap v18: 121 pgs: 121 active+clean; 195 MiB data, 852 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.conf 2024-03-20T08:04:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: osdmap e231: 8 total, 8 up, 8 in 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: Updating smithi118:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: pgmap v18: 121 pgs: 121 active+clean; 195 MiB data, 852 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:04:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:08 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["c"]}]: dispatch 2024-03-20T08:04:09.751 INFO:tasks.rados.rados.0.smithi118.stdout:840: done (1 left) 2024-03-20T08:04:09.751 INFO:tasks.rados.rados.0.smithi118.stdout:841: done (0 left) 2024-03-20T08:04:09.751 INFO:tasks.rados.rados.0.smithi118.stdout:842: snap_create 2024-03-20T08:04:10.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:09 smithi196 ceph-mon[31033]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:04:10.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:09 smithi196 ceph-mon[31033]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:04:10.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:09 smithi196 ceph-mon[31033]: Upgrade: It appears safe to stop mon.c 2024-03-20T08:04:10.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:09 smithi196 ceph-mon[31033]: osdmap e232: 8 total, 8 up, 8 in 2024-03-20T08:04:10.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:09 smithi196 ceph-mon[31033]: osdmap e233: 8 total, 8 up, 8 in 2024-03-20T08:04:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[93121]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:04:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[93121]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:04:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[93121]: Upgrade: It appears safe to stop mon.c 2024-03-20T08:04:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[93121]: osdmap e232: 8 total, 8 up, 8 in 2024-03-20T08:04:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[93121]: osdmap e233: 8 total, 8 up, 8 in 2024-03-20T08:04:10.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[38981]: Updating smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:04:10.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[38981]: Updating smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-20T08:04:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[38981]: Upgrade: It appears safe to stop mon.c 2024-03-20T08:04:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[38981]: osdmap e232: 8 total, 8 up, 8 in 2024-03-20T08:04:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:09 smithi118 ceph-mon[38981]: osdmap e233: 8 total, 8 up, 8 in 2024-03-20T08:04:10.752 INFO:tasks.rados.rados.0.smithi118.stdout:842: done (0 left) 2024-03-20T08:04:10.753 INFO:tasks.rados.rados.0.smithi118.stdout:843: read oid 46 snap -1 2024-03-20T08:04:10.753 INFO:tasks.rados.rados.0.smithi118.stdout:843: expect deleted 2024-03-20T08:04:10.754 INFO:tasks.rados.rados.0.smithi118.stdout:844: read oid 12 snap -1 2024-03-20T08:04:10.754 INFO:tasks.rados.rados.0.smithi118.stdout:844: expect (ObjNum 242 snap 85 seq_num 242) 2024-03-20T08:04:10.754 INFO:tasks.rados.rados.0.smithi118.stdout:845: copy_from oid 19 from oid 41 current snap is 90 2024-03-20T08:04:10.754 INFO:tasks.rados.rados.0.smithi118.stdout:846: copy_from oid 49 from oid 18 current snap is 90 2024-03-20T08:04:10.754 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 90 2024-03-20T08:04:10.754 INFO:tasks.rados.rados.0.smithi118.stdout:847: seq_num 260 ranges {0=761856,761856=417792,1179648=786432} 2024-03-20T08:04:10.756 INFO:tasks.rados.rados.0.smithi118.stdout:847: writing smithi11881747-35 from 0 to 761856 tid 1 2024-03-20T08:04:10.759 INFO:tasks.rados.rados.0.smithi118.stdout:847: writing smithi11881747-35 from 761856 to 1179648 tid 2 2024-03-20T08:04:10.765 INFO:tasks.rados.rados.0.smithi118.stdout:847: writing smithi11881747-35 from 1179648 to 1966080 tid 3 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:848: rollback oid 15 current snap is 90 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 15 to 77 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:847: finishing write tid 1 to smithi11881747-35 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:847: finishing write tid 2 to smithi11881747-35 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:843: done (5 left) 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:849: rollback oid 27 current snap is 90 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 62 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:850: rollback oid 36 current snap is 90 2024-03-20T08:04:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 36 to 64 2024-03-20T08:04:10.767 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 90 2024-03-20T08:04:10.767 INFO:tasks.rados.rados.0.smithi118.stdout:851: seq_num 261 ranges {0=778240,778240=704512,1482752=499712} 2024-03-20T08:04:10.772 INFO:tasks.rados.rados.0.smithi118.stdout:851: writing smithi11881747-46 from 0 to 778240 tid 1 2024-03-20T08:04:10.777 INFO:tasks.rados.rados.0.smithi118.stdout:851: writing smithi11881747-46 from 778240 to 1482752 tid 2 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:851: writing smithi11881747-46 from 1482752 to 1982464 tid 3 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:852: rollback oid 32 current snap is 90 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 32 to 66 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:848: finishing rollback tid 1 to smithi11881747-15 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 77 (ObjNum 177 snap 0 seq_num 322580336) dirty dne 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:846: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:846: finishing copy_from to smithi11881747-49 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:04:10.782 INFO:tasks.rados.rados.0.smithi118.stdout:849: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:04:10.783 INFO:tasks.rados.rados.0.smithi118.stdout:846: done (8 left) 2024-03-20T08:04:10.783 INFO:tasks.rados.rados.0.smithi118.stdout:848: done (7 left) 2024-03-20T08:04:10.783 INFO:tasks.rados.rados.0.smithi118.stdout:853: read oid 39 snap -1 2024-03-20T08:04:10.783 INFO:tasks.rados.rados.0.smithi118.stdout:853: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:04:10.783 INFO:tasks.rados.rados.0.smithi118.stdout:854: delete oid 44 current snap is 90 2024-03-20T08:04:10.783 INFO:tasks.rados.rados.0.smithi118.stdout:844: expect (ObjNum 242 snap 85 seq_num 242) 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:849: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 454 (ObjNum 193 snap 0 seq_num 322475632) dirty dne 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:850: finishing rollback tid 0 to smithi11881747-36 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:850: finishing rollback tid 1 to smithi11881747-36 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:850: finishing rollback tid 2 to smithi11881747-36 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 172 (ObjNum 181 snap 54 seq_num 181) dirty exists 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:847: finishing write tid 3 to smithi11881747-35 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:847: finishing write tid 4 to smithi11881747-35 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:851: finishing write tid 1 to smithi11881747-46 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:851: finishing write tid 2 to smithi11881747-46 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:844: done (8 left) 2024-03-20T08:04:11.001 INFO:tasks.rados.rados.0.smithi118.stdout:849: done (7 left) 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:850: done (6 left) 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:854: done (5 left) 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:855: setattr oid 13 current snap is 90 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:852: finishing rollback tid 1 to smithi11881747-32 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:847: finishing write tid 5 to smithi11881747-35 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:847: oid 35 updating version 0 to 380 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:847: oid 35 updating version 380 to 383 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:847: oid 35 version 383 is already newer than 382 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:847: oid 35 version 383 is already newer than 381 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 383 (ObjNum 260 snap 90 seq_num 260) dirty exists 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:847: left oid 35 (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:852: finishing rollback tid 2 to smithi11881747-32 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 541 (ObjNum 192 snap 63 seq_num 192) dirty exists 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:851: finishing write tid 3 to smithi11881747-46 2024-03-20T08:04:11.002 INFO:tasks.rados.rados.0.smithi118.stdout:851: finishing write tid 4 to smithi11881747-46 2024-03-20T08:04:11.003 INFO:tasks.rados.rados.0.smithi118.stdout:853: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:851: finishing write tid 5 to smithi11881747-46 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:851: oid 46 updating version 0 to 385 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:851: oid 46 updating version 385 to 387 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:851: oid 46 version 387 is already newer than 384 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:851: oid 46 version 387 is already newer than 386 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 387 (ObjNum 261 snap 90 seq_num 261) dirty exists 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:851: left oid 46 (ObjNum 261 snap 90 seq_num 261) 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:845: finishing copy_from racing read to smithi11881747-19 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:845: finishing copy_from to smithi11881747-19 2024-03-20T08:04:11.091 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:04:11.092 INFO:tasks.rados.rados.0.smithi118.stdout:845: done (5 left) 2024-03-20T08:04:11.092 INFO:tasks.rados.rados.0.smithi118.stdout:847: done (4 left) 2024-03-20T08:04:11.092 INFO:tasks.rados.rados.0.smithi118.stdout:851: done (3 left) 2024-03-20T08:04:11.092 INFO:tasks.rados.rados.0.smithi118.stdout:852: done (2 left) 2024-03-20T08:04:11.092 INFO:tasks.rados.rados.0.smithi118.stdout:853: done (1 left) 2024-03-20T08:04:11.092 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 90 2024-03-20T08:04:11.092 INFO:tasks.rados.rados.0.smithi118.stdout:856: seq_num 263 ranges {0=409600,409600=475136,884736=557056,1441792=573440} 2024-03-20T08:04:11.092 INFO:tasks.rados.rados.0.smithi118.stdout:856: writing smithi11881747-20 from 0 to 409600 tid 1 2024-03-20T08:04:11.095 INFO:tasks.rados.rados.0.smithi118.stdout:856: writing smithi11881747-20 from 409600 to 884736 tid 2 2024-03-20T08:04:11.098 INFO:tasks.rados.rados.0.smithi118.stdout:856: writing smithi11881747-20 from 884736 to 1441792 tid 3 2024-03-20T08:04:11.100 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[93121]: pgmap v21: 121 pgs: 121 active+clean; 217 MiB data, 879 MiB used, 714 GiB / 715 GiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-03-20T08:04:11.100 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:11.100 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:11.100 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:11.101 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:11.101 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[93121]: osdmap e234: 8 total, 8 up, 8 in 2024-03-20T08:04:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[38981]: pgmap v21: 121 pgs: 121 active+clean; 217 MiB data, 879 MiB used, 714 GiB / 715 GiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-03-20T08:04:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[38981]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[38981]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:11.101 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:10 smithi118 ceph-mon[38981]: osdmap e234: 8 total, 8 up, 8 in 2024-03-20T08:04:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:856: writing smithi11881747-20 from 1441792 to 2015232 tid 4 2024-03-20T08:04:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:857: read oid 38 snap -1 2024-03-20T08:04:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:857: expect (ObjNum 128 snap 34 seq_num 128) 2024-03-20T08:04:11.103 INFO:tasks.rados.rados.0.smithi118.stdout:858: copy_from oid 41 from oid 34 current snap is 90 2024-03-20T08:04:11.103 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 557 (ObjNum 126 snap 34 seq_num 126) dirty exists 2024-03-20T08:04:11.103 INFO:tasks.rados.rados.0.smithi118.stdout:856: finishing write tid 1 to smithi11881747-20 2024-03-20T08:04:11.103 INFO:tasks.rados.rados.0.smithi118.stdout:856: finishing write tid 2 to smithi11881747-20 2024-03-20T08:04:11.103 INFO:tasks.rados.rados.0.smithi118.stdout:856: finishing write tid 3 to smithi11881747-20 2024-03-20T08:04:11.103 INFO:tasks.rados.rados.0.smithi118.stdout:855: done (3 left) 2024-03-20T08:04:11.103 INFO:tasks.rados.rados.0.smithi118.stdout:859: snap_remove snap 57 2024-03-20T08:04:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:10 smithi196 ceph-mon[31033]: pgmap v21: 121 pgs: 121 active+clean; 217 MiB data, 879 MiB used, 714 GiB / 715 GiB avail; 4.7 MiB/s rd, 6.0 MiB/s wr, 50 op/s 2024-03-20T08:04:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:10 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:10 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:10 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:10 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:10 smithi196 ceph-mon[31033]: osdmap e234: 8 total, 8 up, 8 in 2024-03-20T08:04:11.784 INFO:tasks.rados.rados.0.smithi118.stdout:859: done (3 left) 2024-03-20T08:04:11.784 INFO:tasks.rados.rados.0.smithi118.stdout:860: setattr oid 27 current snap is 90 2024-03-20T08:04:11.784 INFO:tasks.rados.rados.0.smithi118.stdout:858: finishing copy_from to smithi11881747-41 2024-03-20T08:04:11.784 INFO:tasks.rados.rados.0.smithi118.stdout:858: got expected ENOENT (src dne) 2024-03-20T08:04:11.784 INFO:tasks.rados.rados.0.smithi118.stdout:858: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: finishing write tid 4 to smithi11881747-20 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: finishing write tid 5 to smithi11881747-20 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: finishing write tid 6 to smithi11881747-20 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: oid 20 updating version 0 to 455 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: oid 20 updating version 455 to 458 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: oid 20 updating version 458 to 459 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: oid 20 version 459 is already newer than 457 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: oid 20 version 459 is already newer than 456 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 459 (ObjNum 263 snap 90 seq_num 263) dirty exists 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:856: left oid 20 (ObjNum 263 snap 90 seq_num 263) 2024-03-20T08:04:11.785 INFO:tasks.rados.rados.0.smithi118.stdout:857: expect (ObjNum 128 snap 34 seq_num 128) 2024-03-20T08:04:11.873 INFO:tasks.rados.rados.0.smithi118.stdout:856: done (3 left) 2024-03-20T08:04:11.874 INFO:tasks.rados.rados.0.smithi118.stdout:857: done (2 left) 2024-03-20T08:04:11.874 INFO:tasks.rados.rados.0.smithi118.stdout:858: done (1 left) 2024-03-20T08:04:11.874 INFO:tasks.rados.rados.0.smithi118.stdout:861: snap_create 2024-03-20T08:04:11.875 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 466 (ObjNum 193 snap 0 seq_num 322657280) dirty exists 2024-03-20T08:04:12.030 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:11 smithi118 ceph-mon[93121]: Upgrade: Updating mon.c 2024-03-20T08:04:12.030 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:11 smithi118 ceph-mon[93121]: Deploying daemon mon.c on smithi118 2024-03-20T08:04:12.030 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:11 smithi118 ceph-mon[38981]: Upgrade: Updating mon.c 2024-03-20T08:04:12.030 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:11 smithi118 ceph-mon[38981]: Deploying daemon mon.c on smithi118 2024-03-20T08:04:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:11 smithi196 ceph-mon[31033]: Upgrade: Updating mon.c 2024-03-20T08:04:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:11 smithi196 ceph-mon[31033]: Deploying daemon mon.c on smithi118 2024-03-20T08:04:12.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:12 smithi118 systemd[1]: Stopping Ceph mon.c for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:04:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:12.902 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c[38958]: 2024-03-20T08:04:12.562+0000 7f32af051700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:04:12.902 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c[38958]: 2024-03-20T08:04:12.562+0000 7f32af051700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-03-20T08:04:12.902 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:12 smithi118 podman[95491]: 2024-03-20 08:04:12.634630807 +0000 UTC m=+0.180527913 container died 8193e23658db475ccabcc876eca7a92d1fbfd39a01f8dca11b7e350657fab00e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-78dbc53, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-20T08:04:12.903 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:12] ENGINE Bus STOPPING 2024-03-20T08:04:12.903 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:04:12.903 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:12] ENGINE Bus STOPPED 2024-03-20T08:04:12.903 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:12] ENGINE Bus STARTING 2024-03-20T08:04:12.903 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:12] ENGINE Serving on http://:::9283 2024-03-20T08:04:12.903 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:12] ENGINE Bus STARTED 2024-03-20T08:04:13.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 podman[95491]: 2024-03-20 08:04:13.110584208 +0000 UTC m=+0.656481314 container cleanup 8193e23658db475ccabcc876eca7a92d1fbfd39a01f8dca11b7e350657fab00e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, RELEASE=reef-78dbc53, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:04:13.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 bash[95491]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c 2024-03-20T08:04:13.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:04:13] "GET /metrics HTTP/1.1" 200 33354 "" "Prometheus/2.43.0" 2024-03-20T08:04:13.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 podman[95521]: 2024-03-20 08:04:13.387105947 +0000 UTC m=+0.749651256 container remove 8193e23658db475ccabcc876eca7a92d1fbfd39a01f8dca11b7e350657fab00e (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=reef-78dbc53, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-20T08:04:13.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-20T08:04:13.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 systemd[1]: Stopped Ceph mon.c for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:04:13.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.c.service: Consumed 8.229s CPU time. 2024-03-20T08:04:13.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:13.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:14.039 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 systemd[1]: Starting Ceph mon.c for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:04:14.039 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 podman[95655]: 2024-03-20 08:04:13.836194994 +0000 UTC m=+0.017545762 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:04:14.039 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:13 smithi118 podman[95655]: 2024-03-20 08:04:13.937992769 +0000 UTC m=+0.119343519 container create 1ea7cd20eda3de054e099c193a826ae99ba228da82b4f6d3e86d0966cf8e4242 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 podman[95655]: 2024-03-20 08:04:14.1410033 +0000 UTC m=+0.322354060 container init 1ea7cd20eda3de054e099c193a826ae99ba228da82b4f6d3e86d0966cf8e4242 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 podman[95655]: 2024-03-20 08:04:14.146568152 +0000 UTC m=+0.327918899 container start 1ea7cd20eda3de054e099c193a826ae99ba228da82b4f6d3e86d0966cf8e4242 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: set uid:gid to 167:167 (ceph:ceph) 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev), process ceph-mon, pid 7 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: pidfile_write: ignore empty --pid-file 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: load: jerasure load: lrc 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: RocksDB version: 7.9.2 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Git sha 0 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Compile date 2024-03-18 19:18:37 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: DB SUMMARY 2024-03-20T08:04:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: DB Session ID: 2RHF7SDLYVYJK9PCRQ60 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: CURRENT file: CURRENT 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: IDENTITY file: IDENTITY 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1161 Bytes 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 1, files: 000024.sst 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000022.log size: 8955373 ; 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.error_if_exists: 0 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.create_if_missing: 0 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.paranoid_checks: 1 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.env: 0x55986cc6ab20 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.fs: PosixFileSystem 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.info_log: 0x55986ebf9920 2024-03-20T08:04:14.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_file_opening_threads: 16 2024-03-20T08:04:14.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.statistics: (nil) 2024-03-20T08:04:14.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.use_fsync: 0 2024-03-20T08:04:14.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_log_file_size: 0 2024-03-20T08:04:14.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-20T08:04:14.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-20T08:04:14.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.keep_log_file_num: 1000 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.recycle_log_file_num: 0 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.allow_fallocate: 1 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.allow_mmap_reads: 0 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.allow_mmap_writes: 0 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.use_direct_reads: 0 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.create_missing_column_families: 0 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.db_log_dir: 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.wal_dir: 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-20T08:04:14.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.advise_random_on_open: 1 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.db_write_buffer_size: 0 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.write_buffer_manager: 0x55986ebcc820 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.rate_limiter: (nil) 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.wal_recovery_mode: 2 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.enable_thread_tracking: 0 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.enable_pipelined_write: 0 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.unordered_write: 0 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-20T08:04:14.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.row_cache: None 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.wal_filter: None 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.allow_ingest_behind: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.two_write_queues: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.manual_wal_flush: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.wal_compression: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.atomic_flush: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-20T08:04:14.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.log_readahead_size: 0 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.best_efforts_recovery: 0 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.allow_data_in_errors: 0 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.db_host_id: __hostname__ 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_background_jobs: 2 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_background_compactions: -1 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_subcompactions: 1 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_total_wal_size: 0 2024-03-20T08:04:14.430 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_open_files: -1 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bytes_per_sync: 0 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_readahead_size: 0 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_background_flushes: -1 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Compression algorithms supported: 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: kZSTD supported: 0 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: kXpressCompression supported: 0 2024-03-20T08:04:14.431 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: kLZ4HCCompression supported: 1 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: kZlibCompression supported: 1 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: kSnappyCompression supported: 1 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: kLZ4Compression supported: 1 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: kBZip2Compression supported: 0 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000010 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.merge_operator: 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_filter: None 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_filter_factory: None 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.sst_partitioner_factory: None 2024-03-20T08:04:14.432 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55986ebf9a20) 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: cache_index_and_filter_blocks: 1 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: pin_top_level_index_and_filter: 1 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_type: 0 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_block_index_type: 0 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_shortening: 1 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: checksum: 4 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: no_block_cache: 0 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache: 0x55986ebff090 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_name: BinnedLRUCache 2024-03-20T08:04:14.433 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_options: 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: capacity : 536870912 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: num_shard_bits : 4 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: strict_capacity_limit : 0 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: high_pri_pool_ratio: 0.000 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_cache_compressed: (nil) 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: persistent_cache: (nil) 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_size: 4096 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_size_deviation: 10 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_restart_interval: 16 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: index_block_restart_interval: 1 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: metadata_block_size: 4096 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: partition_filters: 0 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: use_delta_encoding: 1 2024-03-20T08:04:14.434 INFO:journalctl@ceph.mon.c.smithi118.stdout: filter_policy: bloomfilter 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: whole_key_filtering: 1 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: verify_compression: 0 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: read_amp_bytes_per_bit: 0 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: format_version: 5 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: enable_index_compression: 1 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: block_align: 0 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: max_auto_readahead_size: 262144 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: prepopulate_block_cache: 0 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: initial_auto_readahead_size: 8192 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout: num_file_reads_for_auto_readahead: 2 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.write_buffer_size: 33554432 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_write_buffer_number: 2 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression: NoCompression 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression: Disabled 2024-03-20T08:04:14.435 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.prefix_extractor: nullptr 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.num_levels: 7 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-20T08:04:14.436 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.level: 32767 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.strategy: 0 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.enabled: false 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.target_file_size_base: 67108864 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-20T08:04:14.437 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.arena_block_size: 1048576 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-20T08:04:14.438 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.disable_auto_compactions: 0 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.table_properties_collectors: 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.inplace_update_support: 0 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-20T08:04:14.439 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.bloom_locality: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.max_successive_merges: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.paranoid_file_checks: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.force_consistency_checks: 1 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.report_bg_io_stats: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.ttl: 2592000 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.enable_blob_files: false 2024-03-20T08:04:14.440 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.min_blob_size: 0 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.blob_file_size: 268435456 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.blob_file_starting_level: 0 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 26, last_sequence is 13541, log_number is 22,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 22 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 22 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6410052f-dd47-4b1a-833f-2e4c11ba051f 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921854176158, "job": 1, "event": "recovery_started", "wal_files": [22]} 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #22 mode 2 2024-03-20T08:04:14.441 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921854196523, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 27, "file_size": 5492537, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 13546, "largest_seqno": 16704, "table_properties": {"data_size": 5478789, "index_size": 9271, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3461, "raw_key_size": 31312, "raw_average_key_size": 22, "raw_value_size": 5450048, "raw_average_value_size": 3946, "num_data_blocks": 406, "num_entries": 1381, "num_filter_entries": 1381, "num_deletions": 287, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710921854, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6410052f-dd47-4b1a-833f-2e4c11ba051f", "db_session_id": "2RHF7SDLYVYJK9PCRQ60", "orig_file_number": 27, "seqno_to_time_mapping": "N/A"}} 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921854196605, "job": 1, "event": "recovery_finished"} 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/version_set.cc:5047] Creating manifest 29 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000022.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55986ecf2000 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: DB pointer 0x55986ecdc000 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** DB Stats ** 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T08:04:14.442 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** Compaction Stats [default] ** 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: L0 1/0 5.24 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 622.6 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: L6 1/0 7.27 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: Sum 2/0 12.51 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 622.6 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 622.6 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** Compaction Stats [default] ** 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-20T08:04:14.443 INFO:journalctl@ceph.mon.c.smithi118.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 622.6 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: Cumulative compaction: 0.01 GB write, 227.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: Interval compaction: 0.01 GB write, 227.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: Block cache BinnedLRUCache@0x55986ebff090#7 capacity: 512.00 MB usage: 14.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-20T08:04:14.444 INFO:journalctl@ceph.mon.c.smithi118.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: starting mon.c rank 2 at public addrs [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] at bind addrs [v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: mon.c@-1(???) e3 preinit fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: mon.c@-1(???).mds e6 new map 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: mon.c@-1(???).mds e6 print_map 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: e6 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: legacy client fscid: 1 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: Filesystem 'cephfs' (1) 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: fs_name cephfs 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: epoch 6 2024-03-20T08:04:14.445 INFO:journalctl@ceph.mon.c.smithi118.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: created 2024-03-20T07:58:26.102802+0000 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: modified 2024-03-20T07:58:28.113441+0000 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: tableserver 0 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: root 0 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: session_timeout 60 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: session_autoclose 300 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: max_file_size 1099511627776 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: max_xattr_size 65536 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: required_client_features {} 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: last_failure 0 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: last_failure_osd_epoch 0 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: max_mds 1 2024-03-20T08:04:14.446 INFO:journalctl@ceph.mon.c.smithi118.stdout: in 0 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: up {0=24343} 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: failed 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: damaged 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: stopped 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: data_pools [3] 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: metadata_pool 2 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: inline_data disabled 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: balancer 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: bal_rank_mask -1 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: standby_count_wanted 0 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: qdb_cluster leader: 0 members: 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: [mds.b{0:24343} state up:active seq 3 addr [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] compat {c=[1],r=[1],i=[7ff]}] 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.447 INFO:journalctl@ceph.mon.c.smithi118.stdout: Standby daemons: 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout: 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout: [mds.a{-1:14430} state up:standby seq 1 addr [v2:172.21.15.118:6834/3215808514,v1:172.21.15.118:6835/3215808514] compat {c=[1],r=[1],i=[7ff]}] 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: mon.c@-1(???).osd e235 crush map has features 3314933069573799936, adjusting msgr requires 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: mon.c@-1(???).osd e235 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: mon.c@-1(???).osd e235 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: mon.c@-1(???).osd e235 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 ceph-mon[95692]: mon.c@-1(???).paxosservice(auth 1..19) refresh upgraded, format 0 -> 3 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 bash[95655]: 1ea7cd20eda3de054e099c193a826ae99ba228da82b4f6d3e86d0966cf8e4242 2024-03-20T08:04:14.448 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:14 smithi118 systemd[1]: Started Ceph mon.c for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:04:19.756 INFO:tasks.rados.rados.0.smithi118.stdout:860: done (1 left) 2024-03-20T08:04:19.756 INFO:tasks.rados.rados.0.smithi118.stdout:861: done (0 left) 2024-03-20T08:04:19.756 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 91 2024-03-20T08:04:19.756 INFO:tasks.rados.rados.0.smithi118.stdout:862: seq_num 264 ranges {0=425984,425984=761856,1187840=770048,1957888=73728} 2024-03-20T08:04:19.758 INFO:tasks.rados.rados.0.smithi118.stdout:862: writing smithi11881747-37 from 0 to 425984 tid 1 2024-03-20T08:04:19.763 INFO:tasks.rados.rados.0.smithi118.stdout:862: writing smithi11881747-37 from 425984 to 1187840 tid 2 2024-03-20T08:04:19.767 INFO:tasks.rados.rados.0.smithi118.stdout:862: writing smithi11881747-37 from 1187840 to 1957888 tid 3 2024-03-20T08:04:19.768 INFO:tasks.rados.rados.0.smithi118.stdout:862: writing smithi11881747-37 from 1957888 to 2031616 tid 4 2024-03-20T08:04:19.768 INFO:tasks.rados.rados.0.smithi118.stdout:863: snap_create 2024-03-20T08:04:19.768 INFO:tasks.rados.rados.0.smithi118.stdout:862: finishing write tid 1 to smithi11881747-37 2024-03-20T08:04:19.768 INFO:tasks.rados.rados.0.smithi118.stdout:862: finishing write tid 2 to smithi11881747-37 2024-03-20T08:04:19.771 INFO:tasks.rados.rados.0.smithi118.stdout:862: finishing write tid 3 to smithi11881747-37 2024-03-20T08:04:19.771 INFO:tasks.rados.rados.0.smithi118.stdout:862: finishing write tid 4 to smithi11881747-37 2024-03-20T08:04:19.771 INFO:tasks.rados.rados.0.smithi118.stdout:862: finishing write tid 5 to smithi11881747-37 2024-03-20T08:04:19.772 INFO:tasks.rados.rados.0.smithi118.stdout:862: finishing write tid 6 to smithi11881747-37 2024-03-20T08:04:19.772 INFO:tasks.rados.rados.0.smithi118.stdout:862: oid 37 updating version 0 to 243 2024-03-20T08:04:19.772 INFO:tasks.rados.rados.0.smithi118.stdout:862: oid 37 version 243 is already newer than 240 2024-03-20T08:04:19.773 INFO:tasks.rados.rados.0.smithi118.stdout:862: oid 37 version 243 is already newer than 241 2024-03-20T08:04:19.773 INFO:tasks.rados.rados.0.smithi118.stdout:862: oid 37 updating version 243 to 244 2024-03-20T08:04:19.773 INFO:tasks.rados.rados.0.smithi118.stdout:862: oid 37 version 244 is already newer than 242 2024-03-20T08:04:19.773 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 244 (ObjNum 264 snap 91 seq_num 264) dirty exists 2024-03-20T08:04:19.773 INFO:tasks.rados.rados.0.smithi118.stdout:862: left oid 37 (ObjNum 264 snap 91 seq_num 264) 2024-03-20T08:04:21.050 INFO:tasks.rados.rados.0.smithi118.stdout:862: done (1 left) 2024-03-20T08:04:21.050 INFO:tasks.rados.rados.0.smithi118.stdout:863: done (0 left) 2024-03-20T08:04:21.050 INFO:tasks.rados.rados.0.smithi118.stdout:864: read oid 11 snap 89 2024-03-20T08:04:21.050 INFO:tasks.rados.rados.0.smithi118.stdout:864: expect (ObjNum 240 snap 84 seq_num 240) 2024-03-20T08:04:21.050 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 92 2024-03-20T08:04:21.050 INFO:tasks.rados.rados.0.smithi118.stdout:865: seq_num 265 ranges {1933312=442368,2375680=647168,3022848=573440,3596288=385024} 2024-03-20T08:04:21.063 INFO:tasks.rados.rados.0.smithi118.stdout:865: writing smithi11881747-2 from 1933312 to 2375680 tid 1 2024-03-20T08:04:21.067 INFO:tasks.rados.rados.0.smithi118.stdout:865: writing smithi11881747-2 from 2375680 to 3022848 tid 2 2024-03-20T08:04:21.070 INFO:tasks.rados.rados.0.smithi118.stdout:865: writing smithi11881747-2 from 3022848 to 3596288 tid 3 2024-03-20T08:04:21.073 INFO:tasks.rados.rados.0.smithi118.stdout:865: writing smithi11881747-2 from 3596288 to 3981312 tid 4 2024-03-20T08:04:21.073 INFO:tasks.rados.rados.0.smithi118.stdout:866: setattr oid 36 current snap is 92 2024-03-20T08:04:21.073 INFO:tasks.rados.rados.0.smithi118.stdout:864: expect (ObjNum 240 snap 84 seq_num 240) 2024-03-20T08:04:21.243 INFO:tasks.rados.rados.0.smithi118.stdout:865: finishing write tid 1 to smithi11881747-2 2024-03-20T08:04:21.243 INFO:tasks.rados.rados.0.smithi118.stdout:865: finishing write tid 2 to smithi11881747-2 2024-03-20T08:04:21.243 INFO:tasks.rados.rados.0.smithi118.stdout:865: finishing write tid 3 to smithi11881747-2 2024-03-20T08:04:21.243 INFO:tasks.rados.rados.0.smithi118.stdout:865: finishing write tid 4 to smithi11881747-2 2024-03-20T08:04:21.243 INFO:tasks.rados.rados.0.smithi118.stdout:865: finishing write tid 5 to smithi11881747-2 2024-03-20T08:04:21.243 INFO:tasks.rados.rados.0.smithi118.stdout:865: finishing write tid 6 to smithi11881747-2 2024-03-20T08:04:21.243 INFO:tasks.rados.rados.0.smithi118.stdout:865: oid 2 updating version 0 to 391 2024-03-20T08:04:21.243 INFO:tasks.rados.rados.0.smithi118.stdout:865: oid 2 version 391 is already newer than 389 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:865: oid 2 version 391 is already newer than 388 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:865: oid 2 version 391 is already newer than 390 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:865: oid 2 updating version 391 to 392 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 392 (ObjNum 265 snap 92 seq_num 265) dirty exists 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:865: left oid 2 (ObjNum 265 snap 92 seq_num 265) 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:864: done (2 left) 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:865: done (1 left) 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 92 2024-03-20T08:04:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:867: seq_num 266 ranges {0=458752,458752=524288,983040=729088,1712128=352256} 2024-03-20T08:04:21.245 INFO:tasks.rados.rados.0.smithi118.stdout:867: writing smithi11881747-26 from 0 to 458752 tid 1 2024-03-20T08:04:21.248 INFO:tasks.rados.rados.0.smithi118.stdout:867: writing smithi11881747-26 from 458752 to 983040 tid 2 2024-03-20T08:04:21.252 INFO:tasks.rados.rados.0.smithi118.stdout:867: writing smithi11881747-26 from 983040 to 1712128 tid 3 2024-03-20T08:04:21.254 INFO:tasks.rados.rados.0.smithi118.stdout:867: writing smithi11881747-26 from 1712128 to 2064384 tid 4 2024-03-20T08:04:21.255 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 92 2024-03-20T08:04:21.255 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 178 (ObjNum 181 snap 54 seq_num 181) dirty exists 2024-03-20T08:04:21.255 INFO:tasks.rados.rados.0.smithi118.stdout:867: finishing write tid 1 to smithi11881747-26 2024-03-20T08:04:21.255 INFO:tasks.rados.rados.0.smithi118.stdout:867: finishing write tid 2 to smithi11881747-26 2024-03-20T08:04:21.255 INFO:tasks.rados.rados.0.smithi118.stdout:868: seq_num 267 ranges {1867776=475136,2342912=458752,2801664=540672,3342336=606208} 2024-03-20T08:04:21.267 INFO:tasks.rados.rados.0.smithi118.stdout:868: writing smithi11881747-10 from 1867776 to 2342912 tid 1 2024-03-20T08:04:21.270 INFO:tasks.rados.rados.0.smithi118.stdout:868: writing smithi11881747-10 from 2342912 to 2801664 tid 2 2024-03-20T08:04:21.273 INFO:tasks.rados.rados.0.smithi118.stdout:868: writing smithi11881747-10 from 2801664 to 3342336 tid 3 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:868: writing smithi11881747-10 from 3342336 to 3948544 tid 4 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:866: done (2 left) 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:869: delete oid 12 current snap is 92 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: finishing write tid 3 to smithi11881747-26 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: finishing write tid 4 to smithi11881747-26 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: finishing write tid 5 to smithi11881747-26 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: finishing write tid 6 to smithi11881747-26 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: oid 26 updating version 0 to 315 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: oid 26 version 315 is already newer than 313 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: oid 26 version 315 is already newer than 314 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: oid 26 version 315 is already newer than 311 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: oid 26 version 315 is already newer than 312 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 315 (ObjNum 266 snap 92 seq_num 266) dirty exists 2024-03-20T08:04:21.277 INFO:tasks.rados.rados.0.smithi118.stdout:867: left oid 26 (ObjNum 266 snap 92 seq_num 266) 2024-03-20T08:04:21.278 INFO:tasks.rados.rados.0.smithi118.stdout:868: finishing write tid 1 to smithi11881747-10 2024-03-20T08:04:21.278 INFO:tasks.rados.rados.0.smithi118.stdout:868: finishing write tid 2 to smithi11881747-10 2024-03-20T08:04:21.278 INFO:tasks.rados.rados.0.smithi118.stdout:868: finishing write tid 3 to smithi11881747-10 2024-03-20T08:04:21.279 INFO:tasks.rados.rados.0.smithi118.stdout:867: done (2 left) 2024-03-20T08:04:21.279 INFO:tasks.rados.rados.0.smithi118.stdout:869: done (1 left) 2024-03-20T08:04:21.279 INFO:tasks.rados.rados.0.smithi118.stdout:870: read oid 41 snap -1 2024-03-20T08:04:21.279 INFO:tasks.rados.rados.0.smithi118.stdout:870: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:04:21.279 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 92 2024-03-20T08:04:21.280 INFO:tasks.rados.rados.0.smithi118.stdout:871: seq_num 269 ranges {2654208=507904,3162112=630784,3792896=557056,4349952=417792} 2024-03-20T08:04:21.297 INFO:tasks.rados.rados.0.smithi118.stdout:871: writing smithi11881747-29 from 2654208 to 3162112 tid 1 2024-03-20T08:04:21.300 INFO:tasks.rados.rados.0.smithi118.stdout:871: writing smithi11881747-29 from 3162112 to 3792896 tid 2 2024-03-20T08:04:21.303 INFO:tasks.rados.rados.0.smithi118.stdout:871: writing smithi11881747-29 from 3792896 to 4349952 tid 3 2024-03-20T08:04:21.307 INFO:tasks.rados.rados.0.smithi118.stdout:871: writing smithi11881747-29 from 4349952 to 4767744 tid 4 2024-03-20T08:04:21.307 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 92 2024-03-20T08:04:21.307 INFO:tasks.rados.rados.0.smithi118.stdout:868: finishing write tid 4 to smithi11881747-10 2024-03-20T08:04:21.307 INFO:tasks.rados.rados.0.smithi118.stdout:868: finishing write tid 5 to smithi11881747-10 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:868: finishing write tid 6 to smithi11881747-10 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:868: oid 10 updating version 0 to 372 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:868: oid 10 updating version 372 to 376 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:868: oid 10 version 376 is already newer than 373 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:868: oid 10 version 376 is already newer than 375 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:868: oid 10 version 376 is already newer than 374 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 376 (ObjNum 267 snap 92 seq_num 267) dirty exists 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:868: left oid 10 (ObjNum 267 snap 92 seq_num 267) 2024-03-20T08:04:21.308 INFO:tasks.rados.rados.0.smithi118.stdout:870: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:04:21.360 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: mon.c calling monitor election 2024-03-20T08:04:21.360 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: mon.a calling monitor election 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: mon.b calling monitor election 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: osdmap e236: 8 total, 8 up, 8 in 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: mgrmap e38: y(active, since 22s), standbys: x 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: Cluster is now healthy 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[93121]: overall HEALTH_OK 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: mon.c calling monitor election 2024-03-20T08:04:21.361 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: mon.a calling monitor election 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: mon.b calling monitor election 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: osdmap e236: 8 total, 8 up, 8 in 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: mgrmap e38: y(active, since 22s), standbys: x 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: Cluster is now healthy 2024-03-20T08:04:21.362 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:21 smithi118 ceph-mon[95692]: overall HEALTH_OK 2024-03-20T08:04:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: mon.c calling monitor election 2024-03-20T08:04:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: mon.a calling monitor election 2024-03-20T08:04:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: mon.b calling monitor election 2024-03-20T08:04:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:04:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: monmap e3: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:04:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:04:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: osdmap e236: 8 total, 8 up, 8 in 2024-03-20T08:04:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: mgrmap e38: y(active, since 22s), standbys: x 2024-03-20T08:04:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-20T08:04:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: Cluster is now healthy 2024-03-20T08:04:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:21 smithi196 ceph-mon[31033]: overall HEALTH_OK 2024-03-20T08:04:21.420 INFO:tasks.rados.rados.0.smithi118.stdout:871: finishing write tid 1 to smithi11881747-29 2024-03-20T08:04:21.420 INFO:tasks.rados.rados.0.smithi118.stdout:871: finishing write tid 2 to smithi11881747-29 2024-03-20T08:04:21.420 INFO:tasks.rados.rados.0.smithi118.stdout:872: seq_num 270 ranges {2146304=524288,2670592=516096,3186688=761856,3948544=327680} 2024-03-20T08:04:21.433 INFO:tasks.rados.rados.0.smithi118.stdout:872: writing smithi11881747-39 from 2146304 to 2670592 tid 1 2024-03-20T08:04:21.437 INFO:tasks.rados.rados.0.smithi118.stdout:872: writing smithi11881747-39 from 2670592 to 3186688 tid 2 2024-03-20T08:04:21.441 INFO:tasks.rados.rados.0.smithi118.stdout:872: writing smithi11881747-39 from 3186688 to 3948544 tid 3 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:872: writing smithi11881747-39 from 3948544 to 4276224 tid 4 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:868: done (3 left) 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:870: done (2 left) 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:873: delete oid 15 current snap is 92 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:871: finishing write tid 3 to smithi11881747-29 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:871: finishing write tid 4 to smithi11881747-29 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:871: finishing write tid 5 to smithi11881747-29 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:871: finishing write tid 6 to smithi11881747-29 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:871: oid 29 updating version 0 to 643 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:871: oid 29 version 643 is already newer than 641 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:871: oid 29 updating version 643 to 644 2024-03-20T08:04:21.444 INFO:tasks.rados.rados.0.smithi118.stdout:871: oid 29 version 644 is already newer than 642 2024-03-20T08:04:21.445 INFO:tasks.rados.rados.0.smithi118.stdout:871: oid 29 updating version 644 to 645 2024-03-20T08:04:21.445 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 645 (ObjNum 269 snap 92 seq_num 269) dirty exists 2024-03-20T08:04:21.445 INFO:tasks.rados.rados.0.smithi118.stdout:871: left oid 29 (ObjNum 269 snap 92 seq_num 269) 2024-03-20T08:04:21.445 INFO:tasks.rados.rados.0.smithi118.stdout:872: finishing write tid 1 to smithi11881747-39 2024-03-20T08:04:21.445 INFO:tasks.rados.rados.0.smithi118.stdout:872: finishing write tid 2 to smithi11881747-39 2024-03-20T08:04:21.445 INFO:tasks.rados.rados.0.smithi118.stdout:871: done (2 left) 2024-03-20T08:04:21.445 INFO:tasks.rados.rados.0.smithi118.stdout:873: done (1 left) 2024-03-20T08:04:21.446 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 92 2024-03-20T08:04:21.446 INFO:tasks.rados.rados.0.smithi118.stdout:874: seq_num 272 ranges {2777088=565248,3342336=688128,4030464=729088,4759552=188416} 2024-03-20T08:04:21.463 INFO:tasks.rados.rados.0.smithi118.stdout:874: writing smithi11881747-38 from 2777088 to 3342336 tid 1 2024-03-20T08:04:21.466 INFO:tasks.rados.rados.0.smithi118.stdout:874: writing smithi11881747-38 from 3342336 to 4030464 tid 2 2024-03-20T08:04:21.471 INFO:tasks.rados.rados.0.smithi118.stdout:874: writing smithi11881747-38 from 4030464 to 4759552 tid 3 2024-03-20T08:04:21.472 INFO:tasks.rados.rados.0.smithi118.stdout:874: writing smithi11881747-38 from 4759552 to 4947968 tid 4 2024-03-20T08:04:21.472 INFO:tasks.rados.rados.0.smithi118.stdout:875: copy_from oid 34 from oid 41 current snap is 92 2024-03-20T08:04:21.472 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 92 2024-03-20T08:04:21.472 INFO:tasks.rados.rados.0.smithi118.stdout:876: seq_num 273 ranges {5414912=581632,5996544=565248,6561792=532480,7094272=507904} 2024-03-20T08:04:21.504 INFO:tasks.rados.rados.0.smithi118.stdout:876: writing smithi11881747-6 from 5414912 to 5996544 tid 1 2024-03-20T08:04:21.507 INFO:tasks.rados.rados.0.smithi118.stdout:876: writing smithi11881747-6 from 5996544 to 6561792 tid 2 2024-03-20T08:04:21.510 INFO:tasks.rados.rados.0.smithi118.stdout:876: writing smithi11881747-6 from 6561792 to 7094272 tid 3 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:876: writing smithi11881747-6 from 7094272 to 7602176 tid 4 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:877: delete oid 33 current snap is 92 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: finishing write tid 3 to smithi11881747-39 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: finishing write tid 4 to smithi11881747-39 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: finishing write tid 5 to smithi11881747-39 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: finishing write tid 6 to smithi11881747-39 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: oid 39 updating version 0 to 470 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: oid 39 version 470 is already newer than 468 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: oid 39 version 470 is already newer than 469 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: oid 39 updating version 470 to 472 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:872: oid 39 version 472 is already newer than 471 2024-03-20T08:04:21.514 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 472 (ObjNum 270 snap 92 seq_num 270) dirty exists 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:872: left oid 39 (ObjNum 270 snap 92 seq_num 270) 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: finishing write tid 1 to smithi11881747-38 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: finishing write tid 2 to smithi11881747-38 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: finishing write tid 3 to smithi11881747-38 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: finishing write tid 4 to smithi11881747-38 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: finishing write tid 5 to smithi11881747-38 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: finishing write tid 6 to smithi11881747-38 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: oid 38 updating version 0 to 318 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: oid 38 updating version 318 to 319 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: oid 38 updating version 319 to 321 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: oid 38 version 321 is already newer than 320 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: oid 38 version 321 is already newer than 317 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 321 (ObjNum 272 snap 92 seq_num 272) dirty exists 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:874: left oid 38 (ObjNum 272 snap 92 seq_num 272) 2024-03-20T08:04:21.515 INFO:tasks.rados.rados.0.smithi118.stdout:875: finishing copy_from racing read to smithi11881747-34 2024-03-20T08:04:21.516 INFO:tasks.rados.rados.0.smithi118.stdout:875: finishing copy_from to smithi11881747-34 2024-03-20T08:04:21.516 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:04:21.516 INFO:tasks.rados.rados.0.smithi118.stdout:876: finishing write tid 1 to smithi11881747-6 2024-03-20T08:04:21.516 INFO:tasks.rados.rados.0.smithi118.stdout:876: finishing write tid 2 to smithi11881747-6 2024-03-20T08:04:21.516 INFO:tasks.rados.rados.0.smithi118.stdout:876: finishing write tid 3 to smithi11881747-6 2024-03-20T08:04:21.516 INFO:tasks.rados.rados.0.smithi118.stdout:872: done (4 left) 2024-03-20T08:04:21.516 INFO:tasks.rados.rados.0.smithi118.stdout:874: done (3 left) 2024-03-20T08:04:21.517 INFO:tasks.rados.rados.0.smithi118.stdout:875: done (2 left) 2024-03-20T08:04:21.517 INFO:tasks.rados.rados.0.smithi118.stdout:877: done (1 left) 2024-03-20T08:04:21.517 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 92 2024-03-20T08:04:21.517 INFO:tasks.rados.rados.0.smithi118.stdout:878: seq_num 275 ranges {0=614400,614400=786432,1400832=499712,1900544=319488} 2024-03-20T08:04:21.518 INFO:tasks.rados.rados.0.smithi118.stdout:878: writing smithi11881747-47 from 0 to 614400 tid 1 2024-03-20T08:04:21.523 INFO:tasks.rados.rados.0.smithi118.stdout:878: writing smithi11881747-47 from 614400 to 1400832 tid 2 2024-03-20T08:04:21.526 INFO:tasks.rados.rados.0.smithi118.stdout:878: writing smithi11881747-47 from 1400832 to 1900544 tid 3 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:878: writing smithi11881747-47 from 1900544 to 2220032 tid 4 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:879: snap_create 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: finishing write tid 4 to smithi11881747-6 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: finishing write tid 5 to smithi11881747-6 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: finishing write tid 6 to smithi11881747-6 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: oid 6 updating version 0 to 397 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: oid 6 version 397 is already newer than 393 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: oid 6 version 397 is already newer than 395 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: oid 6 version 397 is already newer than 396 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: oid 6 version 397 is already newer than 394 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 397 (ObjNum 273 snap 92 seq_num 273) dirty exists 2024-03-20T08:04:21.529 INFO:tasks.rados.rados.0.smithi118.stdout:876: left oid 6 (ObjNum 273 snap 92 seq_num 273) 2024-03-20T08:04:21.530 INFO:tasks.rados.rados.0.smithi118.stdout:878: finishing write tid 1 to smithi11881747-47 2024-03-20T08:04:21.530 INFO:tasks.rados.rados.0.smithi118.stdout:878: finishing write tid 2 to smithi11881747-47 2024-03-20T08:04:21.530 INFO:tasks.rados.rados.0.smithi118.stdout:878: finishing write tid 3 to smithi11881747-47 2024-03-20T08:04:21.530 INFO:tasks.rados.rados.0.smithi118.stdout:878: finishing write tid 4 to smithi11881747-47 2024-03-20T08:04:21.530 INFO:tasks.rados.rados.0.smithi118.stdout:878: finishing write tid 5 to smithi11881747-47 2024-03-20T08:04:21.531 INFO:tasks.rados.rados.0.smithi118.stdout:878: finishing write tid 6 to smithi11881747-47 2024-03-20T08:04:21.531 INFO:tasks.rados.rados.0.smithi118.stdout:878: oid 47 updating version 0 to 648 2024-03-20T08:04:21.531 INFO:tasks.rados.rados.0.smithi118.stdout:878: oid 47 version 648 is already newer than 647 2024-03-20T08:04:21.532 INFO:tasks.rados.rados.0.smithi118.stdout:878: oid 47 version 648 is already newer than 646 2024-03-20T08:04:21.532 INFO:tasks.rados.rados.0.smithi118.stdout:878: oid 47 updating version 648 to 649 2024-03-20T08:04:21.532 INFO:tasks.rados.rados.0.smithi118.stdout:878: oid 47 updating version 649 to 650 2024-03-20T08:04:21.532 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 650 (ObjNum 275 snap 92 seq_num 275) dirty exists 2024-03-20T08:04:21.532 INFO:tasks.rados.rados.0.smithi118.stdout:878: left oid 47 (ObjNum 275 snap 92 seq_num 275) 2024-03-20T08:04:22.058 INFO:tasks.rados.rados.0.smithi118.stdout:876: done (2 left) 2024-03-20T08:04:22.059 INFO:tasks.rados.rados.0.smithi118.stdout:878: done (1 left) 2024-03-20T08:04:22.059 INFO:tasks.rados.rados.0.smithi118.stdout:879: done (0 left) 2024-03-20T08:04:22.059 INFO:tasks.rados.rados.0.smithi118.stdout:880: snap_remove snap 58 2024-03-20T08:04:22.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:22 smithi196 ceph-mon[31033]: osdmap e237: 8 total, 8 up, 8 in 2024-03-20T08:04:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:22 smithi118 ceph-mon[93121]: osdmap e237: 8 total, 8 up, 8 in 2024-03-20T08:04:22.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:22 smithi118 ceph-mon[95692]: osdmap e237: 8 total, 8 up, 8 in 2024-03-20T08:04:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:880: done (0 left) 2024-03-20T08:04:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:881: rmattr oid 46 current snap is 93 2024-03-20T08:04:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:881: done (0 left) 2024-03-20T08:04:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 93 2024-03-20T08:04:23.064 INFO:tasks.rados.rados.0.smithi118.stdout:882: seq_num 276 ranges {1056768=630784,1687552=671744,2359296=704512,3063808=229376} 2024-03-20T08:04:23.074 INFO:tasks.rados.rados.0.smithi118.stdout:882: writing smithi11881747-21 from 1056768 to 1687552 tid 1 2024-03-20T08:04:23.079 INFO:tasks.rados.rados.0.smithi118.stdout:882: writing smithi11881747-21 from 1687552 to 2359296 tid 2 2024-03-20T08:04:23.083 INFO:tasks.rados.rados.0.smithi118.stdout:882: writing smithi11881747-21 from 2359296 to 3063808 tid 3 2024-03-20T08:04:23.085 INFO:tasks.rados.rados.0.smithi118.stdout:882: writing smithi11881747-21 from 3063808 to 3293184 tid 4 2024-03-20T08:04:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:883: delete oid 39 current snap is 93 2024-03-20T08:04:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:882: finishing write tid 1 to smithi11881747-21 2024-03-20T08:04:23.086 INFO:tasks.rados.rados.0.smithi118.stdout:882: finishing write tid 2 to smithi11881747-21 2024-03-20T08:04:23.088 INFO:tasks.rados.rados.0.smithi118.stdout:883: done (1 left) 2024-03-20T08:04:23.088 INFO:tasks.rados.rados.0.smithi118.stdout:884: read oid 49 snap -1 2024-03-20T08:04:23.088 INFO:tasks.rados.rados.0.smithi118.stdout:884: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:04:23.088 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 93 2024-03-20T08:04:23.088 INFO:tasks.rados.rados.0.smithi118.stdout:885: seq_num 278 ranges {2998272=663552,3661824=442368,4104192=720896,4825088=442368} 2024-03-20T08:04:23.108 INFO:tasks.rados.rados.0.smithi118.stdout:885: writing smithi11881747-28 from 2998272 to 3661824 tid 1 2024-03-20T08:04:23.111 INFO:tasks.rados.rados.0.smithi118.stdout:885: writing smithi11881747-28 from 3661824 to 4104192 tid 2 2024-03-20T08:04:23.115 INFO:tasks.rados.rados.0.smithi118.stdout:885: writing smithi11881747-28 from 4104192 to 4825088 tid 3 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:885: writing smithi11881747-28 from 4825088 to 5267456 tid 4 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:886: read oid 39 snap -1 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:886: expect deleted 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:887: rmattr oid 12 current snap is 93 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:887: done (4 left) 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:888: delete oid 3 current snap is 93 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:882: finishing write tid 3 to smithi11881747-21 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:882: finishing write tid 4 to smithi11881747-21 2024-03-20T08:04:23.118 INFO:tasks.rados.rados.0.smithi118.stdout:882: finishing write tid 5 to smithi11881747-21 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:882: finishing write tid 6 to smithi11881747-21 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:882: oid 21 updating version 0 to 332 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:882: oid 21 updating version 332 to 333 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:882: oid 21 version 333 is already newer than 327 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:882: oid 21 version 333 is already newer than 328 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:882: oid 21 version 333 is already newer than 331 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 333 (ObjNum 276 snap 93 seq_num 276) dirty exists 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:882: left oid 21 (ObjNum 276 snap 93 seq_num 276) 2024-03-20T08:04:23.119 INFO:tasks.rados.rados.0.smithi118.stdout:884: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:885: finishing write tid 1 to smithi11881747-28 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:885: finishing write tid 2 to smithi11881747-28 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:882: done (4 left) 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:884: done (3 left) 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:888: done (2 left) 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:889: copy_from oid 43 from oid 35 current snap is 93 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:885: finishing write tid 3 to smithi11881747-28 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:885: finishing write tid 4 to smithi11881747-28 2024-03-20T08:04:23.230 INFO:tasks.rados.rados.0.smithi118.stdout:885: finishing write tid 5 to smithi11881747-28 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:885: finishing write tid 6 to smithi11881747-28 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:885: oid 28 updating version 0 to 665 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:885: oid 28 version 665 is already newer than 663 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:885: oid 28 version 665 is already newer than 664 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:885: oid 28 version 665 is already newer than 662 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:885: oid 28 updating version 665 to 666 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 666 (ObjNum 278 snap 93 seq_num 278) dirty exists 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:885: left oid 28 (ObjNum 278 snap 93 seq_num 278) 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:885: done (2 left) 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:886: done (1 left) 2024-03-20T08:04:23.231 INFO:tasks.rados.rados.0.smithi118.stdout:890: snap_remove snap 82 2024-03-20T08:04:23.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:23 smithi196 ceph-mon[31033]: osdmap e238: 8 total, 8 up, 8 in 2024-03-20T08:04:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:23 smithi196 ceph-mon[31033]: pgmap v32: 121 pgs: 121 active+clean; 230 MiB data, 979 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:23 smithi118 ceph-mon[93121]: osdmap e238: 8 total, 8 up, 8 in 2024-03-20T08:04:23.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:23 smithi118 ceph-mon[93121]: pgmap v32: 121 pgs: 121 active+clean; 230 MiB data, 979 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:23 smithi118 ceph-mon[95692]: osdmap e238: 8 total, 8 up, 8 in 2024-03-20T08:04:23.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:23 smithi118 ceph-mon[95692]: pgmap v32: 121 pgs: 121 active+clean; 230 MiB data, 979 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:23.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:23.506Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:04:23] "GET /metrics HTTP/1.1" 200 36067 "" "Prometheus/2.43.0" 2024-03-20T08:04:24.066 INFO:tasks.rados.rados.0.smithi118.stdout:890: done (1 left) 2024-03-20T08:04:24.066 INFO:tasks.rados.rados.0.smithi118.stdout:891: snap_create 2024-03-20T08:04:24.066 INFO:tasks.rados.rados.0.smithi118.stdout:889: finishing copy_from racing read to smithi11881747-43 2024-03-20T08:04:24.066 INFO:tasks.rados.rados.0.smithi118.stdout:889: finishing copy_from to smithi11881747-43 2024-03-20T08:04:24.066 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 383 (ObjNum 260 snap 90 seq_num 260) dirty exists 2024-03-20T08:04:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:24 smithi196 ceph-mon[31033]: osdmap e239: 8 total, 8 up, 8 in 2024-03-20T08:04:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:24 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:24 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:24 smithi118 ceph-mon[93121]: osdmap e239: 8 total, 8 up, 8 in 2024-03-20T08:04:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:24 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:24 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:24.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:24 smithi118 ceph-mon[95692]: osdmap e239: 8 total, 8 up, 8 in 2024-03-20T08:04:24.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:24 smithi118 ceph-mon[95692]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:24.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:24 smithi118 ceph-mon[95692]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:24.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:04:24.205+0000 7ff9af1ba700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-20T08:04:25.076 INFO:tasks.rados.rados.0.smithi118.stdout:889: done (1 left) 2024-03-20T08:04:25.076 INFO:tasks.rados.rados.0.smithi118.stdout:891: done (0 left) 2024-03-20T08:04:25.076 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 94 2024-03-20T08:04:25.077 INFO:tasks.rados.rados.0.smithi118.stdout:892: seq_num 280 ranges {3293184=696320,3989504=606208,4595712=688128,5283840=311296} 2024-03-20T08:04:25.096 INFO:tasks.rados.rados.0.smithi118.stdout:892: writing smithi11881747-21 from 3293184 to 3989504 tid 1 2024-03-20T08:04:25.100 INFO:tasks.rados.rados.0.smithi118.stdout:892: writing smithi11881747-21 from 3989504 to 4595712 tid 2 2024-03-20T08:04:25.104 INFO:tasks.rados.rados.0.smithi118.stdout:892: writing smithi11881747-21 from 4595712 to 5283840 tid 3 2024-03-20T08:04:25.106 INFO:tasks.rados.rados.0.smithi118.stdout:892: writing smithi11881747-21 from 5283840 to 5595136 tid 4 2024-03-20T08:04:25.106 INFO:tasks.rados.rados.0.smithi118.stdout:893: rollback oid 42 current snap is 94 2024-03-20T08:04:25.106 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 64 2024-03-20T08:04:25.107 INFO:tasks.rados.rados.0.smithi118.stdout:892: finishing write tid 1 to smithi11881747-21 2024-03-20T08:04:25.107 INFO:tasks.rados.rados.0.smithi118.stdout:892: finishing write tid 2 to smithi11881747-21 2024-03-20T08:04:25.107 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 94 2024-03-20T08:04:25.107 INFO:tasks.rados.rados.0.smithi118.stdout:894: seq_num 281 ranges {2220032=712704,2932736=491520,3424256=491520,3915776=622592} 2024-03-20T08:04:25.122 INFO:tasks.rados.rados.0.smithi118.stdout:894: writing smithi11881747-47 from 2220032 to 2932736 tid 1 2024-03-20T08:04:25.125 INFO:tasks.rados.rados.0.smithi118.stdout:894: writing smithi11881747-47 from 2932736 to 3424256 tid 2 2024-03-20T08:04:25.128 INFO:tasks.rados.rados.0.smithi118.stdout:894: writing smithi11881747-47 from 3424256 to 3915776 tid 3 2024-03-20T08:04:25.133 INFO:tasks.rados.rados.0.smithi118.stdout:894: writing smithi11881747-47 from 3915776 to 4538368 tid 4 2024-03-20T08:04:25.133 INFO:tasks.rados.rados.0.smithi118.stdout:895: copy_from oid 13 from oid 7 current snap is 94 2024-03-20T08:04:25.133 INFO:tasks.rados.rados.0.smithi118.stdout:892: finishing write tid 3 to smithi11881747-21 2024-03-20T08:04:25.133 INFO:tasks.rados.rados.0.smithi118.stdout:892: finishing write tid 4 to smithi11881747-21 2024-03-20T08:04:25.133 INFO:tasks.rados.rados.0.smithi118.stdout:896: snap_create 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:893: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:892: finishing write tid 5 to smithi11881747-21 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:893: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:893: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 560 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:892: finishing write tid 6 to smithi11881747-21 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:892: oid 21 updating version 0 to 344 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:892: oid 21 version 344 is already newer than 343 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:892: oid 21 version 344 is already newer than 341 2024-03-20T08:04:25.134 INFO:tasks.rados.rados.0.smithi118.stdout:892: oid 21 version 344 is already newer than 342 2024-03-20T08:04:25.135 INFO:tasks.rados.rados.0.smithi118.stdout:892: oid 21 updating version 344 to 345 2024-03-20T08:04:25.135 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 345 (ObjNum 280 snap 94 seq_num 280) dirty exists 2024-03-20T08:04:25.135 INFO:tasks.rados.rados.0.smithi118.stdout:892: left oid 21 (ObjNum 280 snap 94 seq_num 280) 2024-03-20T08:04:25.135 INFO:tasks.rados.rados.0.smithi118.stdout:894: finishing write tid 1 to smithi11881747-47 2024-03-20T08:04:25.135 INFO:tasks.rados.rados.0.smithi118.stdout:894: finishing write tid 2 to smithi11881747-47 2024-03-20T08:04:25.135 INFO:tasks.rados.rados.0.smithi118.stdout:894: finishing write tid 3 to smithi11881747-47 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:894: finishing write tid 4 to smithi11881747-47 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:894: finishing write tid 5 to smithi11881747-47 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:895: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:894: finishing write tid 6 to smithi11881747-47 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:894: oid 47 updating version 0 to 682 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:894: oid 47 version 682 is already newer than 681 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:894: oid 47 version 682 is already newer than 680 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:894: oid 47 version 682 is already newer than 679 2024-03-20T08:04:25.136 INFO:tasks.rados.rados.0.smithi118.stdout:894: oid 47 version 682 is already newer than 678 2024-03-20T08:04:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 682 (ObjNum 281 snap 94 seq_num 281) dirty exists 2024-03-20T08:04:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:894: left oid 47 (ObjNum 281 snap 94 seq_num 281) 2024-03-20T08:04:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:895: finishing copy_from to smithi11881747-13 2024-03-20T08:04:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 60 (ObjNum 145 snap 41 seq_num 145) dirty exists 2024-03-20T08:04:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:25 smithi196 ceph-mon[31033]: osdmap e240: 8 total, 8 up, 8 in 2024-03-20T08:04:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:25 smithi196 ceph-mon[31033]: pgmap v35: 121 pgs: 121 active+clean; 230 MiB data, 979 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:25 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:25 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:25 smithi118 ceph-mon[93121]: osdmap e240: 8 total, 8 up, 8 in 2024-03-20T08:04:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:25 smithi118 ceph-mon[93121]: pgmap v35: 121 pgs: 121 active+clean; 230 MiB data, 979 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:25 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:25.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:25 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:25.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:25 smithi118 ceph-mon[95692]: osdmap e240: 8 total, 8 up, 8 in 2024-03-20T08:04:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:25 smithi118 ceph-mon[95692]: pgmap v35: 121 pgs: 121 active+clean; 230 MiB data, 979 MiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:25 smithi118 ceph-mon[95692]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:25 smithi118 ceph-mon[95692]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:892: done (4 left) 2024-03-20T08:04:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:893: done (3 left) 2024-03-20T08:04:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:894: done (2 left) 2024-03-20T08:04:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:895: done (1 left) 2024-03-20T08:04:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:896: done (0 left) 2024-03-20T08:04:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 95 2024-03-20T08:04:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:897: seq_num 282 ranges {0=729088,729088=425984,1155072=704512,1859584=475136} 2024-03-20T08:04:26.083 INFO:tasks.rados.rados.0.smithi118.stdout:897: writing smithi11881747-15 from 0 to 729088 tid 1 2024-03-20T08:04:26.085 INFO:tasks.rados.rados.0.smithi118.stdout:897: writing smithi11881747-15 from 729088 to 1155072 tid 2 2024-03-20T08:04:26.090 INFO:tasks.rados.rados.0.smithi118.stdout:897: writing smithi11881747-15 from 1155072 to 1859584 tid 3 2024-03-20T08:04:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:897: writing smithi11881747-15 from 1859584 to 2334720 tid 4 2024-03-20T08:04:26.094 INFO:tasks.rados.rados.0.smithi118.stdout:898: delete oid 31 current snap is 95 2024-03-20T08:04:26.094 INFO:tasks.rados.rados.0.smithi118.stdout:897: finishing write tid 1 to smithi11881747-15 2024-03-20T08:04:26.094 INFO:tasks.rados.rados.0.smithi118.stdout:897: finishing write tid 2 to smithi11881747-15 2024-03-20T08:04:26.094 INFO:tasks.rados.rados.0.smithi118.stdout:897: finishing write tid 3 to smithi11881747-15 2024-03-20T08:04:26.096 INFO:tasks.rados.rados.0.smithi118.stdout:897: finishing write tid 4 to smithi11881747-15 2024-03-20T08:04:26.098 INFO:tasks.rados.rados.0.smithi118.stdout:897: finishing write tid 5 to smithi11881747-15 2024-03-20T08:04:26.100 INFO:tasks.rados.rados.0.smithi118.stdout:897: finishing write tid 6 to smithi11881747-15 2024-03-20T08:04:26.100 INFO:tasks.rados.rados.0.smithi118.stdout:897: oid 15 updating version 0 to 178 2024-03-20T08:04:26.100 INFO:tasks.rados.rados.0.smithi118.stdout:897: oid 15 version 178 is already newer than 175 2024-03-20T08:04:26.100 INFO:tasks.rados.rados.0.smithi118.stdout:897: oid 15 version 178 is already newer than 177 2024-03-20T08:04:26.100 INFO:tasks.rados.rados.0.smithi118.stdout:897: oid 15 version 178 is already newer than 176 2024-03-20T08:04:26.100 INFO:tasks.rados.rados.0.smithi118.stdout:897: oid 15 version 178 is already newer than 174 2024-03-20T08:04:26.100 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 178 (ObjNum 282 snap 95 seq_num 282) dirty exists 2024-03-20T08:04:26.101 INFO:tasks.rados.rados.0.smithi118.stdout:897: left oid 15 (ObjNum 282 snap 95 seq_num 282) 2024-03-20T08:04:26.101 INFO:tasks.rados.rados.0.smithi118.stdout:897: done (1 left) 2024-03-20T08:04:26.101 INFO:tasks.rados.rados.0.smithi118.stdout:898: done (0 left) 2024-03-20T08:04:26.101 INFO:tasks.rados.rados.0.smithi118.stdout:899: snap_create 2024-03-20T08:04:26.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:26 smithi118 ceph-mon[93121]: osdmap e241: 8 total, 8 up, 8 in 2024-03-20T08:04:26.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:26 smithi118 ceph-mon[95692]: osdmap e241: 8 total, 8 up, 8 in 2024-03-20T08:04:26.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:26 smithi196 ceph-mon[31033]: osdmap e241: 8 total, 8 up, 8 in 2024-03-20T08:04:27.086 INFO:tasks.rados.rados.0.smithi118.stdout:899: done (0 left) 2024-03-20T08:04:27.086 INFO:tasks.rados.rados.0.smithi118.stdout:900: copy_from oid 33 from oid 44 current snap is 96 2024-03-20T08:04:27.086 INFO:tasks.rados.rados.0.smithi118.stdout:901: delete oid 3 current snap is 96 2024-03-20T08:04:27.089 INFO:tasks.rados.rados.0.smithi118.stdout:900: finishing copy_from to smithi11881747-33 2024-03-20T08:04:27.089 INFO:tasks.rados.rados.0.smithi118.stdout:900: got expected ENOENT (src dne) 2024-03-20T08:04:27.089 INFO:tasks.rados.rados.0.smithi118.stdout:900: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:04:27.089 INFO:tasks.rados.rados.0.smithi118.stdout:900: done (1 left) 2024-03-20T08:04:27.089 INFO:tasks.rados.rados.0.smithi118.stdout:901: done (0 left) 2024-03-20T08:04:27.089 INFO:tasks.rados.rados.0.smithi118.stdout:902: snap_remove snap 93 2024-03-20T08:04:27.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:27 smithi196 ceph-mon[31033]: osdmap e242: 8 total, 8 up, 8 in 2024-03-20T08:04:27.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:27 smithi196 ceph-mon[31033]: pgmap v38: 121 pgs: 121 active+clean; 253 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 6.9 MiB/s wr, 26 op/s 2024-03-20T08:04:27.416 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:27 smithi118 ceph-mon[93121]: osdmap e242: 8 total, 8 up, 8 in 2024-03-20T08:04:27.416 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:27 smithi118 ceph-mon[93121]: pgmap v38: 121 pgs: 121 active+clean; 253 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 6.9 MiB/s wr, 26 op/s 2024-03-20T08:04:27.416 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:27 smithi118 ceph-mon[95692]: osdmap e242: 8 total, 8 up, 8 in 2024-03-20T08:04:27.416 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:27 smithi118 ceph-mon[95692]: pgmap v38: 121 pgs: 121 active+clean; 253 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 6.9 MiB/s wr, 26 op/s 2024-03-20T08:04:28.092 INFO:tasks.rados.rados.0.smithi118.stdout:902: done (0 left) 2024-03-20T08:04:28.092 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 96 2024-03-20T08:04:28.092 INFO:tasks.rados.rados.0.smithi118.stdout:903: seq_num 285 ranges {3391488=778240,4169728=475136,4644864=475136,5120000=655360} 2024-03-20T08:04:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:903: writing smithi11881747-34 from 3391488 to 4169728 tid 1 2024-03-20T08:04:28.117 INFO:tasks.rados.rados.0.smithi118.stdout:903: writing smithi11881747-34 from 4169728 to 4644864 tid 2 2024-03-20T08:04:28.120 INFO:tasks.rados.rados.0.smithi118.stdout:903: writing smithi11881747-34 from 4644864 to 5120000 tid 3 2024-03-20T08:04:28.124 INFO:tasks.rados.rados.0.smithi118.stdout:903: writing smithi11881747-34 from 5120000 to 5775360 tid 4 2024-03-20T08:04:28.124 INFO:tasks.rados.rados.0.smithi118.stdout:904: read oid 25 snap -1 2024-03-20T08:04:28.124 INFO:tasks.rados.rados.0.smithi118.stdout:904: expect (ObjNum 253 snap 89 seq_num 253) 2024-03-20T08:04:28.124 INFO:tasks.rados.rados.0.smithi118.stdout:905: snap_remove snap 91 2024-03-20T08:04:28.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:28 smithi196 ceph-mon[31033]: osdmap e243: 8 total, 8 up, 8 in 2024-03-20T08:04:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:28 smithi118 ceph-mon[93121]: osdmap e243: 8 total, 8 up, 8 in 2024-03-20T08:04:28.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:28 smithi118 ceph-mon[95692]: osdmap e243: 8 total, 8 up, 8 in 2024-03-20T08:04:29.098 INFO:tasks.rados.rados.0.smithi118.stdout:905: done (2 left) 2024-03-20T08:04:29.098 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 96 2024-03-20T08:04:29.098 INFO:tasks.rados.rados.0.smithi118.stdout:906: seq_num 286 ranges {5595136=794624,6389760=761856,7151616=679936,7831552=163840} 2024-03-20T08:04:29.132 INFO:tasks.rados.rados.0.smithi118.stdout:906: writing smithi11881747-21 from 5595136 to 6389760 tid 1 2024-03-20T08:04:29.136 INFO:tasks.rados.rados.0.smithi118.stdout:906: writing smithi11881747-21 from 6389760 to 7151616 tid 2 2024-03-20T08:04:29.140 INFO:tasks.rados.rados.0.smithi118.stdout:906: writing smithi11881747-21 from 7151616 to 7831552 tid 3 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:906: writing smithi11881747-21 from 7831552 to 7995392 tid 4 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 96 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:907: seq_num 287 ranges {0=8192} 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:907: writing smithi11881747-50 from 0 to 8192 tid 1 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:908: rollback oid 5 current snap is 96 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 78 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:903: finishing write tid 1 to smithi11881747-34 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:903: finishing write tid 2 to smithi11881747-34 2024-03-20T08:04:29.142 INFO:tasks.rados.rados.0.smithi118.stdout:903: finishing write tid 3 to smithi11881747-34 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: finishing write tid 4 to smithi11881747-34 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: finishing write tid 5 to smithi11881747-34 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: finishing write tid 6 to smithi11881747-34 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: oid 34 updating version 0 to 193 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: oid 34 version 193 is already newer than 192 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: oid 34 version 193 is already newer than 189 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: oid 34 version 193 is already newer than 191 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: oid 34 version 193 is already newer than 190 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 193 (ObjNum 285 snap 96 seq_num 285) dirty exists 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:903: left oid 34 (ObjNum 285 snap 96 seq_num 285) 2024-03-20T08:04:29.143 INFO:tasks.rados.rados.0.smithi118.stdout:904: expect (ObjNum 253 snap 89 seq_num 253) 2024-03-20T08:04:29.194 INFO:tasks.rados.rados.0.smithi118.stdout:906: finishing write tid 1 to smithi11881747-21 2024-03-20T08:04:29.194 INFO:tasks.rados.rados.0.smithi118.stdout:906: finishing write tid 2 to smithi11881747-21 2024-03-20T08:04:29.194 INFO:tasks.rados.rados.0.smithi118.stdout:907: finishing write tid 1 to smithi11881747-50 2024-03-20T08:04:29.194 INFO:tasks.rados.rados.0.smithi118.stdout:906: finishing write tid 3 to smithi11881747-21 2024-03-20T08:04:29.194 INFO:tasks.rados.rados.0.smithi118.stdout:907: finishing write tid 2 to smithi11881747-50 2024-03-20T08:04:29.194 INFO:tasks.rados.rados.0.smithi118.stdout:908: finishing rollback tid 0 to smithi11881747-5 2024-03-20T08:04:29.194 INFO:tasks.rados.rados.0.smithi118.stdout:906: finishing write tid 4 to smithi11881747-21 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:908: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:906: finishing write tid 5 to smithi11881747-21 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:908: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 563 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:906: finishing write tid 6 to smithi11881747-21 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:906: oid 21 updating version 0 to 353 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:906: oid 21 updating version 353 to 357 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:906: oid 21 version 357 is already newer than 356 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:906: oid 21 version 357 is already newer than 355 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:906: oid 21 version 357 is already newer than 354 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 357 (ObjNum 286 snap 96 seq_num 286) dirty exists 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:906: left oid 21 (ObjNum 286 snap 96 seq_num 286) 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:907: finishing write tid 3 to smithi11881747-50 2024-03-20T08:04:29.195 INFO:tasks.rados.rados.0.smithi118.stdout:907: oid 50 updating version 0 to 327 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:907: oid 50 version 327 is already newer than 326 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 327 (ObjNum 287 snap 96 seq_num 287) dirty exists 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:907: left oid 50 (ObjNum 287 snap 96 seq_num 287) 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:903: done (4 left) 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:904: done (3 left) 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:906: done (2 left) 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:907: done (1 left) 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:908: done (0 left) 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:909: rollback oid 28 current snap is 96 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 28 to 86 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:910: rmattr oid 44 current snap is 96 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:910: done (1 left) 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:911: read oid 16 snap -1 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:911: expect (ObjNum 257 snap 89 seq_num 257) 2024-03-20T08:04:29.196 INFO:tasks.rados.rados.0.smithi118.stdout:912: snap_create 2024-03-20T08:04:29.197 INFO:tasks.rados.rados.0.smithi118.stdout:909: finishing rollback tid 0 to smithi11881747-28 2024-03-20T08:04:29.197 INFO:tasks.rados.rados.0.smithi118.stdout:909: finishing rollback tid 1 to smithi11881747-28 2024-03-20T08:04:29.197 INFO:tasks.rados.rados.0.smithi118.stdout:909: finishing rollback tid 2 to smithi11881747-28 2024-03-20T08:04:29.197 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 692 (ObjNum 239 snap 84 seq_num 239) dirty exists 2024-03-20T08:04:29.207 INFO:tasks.rados.rados.0.smithi118.stdout:911: expect (ObjNum 257 snap 89 seq_num 257) 2024-03-20T08:04:29.223 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: osdmap e244: 8 total, 8 up, 8 in 2024-03-20T08:04:29.223 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: pgmap v41: 121 pgs: 121 active+clean; 253 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 6.9 MiB/s wr, 26 op/s 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:04:29.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:29 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-20T08:04:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: osdmap e244: 8 total, 8 up, 8 in 2024-03-20T08:04:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: pgmap v41: 121 pgs: 121 active+clean; 253 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 6.9 MiB/s wr, 26 op/s 2024-03-20T08:04:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: osdmap e244: 8 total, 8 up, 8 in 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: pgmap v41: 121 pgs: 121 active+clean; 253 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 7.6 MiB/s rd, 6.9 MiB/s wr, 26 op/s 2024-03-20T08:04:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-20T08:04:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-20T08:04:29.753 INFO:tasks.rados.rados.0.smithi118.stdout:909: done (2 left) 2024-03-20T08:04:29.754 INFO:tasks.rados.rados.0.smithi118.stdout:911: done (1 left) 2024-03-20T08:04:29.754 INFO:tasks.rados.rados.0.smithi118.stdout:912: done (0 left) 2024-03-20T08:04:29.754 INFO:tasks.rados.rados.0.smithi118.stdout:913: snap_remove snap 90 2024-03-20T08:04:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:30 smithi196 ceph-mon[31033]: Upgrade: It appears safe to stop mon.b 2024-03-20T08:04:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:30 smithi196 ceph-mon[31033]: osdmap e245: 8 total, 8 up, 8 in 2024-03-20T08:04:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:30 smithi196 ceph-mon[31033]: osdmap e246: 8 total, 8 up, 8 in 2024-03-20T08:04:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:30 smithi196 ceph-mon[31033]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:30 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:30 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:30 smithi196 ceph-mon[31033]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[93121]: Upgrade: It appears safe to stop mon.b 2024-03-20T08:04:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[93121]: osdmap e245: 8 total, 8 up, 8 in 2024-03-20T08:04:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[93121]: osdmap e246: 8 total, 8 up, 8 in 2024-03-20T08:04:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[93121]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[95692]: Upgrade: It appears safe to stop mon.b 2024-03-20T08:04:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[95692]: osdmap e245: 8 total, 8 up, 8 in 2024-03-20T08:04:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[95692]: osdmap e246: 8 total, 8 up, 8 in 2024-03-20T08:04:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[95692]: from='mgr.34107 ' entity='mgr.y' 2024-03-20T08:04:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:30 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:30.756 INFO:tasks.rados.rados.0.smithi118.stdout:913: done (0 left) 2024-03-20T08:04:30.756 INFO:tasks.rados.rados.0.smithi118.stdout:914: read oid 47 snap -1 2024-03-20T08:04:30.756 INFO:tasks.rados.rados.0.smithi118.stdout:914: expect (ObjNum 281 snap 94 seq_num 281) 2024-03-20T08:04:30.756 INFO:tasks.rados.rados.0.smithi118.stdout:915: read oid 35 snap -1 2024-03-20T08:04:30.757 INFO:tasks.rados.rados.0.smithi118.stdout:915: expect (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:04:30.757 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 97 2024-03-20T08:04:30.757 INFO:tasks.rados.rados.0.smithi118.stdout:916: seq_num 288 ranges {1564672=24576} 2024-03-20T08:04:30.765 INFO:tasks.rados.rados.0.smithi118.stdout:916: writing smithi11881747-4 from 1564672 to 1589248 tid 1 2024-03-20T08:04:30.765 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 97 2024-03-20T08:04:30.766 INFO:tasks.rados.rados.0.smithi118.stdout:917: seq_num 289 ranges {1318912=40960} 2024-03-20T08:04:30.774 INFO:tasks.rados.rados.0.smithi118.stdout:917: writing smithi11881747-36 from 1318912 to 1359872 tid 1 2024-03-20T08:04:30.774 INFO:tasks.rados.rados.0.smithi118.stdout:918: copy_from oid 8 from oid 27 current snap is 97 2024-03-20T08:04:30.774 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 97 2024-03-20T08:04:30.774 INFO:tasks.rados.rados.0.smithi118.stdout:919: seq_num 290 ranges {3391488=57344} 2024-03-20T08:04:30.793 INFO:tasks.rados.rados.0.smithi118.stdout:919: writing smithi11881747-48 from 3391488 to 3448832 tid 1 2024-03-20T08:04:30.793 INFO:tasks.rados.rados.0.smithi118.stdout:920: rollback oid 18 current snap is 97 2024-03-20T08:04:30.793 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 18 to 62 2024-03-20T08:04:30.793 INFO:tasks.rados.rados.0.smithi118.stdout:915: expect (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:04:30.844 INFO:tasks.rados.rados.0.smithi118.stdout:916: finishing write tid 1 to smithi11881747-4 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:916: finishing write tid 2 to smithi11881747-4 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:916: finishing write tid 3 to smithi11881747-4 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:916: oid 4 updating version 0 to 285 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:916: oid 4 version 285 is already newer than 284 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 285 (ObjNum 288 snap 97 seq_num 288) dirty exists 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:916: left oid 4 (ObjNum 288 snap 97 seq_num 288) 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:915: done (6 left) 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:916: done (5 left) 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:921: delete oid 28 current snap is 97 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:917: finishing write tid 1 to smithi11881747-36 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:918: finishing copy_from racing read to smithi11881747-8 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:917: finishing write tid 2 to smithi11881747-36 2024-03-20T08:04:30.845 INFO:tasks.rados.rados.0.smithi118.stdout:917: finishing write tid 3 to smithi11881747-36 2024-03-20T08:04:30.846 INFO:tasks.rados.rados.0.smithi118.stdout:917: oid 36 updating version 0 to 200 2024-03-20T08:04:30.846 INFO:tasks.rados.rados.0.smithi118.stdout:917: oid 36 version 200 is already newer than 199 2024-03-20T08:04:30.846 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 200 (ObjNum 289 snap 97 seq_num 289) dirty exists 2024-03-20T08:04:30.846 INFO:tasks.rados.rados.0.smithi118.stdout:917: left oid 36 (ObjNum 289 snap 97 seq_num 289) 2024-03-20T08:04:30.846 INFO:tasks.rados.rados.0.smithi118.stdout:918: finishing copy_from to smithi11881747-8 2024-03-20T08:04:30.846 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 466 (ObjNum 65 snap 0 seq_num 1532453972) dirty exists 2024-03-20T08:04:30.846 INFO:tasks.rados.rados.0.smithi118.stdout:914: expect (ObjNum 281 snap 94 seq_num 281) 2024-03-20T08:04:30.994 INFO:tasks.rados.rados.0.smithi118.stdout:920: finishing rollback tid 0 to smithi11881747-18 2024-03-20T08:04:30.994 INFO:tasks.rados.rados.0.smithi118.stdout:919: finishing write tid 1 to smithi11881747-48 2024-03-20T08:04:30.994 INFO:tasks.rados.rados.0.smithi118.stdout:920: finishing rollback tid 1 to smithi11881747-18 2024-03-20T08:04:30.994 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 491 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:04:30.994 INFO:tasks.rados.rados.0.smithi118.stdout:919: finishing write tid 2 to smithi11881747-48 2024-03-20T08:04:30.994 INFO:tasks.rados.rados.0.smithi118.stdout:914: done (5 left) 2024-03-20T08:04:30.994 INFO:tasks.rados.rados.0.smithi118.stdout:917: done (4 left) 2024-03-20T08:04:30.994 INFO:tasks.rados.rados.0.smithi118.stdout:918: done (3 left) 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:920: done (2 left) 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:921: done (1 left) 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 97 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:919: finishing write tid 3 to smithi11881747-48 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:919: oid 48 updating version 0 to 363 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:919: oid 48 updating version 363 to 364 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 364 (ObjNum 290 snap 97 seq_num 290) dirty exists 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:919: left oid 48 (ObjNum 290 snap 97 seq_num 290) 2024-03-20T08:04:30.995 INFO:tasks.rados.rados.0.smithi118.stdout:922: seq_num 292 ranges {819200=98304} 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:922: writing smithi11881747-32 from 819200 to 917504 tid 1 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:919: done (1 left) 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:923: read oid 45 snap -1 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:923: expect (ObjNum 196 snap 64 seq_num 196) 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:924: rollback oid 10 current snap is 97 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 89 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:925: rmattr oid 42 current snap is 97 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:925: done (3 left) 2024-03-20T08:04:30.999 INFO:tasks.rados.rados.0.smithi118.stdout:926: snap_remove snap 66 2024-03-20T08:04:31.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:31 smithi196 ceph-mon[31033]: Upgrade: Updating mon.b 2024-03-20T08:04:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:31 smithi196 ceph-mon[31033]: Deploying daemon mon.b on smithi196 2024-03-20T08:04:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:31 smithi196 ceph-mon[31033]: pgmap v44: 121 pgs: 121 active+clean; 265 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 11 op/s 2024-03-20T08:04:31.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:31 smithi196 ceph-mon[31033]: osdmap e247: 8 total, 8 up, 8 in 2024-03-20T08:04:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:31 smithi118 ceph-mon[93121]: Upgrade: Updating mon.b 2024-03-20T08:04:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:31 smithi118 ceph-mon[93121]: Deploying daemon mon.b on smithi196 2024-03-20T08:04:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:31 smithi118 ceph-mon[93121]: pgmap v44: 121 pgs: 121 active+clean; 265 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 11 op/s 2024-03-20T08:04:31.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:31 smithi118 ceph-mon[93121]: osdmap e247: 8 total, 8 up, 8 in 2024-03-20T08:04:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:31 smithi118 ceph-mon[95692]: Upgrade: Updating mon.b 2024-03-20T08:04:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:31 smithi118 ceph-mon[95692]: Deploying daemon mon.b on smithi196 2024-03-20T08:04:31.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:31 smithi118 ceph-mon[95692]: pgmap v44: 121 pgs: 121 active+clean; 265 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 11 op/s 2024-03-20T08:04:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:31 smithi118 ceph-mon[95692]: osdmap e247: 8 total, 8 up, 8 in 2024-03-20T08:04:31.760 INFO:tasks.rados.rados.0.smithi118.stdout:926: done (3 left) 2024-03-20T08:04:31.760 INFO:tasks.rados.rados.0.smithi118.stdout:927: snap_create 2024-03-20T08:04:31.760 INFO:tasks.rados.rados.0.smithi118.stdout:922: finishing write tid 1 to smithi11881747-32 2024-03-20T08:04:31.760 INFO:tasks.rados.rados.0.smithi118.stdout:922: finishing write tid 2 to smithi11881747-32 2024-03-20T08:04:31.760 INFO:tasks.rados.rados.0.smithi118.stdout:924: finishing rollback tid 0 to smithi11881747-10 2024-03-20T08:04:31.760 INFO:tasks.rados.rados.0.smithi118.stdout:924: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:04:31.760 INFO:tasks.rados.rados.0.smithi118.stdout:924: finishing rollback tid 2 to smithi11881747-10 2024-03-20T08:04:31.761 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 408 (ObjNum 254 snap 89 seq_num 254) dirty exists 2024-03-20T08:04:31.761 INFO:tasks.rados.rados.0.smithi118.stdout:922: finishing write tid 3 to smithi11881747-32 2024-03-20T08:04:31.761 INFO:tasks.rados.rados.0.smithi118.stdout:922: oid 32 updating version 0 to 542 2024-03-20T08:04:31.761 INFO:tasks.rados.rados.0.smithi118.stdout:922: oid 32 updating version 542 to 543 2024-03-20T08:04:31.761 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 543 (ObjNum 292 snap 97 seq_num 292) dirty exists 2024-03-20T08:04:31.761 INFO:tasks.rados.rados.0.smithi118.stdout:922: left oid 32 (ObjNum 292 snap 97 seq_num 292) 2024-03-20T08:04:31.761 INFO:tasks.rados.rados.0.smithi118.stdout:923: expect (ObjNum 196 snap 64 seq_num 196) 2024-03-20T08:04:32.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:31 smithi196 systemd[1]: Stopping Ceph mon.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:04:32.312 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b[31010]: 2024-03-20T08:04:32.042+0000 7fad32f0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:04:32.312 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b[31010]: 2024-03-20T08:04:32.042+0000 7fad32f0e700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-03-20T08:04:32.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:32 smithi196 podman[86887]: 2024-03-20 08:04:32.247598222 +0000 UTC m=+0.351497717 container died fd26544c725b0c066670c5030b50b72b7c8f39b7260cc731748b15af91f6aa7a (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-78dbc53) 2024-03-20T08:04:32.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:32] ENGINE Bus STOPPING 2024-03-20T08:04:32.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:32] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:04:32.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:32] ENGINE Bus STOPPED 2024-03-20T08:04:32.422 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:32] ENGINE Bus STARTING 2024-03-20T08:04:32.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:32] ENGINE Serving on http://:::9283 2024-03-20T08:04:32.423 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:04:32] ENGINE Bus STARTED 2024-03-20T08:04:32.597 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:32.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:32 smithi196 podman[86887]: 2024-03-20 08:04:32.598620422 +0000 UTC m=+0.702519917 container cleanup fd26544c725b0c066670c5030b50b72b7c8f39b7260cc731748b15af91f6aa7a (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=reef-78dbc53) 2024-03-20T08:04:32.873 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:32 smithi196 bash[86887]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b 2024-03-20T08:04:33.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:32 smithi196 podman[86906]: 2024-03-20 08:04:32.874916528 +0000 UTC m=+0.632173753 container remove fd26544c725b0c066670c5030b50b72b7c8f39b7260cc731748b15af91f6aa7a (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=reef-78dbc53, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-20T08:04:33.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-20T08:04:33.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 systemd[1]: Stopped Ceph mon.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:04:33.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.b.service: Consumed 9.585s CPU time. 2024-03-20T08:04:33.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 systemd[1]: Starting Ceph mon.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:04:33.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 podman[87042]: 2024-03-20 08:04:33.321437583 +0000 UTC m=+0.018304608 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:04:33.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 podman[87042]: 2024-03-20 08:04:33.426369758 +0000 UTC m=+0.123236821 container create bbde40d464eac0b6054de62ee2e0cbace7f4ededd480a05fd10988ac12846a25 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:04:33.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:33.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:33.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 podman[87042]: 2024-03-20 08:04:33.629022717 +0000 UTC m=+0.325889738 container init bbde40d464eac0b6054de62ee2e0cbace7f4ededd480a05fd10988ac12846a25 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-20T08:04:33.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 podman[87042]: 2024-03-20 08:04:33.63468746 +0000 UTC m=+0.331554481 container start bbde40d464eac0b6054de62ee2e0cbace7f4ededd480a05fd10988ac12846a25 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-20T08:04:33.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: set uid:gid to 167:167 (ceph:ceph) 2024-03-20T08:04:33.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev), process ceph-mon, pid 7 2024-03-20T08:04:33.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: pidfile_write: ignore empty --pid-file 2024-03-20T08:04:33.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: load: jerasure load: lrc 2024-03-20T08:04:33.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: RocksDB version: 7.9.2 2024-03-20T08:04:33.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Git sha 0 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Compile date 2024-03-18 19:18:37 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: DB SUMMARY 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: DB Session ID: FT1JZAORQPK6O8M0J78G 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: CURRENT file: CURRENT 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: IDENTITY file: IDENTITY 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1161 Bytes 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 1, files: 000024.sst 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000022.log size: 9927141 ; 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.error_if_exists: 0 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.create_if_missing: 0 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.paranoid_checks: 1 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-20T08:04:33.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.env: 0x556a753aab20 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.fs: PosixFileSystem 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.info_log: 0x556a77b63920 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_file_opening_threads: 16 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.statistics: (nil) 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.use_fsync: 0 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_log_file_size: 0 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.keep_log_file_num: 1000 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.recycle_log_file_num: 0 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.allow_fallocate: 1 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.allow_mmap_reads: 0 2024-03-20T08:04:33.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.allow_mmap_writes: 0 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.use_direct_reads: 0 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.create_missing_column_families: 0 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.db_log_dir: 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.wal_dir: 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.advise_random_on_open: 1 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.db_write_buffer_size: 0 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.write_buffer_manager: 0x556a77b36820 2024-03-20T08:04:33.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.rate_limiter: (nil) 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.wal_recovery_mode: 2 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.enable_thread_tracking: 0 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.enable_pipelined_write: 0 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.unordered_write: 0 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.row_cache: None 2024-03-20T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.wal_filter: None 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.allow_ingest_behind: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.two_write_queues: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.manual_wal_flush: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.wal_compression: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.atomic_flush: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.log_readahead_size: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.best_efforts_recovery: 0 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-20T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.allow_data_in_errors: 0 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.db_host_id: __hostname__ 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_background_jobs: 2 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_background_compactions: -1 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_subcompactions: 1 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_total_wal_size: 0 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-20T08:04:33.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_open_files: -1 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bytes_per_sync: 0 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_readahead_size: 0 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_background_flushes: -1 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Compression algorithms supported: 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: kZSTD supported: 0 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: kXpressCompression supported: 0 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: kLZ4HCCompression supported: 1 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: kZlibCompression supported: 1 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: kSnappyCompression supported: 1 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: kLZ4Compression supported: 1 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: kBZip2Compression supported: 0 2024-03-20T08:04:33.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.merge_operator: 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_filter: None 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_filter_factory: None 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.sst_partitioner_factory: None 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556a77b63a20) 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-20T08:04:33.898 INFO:journalctl@ceph.mon.b.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_type: 0 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: data_block_index_type: 0 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_shortening: 1 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: checksum: 4 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: no_block_cache: 0 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache: 0x556a77b69090 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_options: 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: capacity : 536870912 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: num_shard_bits : 4 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: strict_capacity_limit : 0 2024-03-20T08:04:33.899 INFO:journalctl@ceph.mon.b.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_cache_compressed: (nil) 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: persistent_cache: (nil) 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_size: 4096 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_size_deviation: 10 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_restart_interval: 16 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: index_block_restart_interval: 1 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: metadata_block_size: 4096 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: partition_filters: 0 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: use_delta_encoding: 1 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: filter_policy: bloomfilter 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: whole_key_filtering: 1 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: verify_compression: 0 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: format_version: 5 2024-03-20T08:04:33.900 INFO:journalctl@ceph.mon.b.smithi196.stdout: enable_index_compression: 1 2024-03-20T08:04:33.901 INFO:journalctl@ceph.mon.b.smithi196.stdout: block_align: 0 2024-03-20T08:04:33.901 INFO:journalctl@ceph.mon.b.smithi196.stdout: max_auto_readahead_size: 262144 2024-03-20T08:04:33.901 INFO:journalctl@ceph.mon.b.smithi196.stdout: prepopulate_block_cache: 0 2024-03-20T08:04:33.901 INFO:journalctl@ceph.mon.b.smithi196.stdout: initial_auto_readahead_size: 8192 2024-03-20T08:04:33.901 INFO:journalctl@ceph.mon.b.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-03-20T08:04:33.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.write_buffer_size: 33554432 2024-03-20T08:04:33.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_write_buffer_number: 2 2024-03-20T08:04:33.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression: NoCompression 2024-03-20T08:04:33.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression: Disabled 2024-03-20T08:04:33.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.prefix_extractor: nullptr 2024-03-20T08:04:33.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-20T08:04:33.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.num_levels: 7 2024-03-20T08:04:33.903 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.level: 32767 2024-03-20T08:04:33.904 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.strategy: 0 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.enabled: false 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.target_file_size_base: 67108864 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-20T08:04:33.905 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.arena_block_size: 1048576 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-20T08:04:33.906 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.disable_auto_compactions: 0 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.table_properties_collectors: 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.inplace_update_support: 0 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-20T08:04:33.907 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.bloom_locality: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.max_successive_merges: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.paranoid_file_checks: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.force_consistency_checks: 1 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.report_bg_io_stats: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.ttl: 2592000 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.enable_blob_files: false 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.min_blob_size: 0 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.blob_file_size: 268435456 2024-03-20T08:04:33.908 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.blob_file_starting_level: 0 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 26, last_sequence is 13595, log_number is 22,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 22 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 22 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fb98c469-8583-4c6a-ada9-e63edf2c67e5 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921873664291, "job": 1, "event": "recovery_started", "wal_files": [22]} 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #22 mode 2 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921873688999, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 27, "file_size": 6059738, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 13600, "largest_seqno": 17221, "table_properties": {"data_size": 6044142, "index_size": 10735, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3845, "raw_key_size": 34852, "raw_average_key_size": 22, "raw_value_size": 6011810, "raw_average_value_size": 3913, "num_data_blocks": 469, "num_entries": 1536, "num_filter_entries": 1536, "num_deletions": 291, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710921873, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fb98c469-8583-4c6a-ada9-e63edf2c67e5", "db_session_id": "FT1JZAORQPK6O8M0J78G", "orig_file_number": 27, "seqno_to_time_mapping": "N/A"}} 2024-03-20T08:04:33.909 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710921873689113, "job": 1, "event": "recovery_finished"} 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/version_set.cc:5047] Creating manifest 29 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000022.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556a77c5c000 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: DB pointer 0x556a77c46000 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** DB Stats ** 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-20T08:04:33.910 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** Compaction Stats [default] ** 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: L0 1/0 5.78 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 591.7 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: L6 1/0 7.27 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: Sum 2/0 13.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 591.7 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 591.7 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** Compaction Stats [default] ** 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-20T08:04:33.911 INFO:journalctl@ceph.mon.b.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 591.7 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: Flush(GB): cumulative 0.006, interval 0.006 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: Cumulative compaction: 0.01 GB write, 198.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: Interval compaction: 0.01 GB write, 198.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: Block cache BinnedLRUCache@0x556a77b69090#7 capacity: 512.00 MB usage: 16.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-20T08:04:33.912 INFO:journalctl@ceph.mon.b.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,12.08 KB,0.00230372%) Misc(1,0.00 KB,0%) 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: starting mon.b rank 1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: mon.b@-1(???) e3 preinit fsid bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: mon.b@-1(???).mds e6 new map 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: mon.b@-1(???).mds e6 print_map 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: e6 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: legacy client fscid: 1 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: Filesystem 'cephfs' (1) 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: fs_name cephfs 2024-03-20T08:04:33.913 INFO:journalctl@ceph.mon.b.smithi196.stdout: epoch 6 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: created 2024-03-20T07:58:26.102802+0000 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: modified 2024-03-20T07:58:28.113441+0000 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: tableserver 0 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: root 0 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: session_timeout 60 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: session_autoclose 300 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: max_file_size 1099511627776 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: max_xattr_size 65536 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: required_client_features {} 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: last_failure 0 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: last_failure_osd_epoch 0 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-20T08:04:33.914 INFO:journalctl@ceph.mon.b.smithi196.stdout: max_mds 1 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: in 0 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: up {0=24343} 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: failed 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: damaged 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: stopped 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: data_pools [3] 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: metadata_pool 2 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: inline_data disabled 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: balancer 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: bal_rank_mask -1 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: standby_count_wanted 0 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: qdb_cluster leader: 0 members: 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: [mds.b{0:24343} state up:active seq 3 addr [v2:172.21.15.196:6832/5972345,v1:172.21.15.196:6833/5972345] compat {c=[1],r=[1],i=[7ff]}] 2024-03-20T08:04:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout: Standby daemons: 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout: 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout: [mds.a{-1:14430} state up:standby seq 1 addr [v2:172.21.15.118:6834/3215808514,v1:172.21.15.118:6835/3215808514] compat {c=[1],r=[1],i=[7ff]}] 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: mon.b@-1(???).osd e248 crush map has features 3314933069573799936, adjusting msgr requires 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: mon.b@-1(???).osd e248 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: mon.b@-1(???).osd e248 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: mon.b@-1(???).osd e248 crush map has features 432629308056666112, adjusting msgr requires 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 ceph-mon[87080]: mon.b@-1(???).paxosservice(auth 1..20) refresh upgraded, format 0 -> 3 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 bash[87042]: bbde40d464eac0b6054de62ee2e0cbace7f4ededd480a05fd10988ac12846a25 2024-03-20T08:04:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:33 smithi196 systemd[1]: Started Ceph mon.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:04:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:04:33] "GET /metrics HTTP/1.1" 200 36072 "" "Prometheus/2.43.0" 2024-03-20T08:04:34.125 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:04:34.445 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:04:34.445 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (4m) 10s ago 5m 28.5M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:04:34.445 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (4m) 31s ago 4m 89.5M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:04:34.445 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (6m) 10s ago 6m 14.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (6m) 31s ago 6m 18.8M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (60s) 31s ago 9m 447M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (90s) 10s ago 11m 494M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (45s) 10s ago 11m 38.2M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 starting - - - 2048M 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (20s) 10s ago 10m 27.1M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (4m) 10s ago 4m 14.4M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (4m) 31s ago 4m 15.5M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (9m) 10s ago 9m 151M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 481bba84ab31 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (9m) 10s ago 9m 157M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (8m) 10s ago 8m 130M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (8m) 10s ago 8m 126M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:04:34.446 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (7m) 31s ago 7m 157M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:04:34.447 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (7m) 31s ago 7m 114M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:04:34.447 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (6m) 31s ago 6m 93.1M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:04:34.447 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (6m) 31s ago 6m 98.2M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:04:34.447 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (65s) 31s ago 5m 44.8M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 1, 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:04:34.837 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 11, 2024-03-20T08:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 4 2024-03-20T08:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:04:34.838 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:04:36.196 INFO:tasks.rados.rados.0.smithi118.stdout:922: done (3 left) 2024-03-20T08:04:36.196 INFO:tasks.rados.rados.0.smithi118.stdout:923: done (2 left) 2024-03-20T08:04:36.196 INFO:tasks.rados.rados.0.smithi118.stdout:924: done (1 left) 2024-03-20T08:04:36.197 INFO:tasks.rados.rados.0.smithi118.stdout:927: done (0 left) 2024-03-20T08:04:36.197 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 98 2024-03-20T08:04:36.197 INFO:tasks.rados.rados.0.smithi118.stdout:928: seq_num 293 ranges {1744896=114688} 2024-03-20T08:04:36.206 INFO:tasks.rados.rados.0.smithi118.stdout:928: writing smithi11881747-1 from 1744896 to 1859584 tid 1 2024-03-20T08:04:36.206 INFO:tasks.rados.rados.0.smithi118.stdout:929: read oid 35 snap -1 2024-03-20T08:04:36.206 INFO:tasks.rados.rados.0.smithi118.stdout:929: expect (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:04:36.207 INFO:tasks.rados.rados.0.smithi118.stdout:930: snap_create 2024-03-20T08:04:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:928: finishing write tid 1 to smithi11881747-1 2024-03-20T08:04:36.212 INFO:tasks.rados.rados.0.smithi118.stdout:928: finishing write tid 2 to smithi11881747-1 2024-03-20T08:04:36.214 INFO:tasks.rados.rados.0.smithi118.stdout:928: finishing write tid 3 to smithi11881747-1 2024-03-20T08:04:36.214 INFO:tasks.rados.rados.0.smithi118.stdout:928: oid 1 updating version 0 to 416 2024-03-20T08:04:36.214 INFO:tasks.rados.rados.0.smithi118.stdout:928: oid 1 updating version 416 to 417 2024-03-20T08:04:36.214 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 417 (ObjNum 293 snap 98 seq_num 293) dirty exists 2024-03-20T08:04:36.214 INFO:tasks.rados.rados.0.smithi118.stdout:928: left oid 1 (ObjNum 293 snap 98 seq_num 293) 2024-03-20T08:04:36.219 INFO:tasks.rados.rados.0.smithi118.stdout:929: expect (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: from='client.34173 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: mon.b calling monitor election 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: mon.c calling monitor election 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: mon.a calling monitor election 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:04:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: monmap e4: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:04:36.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:04:36.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: osdmap e248: 8 total, 8 up, 8 in 2024-03-20T08:04:36.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: mgrmap e38: y(active, since 37s), standbys: x 2024-03-20T08:04:36.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: overall HEALTH_OK 2024-03-20T08:04:36.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:36.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:36.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:36 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: from='client.34173 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: mon.b calling monitor election 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: mon.c calling monitor election 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: mon.a calling monitor election 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: monmap e4: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: osdmap e248: 8 total, 8 up, 8 in 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: mgrmap e38: y(active, since 37s), standbys: x 2024-03-20T08:04:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: overall HEALTH_OK 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: from='client.34173 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: mon.b calling monitor election 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: mon.c calling monitor election 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-20T08:04:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: mon.a calling monitor election 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: monmap e4: 3 mons at {a=[v2:172.21.15.118:3300/0,v1:172.21.15.118:6789/0],b=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0],c=[v2:172.21.15.118:3301/0,v1:172.21.15.118:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: osdmap e248: 8 total, 8 up, 8 in 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: mgrmap e38: y(active, since 37s), standbys: x 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: overall HEALTH_OK 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:36.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:36 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:04:37.201 INFO:tasks.rados.rados.0.smithi118.stdout:928: done (2 left) 2024-03-20T08:04:37.202 INFO:tasks.rados.rados.0.smithi118.stdout:929: done (1 left) 2024-03-20T08:04:37.202 INFO:tasks.rados.rados.0.smithi118.stdout:930: done (0 left) 2024-03-20T08:04:37.202 INFO:tasks.rados.rados.0.smithi118.stdout:931: rollback oid 37 current snap is 99 2024-03-20T08:04:37.202 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 81 2024-03-20T08:04:37.202 INFO:tasks.rados.rados.0.smithi118.stdout:932: delete oid 44 current snap is 99 2024-03-20T08:04:37.207 INFO:tasks.rados.rados.0.smithi118.stdout:932: done (1 left) 2024-03-20T08:04:37.207 INFO:tasks.rados.rados.0.smithi118.stdout:933: copy_from oid 39 from oid 8 current snap is 99 2024-03-20T08:04:37.207 INFO:tasks.rados.rados.0.smithi118.stdout:934: read oid 46 snap 86 2024-03-20T08:04:37.208 INFO:tasks.rados.rados.0.smithi118.stdout:934: expect deleted 2024-03-20T08:04:37.208 INFO:tasks.rados.rados.0.smithi118.stdout:935: rollback oid 29 current snap is 99 2024-03-20T08:04:37.208 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 73 2024-03-20T08:04:37.208 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 99 2024-03-20T08:04:37.208 INFO:tasks.rados.rados.0.smithi118.stdout:936: seq_num 295 ranges {8192=147456} 2024-03-20T08:04:37.208 INFO:tasks.rados.rados.0.smithi118.stdout:936: writing smithi11881747-50 from 8192 to 155648 tid 1 2024-03-20T08:04:37.209 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 99 2024-03-20T08:04:37.209 INFO:tasks.rados.rados.0.smithi118.stdout:937: seq_num 296 ranges {32768=163840} 2024-03-20T08:04:37.210 INFO:tasks.rados.rados.0.smithi118.stdout:937: writing smithi11881747-24 from 32768 to 196608 tid 1 2024-03-20T08:04:37.210 INFO:tasks.rados.rados.0.smithi118.stdout:938: snap_remove snap 84 2024-03-20T08:04:37.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:37 smithi196 ceph-mon[87080]: pgmap v49: 121 pgs: 121 active+clean; 265 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.8 MiB/s rd, 34 KiB/s wr, 5 op/s 2024-03-20T08:04:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:37 smithi196 ceph-mon[87080]: osdmap e249: 8 total, 8 up, 8 in 2024-03-20T08:04:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:37 smithi118 ceph-mon[93121]: pgmap v49: 121 pgs: 121 active+clean; 265 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.8 MiB/s rd, 34 KiB/s wr, 5 op/s 2024-03-20T08:04:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:37 smithi118 ceph-mon[93121]: osdmap e249: 8 total, 8 up, 8 in 2024-03-20T08:04:37.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:37 smithi118 ceph-mon[95692]: pgmap v49: 121 pgs: 121 active+clean; 265 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.8 MiB/s rd, 34 KiB/s wr, 5 op/s 2024-03-20T08:04:37.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:37 smithi118 ceph-mon[95692]: osdmap e249: 8 total, 8 up, 8 in 2024-03-20T08:04:38.210 INFO:tasks.rados.rados.0.smithi118.stdout:938: done (6 left) 2024-03-20T08:04:38.210 INFO:tasks.rados.rados.0.smithi118.stdout:939: rmattr oid 28 current snap is 99 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:939: done (6 left) 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:940: setattr oid 44 current snap is 99 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:931: finishing rollback tid 0 to smithi11881747-37 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:931: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 249 (ObjNum 33 snap 0 seq_num 322557168) dirty dne 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:933: finishing copy_from racing read to smithi11881747-39 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:935: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:936: finishing write tid 1 to smithi11881747-50 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:936: finishing write tid 2 to smithi11881747-50 2024-03-20T08:04:38.211 INFO:tasks.rados.rados.0.smithi118.stdout:933: finishing copy_from to smithi11881747-39 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 466 (ObjNum 129 snap 0 seq_num 322000512) dirty exists 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:937: finishing write tid 1 to smithi11881747-24 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:937: finishing write tid 2 to smithi11881747-24 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:935: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:935: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 710 (ObjNum 209 snap 67 seq_num 209) dirty exists 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:936: finishing write tid 3 to smithi11881747-50 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:936: oid 50 updating version 0 to 335 2024-03-20T08:04:38.212 INFO:tasks.rados.rados.0.smithi118.stdout:936: oid 50 version 335 is already newer than 334 2024-03-20T08:04:38.213 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 335 (ObjNum 295 snap 99 seq_num 295) dirty exists 2024-03-20T08:04:38.213 INFO:tasks.rados.rados.0.smithi118.stdout:936: left oid 50 (ObjNum 295 snap 99 seq_num 295) 2024-03-20T08:04:38.213 INFO:tasks.rados.rados.0.smithi118.stdout:937: finishing write tid 3 to smithi11881747-24 2024-03-20T08:04:38.213 INFO:tasks.rados.rados.0.smithi118.stdout:937: oid 24 updating version 0 to 337 2024-03-20T08:04:38.213 INFO:tasks.rados.rados.0.smithi118.stdout:937: oid 24 updating version 337 to 338 2024-03-20T08:04:38.213 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 338 (ObjNum 296 snap 99 seq_num 296) dirty exists 2024-03-20T08:04:38.213 INFO:tasks.rados.rados.0.smithi118.stdout:937: left oid 24 (ObjNum 296 snap 99 seq_num 296) 2024-03-20T08:04:38.214 INFO:tasks.rados.rados.0.smithi118.stdout:931: done (6 left) 2024-03-20T08:04:38.214 INFO:tasks.rados.rados.0.smithi118.stdout:933: done (5 left) 2024-03-20T08:04:38.215 INFO:tasks.rados.rados.0.smithi118.stdout:934: done (4 left) 2024-03-20T08:04:38.215 INFO:tasks.rados.rados.0.smithi118.stdout:935: done (3 left) 2024-03-20T08:04:38.215 INFO:tasks.rados.rados.0.smithi118.stdout:936: done (2 left) 2024-03-20T08:04:38.215 INFO:tasks.rados.rados.0.smithi118.stdout:937: done (1 left) 2024-03-20T08:04:38.215 INFO:tasks.rados.rados.0.smithi118.stdout:941: snap_remove snap 65 2024-03-20T08:04:38.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:38 smithi196 ceph-mon[87080]: osdmap e250: 8 total, 8 up, 8 in 2024-03-20T08:04:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:38 smithi118 ceph-mon[93121]: osdmap e250: 8 total, 8 up, 8 in 2024-03-20T08:04:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:38 smithi118 ceph-mon[95692]: osdmap e250: 8 total, 8 up, 8 in 2024-03-20T08:04:39.172 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:04:38.698+0000 7ff9af1ba700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-20T08:04:39.215 INFO:tasks.rados.rados.0.smithi118.stdout:941: done (1 left) 2024-03-20T08:04:39.215 INFO:tasks.rados.rados.0.smithi118.stdout:942: rollback oid 16 current snap is 99 2024-03-20T08:04:39.215 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 86 2024-03-20T08:04:39.215 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 339 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-03-20T08:04:39.215 INFO:tasks.rados.rados.0.smithi118.stdout:940: done (1 left) 2024-03-20T08:04:39.215 INFO:tasks.rados.rados.0.smithi118.stdout:943: snap_remove snap 87 2024-03-20T08:04:39.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:39 smithi196 ceph-mon[87080]: pgmap v52: 121 pgs: 121 active+clean; 265 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.8 MiB/s rd, 34 KiB/s wr, 5 op/s 2024-03-20T08:04:39.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:39 smithi196 ceph-mon[87080]: osdmap e251: 8 total, 8 up, 8 in 2024-03-20T08:04:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:39 smithi118 ceph-mon[93121]: pgmap v52: 121 pgs: 121 active+clean; 265 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.8 MiB/s rd, 34 KiB/s wr, 5 op/s 2024-03-20T08:04:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:39 smithi118 ceph-mon[93121]: osdmap e251: 8 total, 8 up, 8 in 2024-03-20T08:04:39.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:39 smithi118 ceph-mon[95692]: pgmap v52: 121 pgs: 121 active+clean; 265 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 5.8 MiB/s rd, 34 KiB/s wr, 5 op/s 2024-03-20T08:04:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:39 smithi118 ceph-mon[95692]: osdmap e251: 8 total, 8 up, 8 in 2024-03-20T08:04:39.757 INFO:tasks.rados.rados.0.smithi118.stdout:943: done (1 left) 2024-03-20T08:04:39.757 INFO:tasks.rados.rados.0.smithi118.stdout:944: read oid 46 snap -1 2024-03-20T08:04:39.757 INFO:tasks.rados.rados.0.smithi118.stdout:944: expect (ObjNum 261 snap 90 seq_num 261) 2024-03-20T08:04:39.757 INFO:tasks.rados.rados.0.smithi118.stdout:945: copy_from oid 21 from oid 11 current snap is 99 2024-03-20T08:04:39.757 INFO:tasks.rados.rados.0.smithi118.stdout:946: snap_remove snap 92 2024-03-20T08:04:40.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:40 smithi196 ceph-mon[87080]: osdmap e252: 8 total, 8 up, 8 in 2024-03-20T08:04:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:40 smithi196 ceph-mon[87080]: osdmap e253: 8 total, 8 up, 8 in 2024-03-20T08:04:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:40 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:40 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:40 smithi118 ceph-mon[95692]: osdmap e252: 8 total, 8 up, 8 in 2024-03-20T08:04:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:40 smithi118 ceph-mon[95692]: osdmap e253: 8 total, 8 up, 8 in 2024-03-20T08:04:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:40 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:40 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:40.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:40 smithi118 ceph-mon[93121]: osdmap e252: 8 total, 8 up, 8 in 2024-03-20T08:04:40.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:40 smithi118 ceph-mon[93121]: osdmap e253: 8 total, 8 up, 8 in 2024-03-20T08:04:40.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:40 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:40.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:40 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:40.758 INFO:tasks.rados.rados.0.smithi118.stdout:946: done (3 left) 2024-03-20T08:04:40.758 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 99 2024-03-20T08:04:40.758 INFO:tasks.rados.rados.0.smithi118.stdout:942: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:04:40.758 INFO:tasks.rados.rados.0.smithi118.stdout:947: seq_num 297 ranges {0=180224} 2024-03-20T08:04:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:947: writing smithi11881747-18 from 0 to 180224 tid 1 2024-03-20T08:04:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:948: delete oid 12 current snap is 99 2024-03-20T08:04:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:942: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:04:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:942: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:04:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 724 (ObjNum 211 snap 67 seq_num 211) dirty exists 2024-03-20T08:04:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:944: expect (ObjNum 261 snap 90 seq_num 261) 2024-03-20T08:04:40.819 INFO:tasks.rados.rados.0.smithi118.stdout:945: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:04:40.820 INFO:tasks.rados.rados.0.smithi118.stdout:945: finishing copy_from to smithi11881747-21 2024-03-20T08:04:40.820 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 552 (ObjNum 240 snap 84 seq_num 240) dirty exists 2024-03-20T08:04:40.820 INFO:tasks.rados.rados.0.smithi118.stdout:947: finishing write tid 1 to smithi11881747-18 2024-03-20T08:04:40.820 INFO:tasks.rados.rados.0.smithi118.stdout:947: finishing write tid 2 to smithi11881747-18 2024-03-20T08:04:40.820 INFO:tasks.rados.rados.0.smithi118.stdout:942: done (4 left) 2024-03-20T08:04:40.820 INFO:tasks.rados.rados.0.smithi118.stdout:944: done (3 left) 2024-03-20T08:04:40.821 INFO:tasks.rados.rados.0.smithi118.stdout:945: done (2 left) 2024-03-20T08:04:40.821 INFO:tasks.rados.rados.0.smithi118.stdout:948: done (1 left) 2024-03-20T08:04:40.821 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 99 2024-03-20T08:04:40.821 INFO:tasks.rados.rados.0.smithi118.stdout:947: finishing write tid 3 to smithi11881747-18 2024-03-20T08:04:40.821 INFO:tasks.rados.rados.0.smithi118.stdout:947: oid 18 updating version 0 to 521 2024-03-20T08:04:40.821 INFO:tasks.rados.rados.0.smithi118.stdout:947: oid 18 updating version 521 to 522 2024-03-20T08:04:40.822 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 522 (ObjNum 297 snap 99 seq_num 297) dirty exists 2024-03-20T08:04:40.822 INFO:tasks.rados.rados.0.smithi118.stdout:947: left oid 18 (ObjNum 297 snap 99 seq_num 297) 2024-03-20T08:04:40.822 INFO:tasks.rados.rados.0.smithi118.stdout:949: seq_num 299 ranges {3981312=212992} 2024-03-20T08:04:40.841 INFO:tasks.rados.rados.0.smithi118.stdout:949: writing smithi11881747-2 from 3981312 to 4194304 tid 1 2024-03-20T08:04:40.841 INFO:tasks.rados.rados.0.smithi118.stdout:947: done (1 left) 2024-03-20T08:04:40.841 INFO:tasks.rados.rados.0.smithi118.stdout:950: rollback oid 47 current snap is 99 2024-03-20T08:04:40.841 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 81 2024-03-20T08:04:40.841 INFO:tasks.rados.rados.0.smithi118.stdout:951: snap_create 2024-03-20T08:04:40.843 INFO:tasks.rados.rados.0.smithi118.stdout:950: finishing rollback tid 0 to smithi11881747-47 2024-03-20T08:04:40.844 INFO:tasks.rados.rados.0.smithi118.stdout:950: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:04:40.844 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 753 (ObjNum 129 snap 0 seq_num 322408224) dirty dne 2024-03-20T08:04:40.844 INFO:tasks.rados.rados.0.smithi118.stdout:949: finishing write tid 1 to smithi11881747-2 2024-03-20T08:04:40.845 INFO:tasks.rados.rados.0.smithi118.stdout:949: finishing write tid 2 to smithi11881747-2 2024-03-20T08:04:40.845 INFO:tasks.rados.rados.0.smithi118.stdout:949: finishing write tid 3 to smithi11881747-2 2024-03-20T08:04:40.846 INFO:tasks.rados.rados.0.smithi118.stdout:949: oid 2 updating version 0 to 399 2024-03-20T08:04:40.846 INFO:tasks.rados.rados.0.smithi118.stdout:949: oid 2 version 399 is already newer than 398 2024-03-20T08:04:40.846 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 399 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:04:40.846 INFO:tasks.rados.rados.0.smithi118.stdout:949: left oid 2 (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:04:41.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:41 smithi196 ceph-mon[87080]: pgmap v56: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 104 KiB/s wr, 9 op/s 2024-03-20T08:04:41.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:41 smithi196 ceph-mon[87080]: osdmap e254: 8 total, 8 up, 8 in 2024-03-20T08:04:41.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:41 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:41.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:41 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:41.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:41 smithi118 ceph-mon[95692]: pgmap v56: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 104 KiB/s wr, 9 op/s 2024-03-20T08:04:41.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:41 smithi118 ceph-mon[95692]: osdmap e254: 8 total, 8 up, 8 in 2024-03-20T08:04:41.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:41 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:41 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:41.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:41 smithi118 ceph-mon[93121]: pgmap v56: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 104 KiB/s wr, 9 op/s 2024-03-20T08:04:41.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:41 smithi118 ceph-mon[93121]: osdmap e254: 8 total, 8 up, 8 in 2024-03-20T08:04:41.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:41 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:41.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:41 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:41.759 INFO:tasks.rados.rados.0.smithi118.stdout:949: done (2 left) 2024-03-20T08:04:41.759 INFO:tasks.rados.rados.0.smithi118.stdout:950: done (1 left) 2024-03-20T08:04:41.759 INFO:tasks.rados.rados.0.smithi118.stdout:951: done (0 left) 2024-03-20T08:04:41.760 INFO:tasks.rados.rados.0.smithi118.stdout:952: rmattr oid 32 current snap is 100 2024-03-20T08:04:41.760 INFO:tasks.rados.rados.0.smithi118.stdout:952: done (0 left) 2024-03-20T08:04:41.760 INFO:tasks.rados.rados.0.smithi118.stdout:953: read oid 50 snap -1 2024-03-20T08:04:41.760 INFO:tasks.rados.rados.0.smithi118.stdout:953: expect (ObjNum 295 snap 99 seq_num 295) 2024-03-20T08:04:41.760 INFO:tasks.rados.rados.0.smithi118.stdout:954: rollback oid 48 current snap is 100 2024-03-20T08:04:41.761 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 48 to 97 2024-03-20T08:04:41.761 INFO:tasks.rados.rados.0.smithi118.stdout:955: delete oid 5 current snap is 100 2024-03-20T08:04:41.761 INFO:tasks.rados.rados.0.smithi118.stdout:955: done (2 left) 2024-03-20T08:04:41.762 INFO:tasks.rados.rados.0.smithi118.stdout:956: delete oid 36 current snap is 100 2024-03-20T08:04:41.764 INFO:tasks.rados.rados.0.smithi118.stdout:953: expect (ObjNum 295 snap 99 seq_num 295) 2024-03-20T08:04:41.770 INFO:tasks.rados.rados.0.smithi118.stdout:954: finishing rollback tid 0 to smithi11881747-48 2024-03-20T08:04:41.770 INFO:tasks.rados.rados.0.smithi118.stdout:954: finishing rollback tid 1 to smithi11881747-48 2024-03-20T08:04:41.770 INFO:tasks.rados.rados.0.smithi118.stdout:954: finishing rollback tid 2 to smithi11881747-48 2024-03-20T08:04:41.770 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 555 (ObjNum 290 snap 97 seq_num 290) dirty exists 2024-03-20T08:04:41.771 INFO:tasks.rados.rados.0.smithi118.stdout:953: done (2 left) 2024-03-20T08:04:41.771 INFO:tasks.rados.rados.0.smithi118.stdout:954: done (1 left) 2024-03-20T08:04:41.771 INFO:tasks.rados.rados.0.smithi118.stdout:956: done (0 left) 2024-03-20T08:04:41.771 INFO:tasks.rados.rados.0.smithi118.stdout:957: copy_from oid 25 from oid 31 current snap is 100 2024-03-20T08:04:41.771 INFO:tasks.rados.rados.0.smithi118.stdout:958: read oid 49 snap 78 2024-03-20T08:04:41.771 INFO:tasks.rados.rados.0.smithi118.stdout:958: expect (ObjNum 176 snap 52 seq_num 176) 2024-03-20T08:04:41.772 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 100 2024-03-20T08:04:41.772 INFO:tasks.rados.rados.0.smithi118.stdout:959: seq_num 302 ranges {3407872=262144} 2024-03-20T08:04:41.790 INFO:tasks.rados.rados.0.smithi118.stdout:959: writing smithi11881747-23 from 3407872 to 3670016 tid 1 2024-03-20T08:04:41.790 INFO:tasks.rados.rados.0.smithi118.stdout:960: copy_from oid 32 from oid 42 current snap is 100 2024-03-20T08:04:41.790 INFO:tasks.rados.rados.0.smithi118.stdout:957: finishing copy_from to smithi11881747-25 2024-03-20T08:04:41.790 INFO:tasks.rados.rados.0.smithi118.stdout:957: got expected ENOENT (src dne) 2024-03-20T08:04:41.790 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 100 2024-03-20T08:04:41.790 INFO:tasks.rados.rados.0.smithi118.stdout:957: finishing copy_from racing read to smithi11881747-25 2024-03-20T08:04:41.790 INFO:tasks.rados.rados.0.smithi118.stdout:958: expect (ObjNum 176 snap 52 seq_num 176) 2024-03-20T08:04:41.990 INFO:tasks.rados.rados.0.smithi118.stdout:959: finishing write tid 1 to smithi11881747-23 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:959: finishing write tid 2 to smithi11881747-23 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:960: finishing copy_from racing read to smithi11881747-32 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:960: finishing copy_from to smithi11881747-32 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 560 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:959: finishing write tid 3 to smithi11881747-23 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:959: oid 23 updating version 0 to 755 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:959: oid 23 updating version 755 to 756 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 756 (ObjNum 302 snap 100 seq_num 302) dirty exists 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:959: left oid 23 (ObjNum 302 snap 100 seq_num 302) 2024-03-20T08:04:41.991 INFO:tasks.rados.rados.0.smithi118.stdout:961: seq_num 303 ranges {1589248=278528} 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:961: writing smithi11881747-4 from 1589248 to 1867776 tid 1 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:957: done (4 left) 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:958: done (3 left) 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:959: done (2 left) 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:960: done (1 left) 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:962: read oid 19 snap 99 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:962: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:963: read oid 42 snap -1 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:963: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:964: read oid 7 snap 75 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:964: expect (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 100 2024-03-20T08:04:42.001 INFO:tasks.rados.rados.0.smithi118.stdout:965: seq_num 304 ranges {32768=294912} 2024-03-20T08:04:42.002 INFO:tasks.rados.rados.0.smithi118.stdout:965: writing smithi11881747-13 from 32768 to 327680 tid 1 2024-03-20T08:04:42.003 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 100 2024-03-20T08:04:42.003 INFO:tasks.rados.rados.0.smithi118.stdout:966: seq_num 305 ranges {1851392=311296} 2024-03-20T08:04:42.015 INFO:tasks.rados.rados.0.smithi118.stdout:966: writing smithi11881747-25 from 1851392 to 2162688 tid 1 2024-03-20T08:04:42.015 INFO:tasks.rados.rados.0.smithi118.stdout:967: setattr oid 26 current snap is 100 2024-03-20T08:04:42.015 INFO:tasks.rados.rados.0.smithi118.stdout:964: expect (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:04:42.021 INFO:tasks.rados.rados.0.smithi118.stdout:961: finishing write tid 1 to smithi11881747-4 2024-03-20T08:04:42.021 INFO:tasks.rados.rados.0.smithi118.stdout:963: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:961: finishing write tid 2 to smithi11881747-4 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:961: finishing write tid 3 to smithi11881747-4 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:961: oid 4 updating version 0 to 286 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:961: oid 4 updating version 286 to 287 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 287 (ObjNum 303 snap 100 seq_num 303) dirty exists 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:961: left oid 4 (ObjNum 303 snap 100 seq_num 303) 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:965: finishing write tid 1 to smithi11881747-13 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:965: finishing write tid 2 to smithi11881747-13 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:965: finishing write tid 3 to smithi11881747-13 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:965: oid 13 updating version 0 to 566 2024-03-20T08:04:42.040 INFO:tasks.rados.rados.0.smithi118.stdout:965: oid 13 version 566 is already newer than 565 2024-03-20T08:04:42.041 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 566 (ObjNum 304 snap 100 seq_num 304) dirty exists 2024-03-20T08:04:42.041 INFO:tasks.rados.rados.0.smithi118.stdout:965: left oid 13 (ObjNum 304 snap 100 seq_num 304) 2024-03-20T08:04:42.041 INFO:tasks.rados.rados.0.smithi118.stdout:962: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:04:42.150 INFO:tasks.rados.rados.0.smithi118.stdout:966: finishing write tid 1 to smithi11881747-25 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:966: finishing write tid 2 to smithi11881747-25 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:966: finishing write tid 3 to smithi11881747-25 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:966: oid 25 updating version 0 to 568 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:966: oid 25 version 568 is already newer than 567 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 568 (ObjNum 305 snap 100 seq_num 305) dirty exists 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:966: left oid 25 (ObjNum 305 snap 100 seq_num 305) 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:961: done (6 left) 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:962: done (5 left) 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:963: done (4 left) 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:964: done (3 left) 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:965: done (2 left) 2024-03-20T08:04:42.151 INFO:tasks.rados.rados.0.smithi118.stdout:966: done (1 left) 2024-03-20T08:04:42.152 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 40 current snap is 100 2024-03-20T08:04:42.152 INFO:tasks.rados.rados.0.smithi118.stdout:968: seq_num 306 ranges {1597440=327680} 2024-03-20T08:04:42.160 INFO:tasks.rados.rados.0.smithi118.stdout:968: writing smithi11881747-40 from 1597440 to 1925120 tid 1 2024-03-20T08:04:42.160 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 100 2024-03-20T08:04:42.160 INFO:tasks.rados.rados.0.smithi118.stdout:969: seq_num 307 ranges {3776512=344064} 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:969: writing smithi11881747-11 from 3776512 to 4120576 tid 1 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 100 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 355 (ObjNum 266 snap 92 seq_num 266) dirty exists 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:968: finishing write tid 1 to smithi11881747-40 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:968: finishing write tid 2 to smithi11881747-40 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:968: finishing write tid 3 to smithi11881747-40 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:968: oid 40 updating version 0 to 254 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:968: oid 40 version 254 is already newer than 253 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 254 (ObjNum 306 snap 100 seq_num 306) dirty exists 2024-03-20T08:04:42.183 INFO:tasks.rados.rados.0.smithi118.stdout:968: left oid 40 (ObjNum 306 snap 100 seq_num 306) 2024-03-20T08:04:42.184 INFO:tasks.rados.rados.0.smithi118.stdout:970: seq_num 308 ranges {0=360448} 2024-03-20T08:04:42.184 INFO:tasks.rados.rados.0.smithi118.stdout:970: writing smithi11881747-3 from 0 to 360448 tid 1 2024-03-20T08:04:42.185 INFO:tasks.rados.rados.0.smithi118.stdout:967: done (3 left) 2024-03-20T08:04:42.185 INFO:tasks.rados.rados.0.smithi118.stdout:968: done (2 left) 2024-03-20T08:04:42.185 INFO:tasks.rados.rados.0.smithi118.stdout:971: snap_remove snap 81 2024-03-20T08:04:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:42.764 INFO:tasks.rados.rados.0.smithi118.stdout:971: done (2 left) 2024-03-20T08:04:42.764 INFO:tasks.rados.rados.0.smithi118.stdout:972: setattr oid 39 current snap is 100 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:969: finishing write tid 1 to smithi11881747-11 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:969: finishing write tid 2 to smithi11881747-11 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:970: finishing write tid 1 to smithi11881747-3 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:969: finishing write tid 3 to smithi11881747-11 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:969: oid 11 updating version 0 to 570 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:969: oid 11 version 570 is already newer than 569 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 570 (ObjNum 307 snap 100 seq_num 307) dirty exists 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:969: left oid 11 (ObjNum 307 snap 100 seq_num 307) 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:970: finishing write tid 2 to smithi11881747-3 2024-03-20T08:04:42.766 INFO:tasks.rados.rados.0.smithi118.stdout:970: finishing write tid 3 to smithi11881747-3 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:970: oid 3 updating version 0 to 241 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:970: oid 3 updating version 241 to 242 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 242 (ObjNum 308 snap 100 seq_num 308) dirty exists 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:970: left oid 3 (ObjNum 308 snap 100 seq_num 308) 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:969: done (2 left) 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:970: done (1 left) 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:973: rollback oid 24 current snap is 100 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 69 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:974: read oid 32 snap -1 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:974: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:04:42.767 INFO:tasks.rados.rados.0.smithi118.stdout:975: snap_create 2024-03-20T08:04:42.776 INFO:tasks.rados.rados.0.smithi118.stdout:973: finishing rollback tid 0 to smithi11881747-24 2024-03-20T08:04:42.776 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 532 (ObjNum 81 snap 0 seq_num 1650617197) dirty exists 2024-03-20T08:04:42.776 INFO:tasks.rados.rados.0.smithi118.stdout:973: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:04:42.776 INFO:tasks.rados.rados.0.smithi118.stdout:973: finishing rollback tid 2 to smithi11881747-24 2024-03-20T08:04:42.776 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 363 (ObjNum 216 snap 69 seq_num 216) dirty exists 2024-03-20T08:04:42.781 INFO:tasks.rados.rados.0.smithi118.stdout:974: expect (ObjNum 41 snap 0 seq_num 41) 2024-03-20T08:04:42.936 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:42 smithi196 ceph-mon[87080]: osdmap e255: 8 total, 8 up, 8 in 2024-03-20T08:04:42.937 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:42 smithi196 ceph-mon[87080]: pgmap v59: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 104 KiB/s wr, 9 op/s 2024-03-20T08:04:43.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:42 smithi118 ceph-mon[93121]: osdmap e255: 8 total, 8 up, 8 in 2024-03-20T08:04:43.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:42 smithi118 ceph-mon[93121]: pgmap v59: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 104 KiB/s wr, 9 op/s 2024-03-20T08:04:43.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:42 smithi118 ceph-mon[95692]: osdmap e255: 8 total, 8 up, 8 in 2024-03-20T08:04:43.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:42 smithi118 ceph-mon[95692]: pgmap v59: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.4 MiB/s rd, 104 KiB/s wr, 9 op/s 2024-03-20T08:04:43.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:43.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:43.768 INFO:tasks.rados.rados.0.smithi118.stdout:972: done (3 left) 2024-03-20T08:04:43.768 INFO:tasks.rados.rados.0.smithi118.stdout:973: done (2 left) 2024-03-20T08:04:43.768 INFO:tasks.rados.rados.0.smithi118.stdout:974: done (1 left) 2024-03-20T08:04:43.768 INFO:tasks.rados.rados.0.smithi118.stdout:975: done (0 left) 2024-03-20T08:04:43.768 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 101 2024-03-20T08:04:43.769 INFO:tasks.rados.rados.0.smithi118.stdout:976: seq_num 309 ranges {0=376832} 2024-03-20T08:04:43.770 INFO:tasks.rados.rados.0.smithi118.stdout:976: writing smithi11881747-37 from 0 to 376832 tid 1 2024-03-20T08:04:43.770 INFO:tasks.rados.rados.0.smithi118.stdout:977: rmattr oid 2 current snap is 101 2024-03-20T08:04:43.770 INFO:tasks.rados.rados.0.smithi118.stdout:978: read oid 50 snap 64 2024-03-20T08:04:43.771 INFO:tasks.rados.rados.0.smithi118.stdout:978: expect deleted 2024-03-20T08:04:43.771 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 101 2024-03-20T08:04:43.771 INFO:tasks.rados.rados.0.smithi118.stdout:979: seq_num 310 ranges {360448=401408} 2024-03-20T08:04:43.775 INFO:tasks.rados.rados.0.smithi118.stdout:979: writing smithi11881747-3 from 360448 to 761856 tid 1 2024-03-20T08:04:43.775 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 101 2024-03-20T08:04:43.775 INFO:tasks.rados.rados.0.smithi118.stdout:980: seq_num 311 ranges {1867776=417792} 2024-03-20T08:04:43.788 INFO:tasks.rados.rados.0.smithi118.stdout:980: writing smithi11881747-10 from 1867776 to 2285568 tid 1 2024-03-20T08:04:43.788 INFO:tasks.rados.rados.0.smithi118.stdout:981: rmattr oid 27 current snap is 101 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:976: finishing write tid 1 to smithi11881747-37 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 400 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:977: done (5 left) 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:978: done (4 left) 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:982: setattr oid 38 current snap is 101 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:976: finishing write tid 2 to smithi11881747-37 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:976: finishing write tid 3 to smithi11881747-37 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:976: oid 37 updating version 0 to 255 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:976: oid 37 updating version 255 to 256 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 256 (ObjNum 309 snap 101 seq_num 309) dirty exists 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:976: left oid 37 (ObjNum 309 snap 101 seq_num 309) 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:979: finishing write tid 1 to smithi11881747-3 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:979: finishing write tid 2 to smithi11881747-3 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:979: finishing write tid 3 to smithi11881747-3 2024-03-20T08:04:43.789 INFO:tasks.rados.rados.0.smithi118.stdout:979: oid 3 updating version 0 to 248 2024-03-20T08:04:43.790 INFO:tasks.rados.rados.0.smithi118.stdout:979: oid 3 updating version 248 to 249 2024-03-20T08:04:43.790 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 249 (ObjNum 310 snap 101 seq_num 310) dirty exists 2024-03-20T08:04:43.790 INFO:tasks.rados.rados.0.smithi118.stdout:979: left oid 3 (ObjNum 310 snap 101 seq_num 310) 2024-03-20T08:04:43.790 INFO:tasks.rados.rados.0.smithi118.stdout:976: done (4 left) 2024-03-20T08:04:43.790 INFO:tasks.rados.rados.0.smithi118.stdout:979: done (3 left) 2024-03-20T08:04:43.790 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 101 2024-03-20T08:04:43.790 INFO:tasks.rados.rados.0.smithi118.stdout:983: seq_num 312 ranges {180224=434176} 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:983: writing smithi11881747-18 from 180224 to 614400 tid 1 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:984: delete oid 7 current snap is 101 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:980: finishing write tid 1 to smithi11881747-10 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:980: finishing write tid 2 to smithi11881747-10 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 556 (ObjNum 272 snap 92 seq_num 272) dirty exists 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 536 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:980: finishing write tid 3 to smithi11881747-10 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:980: oid 10 updating version 0 to 447 2024-03-20T08:04:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:980: oid 10 updating version 447 to 448 2024-03-20T08:04:43.794 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 448 (ObjNum 311 snap 101 seq_num 311) dirty exists 2024-03-20T08:04:43.794 INFO:tasks.rados.rados.0.smithi118.stdout:980: left oid 10 (ObjNum 311 snap 101 seq_num 311) 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:980: done (4 left) 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:981: done (3 left) 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:982: done (2 left) 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:984: done (1 left) 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:985: read oid 5 snap -1 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:985: expect deleted 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:986: rmattr oid 32 current snap is 101 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:986: done (2 left) 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:987: rmattr oid 47 current snap is 101 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:987: done (2 left) 2024-03-20T08:04:43.796 INFO:tasks.rados.rados.0.smithi118.stdout:988: setattr oid 47 current snap is 101 2024-03-20T08:04:43.797 INFO:tasks.rados.rados.0.smithi118.stdout:983: finishing write tid 1 to smithi11881747-18 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:983: finishing write tid 2 to smithi11881747-18 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:985: done (2 left) 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:989: read oid 39 snap 73 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:989: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:990: delete oid 3 current snap is 101 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:983: finishing write tid 3 to smithi11881747-18 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:983: oid 18 updating version 0 to 539 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:983: oid 18 version 539 is already newer than 538 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 539 (ObjNum 312 snap 101 seq_num 312) dirty exists 2024-03-20T08:04:43.798 INFO:tasks.rados.rados.0.smithi118.stdout:983: left oid 18 (ObjNum 312 snap 101 seq_num 312) 2024-03-20T08:04:43.800 INFO:tasks.rados.rados.0.smithi118.stdout:983: done (3 left) 2024-03-20T08:04:43.800 INFO:tasks.rados.rados.0.smithi118.stdout:990: done (2 left) 2024-03-20T08:04:43.800 INFO:tasks.rados.rados.0.smithi118.stdout:991: copy_from oid 21 from oid 34 current snap is 101 2024-03-20T08:04:43.800 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 101 2024-03-20T08:04:43.800 INFO:tasks.rados.rados.0.smithi118.stdout:992: seq_num 315 ranges {1187840=483328} 2024-03-20T08:04:43.810 INFO:tasks.rados.rados.0.smithi118.stdout:992: writing smithi11881747-16 from 1187840 to 1671168 tid 1 2024-03-20T08:04:43.810 INFO:tasks.rados.rados.0.smithi118.stdout:993: setattr oid 13 current snap is 101 2024-03-20T08:04:43.810 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 767 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-03-20T08:04:43.810 INFO:tasks.rados.rados.0.smithi118.stdout:989: expect (ObjNum 191 snap 61 seq_num 191) 2024-03-20T08:04:43.900 INFO:tasks.rados.rados.0.smithi118.stdout:992: finishing write tid 1 to smithi11881747-16 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:992: finishing write tid 2 to smithi11881747-16 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:992: finishing write tid 3 to smithi11881747-16 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:992: oid 16 updating version 0 to 770 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:992: oid 16 version 770 is already newer than 769 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 770 (ObjNum 315 snap 101 seq_num 315) dirty exists 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:992: left oid 16 (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:991: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:991: finishing copy_from to smithi11881747-21 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 193 (ObjNum 285 snap 96 seq_num 285) dirty exists 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:988: done (4 left) 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:989: done (3 left) 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:991: done (2 left) 2024-03-20T08:04:43.901 INFO:tasks.rados.rados.0.smithi118.stdout:992: done (1 left) 2024-03-20T08:04:43.902 INFO:tasks.rados.rados.0.smithi118.stdout:994: snap_create 2024-03-20T08:04:43.904 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 571 (ObjNum 304 snap 100 seq_num 304) dirty exists 2024-03-20T08:04:43.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:43 smithi118 ceph-mon[93121]: osdmap e256: 8 total, 8 up, 8 in 2024-03-20T08:04:43.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:43.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:04:43] "GET /metrics HTTP/1.1" 200 36063 "" "Prometheus/2.43.0" 2024-03-20T08:04:43.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:43 smithi118 ceph-mon[95692]: osdmap e256: 8 total, 8 up, 8 in 2024-03-20T08:04:43.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:44.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:43 smithi196 ceph-mon[87080]: osdmap e256: 8 total, 8 up, 8 in 2024-03-20T08:04:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:44.775 INFO:tasks.rados.rados.0.smithi118.stdout:993: done (1 left) 2024-03-20T08:04:44.775 INFO:tasks.rados.rados.0.smithi118.stdout:994: done (0 left) 2024-03-20T08:04:44.775 INFO:tasks.rados.rados.0.smithi118.stdout:995: snap_create 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: osdmap e257: 8 total, 8 up, 8 in 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: pgmap v62: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: osdmap e257: 8 total, 8 up, 8 in 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: pgmap v62: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: osdmap e257: 8 total, 8 up, 8 in 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: pgmap v62: 121 pgs: 121 active+clean; 264 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:45.776 INFO:tasks.rados.rados.0.smithi118.stdout:995: done (0 left) 2024-03-20T08:04:45.776 INFO:tasks.rados.rados.0.smithi118.stdout:996: copy_from oid 10 from oid 16 current snap is 103 2024-03-20T08:04:45.776 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 5 current snap is 103 2024-03-20T08:04:45.776 INFO:tasks.rados.rados.0.smithi118.stdout:997: seq_num 316 ranges {0=499712} 2024-03-20T08:04:45.779 INFO:tasks.rados.rados.0.smithi118.stdout:997: writing smithi11881747-5 from 0 to 499712 tid 1 2024-03-20T08:04:45.779 INFO:tasks.rados.rados.0.smithi118.stdout:998: read oid 23 snap 77 2024-03-20T08:04:45.779 INFO:tasks.rados.rados.0.smithi118.stdout:998: expect (ObjNum 213 snap 67 seq_num 213) 2024-03-20T08:04:45.779 INFO:tasks.rados.rados.0.smithi118.stdout:999: read oid 35 snap 73 2024-03-20T08:04:45.780 INFO:tasks.rados.rados.0.smithi118.stdout:999: expect (ObjNum 323097008 snap 21898 seq_num 322552368) 2024-03-20T08:04:45.780 INFO:tasks.rados.rados.0.smithi118.stdout:1000: snap_create 2024-03-20T08:04:45.780 INFO:tasks.rados.rados.0.smithi118.stdout:999: expect (ObjNum 323097008 snap 21898 seq_num 322552368) 2024-03-20T08:04:45.787 INFO:tasks.rados.rados.0.smithi118.stdout:998: expect (ObjNum 213 snap 67 seq_num 213) 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:997: finishing write tid 1 to smithi11881747-5 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:996: finishing copy_from racing read to smithi11881747-10 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:997: finishing write tid 2 to smithi11881747-5 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:996: finishing copy_from to smithi11881747-10 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 770 (ObjNum 315 snap 101 seq_num 315) dirty exists 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:997: finishing write tid 3 to smithi11881747-5 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:997: oid 5 updating version 0 to 573 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:997: oid 5 version 573 is already newer than 572 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 573 (ObjNum 316 snap 103 seq_num 316) dirty exists 2024-03-20T08:04:45.852 INFO:tasks.rados.rados.0.smithi118.stdout:997: left oid 5 (ObjNum 316 snap 103 seq_num 316) 2024-03-20T08:04:46.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:45 smithi196 ceph-mon[87080]: Reconfiguring mon.a (monmap changed)... 2024-03-20T08:04:46.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:45 smithi196 ceph-mon[87080]: Reconfiguring daemon mon.a on smithi118 2024-03-20T08:04:46.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:45 smithi196 ceph-mon[87080]: osdmap e258: 8 total, 8 up, 8 in 2024-03-20T08:04:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:45 smithi118 ceph-mon[93121]: Reconfiguring mon.a (monmap changed)... 2024-03-20T08:04:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:45 smithi118 ceph-mon[93121]: Reconfiguring daemon mon.a on smithi118 2024-03-20T08:04:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:45 smithi118 ceph-mon[93121]: osdmap e258: 8 total, 8 up, 8 in 2024-03-20T08:04:46.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:45 smithi118 ceph-mon[95692]: Reconfiguring mon.a (monmap changed)... 2024-03-20T08:04:46.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:45 smithi118 ceph-mon[95692]: Reconfiguring daemon mon.a on smithi118 2024-03-20T08:04:46.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:45 smithi118 ceph-mon[95692]: osdmap e258: 8 total, 8 up, 8 in 2024-03-20T08:04:46.786 INFO:tasks.rados.rados.0.smithi118.stdout:996: done (4 left) 2024-03-20T08:04:46.787 INFO:tasks.rados.rados.0.smithi118.stdout:997: done (3 left) 2024-03-20T08:04:46.787 INFO:tasks.rados.rados.0.smithi118.stdout:998: done (2 left) 2024-03-20T08:04:46.787 INFO:tasks.rados.rados.0.smithi118.stdout:999: done (1 left) 2024-03-20T08:04:46.787 INFO:tasks.rados.rados.0.smithi118.stdout:1000: done (0 left) 2024-03-20T08:04:46.787 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 104 2024-03-20T08:04:46.788 INFO:tasks.rados.rados.0.smithi118.stdout:1001: seq_num 317 ranges {327680=516096} 2024-03-20T08:04:46.792 INFO:tasks.rados.rados.0.smithi118.stdout:1001: writing smithi11881747-13 from 327680 to 843776 tid 1 2024-03-20T08:04:46.792 INFO:tasks.rados.rados.0.smithi118.stdout:1002: read oid 46 snap -1 2024-03-20T08:04:46.792 INFO:tasks.rados.rados.0.smithi118.stdout:1002: expect (ObjNum 261 snap 90 seq_num 261) 2024-03-20T08:04:46.792 INFO:tasks.rados.rados.0.smithi118.stdout:1003: read oid 41 snap -1 2024-03-20T08:04:46.792 INFO:tasks.rados.rados.0.smithi118.stdout:1003: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:04:46.793 INFO:tasks.rados.rados.0.smithi118.stdout:1004: copy_from oid 4 from oid 19 current snap is 104 2024-03-20T08:04:46.793 INFO:tasks.rados.rados.0.smithi118.stdout:1005: copy_from oid 9 from oid 44 current snap is 104 2024-03-20T08:04:46.793 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 104 2024-03-20T08:04:46.793 INFO:tasks.rados.rados.0.smithi118.stdout:1006: seq_num 318 ranges {2015232=532480} 2024-03-20T08:04:46.806 INFO:tasks.rados.rados.0.smithi118.stdout:1006: writing smithi11881747-20 from 2015232 to 2547712 tid 1 2024-03-20T08:04:46.807 INFO:tasks.rados.rados.0.smithi118.stdout:1007: snap_remove snap 86 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[93121]: osdmap e259: 8 total, 8 up, 8 in 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[93121]: pgmap v65: 121 pgs: 121 active+clean; 276 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[95692]: osdmap e259: 8 total, 8 up, 8 in 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[95692]: pgmap v65: 121 pgs: 121 active+clean; 276 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-03-20T08:04:47.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:47.080 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:47.080 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:04:47.080 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:04:47.080 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:46 smithi196 ceph-mon[87080]: osdmap e259: 8 total, 8 up, 8 in 2024-03-20T08:04:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:46 smithi196 ceph-mon[87080]: pgmap v65: 121 pgs: 121 active+clean; 276 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-03-20T08:04:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:04:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:04:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:47.794 INFO:tasks.rados.rados.0.smithi118.stdout:1007: done (6 left) 2024-03-20T08:04:47.795 INFO:tasks.rados.rados.0.smithi118.stdout:1008: rmattr oid 47 current snap is 104 2024-03-20T08:04:47.795 INFO:tasks.rados.rados.0.smithi118.stdout:1009: rollback oid 38 current snap is 104 2024-03-20T08:04:47.795 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 38 to 94 2024-03-20T08:04:47.795 INFO:tasks.rados.rados.0.smithi118.stdout:1010: delete oid 39 current snap is 104 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1001: finishing write tid 1 to smithi11881747-13 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1001: finishing write tid 2 to smithi11881747-13 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1005: finishing copy_from racing read to smithi11881747-9 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1005: finishing copy_from to smithi11881747-9 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 339 (ObjNum 65 snap 0 seq_num 1902991450) dirty exists 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1001: finishing write tid 3 to smithi11881747-13 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1001: oid 13 updating version 0 to 575 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1001: oid 13 version 575 is already newer than 574 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 575 (ObjNum 317 snap 104 seq_num 317) dirty exists 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1001: left oid 13 (ObjNum 317 snap 104 seq_num 317) 2024-03-20T08:04:47.796 INFO:tasks.rados.rados.0.smithi118.stdout:1002: expect (ObjNum 261 snap 90 seq_num 261) 2024-03-20T08:04:47.856 INFO:tasks.rados.rados.0.smithi118.stdout:1003: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:04:47.968 INFO:tasks.rados.rados.0.smithi118.stdout:1004: finishing copy_from racing read to smithi11881747-4 2024-03-20T08:04:47.968 INFO:tasks.rados.rados.0.smithi118.stdout:1004: finishing copy_from to smithi11881747-4 2024-03-20T08:04:47.968 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1006: finishing write tid 1 to smithi11881747-20 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1006: finishing write tid 2 to smithi11881747-20 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1006: finishing write tid 3 to smithi11881747-20 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1006: oid 20 updating version 0 to 542 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1006: oid 20 version 542 is already newer than 541 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 542 (ObjNum 318 snap 104 seq_num 318) dirty exists 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1006: left oid 20 (ObjNum 318 snap 104 seq_num 318) 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1001: done (8 left) 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1002: done (7 left) 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1003: done (6 left) 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1004: done (5 left) 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1005: done (4 left) 2024-03-20T08:04:47.969 INFO:tasks.rados.rados.0.smithi118.stdout:1006: done (3 left) 2024-03-20T08:04:47.970 INFO:tasks.rados.rados.0.smithi118.stdout:1010: done (2 left) 2024-03-20T08:04:47.970 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 104 2024-03-20T08:04:47.970 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 772 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:04:47.970 INFO:tasks.rados.rados.0.smithi118.stdout:1009: finishing rollback tid 0 to smithi11881747-38 2024-03-20T08:04:47.970 INFO:tasks.rados.rados.0.smithi118.stdout:1009: finishing rollback tid 1 to smithi11881747-38 2024-03-20T08:04:47.970 INFO:tasks.rados.rados.0.smithi118.stdout:1011: seq_num 320 ranges {0=565248} 2024-03-20T08:04:47.971 INFO:tasks.rados.rados.0.smithi118.stdout:1011: writing smithi11881747-30 from 0 to 565248 tid 1 2024-03-20T08:04:47.971 INFO:tasks.rados.rados.0.smithi118.stdout:1008: done (2 left) 2024-03-20T08:04:47.971 INFO:tasks.rados.rados.0.smithi118.stdout:1012: read oid 14 snap -1 2024-03-20T08:04:47.971 INFO:tasks.rados.rados.0.smithi118.stdout:1012: expect (ObjNum 235 snap 82 seq_num 235) 2024-03-20T08:04:47.971 INFO:tasks.rados.rados.0.smithi118.stdout:1013: snap_remove snap 95 2024-03-20T08:04:48.100 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:47 smithi118 ceph-mon[95692]: Reconfiguring mgr.y (monmap changed)... 2024-03-20T08:04:48.100 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:47 smithi118 ceph-mon[95692]: Reconfiguring daemon mgr.y on smithi118 2024-03-20T08:04:48.100 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:47 smithi118 ceph-mon[95692]: osdmap e260: 8 total, 8 up, 8 in 2024-03-20T08:04:48.100 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:47 smithi118 ceph-mon[93121]: Reconfiguring mgr.y (monmap changed)... 2024-03-20T08:04:48.101 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:47 smithi118 ceph-mon[93121]: Reconfiguring daemon mgr.y on smithi118 2024-03-20T08:04:48.101 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:47 smithi118 ceph-mon[93121]: osdmap e260: 8 total, 8 up, 8 in 2024-03-20T08:04:48.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:47 smithi196 ceph-mon[87080]: Reconfiguring mgr.y (monmap changed)... 2024-03-20T08:04:48.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:47 smithi196 ceph-mon[87080]: Reconfiguring daemon mgr.y on smithi118 2024-03-20T08:04:48.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:47 smithi196 ceph-mon[87080]: osdmap e260: 8 total, 8 up, 8 in 2024-03-20T08:04:48.798 INFO:tasks.rados.rados.0.smithi118.stdout:1013: done (3 left) 2024-03-20T08:04:48.798 INFO:tasks.rados.rados.0.smithi118.stdout:1014: read oid 37 snap -1 2024-03-20T08:04:48.798 INFO:tasks.rados.rados.0.smithi118.stdout:1014: expect (ObjNum 309 snap 101 seq_num 309) 2024-03-20T08:04:48.798 INFO:tasks.rados.rados.0.smithi118.stdout:1015: rollback oid 26 current snap is 104 2024-03-20T08:04:48.798 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 94 2024-03-20T08:04:48.798 INFO:tasks.rados.rados.0.smithi118.stdout:1009: finishing rollback tid 2 to smithi11881747-38 2024-03-20T08:04:48.798 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 559 (ObjNum 272 snap 92 seq_num 272) dirty exists 2024-03-20T08:04:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:1011: finishing write tid 1 to smithi11881747-30 2024-03-20T08:04:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:1011: finishing write tid 2 to smithi11881747-30 2024-03-20T08:04:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:1011: finishing write tid 3 to smithi11881747-30 2024-03-20T08:04:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:1011: oid 30 updating version 0 to 288 2024-03-20T08:04:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:1011: oid 30 updating version 288 to 289 2024-03-20T08:04:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 289 (ObjNum 320 snap 104 seq_num 320) dirty exists 2024-03-20T08:04:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:1011: left oid 30 (ObjNum 320 snap 104 seq_num 320) 2024-03-20T08:04:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:1012: expect (ObjNum 235 snap 82 seq_num 235) 2024-03-20T08:04:48.893 INFO:tasks.rados.rados.0.smithi118.stdout:1009: done (4 left) 2024-03-20T08:04:48.893 INFO:tasks.rados.rados.0.smithi118.stdout:1011: done (3 left) 2024-03-20T08:04:48.893 INFO:tasks.rados.rados.0.smithi118.stdout:1012: done (2 left) 2024-03-20T08:04:48.893 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 104 2024-03-20T08:04:48.893 INFO:tasks.rados.rados.0.smithi118.stdout:1016: seq_num 321 ranges {4947968=581632} 2024-03-20T08:04:48.923 INFO:tasks.rados.rados.0.smithi118.stdout:1016: writing smithi11881747-38 from 4947968 to 5529600 tid 1 2024-03-20T08:04:48.923 INFO:tasks.rados.rados.0.smithi118.stdout:1017: setattr oid 27 current snap is 104 2024-03-20T08:04:48.923 INFO:tasks.rados.rados.0.smithi118.stdout:1014: expect (ObjNum 309 snap 101 seq_num 309) 2024-03-20T08:04:48.933 INFO:tasks.rados.rados.0.smithi118.stdout:1015: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1015: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1015: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 371 (ObjNum 266 snap 92 seq_num 266) dirty exists 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1016: finishing write tid 1 to smithi11881747-38 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1016: finishing write tid 2 to smithi11881747-38 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1016: finishing write tid 3 to smithi11881747-38 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1016: oid 38 updating version 0 to 561 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1016: oid 38 version 561 is already newer than 560 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 561 (ObjNum 321 snap 104 seq_num 321) dirty exists 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1016: left oid 38 (ObjNum 321 snap 104 seq_num 321) 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1014: done (3 left) 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1015: done (2 left) 2024-03-20T08:04:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1016: done (1 left) 2024-03-20T08:04:48.935 INFO:tasks.rados.rados.0.smithi118.stdout:1018: setattr oid 9 current snap is 104 2024-03-20T08:04:48.935 INFO:tasks.rados.rados.0.smithi118.stdout:1019: rmattr oid 11 current snap is 104 2024-03-20T08:04:48.935 INFO:tasks.rados.rados.0.smithi118.stdout:1020: read oid 43 snap -1 2024-03-20T08:04:48.935 INFO:tasks.rados.rados.0.smithi118.stdout:1020: expect (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:04:48.936 INFO:tasks.rados.rados.0.smithi118.stdout:1021: rmattr oid 16 current snap is 104 2024-03-20T08:04:48.936 INFO:tasks.rados.rados.0.smithi118.stdout:1021: done (4 left) 2024-03-20T08:04:48.936 INFO:tasks.rados.rados.0.smithi118.stdout:1022: setattr oid 23 current snap is 104 2024-03-20T08:04:48.937 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 560 (ObjNum 65 snap 0 seq_num 1601066321) dirty exists 2024-03-20T08:04:48.937 INFO:tasks.rados.rados.0.smithi118.stdout:1017: done (4 left) 2024-03-20T08:04:48.937 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 104 2024-03-20T08:04:48.937 INFO:tasks.rados.rados.0.smithi118.stdout:1023: seq_num 322 ranges {3391488=598016} 2024-03-20T08:04:48.959 INFO:tasks.rados.rados.0.smithi118.stdout:1023: writing smithi11881747-49 from 3391488 to 3989504 tid 1 2024-03-20T08:04:48.960 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 104 2024-03-20T08:04:48.960 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 590 (ObjNum 65 snap 0 seq_num 1885893734) dirty exists 2024-03-20T08:04:48.960 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 592 (ObjNum 307 snap 100 seq_num 307) dirty exists 2024-03-20T08:04:48.960 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 794 (ObjNum 302 snap 100 seq_num 302) dirty exists 2024-03-20T08:04:48.960 INFO:tasks.rados.rados.0.smithi118.stdout:1024: seq_num 323 ranges {843776=614400} 2024-03-20T08:04:48.968 INFO:tasks.rados.rados.0.smithi118.stdout:1024: writing smithi11881747-13 from 843776 to 1458176 tid 1 2024-03-20T08:04:48.968 INFO:tasks.rados.rados.0.smithi118.stdout:1018: done (5 left) 2024-03-20T08:04:48.968 INFO:tasks.rados.rados.0.smithi118.stdout:1019: done (4 left) 2024-03-20T08:04:48.968 INFO:tasks.rados.rados.0.smithi118.stdout:1022: done (3 left) 2024-03-20T08:04:48.968 INFO:tasks.rados.rados.0.smithi118.stdout:1025: setattr oid 11 current snap is 104 2024-03-20T08:04:48.968 INFO:tasks.rados.rados.0.smithi118.stdout:1020: expect (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:04:49.020 INFO:tasks.rados.rados.0.smithi118.stdout:1023: finishing write tid 1 to smithi11881747-49 2024-03-20T08:04:49.020 INFO:tasks.rados.rados.0.smithi118.stdout:1023: finishing write tid 2 to smithi11881747-49 2024-03-20T08:04:49.020 INFO:tasks.rados.rados.0.smithi118.stdout:1023: finishing write tid 3 to smithi11881747-49 2024-03-20T08:04:49.020 INFO:tasks.rados.rados.0.smithi118.stdout:1023: oid 49 updating version 0 to 563 2024-03-20T08:04:49.020 INFO:tasks.rados.rados.0.smithi118.stdout:1023: oid 49 version 563 is already newer than 562 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 563 (ObjNum 322 snap 104 seq_num 322) dirty exists 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1023: left oid 49 (ObjNum 322 snap 104 seq_num 322) 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1024: finishing write tid 1 to smithi11881747-13 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1024: finishing write tid 2 to smithi11881747-13 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1024: finishing write tid 3 to smithi11881747-13 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1024: oid 13 updating version 0 to 594 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1024: oid 13 version 594 is already newer than 593 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 594 (ObjNum 323 snap 104 seq_num 323) dirty exists 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1024: left oid 13 (ObjNum 323 snap 104 seq_num 323) 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1020: done (3 left) 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1023: done (2 left) 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1024: done (1 left) 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1026: read oid 45 snap -1 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1026: expect (ObjNum 196 snap 64 seq_num 196) 2024-03-20T08:04:49.021 INFO:tasks.rados.rados.0.smithi118.stdout:1027: delete oid 29 current snap is 104 2024-03-20T08:04:49.022 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 595 (ObjNum 307 snap 100 seq_num 307) dirty exists 2024-03-20T08:04:49.023 INFO:tasks.rados.rados.0.smithi118.stdout:1025: done (2 left) 2024-03-20T08:04:49.023 INFO:tasks.rados.rados.0.smithi118.stdout:1027: done (1 left) 2024-03-20T08:04:49.023 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 104 2024-03-20T08:04:49.023 INFO:tasks.rados.rados.0.smithi118.stdout:1028: seq_num 325 ranges {0=647168} 2024-03-20T08:04:49.027 INFO:tasks.rados.rados.0.smithi118.stdout:1028: writing smithi11881747-9 from 0 to 647168 tid 1 2024-03-20T08:04:49.027 INFO:tasks.rados.rados.0.smithi118.stdout:1029: snap_create 2024-03-20T08:04:49.029 INFO:tasks.rados.rados.0.smithi118.stdout:1026: expect (ObjNum 196 snap 64 seq_num 196) 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: osdmap e261: 8 total, 8 up, 8 in 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: Reconfiguring mon.c (monmap changed)... 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: Reconfiguring daemon mon.c on smithi118 2024-03-20T08:04:49.037 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[93121]: pgmap v68: 121 pgs: 121 active+clean; 276 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: osdmap e261: 8 total, 8 up, 8 in 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: Reconfiguring mon.c (monmap changed)... 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: Reconfiguring daemon mon.c on smithi118 2024-03-20T08:04:49.038 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:48 smithi118 ceph-mon[95692]: pgmap v68: 121 pgs: 121 active+clean; 276 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: osdmap e261: 8 total, 8 up, 8 in 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: Reconfiguring mon.c (monmap changed)... 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: Reconfiguring daemon mon.c on smithi118 2024-03-20T08:04:49.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:48 smithi196 ceph-mon[87080]: pgmap v68: 121 pgs: 121 active+clean; 276 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 9.2 MiB/s rd, 3.6 MiB/s wr, 51 op/s 2024-03-20T08:04:49.174 INFO:tasks.rados.rados.0.smithi118.stdout:1028: finishing write tid 1 to smithi11881747-9 2024-03-20T08:04:49.175 INFO:tasks.rados.rados.0.smithi118.stdout:1028: finishing write tid 2 to smithi11881747-9 2024-03-20T08:04:49.175 INFO:tasks.rados.rados.0.smithi118.stdout:1028: finishing write tid 3 to smithi11881747-9 2024-03-20T08:04:49.175 INFO:tasks.rados.rados.0.smithi118.stdout:1028: oid 9 updating version 0 to 596 2024-03-20T08:04:49.175 INFO:tasks.rados.rados.0.smithi118.stdout:1028: oid 9 updating version 596 to 597 2024-03-20T08:04:49.175 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 597 (ObjNum 325 snap 104 seq_num 325) dirty exists 2024-03-20T08:04:49.175 INFO:tasks.rados.rados.0.smithi118.stdout:1028: left oid 9 (ObjNum 325 snap 104 seq_num 325) 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1026: done (2 left) 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1028: done (1 left) 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1029: done (0 left) 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1030: read oid 48 snap -1 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1030: expect (ObjNum 290 snap 97 seq_num 290) 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1031: read oid 35 snap 75 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1031: expect (ObjNum 322129520 snap 21898 seq_num 321663408) 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1032: copy_from oid 14 from oid 10 current snap is 105 2024-03-20T08:04:49.759 INFO:tasks.rados.rados.0.smithi118.stdout:1033: snap_remove snap 74 2024-03-20T08:04:50.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:49 smithi196 ceph-mon[87080]: osdmap e262: 8 total, 8 up, 8 in 2024-03-20T08:04:50.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:49 smithi196 ceph-mon[87080]: osdmap e263: 8 total, 8 up, 8 in 2024-03-20T08:04:50.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:49 smithi118 ceph-mon[93121]: osdmap e262: 8 total, 8 up, 8 in 2024-03-20T08:04:50.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:49 smithi118 ceph-mon[93121]: osdmap e263: 8 total, 8 up, 8 in 2024-03-20T08:04:50.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:49 smithi118 ceph-mon[95692]: osdmap e262: 8 total, 8 up, 8 in 2024-03-20T08:04:50.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:49 smithi118 ceph-mon[95692]: osdmap e263: 8 total, 8 up, 8 in 2024-03-20T08:04:50.760 INFO:tasks.rados.rados.0.smithi118.stdout:1033: done (3 left) 2024-03-20T08:04:50.760 INFO:tasks.rados.rados.0.smithi118.stdout:1034: copy_from oid 21 from oid 24 current snap is 105 2024-03-20T08:04:50.760 INFO:tasks.rados.rados.0.smithi118.stdout:1035: snap_remove snap 94 2024-03-20T08:04:51.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:51.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:51.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[95692]: Reconfiguring osd.0 (monmap changed)... 2024-03-20T08:04:51.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T08:04:51.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:51.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[95692]: Reconfiguring daemon osd.0 on smithi118 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[95692]: pgmap v71: 121 pgs: 121 active+clean; 285 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[95692]: osdmap e264: 8 total, 8 up, 8 in 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[93121]: Reconfiguring osd.0 (monmap changed)... 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[93121]: Reconfiguring daemon osd.0 on smithi118 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[93121]: pgmap v71: 121 pgs: 121 active+clean; 285 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2024-03-20T08:04:51.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:50 smithi118 ceph-mon[93121]: osdmap e264: 8 total, 8 up, 8 in 2024-03-20T08:04:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:50 smithi196 ceph-mon[87080]: Reconfiguring osd.0 (monmap changed)... 2024-03-20T08:04:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T08:04:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:51.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:50 smithi196 ceph-mon[87080]: Reconfiguring daemon osd.0 on smithi118 2024-03-20T08:04:51.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:50 smithi196 ceph-mon[87080]: pgmap v71: 121 pgs: 121 active+clean; 285 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2024-03-20T08:04:51.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:50 smithi196 ceph-mon[87080]: osdmap e264: 8 total, 8 up, 8 in 2024-03-20T08:04:51.760 INFO:tasks.rados.rados.0.smithi118.stdout:1035: done (4 left) 2024-03-20T08:04:51.760 INFO:tasks.rados.rados.0.smithi118.stdout:1036: snap_create 2024-03-20T08:04:51.761 INFO:tasks.rados.rados.0.smithi118.stdout:1031: expect (ObjNum 322129520 snap 21898 seq_num 321663408) 2024-03-20T08:04:51.764 INFO:tasks.rados.rados.0.smithi118.stdout:1032: finishing copy_from racing read to smithi11881747-14 2024-03-20T08:04:51.764 INFO:tasks.rados.rados.0.smithi118.stdout:1032: finishing copy_from to smithi11881747-14 2024-03-20T08:04:51.764 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 770 (ObjNum 315 snap 101 seq_num 315) dirty exists 2024-03-20T08:04:51.764 INFO:tasks.rados.rados.0.smithi118.stdout:1030: expect (ObjNum 290 snap 97 seq_num 290) 2024-03-20T08:04:51.945 INFO:tasks.rados.rados.0.smithi118.stdout:1034: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:04:51.945 INFO:tasks.rados.rados.0.smithi118.stdout:1034: finishing copy_from to smithi11881747-21 2024-03-20T08:04:51.945 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 363 (ObjNum 216 snap 69 seq_num 216) dirty exists 2024-03-20T08:04:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:52.764 INFO:tasks.rados.rados.0.smithi118.stdout:1030: done (4 left) 2024-03-20T08:04:52.765 INFO:tasks.rados.rados.0.smithi118.stdout:1031: done (3 left) 2024-03-20T08:04:52.765 INFO:tasks.rados.rados.0.smithi118.stdout:1032: done (2 left) 2024-03-20T08:04:52.765 INFO:tasks.rados.rados.0.smithi118.stdout:1034: done (1 left) 2024-03-20T08:04:52.765 INFO:tasks.rados.rados.0.smithi118.stdout:1036: done (0 left) 2024-03-20T08:04:52.765 INFO:tasks.rados.rados.0.smithi118.stdout:1037: snap_remove snap 88 2024-03-20T08:04:52.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:52.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:52.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[95692]: osdmap e265: 8 total, 8 up, 8 in 2024-03-20T08:04:52.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[95692]: Reconfiguring osd.1 (monmap changed)... 2024-03-20T08:04:52.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T08:04:52.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[95692]: Reconfiguring daemon osd.1 on smithi118 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[95692]: pgmap v74: 121 pgs: 121 active+clean; 285 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[93121]: osdmap e265: 8 total, 8 up, 8 in 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[93121]: Reconfiguring osd.1 (monmap changed)... 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[93121]: Reconfiguring daemon osd.1 on smithi118 2024-03-20T08:04:52.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:52 smithi118 ceph-mon[93121]: pgmap v74: 121 pgs: 121 active+clean; 285 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2024-03-20T08:04:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-mon[87080]: osdmap e265: 8 total, 8 up, 8 in 2024-03-20T08:04:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-mon[87080]: Reconfiguring osd.1 (monmap changed)... 2024-03-20T08:04:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T08:04:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-mon[87080]: Reconfiguring daemon osd.1 on smithi118 2024-03-20T08:04:53.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:52 smithi196 ceph-mon[87080]: pgmap v74: 121 pgs: 121 active+clean; 285 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 42 op/s 2024-03-20T08:04:53.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:04:53] "GET /metrics HTTP/1.1" 200 36063 "" "Prometheus/2.43.0" 2024-03-20T08:04:53.767 INFO:tasks.rados.rados.0.smithi118.stdout:1037: done (0 left) 2024-03-20T08:04:53.767 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 106 2024-03-20T08:04:53.767 INFO:tasks.rados.rados.0.smithi118.stdout:1038: seq_num 326 ranges {2162688=663552} 2024-03-20T08:04:53.770 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:04:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:04:53.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:04:53.785 INFO:tasks.rados.rados.0.smithi118.stdout:1038: writing smithi11881747-25 from 2162688 to 2826240 tid 1 2024-03-20T08:04:53.786 INFO:tasks.rados.rados.0.smithi118.stdout:1039: copy_from oid 1 from oid 10 current snap is 106 2024-03-20T08:04:53.786 INFO:tasks.rados.rados.0.smithi118.stdout:1040: read oid 46 snap -1 2024-03-20T08:04:53.786 INFO:tasks.rados.rados.0.smithi118.stdout:1040: expect (ObjNum 261 snap 90 seq_num 261) 2024-03-20T08:04:53.786 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 106 2024-03-20T08:04:53.786 INFO:tasks.rados.rados.0.smithi118.stdout:1041: seq_num 327 ranges {1228800=679936} 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:1041: writing smithi11881747-24 from 1228800 to 1908736 tid 1 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 5 current snap is 106 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:1038: finishing write tid 1 to smithi11881747-25 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:1038: finishing write tid 2 to smithi11881747-25 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:1038: finishing write tid 3 to smithi11881747-25 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:1038: oid 25 updating version 0 to 619 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:1038: oid 25 updating version 619 to 620 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 620 (ObjNum 326 snap 106 seq_num 326) dirty exists 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:1038: left oid 25 (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:04:53.798 INFO:tasks.rados.rados.0.smithi118.stdout:1042: seq_num 328 ranges {499712=696320} 2024-03-20T08:04:53.804 INFO:tasks.rados.rados.0.smithi118.stdout:1042: writing smithi11881747-5 from 499712 to 1196032 tid 1 2024-03-20T08:04:53.804 INFO:tasks.rados.rados.0.smithi118.stdout:1038: done (4 left) 2024-03-20T08:04:53.804 INFO:tasks.rados.rados.0.smithi118.stdout:1043: rollback oid 31 current snap is 106 2024-03-20T08:04:53.804 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 85 2024-03-20T08:04:53.805 INFO:tasks.rados.rados.0.smithi118.stdout:1039: finishing copy_from racing read to smithi11881747-1 2024-03-20T08:04:53.805 INFO:tasks.rados.rados.0.smithi118.stdout:1039: finishing copy_from to smithi11881747-1 2024-03-20T08:04:53.805 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 770 (ObjNum 315 snap 101 seq_num 315) dirty exists 2024-03-20T08:04:53.805 INFO:tasks.rados.rados.0.smithi118.stdout:1039: done (4 left) 2024-03-20T08:04:53.805 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 106 2024-03-20T08:04:53.805 INFO:tasks.rados.rados.0.smithi118.stdout:1044: seq_num 329 ranges {1671168=712704} 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1044: writing smithi11881747-10 from 1671168 to 2383872 tid 1 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1045: copy_from oid 9 from oid 8 current snap is 106 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1041: finishing write tid 1 to smithi11881747-24 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1041: finishing write tid 2 to smithi11881747-24 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1041: finishing write tid 3 to smithi11881747-24 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1041: oid 24 updating version 0 to 389 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1041: oid 24 updating version 389 to 390 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 390 (ObjNum 327 snap 106 seq_num 327) dirty exists 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1041: left oid 24 (ObjNum 327 snap 106 seq_num 327) 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1043: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1041: done (5 left) 2024-03-20T08:04:53.818 INFO:tasks.rados.rados.0.smithi118.stdout:1046: copy_from oid 22 from oid 13 current snap is 106 2024-03-20T08:04:53.819 INFO:tasks.rados.rados.0.smithi118.stdout:1040: expect (ObjNum 261 snap 90 seq_num 261) 2024-03-20T08:04:53.872 INFO:tasks.rados.rados.0.smithi118.stdout:1043: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:04:53.872 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 772 (ObjNum 207 snap 67 seq_num 207) dirty exists 2024-03-20T08:04:53.872 INFO:tasks.rados.rados.0.smithi118.stdout:1042: finishing write tid 1 to smithi11881747-5 2024-03-20T08:04:53.872 INFO:tasks.rados.rados.0.smithi118.stdout:1040: done (5 left) 2024-03-20T08:04:53.872 INFO:tasks.rados.rados.0.smithi118.stdout:1043: done (4 left) 2024-03-20T08:04:53.872 INFO:tasks.rados.rados.0.smithi118.stdout:1047: read oid 6 snap -1 2024-03-20T08:04:53.872 INFO:tasks.rados.rados.0.smithi118.stdout:1047: expect (ObjNum 273 snap 92 seq_num 273) 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1048: setattr oid 16 current snap is 106 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1042: finishing write tid 2 to smithi11881747-5 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1042: finishing write tid 3 to smithi11881747-5 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1042: oid 5 updating version 0 to 628 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1042: oid 5 updating version 628 to 629 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 629 (ObjNum 328 snap 106 seq_num 328) dirty exists 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1042: left oid 5 (ObjNum 328 snap 106 seq_num 328) 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1044: finishing write tid 1 to smithi11881747-10 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1045: finishing copy_from racing read to smithi11881747-9 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1044: finishing write tid 2 to smithi11881747-10 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1045: finishing copy_from to smithi11881747-9 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 466 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1044: finishing write tid 3 to smithi11881747-10 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1044: oid 10 updating version 0 to 774 2024-03-20T08:04:53.873 INFO:tasks.rados.rados.0.smithi118.stdout:1044: oid 10 version 774 is already newer than 773 2024-03-20T08:04:53.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 774 (ObjNum 329 snap 106 seq_num 329) dirty exists 2024-03-20T08:04:53.874 INFO:tasks.rados.rados.0.smithi118.stdout:1044: left oid 10 (ObjNum 329 snap 106 seq_num 329) 2024-03-20T08:04:53.874 INFO:tasks.rados.rados.0.smithi118.stdout:1046: finishing copy_from racing read to smithi11881747-22 2024-03-20T08:04:53.874 INFO:tasks.rados.rados.0.smithi118.stdout:1046: finishing copy_from to smithi11881747-22 2024-03-20T08:04:53.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 594 (ObjNum 323 snap 104 seq_num 323) dirty exists 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:1042: done (5 left) 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:1044: done (4 left) 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:1045: done (3 left) 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:1046: done (2 left) 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:1049: rollback oid 21 current snap is 106 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 78 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:1050: rollback oid 26 current snap is 106 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 101 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:1051: read oid 38 snap -1 2024-03-20T08:04:53.875 INFO:tasks.rados.rados.0.smithi118.stdout:1051: expect (ObjNum 321 snap 104 seq_num 321) 2024-03-20T08:04:53.876 INFO:tasks.rados.rados.0.smithi118.stdout:1052: rollback oid 17 current snap is 106 2024-03-20T08:04:53.876 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 103 2024-03-20T08:04:53.876 INFO:tasks.rados.rados.0.smithi118.stdout:1053: delete oid 28 current snap is 106 2024-03-20T08:04:53.877 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 828 (ObjNum 315 snap 101 seq_num 315) dirty exists 2024-03-20T08:04:53.878 INFO:tasks.rados.rados.0.smithi118.stdout:1048: done (6 left) 2024-03-20T08:04:53.878 INFO:tasks.rados.rados.0.smithi118.stdout:1053: done (5 left) 2024-03-20T08:04:53.878 INFO:tasks.rados.rados.0.smithi118.stdout:1054: copy_from oid 43 from oid 35 current snap is 106 2024-03-20T08:04:53.878 INFO:tasks.rados.rados.0.smithi118.stdout:1055: setattr oid 7 current snap is 106 2024-03-20T08:04:53.879 INFO:tasks.rados.rados.0.smithi118.stdout:1050: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:04:53.879 INFO:tasks.rados.rados.0.smithi118.stdout:1052: finishing rollback tid 0 to smithi11881747-17 2024-03-20T08:04:53.879 INFO:tasks.rados.rados.0.smithi118.stdout:1050: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:04:53.879 INFO:tasks.rados.rados.0.smithi118.stdout:1056: snap_create 2024-03-20T08:04:53.880 INFO:tasks.rados.rados.0.smithi118.stdout:1050: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:04:53.880 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 394 (ObjNum 266 snap 92 seq_num 266) dirty exists 2024-03-20T08:04:53.882 INFO:tasks.rados.rados.0.smithi118.stdout:1049: finishing rollback tid 0 to smithi11881747-21 2024-03-20T08:04:53.882 INFO:tasks.rados.rados.0.smithi118.stdout:1049: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:04:53.882 INFO:tasks.rados.rados.0.smithi118.stdout:1049: finishing rollback tid 2 to smithi11881747-21 2024-03-20T08:04:53.882 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 597 (ObjNum 206 snap 67 seq_num 206) dirty exists 2024-03-20T08:04:53.883 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 193 (ObjNum 33 snap 0 seq_num 323042384) dirty exists 2024-03-20T08:04:53.885 INFO:tasks.rados.rados.0.smithi118.stdout:1052: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:04:53.885 INFO:tasks.rados.rados.0.smithi118.stdout:1052: finishing rollback tid 2 to smithi11881747-17 2024-03-20T08:04:53.885 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 190 (ObjNum 128 snap 34 seq_num 128) dirty exists 2024-03-20T08:04:53.906 INFO:tasks.rados.rados.0.smithi118.stdout:1054: finishing copy_from racing read to smithi11881747-43 2024-03-20T08:04:53.907 INFO:tasks.rados.rados.0.smithi118.stdout:1054: finishing copy_from to smithi11881747-43 2024-03-20T08:04:53.907 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 383 (ObjNum 260 snap 90 seq_num 260) dirty exists 2024-03-20T08:04:53.922 INFO:tasks.rados.rados.0.smithi118.stdout:1051: expect (ObjNum 321 snap 104 seq_num 321) 2024-03-20T08:04:54.088 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[95692]: osdmap e266: 8 total, 8 up, 8 in 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[93121]: osdmap e266: 8 total, 8 up, 8 in 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T08:04:54.089 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:53 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:54.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:53 smithi196 ceph-mon[87080]: osdmap e266: 8 total, 8 up, 8 in 2024-03-20T08:04:54.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:53 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:54.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:53 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:54.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:53 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T08:04:54.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:53 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:54.211 INFO:tasks.rados.rados.0.smithi118.stdout:1047: expect (ObjNum 273 snap 92 seq_num 273) 2024-03-20T08:04:54.776 INFO:tasks.rados.rados.0.smithi118.stdout:1047: done (7 left) 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:1049: done (6 left) 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:1050: done (5 left) 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:1051: done (4 left) 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:1052: done (3 left) 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:1054: done (2 left) 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:1055: done (1 left) 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:1056: done (0 left) 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 107 2024-03-20T08:04:54.777 INFO:tasks.rados.rados.0.smithi118.stdout:1057: seq_num 331 ranges {7602176=753664} 2024-03-20T08:04:54.821 INFO:tasks.rados.rados.0.smithi118.stdout:1057: writing smithi11881747-6 from 7602176 to 8355840 tid 1 2024-03-20T08:04:54.821 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 107 2024-03-20T08:04:54.821 INFO:tasks.rados.rados.0.smithi118.stdout:1058: seq_num 332 ranges {0=770048} 2024-03-20T08:04:54.826 INFO:tasks.rados.rados.0.smithi118.stdout:1058: writing smithi11881747-44 from 0 to 770048 tid 1 2024-03-20T08:04:54.826 INFO:tasks.rados.rados.0.smithi118.stdout:1059: read oid 50 snap -1 2024-03-20T08:04:54.826 INFO:tasks.rados.rados.0.smithi118.stdout:1059: expect (ObjNum 295 snap 99 seq_num 295) 2024-03-20T08:04:54.826 INFO:tasks.rados.rados.0.smithi118.stdout:1060: snap_create 2024-03-20T08:04:54.826 INFO:tasks.rados.rados.0.smithi118.stdout:1057: finishing write tid 1 to smithi11881747-6 2024-03-20T08:04:54.826 INFO:tasks.rados.rados.0.smithi118.stdout:1057: finishing write tid 2 to smithi11881747-6 2024-03-20T08:04:54.828 INFO:tasks.rados.rados.0.smithi118.stdout:1057: finishing write tid 3 to smithi11881747-6 2024-03-20T08:04:54.828 INFO:tasks.rados.rados.0.smithi118.stdout:1057: oid 6 updating version 0 to 401 2024-03-20T08:04:54.828 INFO:tasks.rados.rados.0.smithi118.stdout:1057: oid 6 updating version 401 to 402 2024-03-20T08:04:54.828 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 402 (ObjNum 331 snap 107 seq_num 331) dirty exists 2024-03-20T08:04:54.828 INFO:tasks.rados.rados.0.smithi118.stdout:1057: left oid 6 (ObjNum 331 snap 107 seq_num 331) 2024-03-20T08:04:54.833 INFO:tasks.rados.rados.0.smithi118.stdout:1058: finishing write tid 1 to smithi11881747-44 2024-03-20T08:04:54.833 INFO:tasks.rados.rados.0.smithi118.stdout:1058: finishing write tid 2 to smithi11881747-44 2024-03-20T08:04:54.835 INFO:tasks.rados.rados.0.smithi118.stdout:1058: finishing write tid 3 to smithi11881747-44 2024-03-20T08:04:54.836 INFO:tasks.rados.rados.0.smithi118.stdout:1058: oid 44 updating version 0 to 397 2024-03-20T08:04:54.836 INFO:tasks.rados.rados.0.smithi118.stdout:1058: oid 44 version 397 is already newer than 396 2024-03-20T08:04:54.836 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 397 (ObjNum 332 snap 107 seq_num 332) dirty exists 2024-03-20T08:04:54.836 INFO:tasks.rados.rados.0.smithi118.stdout:1058: left oid 44 (ObjNum 332 snap 107 seq_num 332) 2024-03-20T08:04:54.836 INFO:tasks.rados.rados.0.smithi118.stdout:1059: expect (ObjNum 295 snap 99 seq_num 295) 2024-03-20T08:04:55.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:54 smithi196 ceph-mon[87080]: Reconfiguring osd.2 (monmap changed)... 2024-03-20T08:04:55.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:54 smithi196 ceph-mon[87080]: Reconfiguring daemon osd.2 on smithi118 2024-03-20T08:04:55.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:54 smithi196 ceph-mon[87080]: osdmap e267: 8 total, 8 up, 8 in 2024-03-20T08:04:55.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:54 smithi196 ceph-mon[87080]: pgmap v77: 121 pgs: 121 active+clean; 286 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 166 KiB/s wr, 1 op/s 2024-03-20T08:04:55.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:54 smithi118 ceph-mon[93121]: Reconfiguring osd.2 (monmap changed)... 2024-03-20T08:04:55.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:54 smithi118 ceph-mon[93121]: Reconfiguring daemon osd.2 on smithi118 2024-03-20T08:04:55.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:54 smithi118 ceph-mon[93121]: osdmap e267: 8 total, 8 up, 8 in 2024-03-20T08:04:55.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:54 smithi118 ceph-mon[93121]: pgmap v77: 121 pgs: 121 active+clean; 286 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 166 KiB/s wr, 1 op/s 2024-03-20T08:04:55.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:54 smithi118 ceph-mon[95692]: Reconfiguring osd.2 (monmap changed)... 2024-03-20T08:04:55.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:54 smithi118 ceph-mon[95692]: Reconfiguring daemon osd.2 on smithi118 2024-03-20T08:04:55.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:54 smithi118 ceph-mon[95692]: osdmap e267: 8 total, 8 up, 8 in 2024-03-20T08:04:55.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:54 smithi118 ceph-mon[95692]: pgmap v77: 121 pgs: 121 active+clean; 286 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 166 KiB/s wr, 1 op/s 2024-03-20T08:04:55.781 INFO:tasks.rados.rados.0.smithi118.stdout:1057: done (3 left) 2024-03-20T08:04:55.782 INFO:tasks.rados.rados.0.smithi118.stdout:1058: done (2 left) 2024-03-20T08:04:55.782 INFO:tasks.rados.rados.0.smithi118.stdout:1059: done (1 left) 2024-03-20T08:04:55.782 INFO:tasks.rados.rados.0.smithi118.stdout:1060: done (0 left) 2024-03-20T08:04:55.782 INFO:tasks.rados.rados.0.smithi118.stdout:1061: snap_create 2024-03-20T08:04:56.065 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[93121]: osdmap e268: 8 total, 8 up, 8 in 2024-03-20T08:04:56.065 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:56.065 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:56.065 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T08:04:56.065 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:56.066 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[95692]: osdmap e268: 8 total, 8 up, 8 in 2024-03-20T08:04:56.066 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:56.066 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:56.066 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T08:04:56.066 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:55 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:55 smithi196 ceph-mon[87080]: osdmap e268: 8 total, 8 up, 8 in 2024-03-20T08:04:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:55 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:55 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:55 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T08:04:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:55 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:56.786 INFO:tasks.rados.rados.0.smithi118.stdout:1061: done (0 left) 2024-03-20T08:04:56.786 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 5 current snap is 109 2024-03-20T08:04:56.787 INFO:tasks.rados.rados.0.smithi118.stdout:1062: seq_num 333 ranges {1196032=786432} 2024-03-20T08:04:56.797 INFO:tasks.rados.rados.0.smithi118.stdout:1062: writing smithi11881747-5 from 1196032 to 1982464 tid 1 2024-03-20T08:04:56.797 INFO:tasks.rados.rados.0.smithi118.stdout:1063: read oid 22 snap -1 2024-03-20T08:04:56.797 INFO:tasks.rados.rados.0.smithi118.stdout:1063: expect (ObjNum 323 snap 104 seq_num 323) 2024-03-20T08:04:56.797 INFO:tasks.rados.rados.0.smithi118.stdout:1064: snap_remove snap 101 2024-03-20T08:04:57.040 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:56 smithi196 ceph-mon[87080]: Reconfiguring osd.3 (monmap changed)... 2024-03-20T08:04:57.040 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:56 smithi196 ceph-mon[87080]: Reconfiguring daemon osd.3 on smithi118 2024-03-20T08:04:57.040 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:56 smithi196 ceph-mon[87080]: osdmap e269: 8 total, 8 up, 8 in 2024-03-20T08:04:57.041 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:56 smithi196 ceph-mon[87080]: pgmap v80: 121 pgs: 121 active+clean; 303 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2024-03-20T08:04:57.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:56 smithi118 ceph-mon[95692]: Reconfiguring osd.3 (monmap changed)... 2024-03-20T08:04:57.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:56 smithi118 ceph-mon[95692]: Reconfiguring daemon osd.3 on smithi118 2024-03-20T08:04:57.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:56 smithi118 ceph-mon[95692]: osdmap e269: 8 total, 8 up, 8 in 2024-03-20T08:04:57.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:56 smithi118 ceph-mon[95692]: pgmap v80: 121 pgs: 121 active+clean; 303 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2024-03-20T08:04:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:56 smithi118 ceph-mon[93121]: Reconfiguring osd.3 (monmap changed)... 2024-03-20T08:04:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:56 smithi118 ceph-mon[93121]: Reconfiguring daemon osd.3 on smithi118 2024-03-20T08:04:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:56 smithi118 ceph-mon[93121]: osdmap e269: 8 total, 8 up, 8 in 2024-03-20T08:04:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:56 smithi118 ceph-mon[93121]: pgmap v80: 121 pgs: 121 active+clean; 303 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2024-03-20T08:04:57.788 INFO:tasks.rados.rados.0.smithi118.stdout:1064: done (2 left) 2024-03-20T08:04:57.789 INFO:tasks.rados.rados.0.smithi118.stdout:1065: read oid 18 snap -1 2024-03-20T08:04:57.789 INFO:tasks.rados.rados.0.smithi118.stdout:1065: expect (ObjNum 312 snap 101 seq_num 312) 2024-03-20T08:04:57.789 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 109 2024-03-20T08:04:57.789 INFO:tasks.rados.rados.0.smithi118.stdout:1062: finishing write tid 1 to smithi11881747-5 2024-03-20T08:04:57.789 INFO:tasks.rados.rados.0.smithi118.stdout:1066: seq_num 334 ranges {0=802816} 2024-03-20T08:04:57.797 INFO:tasks.rados.rados.0.smithi118.stdout:1066: writing smithi11881747-33 from 0 to 802816 tid 1 2024-03-20T08:04:57.798 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 109 2024-03-20T08:04:57.798 INFO:tasks.rados.rados.0.smithi118.stdout:1062: finishing write tid 2 to smithi11881747-5 2024-03-20T08:04:57.798 INFO:tasks.rados.rados.0.smithi118.stdout:1062: finishing write tid 3 to smithi11881747-5 2024-03-20T08:04:57.798 INFO:tasks.rados.rados.0.smithi118.stdout:1062: oid 5 updating version 0 to 634 2024-03-20T08:04:57.798 INFO:tasks.rados.rados.0.smithi118.stdout:1062: oid 5 version 634 is already newer than 633 2024-03-20T08:04:57.798 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 634 (ObjNum 333 snap 109 seq_num 333) dirty exists 2024-03-20T08:04:57.799 INFO:tasks.rados.rados.0.smithi118.stdout:1062: left oid 5 (ObjNum 333 snap 109 seq_num 333) 2024-03-20T08:04:57.799 INFO:tasks.rados.rados.0.smithi118.stdout:1063: expect (ObjNum 323 snap 104 seq_num 323) 2024-03-20T08:04:57.865 INFO:tasks.rados.rados.0.smithi118.stdout:1065: expect (ObjNum 312 snap 101 seq_num 312) 2024-03-20T08:04:57.884 INFO:tasks.rados.rados.0.smithi118.stdout:1066: finishing write tid 1 to smithi11881747-33 2024-03-20T08:04:57.884 INFO:tasks.rados.rados.0.smithi118.stdout:1066: finishing write tid 2 to smithi11881747-33 2024-03-20T08:04:57.884 INFO:tasks.rados.rados.0.smithi118.stdout:1066: finishing write tid 3 to smithi11881747-33 2024-03-20T08:04:57.884 INFO:tasks.rados.rados.0.smithi118.stdout:1066: oid 33 updating version 0 to 260 2024-03-20T08:04:57.884 INFO:tasks.rados.rados.0.smithi118.stdout:1066: oid 33 version 260 is already newer than 259 2024-03-20T08:04:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 260 (ObjNum 334 snap 109 seq_num 334) dirty exists 2024-03-20T08:04:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:1066: left oid 33 (ObjNum 334 snap 109 seq_num 334) 2024-03-20T08:04:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:1067: seq_num 335 ranges {1056768=417792,1474560=401408} 2024-03-20T08:04:57.892 INFO:tasks.rados.rados.0.smithi118.stdout:1067: writing smithi11881747-21 from 1056768 to 1474560 tid 1 2024-03-20T08:04:57.894 INFO:tasks.rados.rados.0.smithi118.stdout:1067: writing smithi11881747-21 from 1474560 to 1875968 tid 2 2024-03-20T08:04:57.894 INFO:tasks.rados.rados.0.smithi118.stdout:1062: done (4 left) 2024-03-20T08:04:57.895 INFO:tasks.rados.rados.0.smithi118.stdout:1063: done (3 left) 2024-03-20T08:04:57.895 INFO:tasks.rados.rados.0.smithi118.stdout:1065: done (2 left) 2024-03-20T08:04:57.895 INFO:tasks.rados.rados.0.smithi118.stdout:1066: done (1 left) 2024-03-20T08:04:57.895 INFO:tasks.rados.rados.0.smithi118.stdout:1068: delete oid 17 current snap is 109 2024-03-20T08:04:57.897 INFO:tasks.rados.rados.0.smithi118.stdout:1068: done (1 left) 2024-03-20T08:04:57.897 INFO:tasks.rados.rados.0.smithi118.stdout:1069: rollback oid 38 current snap is 109 2024-03-20T08:04:57.897 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 38 to 107 2024-03-20T08:04:57.897 INFO:tasks.rados.rados.0.smithi118.stdout:1070: delete oid 3 current snap is 109 2024-03-20T08:04:57.897 INFO:tasks.rados.rados.0.smithi118.stdout:1067: finishing write tid 1 to smithi11881747-21 2024-03-20T08:04:57.897 INFO:tasks.rados.rados.0.smithi118.stdout:1067: finishing write tid 2 to smithi11881747-21 2024-03-20T08:04:57.898 INFO:tasks.rados.rados.0.smithi118.stdout:1067: finishing write tid 3 to smithi11881747-21 2024-03-20T08:04:57.898 INFO:tasks.rados.rados.0.smithi118.stdout:1070: done (2 left) 2024-03-20T08:04:57.898 INFO:tasks.rados.rados.0.smithi118.stdout:1071: snap_create 2024-03-20T08:04:57.900 INFO:tasks.rados.rados.0.smithi118.stdout:1069: finishing rollback tid 0 to smithi11881747-38 2024-03-20T08:04:57.901 INFO:tasks.rados.rados.0.smithi118.stdout:1069: finishing rollback tid 1 to smithi11881747-38 2024-03-20T08:04:57.901 INFO:tasks.rados.rados.0.smithi118.stdout:1067: finishing write tid 4 to smithi11881747-21 2024-03-20T08:04:57.902 INFO:tasks.rados.rados.0.smithi118.stdout:1067: oid 21 updating version 0 to 600 2024-03-20T08:04:57.902 INFO:tasks.rados.rados.0.smithi118.stdout:1067: oid 21 version 600 is already newer than 598 2024-03-20T08:04:57.902 INFO:tasks.rados.rados.0.smithi118.stdout:1067: oid 21 version 600 is already newer than 599 2024-03-20T08:04:57.902 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 600 (ObjNum 335 snap 109 seq_num 335) dirty exists 2024-03-20T08:04:57.902 INFO:tasks.rados.rados.0.smithi118.stdout:1067: left oid 21 (ObjNum 335 snap 109 seq_num 335) 2024-03-20T08:04:57.902 INFO:tasks.rados.rados.0.smithi118.stdout:1069: finishing rollback tid 2 to smithi11881747-38 2024-03-20T08:04:57.903 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 603 (ObjNum 321 snap 104 seq_num 321) dirty exists 2024-03-20T08:04:58.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:57 smithi196 ceph-mon[87080]: osdmap e270: 8 total, 8 up, 8 in 2024-03-20T08:04:58.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:57 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:58.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:57 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:58.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:57 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:04:58.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:57 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:58.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[93121]: osdmap e270: 8 total, 8 up, 8 in 2024-03-20T08:04:58.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:58.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:58.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:04:58.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[95692]: osdmap e270: 8 total, 8 up, 8 in 2024-03-20T08:04:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:04:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:57 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:1067: done (2 left) 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:1069: done (1 left) 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:1071: done (0 left) 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:1072: read oid 12 snap -1 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:1072: expect deleted 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:1073: rollback oid 47 current snap is 110 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 98 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:1074: copy_from oid 22 from oid 35 current snap is 110 2024-03-20T08:04:58.798 INFO:tasks.rados.rados.0.smithi118.stdout:1075: delete oid 3 current snap is 110 2024-03-20T08:04:58.801 INFO:tasks.rados.rados.0.smithi118.stdout:1072: done (3 left) 2024-03-20T08:04:58.801 INFO:tasks.rados.rados.0.smithi118.stdout:1075: done (2 left) 2024-03-20T08:04:58.801 INFO:tasks.rados.rados.0.smithi118.stdout:1076: rmattr oid 11 current snap is 110 2024-03-20T08:04:58.801 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 110 2024-03-20T08:04:58.801 INFO:tasks.rados.rados.0.smithi118.stdout:1077: seq_num 339 ranges {696320=483328,1179648=401408} 2024-03-20T08:04:58.808 INFO:tasks.rados.rados.0.smithi118.stdout:1077: writing smithi11881747-42 from 696320 to 1179648 tid 1 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:1077: writing smithi11881747-42 from 1179648 to 1581056 tid 2 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:1078: snap_create 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:1073: finishing rollback tid 0 to smithi11881747-47 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:1073: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:1073: finishing rollback tid 2 to smithi11881747-47 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 841 (ObjNum 281 snap 94 seq_num 281) dirty exists 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 642 (ObjNum 307 snap 100 seq_num 307) dirty exists 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:1074: finishing copy_from racing read to smithi11881747-22 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:1074: finishing copy_from to smithi11881747-22 2024-03-20T08:04:58.811 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 383 (ObjNum 260 snap 90 seq_num 260) dirty exists 2024-03-20T08:04:58.813 INFO:tasks.rados.rados.0.smithi118.stdout:1077: finishing write tid 1 to smithi11881747-42 2024-03-20T08:04:58.813 INFO:tasks.rados.rados.0.smithi118.stdout:1077: finishing write tid 2 to smithi11881747-42 2024-03-20T08:04:58.813 INFO:tasks.rados.rados.0.smithi118.stdout:1077: finishing write tid 3 to smithi11881747-42 2024-03-20T08:04:58.814 INFO:tasks.rados.rados.0.smithi118.stdout:1077: finishing write tid 4 to smithi11881747-42 2024-03-20T08:04:58.814 INFO:tasks.rados.rados.0.smithi118.stdout:1077: oid 42 updating version 0 to 646 2024-03-20T08:04:58.814 INFO:tasks.rados.rados.0.smithi118.stdout:1077: oid 42 version 646 is already newer than 645 2024-03-20T08:04:58.814 INFO:tasks.rados.rados.0.smithi118.stdout:1077: oid 42 version 646 is already newer than 644 2024-03-20T08:04:58.814 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 646 (ObjNum 339 snap 110 seq_num 339) dirty exists 2024-03-20T08:04:58.815 INFO:tasks.rados.rados.0.smithi118.stdout:1077: left oid 42 (ObjNum 339 snap 110 seq_num 339) 2024-03-20T08:04:58.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[93121]: Reconfiguring mds.a (monmap changed)... 2024-03-20T08:04:58.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[93121]: Reconfiguring daemon mds.a on smithi118 2024-03-20T08:04:58.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[93121]: osdmap e271: 8 total, 8 up, 8 in 2024-03-20T08:04:58.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[93121]: pgmap v83: 121 pgs: 121 active+clean; 303 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.4 MiB/s wr, 21 op/s 2024-03-20T08:04:58.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:58.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[95692]: Reconfiguring mds.a (monmap changed)... 2024-03-20T08:04:58.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[95692]: Reconfiguring daemon mds.a on smithi118 2024-03-20T08:04:58.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[95692]: osdmap e271: 8 total, 8 up, 8 in 2024-03-20T08:04:58.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[95692]: pgmap v83: 121 pgs: 121 active+clean; 303 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.4 MiB/s wr, 21 op/s 2024-03-20T08:04:58.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:59.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:58 smithi196 ceph-mon[87080]: Reconfiguring mds.a (monmap changed)... 2024-03-20T08:04:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:58 smithi196 ceph-mon[87080]: Reconfiguring daemon mds.a on smithi118 2024-03-20T08:04:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:58 smithi196 ceph-mon[87080]: osdmap e271: 8 total, 8 up, 8 in 2024-03-20T08:04:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:58 smithi196 ceph-mon[87080]: pgmap v83: 121 pgs: 121 active+clean; 303 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 13 MiB/s rd, 2.4 MiB/s wr, 21 op/s 2024-03-20T08:04:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:04:59.762 INFO:tasks.rados.rados.0.smithi118.stdout:1073: done (4 left) 2024-03-20T08:04:59.762 INFO:tasks.rados.rados.0.smithi118.stdout:1074: done (3 left) 2024-03-20T08:04:59.762 INFO:tasks.rados.rados.0.smithi118.stdout:1076: done (2 left) 2024-03-20T08:04:59.762 INFO:tasks.rados.rados.0.smithi118.stdout:1077: done (1 left) 2024-03-20T08:04:59.762 INFO:tasks.rados.rados.0.smithi118.stdout:1078: done (0 left) 2024-03-20T08:04:59.762 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 111 2024-03-20T08:04:59.762 INFO:tasks.rados.rados.0.smithi118.stdout:1079: seq_num 340 ranges {0=499712,499712=401408} 2024-03-20T08:04:59.766 INFO:tasks.rados.rados.0.smithi118.stdout:1079: writing smithi11881747-39 from 0 to 499712 tid 1 2024-03-20T08:04:59.767 INFO:tasks.rados.rados.0.smithi118.stdout:1079: writing smithi11881747-39 from 499712 to 901120 tid 2 2024-03-20T08:04:59.767 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 111 2024-03-20T08:04:59.767 INFO:tasks.rados.rados.0.smithi118.stdout:1080: seq_num 341 ranges {1966080=516096,2482176=401408} 2024-03-20T08:04:59.781 INFO:tasks.rados.rados.0.smithi118.stdout:1080: writing smithi11881747-43 from 1966080 to 2482176 tid 1 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1080: writing smithi11881747-43 from 2482176 to 2883584 tid 2 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1081: read oid 2 snap -1 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1081: expect (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1082: delete oid 48 current snap is 111 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1079: finishing write tid 1 to smithi11881747-39 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1079: finishing write tid 2 to smithi11881747-39 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1079: finishing write tid 3 to smithi11881747-39 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1079: finishing write tid 4 to smithi11881747-39 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1079: oid 39 updating version 0 to 596 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1079: oid 39 version 596 is already newer than 594 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1079: oid 39 version 596 is already newer than 595 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 596 (ObjNum 340 snap 111 seq_num 340) dirty exists 2024-03-20T08:04:59.784 INFO:tasks.rados.rados.0.smithi118.stdout:1079: left oid 39 (ObjNum 340 snap 111 seq_num 340) 2024-03-20T08:04:59.787 INFO:tasks.rados.rados.0.smithi118.stdout:1080: finishing write tid 1 to smithi11881747-43 2024-03-20T08:04:59.789 INFO:tasks.rados.rados.0.smithi118.stdout:1080: finishing write tid 2 to smithi11881747-43 2024-03-20T08:04:59.790 INFO:tasks.rados.rados.0.smithi118.stdout:1080: finishing write tid 3 to smithi11881747-43 2024-03-20T08:04:59.790 INFO:tasks.rados.rados.0.smithi118.stdout:1079: done (3 left) 2024-03-20T08:04:59.790 INFO:tasks.rados.rados.0.smithi118.stdout:1082: done (2 left) 2024-03-20T08:04:59.790 INFO:tasks.rados.rados.0.smithi118.stdout:1083: setattr oid 6 current snap is 111 2024-03-20T08:04:59.792 INFO:tasks.rados.rados.0.smithi118.stdout:1084: snap_create 2024-03-20T08:04:59.792 INFO:tasks.rados.rados.0.smithi118.stdout:1080: finishing write tid 4 to smithi11881747-43 2024-03-20T08:04:59.792 INFO:tasks.rados.rados.0.smithi118.stdout:1080: oid 43 updating version 0 to 775 2024-03-20T08:04:59.792 INFO:tasks.rados.rados.0.smithi118.stdout:1080: oid 43 updating version 775 to 777 2024-03-20T08:04:59.792 INFO:tasks.rados.rados.0.smithi118.stdout:1080: oid 43 version 777 is already newer than 776 2024-03-20T08:04:59.792 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 777 (ObjNum 341 snap 111 seq_num 341) dirty exists 2024-03-20T08:04:59.792 INFO:tasks.rados.rados.0.smithi118.stdout:1080: left oid 43 (ObjNum 341 snap 111 seq_num 341) 2024-03-20T08:04:59.810 INFO:tasks.rados.rados.0.smithi118.stdout:1081: expect (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:04:59.963 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: osdmap e272: 8 total, 8 up, 8 in 2024-03-20T08:04:59.963 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:59.963 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:04:59.963 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: Reconfiguring mon.b (monmap changed)... 2024-03-20T08:04:59.963 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:04:59.963 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:04:59.964 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:04:59.964 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: Reconfiguring daemon mon.b on smithi196 2024-03-20T08:04:59.964 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:04:59 smithi196 ceph-mon[87080]: osdmap e273: 8 total, 8 up, 8 in 2024-03-20T08:05:00.012 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 403 (ObjNum 331 snap 107 seq_num 331) dirty exists 2024-03-20T08:05:00.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: osdmap e272: 8 total, 8 up, 8 in 2024-03-20T08:05:00.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:00.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:00.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: Reconfiguring mon.b (monmap changed)... 2024-03-20T08:05:00.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:05:00.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: Reconfiguring daemon mon.b on smithi196 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[93121]: osdmap e273: 8 total, 8 up, 8 in 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: osdmap e272: 8 total, 8 up, 8 in 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: Reconfiguring mon.b (monmap changed)... 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: Reconfiguring daemon mon.b on smithi196 2024-03-20T08:05:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:04:59 smithi118 ceph-mon[95692]: osdmap e273: 8 total, 8 up, 8 in 2024-03-20T08:05:00.762 INFO:tasks.rados.rados.0.smithi118.stdout:1080: done (3 left) 2024-03-20T08:05:00.763 INFO:tasks.rados.rados.0.smithi118.stdout:1081: done (2 left) 2024-03-20T08:05:00.763 INFO:tasks.rados.rados.0.smithi118.stdout:1083: done (1 left) 2024-03-20T08:05:00.763 INFO:tasks.rados.rados.0.smithi118.stdout:1084: done (0 left) 2024-03-20T08:05:00.763 INFO:tasks.rados.rados.0.smithi118.stdout:1085: read oid 42 snap -1 2024-03-20T08:05:00.763 INFO:tasks.rados.rados.0.smithi118.stdout:1085: expect (ObjNum 339 snap 110 seq_num 339) 2024-03-20T08:05:00.763 INFO:tasks.rados.rados.0.smithi118.stdout:1086: snap_remove snap 75 2024-03-20T08:05:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:00 smithi196 ceph-mon[87080]: pgmap v86: 121 pgs: 121 active+clean; 314 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-03-20T08:05:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:00 smithi196 ceph-mon[87080]: osdmap e274: 8 total, 8 up, 8 in 2024-03-20T08:05:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:05:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:05:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[93121]: pgmap v86: 121 pgs: 121 active+clean; 314 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-03-20T08:05:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[93121]: osdmap e274: 8 total, 8 up, 8 in 2024-03-20T08:05:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:05:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:05:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[95692]: pgmap v86: 121 pgs: 121 active+clean; 314 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-03-20T08:05:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[95692]: osdmap e274: 8 total, 8 up, 8 in 2024-03-20T08:05:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-20T08:05:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-20T08:05:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:01.766 INFO:tasks.rados.rados.0.smithi118.stdout:1086: done (1 left) 2024-03-20T08:05:01.767 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 112 2024-03-20T08:05:01.767 INFO:tasks.rados.rados.0.smithi118.stdout:1087: seq_num 343 ranges {1671168=548864,2220032=401408} 2024-03-20T08:05:01.778 INFO:tasks.rados.rados.0.smithi118.stdout:1087: writing smithi11881747-1 from 1671168 to 2220032 tid 1 2024-03-20T08:05:01.782 INFO:tasks.rados.rados.0.smithi118.stdout:1087: writing smithi11881747-1 from 2220032 to 2621440 tid 2 2024-03-20T08:05:01.782 INFO:tasks.rados.rados.0.smithi118.stdout:1088: copy_from oid 3 from oid 29 current snap is 112 2024-03-20T08:05:01.782 INFO:tasks.rados.rados.0.smithi118.stdout:1085: expect (ObjNum 339 snap 110 seq_num 339) 2024-03-20T08:05:01.838 INFO:tasks.rados.rados.0.smithi118.stdout:1088: finishing copy_from to smithi11881747-3 2024-03-20T08:05:01.838 INFO:tasks.rados.rados.0.smithi118.stdout:1088: got expected ENOENT (src dne) 2024-03-20T08:05:01.838 INFO:tasks.rados.rados.0.smithi118.stdout:1088: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:05:01.838 INFO:tasks.rados.rados.0.smithi118.stdout:1087: finishing write tid 1 to smithi11881747-1 2024-03-20T08:05:01.838 INFO:tasks.rados.rados.0.smithi118.stdout:1087: finishing write tid 2 to smithi11881747-1 2024-03-20T08:05:01.838 INFO:tasks.rados.rados.0.smithi118.stdout:1087: finishing write tid 3 to smithi11881747-1 2024-03-20T08:05:01.838 INFO:tasks.rados.rados.0.smithi118.stdout:1087: finishing write tid 4 to smithi11881747-1 2024-03-20T08:05:01.838 INFO:tasks.rados.rados.0.smithi118.stdout:1087: oid 1 updating version 0 to 780 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:1087: oid 1 version 780 is already newer than 778 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:1087: oid 1 version 780 is already newer than 779 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 780 (ObjNum 343 snap 112 seq_num 343) dirty exists 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:1087: left oid 1 (ObjNum 343 snap 112 seq_num 343) 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:1085: done (2 left) 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:1087: done (1 left) 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:1088: done (0 left) 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 112 2024-03-20T08:05:01.839 INFO:tasks.rados.rados.0.smithi118.stdout:1089: seq_num 344 ranges {4538368=565248,5103616=401408} 2024-03-20T08:05:01.864 INFO:tasks.rados.rados.0.smithi118.stdout:1089: writing smithi11881747-47 from 4538368 to 5103616 tid 1 2024-03-20T08:05:01.866 INFO:tasks.rados.rados.0.smithi118.stdout:1089: writing smithi11881747-47 from 5103616 to 5505024 tid 2 2024-03-20T08:05:01.866 INFO:tasks.rados.rados.0.smithi118.stdout:1090: snap_remove snap 107 2024-03-20T08:05:01.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:01 smithi196 ceph-mon[87080]: Reconfiguring mgr.x (monmap changed)... 2024-03-20T08:05:01.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:01 smithi196 ceph-mon[87080]: Reconfiguring daemon mgr.x on smithi196 2024-03-20T08:05:01.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:01 smithi196 ceph-mon[87080]: osdmap e275: 8 total, 8 up, 8 in 2024-03-20T08:05:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:01 smithi118 ceph-mon[93121]: Reconfiguring mgr.x (monmap changed)... 2024-03-20T08:05:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:01 smithi118 ceph-mon[93121]: Reconfiguring daemon mgr.x on smithi196 2024-03-20T08:05:02.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:01 smithi118 ceph-mon[93121]: osdmap e275: 8 total, 8 up, 8 in 2024-03-20T08:05:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:01 smithi118 ceph-mon[95692]: Reconfiguring mgr.x (monmap changed)... 2024-03-20T08:05:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:01 smithi118 ceph-mon[95692]: Reconfiguring daemon mgr.x on smithi196 2024-03-20T08:05:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:01 smithi118 ceph-mon[95692]: osdmap e275: 8 total, 8 up, 8 in 2024-03-20T08:05:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:02.811 INFO:tasks.rados.rados.0.smithi118.stdout:1090: done (1 left) 2024-03-20T08:05:02.811 INFO:tasks.rados.rados.0.smithi118.stdout:1091: delete oid 9 current snap is 112 2024-03-20T08:05:02.811 INFO:tasks.rados.rados.0.smithi118.stdout:1089: finishing write tid 1 to smithi11881747-47 2024-03-20T08:05:02.811 INFO:tasks.rados.rados.0.smithi118.stdout:1089: finishing write tid 2 to smithi11881747-47 2024-03-20T08:05:02.811 INFO:tasks.rados.rados.0.smithi118.stdout:1089: finishing write tid 3 to smithi11881747-47 2024-03-20T08:05:02.811 INFO:tasks.rados.rados.0.smithi118.stdout:1089: finishing write tid 4 to smithi11881747-47 2024-03-20T08:05:02.811 INFO:tasks.rados.rados.0.smithi118.stdout:1089: oid 47 updating version 0 to 853 2024-03-20T08:05:02.811 INFO:tasks.rados.rados.0.smithi118.stdout:1089: oid 47 updating version 853 to 854 2024-03-20T08:05:02.812 INFO:tasks.rados.rados.0.smithi118.stdout:1089: oid 47 updating version 854 to 855 2024-03-20T08:05:02.812 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 855 (ObjNum 344 snap 112 seq_num 344) dirty exists 2024-03-20T08:05:02.812 INFO:tasks.rados.rados.0.smithi118.stdout:1089: left oid 47 (ObjNum 344 snap 112 seq_num 344) 2024-03-20T08:05:02.822 INFO:tasks.rados.rados.0.smithi118.stdout:1089: done (1 left) 2024-03-20T08:05:02.822 INFO:tasks.rados.rados.0.smithi118.stdout:1091: done (0 left) 2024-03-20T08:05:02.822 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 112 2024-03-20T08:05:02.822 INFO:tasks.rados.rados.0.smithi118.stdout:1092: seq_num 346 ranges {1982464=598016,2580480=401408} 2024-03-20T08:05:02.836 INFO:tasks.rados.rados.0.smithi118.stdout:1092: writing smithi11881747-46 from 1982464 to 2580480 tid 1 2024-03-20T08:05:02.838 INFO:tasks.rados.rados.0.smithi118.stdout:1092: writing smithi11881747-46 from 2580480 to 2981888 tid 2 2024-03-20T08:05:02.838 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 112 2024-03-20T08:05:02.838 INFO:tasks.rados.rados.0.smithi118.stdout:1093: seq_num 347 ranges {2334720=614400,2949120=401408} 2024-03-20T08:05:02.854 INFO:tasks.rados.rados.0.smithi118.stdout:1093: writing smithi11881747-15 from 2334720 to 2949120 tid 1 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1093: writing smithi11881747-15 from 2949120 to 3350528 tid 2 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1094: read oid 31 snap -1 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1094: expect (ObjNum 207 snap 67 seq_num 207) 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1095: copy_from oid 33 from oid 44 current snap is 112 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: finishing write tid 1 to smithi11881747-46 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: finishing write tid 2 to smithi11881747-46 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: finishing write tid 3 to smithi11881747-46 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: finishing write tid 4 to smithi11881747-46 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: oid 46 updating version 0 to 405 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: oid 46 version 405 is already newer than 404 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: oid 46 updating version 405 to 406 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 406 (ObjNum 346 snap 112 seq_num 346) dirty exists 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: left oid 46 (ObjNum 346 snap 112 seq_num 346) 2024-03-20T08:05:02.857 INFO:tasks.rados.rados.0.smithi118.stdout:1092: done (3 left) 2024-03-20T08:05:02.858 INFO:tasks.rados.rados.0.smithi118.stdout:1096: snap_create 2024-03-20T08:05:02.859 INFO:tasks.rados.rados.0.smithi118.stdout:1093: finishing write tid 1 to smithi11881747-15 2024-03-20T08:05:02.859 INFO:tasks.rados.rados.0.smithi118.stdout:1093: finishing write tid 2 to smithi11881747-15 2024-03-20T08:05:02.859 INFO:tasks.rados.rados.0.smithi118.stdout:1093: finishing write tid 3 to smithi11881747-15 2024-03-20T08:05:02.863 INFO:tasks.rados.rados.0.smithi118.stdout:1095: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:05:02.863 INFO:tasks.rados.rados.0.smithi118.stdout:1093: finishing write tid 4 to smithi11881747-15 2024-03-20T08:05:02.863 INFO:tasks.rados.rados.0.smithi118.stdout:1093: oid 15 updating version 0 to 199 2024-03-20T08:05:02.863 INFO:tasks.rados.rados.0.smithi118.stdout:1093: oid 15 updating version 199 to 200 2024-03-20T08:05:02.863 INFO:tasks.rados.rados.0.smithi118.stdout:1093: oid 15 version 200 is already newer than 198 2024-03-20T08:05:02.863 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 200 (ObjNum 347 snap 112 seq_num 347) dirty exists 2024-03-20T08:05:02.863 INFO:tasks.rados.rados.0.smithi118.stdout:1093: left oid 15 (ObjNum 347 snap 112 seq_num 347) 2024-03-20T08:05:02.865 INFO:tasks.rados.rados.0.smithi118.stdout:1095: finishing copy_from to smithi11881747-33 2024-03-20T08:05:02.865 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 397 (ObjNum 332 snap 107 seq_num 332) dirty exists 2024-03-20T08:05:02.899 INFO:tasks.rados.rados.0.smithi118.stdout:1094: expect (ObjNum 207 snap 67 seq_num 207) 2024-03-20T08:05:03.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:02 smithi196 ceph-mon[87080]: pgmap v89: 121 pgs: 121 active+clean; 314 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-03-20T08:05:03.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:03.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:03.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T08:05:03.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:03.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[95692]: pgmap v89: 121 pgs: 121 active+clean; 314 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-03-20T08:05:03.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:03.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:03.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T08:05:03.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:03.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[93121]: pgmap v89: 121 pgs: 121 active+clean; 314 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 13 op/s 2024-03-20T08:05:03.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:03.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:03.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T08:05:03.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:03.808 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:03.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:03.813 INFO:tasks.rados.rados.0.smithi118.stdout:1093: done (3 left) 2024-03-20T08:05:03.813 INFO:tasks.rados.rados.0.smithi118.stdout:1094: done (2 left) 2024-03-20T08:05:03.813 INFO:tasks.rados.rados.0.smithi118.stdout:1095: done (1 left) 2024-03-20T08:05:03.813 INFO:tasks.rados.rados.0.smithi118.stdout:1096: done (0 left) 2024-03-20T08:05:03.814 INFO:tasks.rados.rados.0.smithi118.stdout:1097: snap_create 2024-03-20T08:05:03.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:03 smithi118 ceph-mon[93121]: Reconfiguring osd.4 (monmap changed)... 2024-03-20T08:05:03.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:03 smithi118 ceph-mon[93121]: Reconfiguring daemon osd.4 on smithi196 2024-03-20T08:05:03.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:03 smithi118 ceph-mon[93121]: osdmap e276: 8 total, 8 up, 8 in 2024-03-20T08:05:03.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:05:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:05:03] "GET /metrics HTTP/1.1" 200 36069 "" "Prometheus/2.43.0" 2024-03-20T08:05:03.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:03 smithi118 ceph-mon[95692]: Reconfiguring osd.4 (monmap changed)... 2024-03-20T08:05:03.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:03 smithi118 ceph-mon[95692]: Reconfiguring daemon osd.4 on smithi196 2024-03-20T08:05:03.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:03 smithi118 ceph-mon[95692]: osdmap e276: 8 total, 8 up, 8 in 2024-03-20T08:05:04.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:03 smithi196 ceph-mon[87080]: Reconfiguring osd.4 (monmap changed)... 2024-03-20T08:05:04.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:03 smithi196 ceph-mon[87080]: Reconfiguring daemon osd.4 on smithi196 2024-03-20T08:05:04.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:03 smithi196 ceph-mon[87080]: osdmap e276: 8 total, 8 up, 8 in 2024-03-20T08:05:04.819 INFO:tasks.rados.rados.0.smithi118.stdout:1097: done (0 left) 2024-03-20T08:05:04.819 INFO:tasks.rados.rados.0.smithi118.stdout:1098: snap_remove snap 113 2024-03-20T08:05:04.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:04 smithi196 ceph-mon[87080]: osdmap e277: 8 total, 8 up, 8 in 2024-03-20T08:05:04.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:04.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:04.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:04 smithi196 ceph-mon[87080]: Reconfiguring osd.5 (monmap changed)... 2024-03-20T08:05:04.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T08:05:04.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:04.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:04 smithi196 ceph-mon[87080]: Reconfiguring daemon osd.5 on smithi196 2024-03-20T08:05:04.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:04 smithi196 ceph-mon[87080]: pgmap v92: 121 pgs: 121 active+clean; 305 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-03-20T08:05:05.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[93121]: osdmap e277: 8 total, 8 up, 8 in 2024-03-20T08:05:05.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:05.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:05.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[93121]: Reconfiguring osd.5 (monmap changed)... 2024-03-20T08:05:05.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[93121]: Reconfiguring daemon osd.5 on smithi196 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[93121]: pgmap v92: 121 pgs: 121 active+clean; 305 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[95692]: osdmap e277: 8 total, 8 up, 8 in 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[95692]: Reconfiguring osd.5 (monmap changed)... 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[95692]: Reconfiguring daemon osd.5 on smithi196 2024-03-20T08:05:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:04 smithi118 ceph-mon[95692]: pgmap v92: 121 pgs: 121 active+clean; 305 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 0 op/s 2024-03-20T08:05:05.196 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:05:05.516 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:05:05.516 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (4m) 42s ago 5m 28.5M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:05:05.516 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (5m) 25s ago 5m 86.9M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:05:05.516 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (6m) 42s ago 6m 14.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:05:05.517 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (6m) 25s ago 6m 19.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:05:05.517 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (91s) 25s ago 10m 449M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:05:05.517 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (2m) 42s ago 12m 494M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:05:05.517 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (76s) 42s ago 12m 38.2M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:05:05.517 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (32s) 25s ago 10m 26.6M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:05:05.517 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (51s) 42s ago 10m 27.1M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:05:05.518 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (5m) 42s ago 5m 14.4M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:05:05.518 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (4m) 25s ago 4m 15.8M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:05:05.518 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (10m) 42s ago 9m 151M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 481bba84ab31 2024-03-20T08:05:05.518 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (9m) 42s ago 9m 157M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:05:05.518 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (9m) 42s ago 9m 130M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:05:05.518 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (8m) 42s ago 8m 126M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:05:05.519 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (8m) 25s ago 8m 181M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:05:05.519 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (7m) 25s ago 7m 133M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:05:05.519 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (7m) 25s ago 7m 117M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:05:05.519 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (7m) 25s ago 7m 115M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:05:05.519 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (96s) 25s ago 6m 56.2M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:05:05.821 INFO:tasks.rados.rados.0.smithi118.stdout:1098: done (0 left) 2024-03-20T08:05:05.821 INFO:tasks.rados.rados.0.smithi118.stdout:1099: snap_remove snap 100 2024-03-20T08:05:05.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:05 smithi196 ceph-mon[87080]: osdmap e278: 8 total, 8 up, 8 in 2024-03-20T08:05:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:05 smithi196 ceph-mon[87080]: from='client.44108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T08:05:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:05.921 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:05:05.921 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:05:05.922 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 10, 2024-03-20T08:05:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 5 2024-03-20T08:05:05.923 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:05:05.923 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:05:06.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[95692]: osdmap e278: 8 total, 8 up, 8 in 2024-03-20T08:05:06.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[95692]: from='client.44108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:06.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:06.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:06.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T08:05:06.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:06.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[93121]: osdmap e278: 8 total, 8 up, 8 in 2024-03-20T08:05:06.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[93121]: from='client.44108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:06.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:06.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:06.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T08:05:06.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:06.831 INFO:tasks.rados.rados.0.smithi118.stdout:1099: done (0 left) 2024-03-20T08:05:06.831 INFO:tasks.rados.rados.0.smithi118.stdout:1100: delete oid 6 current snap is 114 2024-03-20T08:05:06.848 INFO:tasks.rados.rados.0.smithi118.stdout:1100: done (0 left) 2024-03-20T08:05:06.848 INFO:tasks.rados.rados.0.smithi118.stdout:1101: read oid 23 snap 99 2024-03-20T08:05:06.848 INFO:tasks.rados.rados.0.smithi118.stdout:1101: expect (ObjNum 230 snap 79 seq_num 230) 2024-03-20T08:05:06.848 INFO:tasks.rados.rados.0.smithi118.stdout:1102: copy_from oid 27 from oid 41 current snap is 114 2024-03-20T08:05:06.848 INFO:tasks.rados.rados.0.smithi118.stdout:1103: delete oid 33 current snap is 114 2024-03-20T08:05:06.852 INFO:tasks.rados.rados.0.smithi118.stdout:1103: done (2 left) 2024-03-20T08:05:06.852 INFO:tasks.rados.rados.0.smithi118.stdout:1104: read oid 43 snap -1 2024-03-20T08:05:06.852 INFO:tasks.rados.rados.0.smithi118.stdout:1104: expect (ObjNum 341 snap 111 seq_num 341) 2024-03-20T08:05:06.852 INFO:tasks.rados.rados.0.smithi118.stdout:1105: rmattr oid 7 current snap is 114 2024-03-20T08:05:06.852 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 114 2024-03-20T08:05:06.852 INFO:tasks.rados.rados.0.smithi118.stdout:1106: seq_num 350 ranges {0=671744,671744=401408} 2024-03-20T08:05:06.857 INFO:tasks.rados.rados.0.smithi118.stdout:1106: writing smithi11881747-36 from 0 to 671744 tid 1 2024-03-20T08:05:06.861 INFO:tasks.rados.rados.0.smithi118.stdout:1106: writing smithi11881747-36 from 671744 to 1073152 tid 2 2024-03-20T08:05:06.861 INFO:tasks.rados.rados.0.smithi118.stdout:1107: delete oid 38 current snap is 114 2024-03-20T08:05:06.861 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 201 (ObjNum 65 snap 0 seq_num 1399349347) dirty exists 2024-03-20T08:05:06.863 INFO:tasks.rados.rados.0.smithi118.stdout:1105: done (5 left) 2024-03-20T08:05:06.864 INFO:tasks.rados.rados.0.smithi118.stdout:1107: done (4 left) 2024-03-20T08:05:06.864 INFO:tasks.rados.rados.0.smithi118.stdout:1108: read oid 29 snap -1 2024-03-20T08:05:06.864 INFO:tasks.rados.rados.0.smithi118.stdout:1108: expect deleted 2024-03-20T08:05:06.864 INFO:tasks.rados.rados.0.smithi118.stdout:1109: snap_remove snap 98 2024-03-20T08:05:07.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:06 smithi196 ceph-mon[87080]: from='client.34191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:07.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:06 smithi196 ceph-mon[87080]: Reconfiguring osd.6 (monmap changed)... 2024-03-20T08:05:07.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:06 smithi196 ceph-mon[87080]: Reconfiguring daemon osd.6 on smithi196 2024-03-20T08:05:07.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:06 smithi196 ceph-mon[87080]: osdmap e279: 8 total, 8 up, 8 in 2024-03-20T08:05:07.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:06 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/4168753183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:07.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:06 smithi196 ceph-mon[87080]: pgmap v95: 121 pgs: 121 active+clean; 311 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 15 op/s 2024-03-20T08:05:07.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[95692]: from='client.34191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:07.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[95692]: Reconfiguring osd.6 (monmap changed)... 2024-03-20T08:05:07.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[95692]: Reconfiguring daemon osd.6 on smithi196 2024-03-20T08:05:07.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[95692]: osdmap e279: 8 total, 8 up, 8 in 2024-03-20T08:05:07.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/4168753183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:07.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[95692]: pgmap v95: 121 pgs: 121 active+clean; 311 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 15 op/s 2024-03-20T08:05:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[93121]: from='client.34191 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[93121]: Reconfiguring osd.6 (monmap changed)... 2024-03-20T08:05:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[93121]: Reconfiguring daemon osd.6 on smithi196 2024-03-20T08:05:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[93121]: osdmap e279: 8 total, 8 up, 8 in 2024-03-20T08:05:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/4168753183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:06 smithi118 ceph-mon[93121]: pgmap v95: 121 pgs: 121 active+clean; 311 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 15 op/s 2024-03-20T08:05:07.834 INFO:tasks.rados.rados.0.smithi118.stdout:1109: done (5 left) 2024-03-20T08:05:07.834 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 114 2024-03-20T08:05:07.835 INFO:tasks.rados.rados.0.smithi118.stdout:1106: finishing write tid 1 to smithi11881747-36 2024-03-20T08:05:07.835 INFO:tasks.rados.rados.0.smithi118.stdout:1110: seq_num 352 ranges {5505024=704512,6209536=401408} 2024-03-20T08:05:07.869 INFO:tasks.rados.rados.0.smithi118.stdout:1110: writing smithi11881747-47 from 5505024 to 6209536 tid 1 2024-03-20T08:05:07.871 INFO:tasks.rados.rados.0.smithi118.stdout:1110: writing smithi11881747-47 from 6209536 to 6610944 tid 2 2024-03-20T08:05:07.871 INFO:tasks.rados.rados.0.smithi118.stdout:1111: read oid 40 snap -1 2024-03-20T08:05:07.871 INFO:tasks.rados.rados.0.smithi118.stdout:1111: expect (ObjNum 306 snap 100 seq_num 306) 2024-03-20T08:05:07.871 INFO:tasks.rados.rados.0.smithi118.stdout:1112: rollback oid 14 current snap is 114 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 29 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:1106: finishing write tid 2 to smithi11881747-36 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:1106: finishing write tid 3 to smithi11881747-36 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:1106: finishing write tid 4 to smithi11881747-36 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:1106: oid 36 updating version 0 to 251 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:1106: oid 36 updating version 251 to 253 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:1106: oid 36 version 253 is already newer than 252 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 253 (ObjNum 350 snap 114 seq_num 350) dirty exists 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:1106: left oid 36 (ObjNum 350 snap 114 seq_num 350) 2024-03-20T08:05:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:1101: expect (ObjNum 230 snap 79 seq_num 230) 2024-03-20T08:05:08.005 INFO:tasks.rados.rados.0.smithi118.stdout:1104: expect (ObjNum 341 snap 111 seq_num 341) 2024-03-20T08:05:08.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:07 smithi196 ceph-mon[87080]: osdmap e280: 8 total, 8 up, 8 in 2024-03-20T08:05:08.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:07 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:08.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:07 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:08.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:07 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T08:05:08.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:07 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1102: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1102: finishing copy_from to smithi11881747-27 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1101: done (7 left) 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1102: done (6 left) 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1104: done (5 left) 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1106: done (4 left) 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1108: done (3 left) 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1113: snap_create 2024-03-20T08:05:08.109 INFO:tasks.rados.rados.0.smithi118.stdout:1110: finishing write tid 1 to smithi11881747-47 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1112: finishing rollback tid 0 to smithi11881747-14 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1110: finishing write tid 2 to smithi11881747-47 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1112: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1112: finishing rollback tid 2 to smithi11881747-14 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 773 (ObjNum 104 snap 23 seq_num 104) dirty exists 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1110: finishing write tid 3 to smithi11881747-47 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1110: finishing write tid 4 to smithi11881747-47 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1110: oid 47 updating version 0 to 875 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1110: oid 47 version 875 is already newer than 873 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1110: oid 47 version 875 is already newer than 874 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 875 (ObjNum 352 snap 114 seq_num 352) dirty exists 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1110: left oid 47 (ObjNum 352 snap 114 seq_num 352) 2024-03-20T08:05:08.110 INFO:tasks.rados.rados.0.smithi118.stdout:1111: expect (ObjNum 306 snap 100 seq_num 306) 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[95692]: osdmap e280: 8 total, 8 up, 8 in 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[93121]: osdmap e280: 8 total, 8 up, 8 in 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:08.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T08:05:08.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:07 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:08.840 INFO:tasks.rados.rados.0.smithi118.stdout:1110: done (3 left) 2024-03-20T08:05:08.840 INFO:tasks.rados.rados.0.smithi118.stdout:1111: done (2 left) 2024-03-20T08:05:08.840 INFO:tasks.rados.rados.0.smithi118.stdout:1112: done (1 left) 2024-03-20T08:05:08.840 INFO:tasks.rados.rados.0.smithi118.stdout:1113: done (0 left) 2024-03-20T08:05:08.840 INFO:tasks.rados.rados.0.smithi118.stdout:1114: rollback oid 29 current snap is 115 2024-03-20T08:05:08.840 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 96 2024-03-20T08:05:08.840 INFO:tasks.rados.rados.0.smithi118.stdout:1115: snap_remove snap 114 2024-03-20T08:05:09.028 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:08 smithi196 ceph-mon[87080]: Reconfiguring osd.7 (monmap changed)... 2024-03-20T08:05:09.028 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:08 smithi196 ceph-mon[87080]: Reconfiguring daemon osd.7 on smithi196 2024-03-20T08:05:09.028 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:08 smithi196 ceph-mon[87080]: osdmap e281: 8 total, 8 up, 8 in 2024-03-20T08:05:09.028 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:08 smithi196 ceph-mon[87080]: pgmap v98: 121 pgs: 121 active+clean; 311 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 14 op/s 2024-03-20T08:05:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:08 smithi118 ceph-mon[93121]: Reconfiguring osd.7 (monmap changed)... 2024-03-20T08:05:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:08 smithi118 ceph-mon[93121]: Reconfiguring daemon osd.7 on smithi196 2024-03-20T08:05:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:08 smithi118 ceph-mon[93121]: osdmap e281: 8 total, 8 up, 8 in 2024-03-20T08:05:09.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:08 smithi118 ceph-mon[93121]: pgmap v98: 121 pgs: 121 active+clean; 311 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 14 op/s 2024-03-20T08:05:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:08 smithi118 ceph-mon[95692]: Reconfiguring osd.7 (monmap changed)... 2024-03-20T08:05:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:08 smithi118 ceph-mon[95692]: Reconfiguring daemon osd.7 on smithi196 2024-03-20T08:05:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:08 smithi118 ceph-mon[95692]: osdmap e281: 8 total, 8 up, 8 in 2024-03-20T08:05:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:08 smithi118 ceph-mon[95692]: pgmap v98: 121 pgs: 121 active+clean; 311 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 14 op/s 2024-03-20T08:05:09.764 INFO:tasks.rados.rados.0.smithi118.stdout:1115: done (1 left) 2024-03-20T08:05:09.764 INFO:tasks.rados.rados.0.smithi118.stdout:1116: read oid 4 snap -1 2024-03-20T08:05:09.764 INFO:tasks.rados.rados.0.smithi118.stdout:1116: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:09.764 INFO:tasks.rados.rados.0.smithi118.stdout:1117: delete oid 20 current snap is 115 2024-03-20T08:05:09.764 INFO:tasks.rados.rados.0.smithi118.stdout:1114: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:05:09.764 INFO:tasks.rados.rados.0.smithi118.stdout:1114: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:05:09.764 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 877 (ObjNum 269 snap 92 seq_num 269) dirty exists 2024-03-20T08:05:09.772 INFO:tasks.rados.rados.0.smithi118.stdout:1114: done (2 left) 2024-03-20T08:05:09.772 INFO:tasks.rados.rados.0.smithi118.stdout:1117: done (1 left) 2024-03-20T08:05:09.773 INFO:tasks.rados.rados.0.smithi118.stdout:1118: snap_remove snap 96 2024-03-20T08:05:10.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:09 smithi196 ceph-mon[87080]: osdmap e282: 8 total, 8 up, 8 in 2024-03-20T08:05:10.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:10.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:10.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:09 smithi196 ceph-mon[87080]: Reconfiguring mds.b (monmap changed)... 2024-03-20T08:05:10.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:05:10.049 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:10.049 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:09 smithi196 ceph-mon[87080]: Reconfiguring daemon mds.b on smithi196 2024-03-20T08:05:10.049 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:09 smithi196 ceph-mon[87080]: osdmap e283: 8 total, 8 up, 8 in 2024-03-20T08:05:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[93121]: osdmap e282: 8 total, 8 up, 8 in 2024-03-20T08:05:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[93121]: Reconfiguring mds.b (monmap changed)... 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[93121]: Reconfiguring daemon mds.b on smithi196 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[93121]: osdmap e283: 8 total, 8 up, 8 in 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[95692]: osdmap e282: 8 total, 8 up, 8 in 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[95692]: Reconfiguring mds.b (monmap changed)... 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[95692]: Reconfiguring daemon mds.b on smithi196 2024-03-20T08:05:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:09 smithi118 ceph-mon[95692]: osdmap e283: 8 total, 8 up, 8 in 2024-03-20T08:05:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:1118: done (1 left) 2024-03-20T08:05:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:1119: delete oid 11 current snap is 115 2024-03-20T08:05:10.766 INFO:tasks.rados.rados.0.smithi118.stdout:1116: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:10.886 INFO:tasks.rados.rados.0.smithi118.stdout:1116: done (1 left) 2024-03-20T08:05:10.886 INFO:tasks.rados.rados.0.smithi118.stdout:1119: done (0 left) 2024-03-20T08:05:10.887 INFO:tasks.rados.rados.0.smithi118.stdout:1120: snap_remove snap 110 2024-03-20T08:05:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:10 smithi196 ceph-mon[87080]: pgmap v101: 121 pgs: 121 active+clean; 322 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:05:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:10 smithi196 ceph-mon[87080]: osdmap e284: 8 total, 8 up, 8 in 2024-03-20T08:05:11.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:10 smithi118 ceph-mon[93121]: pgmap v101: 121 pgs: 121 active+clean; 322 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:05:11.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:10 smithi118 ceph-mon[93121]: osdmap e284: 8 total, 8 up, 8 in 2024-03-20T08:05:11.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:10 smithi118 ceph-mon[95692]: pgmap v101: 121 pgs: 121 active+clean; 322 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:05:11.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:10 smithi118 ceph-mon[95692]: osdmap e284: 8 total, 8 up, 8 in 2024-03-20T08:05:11.888 INFO:tasks.rados.rados.0.smithi118.stdout:1120: done (0 left) 2024-03-20T08:05:11.888 INFO:tasks.rados.rados.0.smithi118.stdout:1121: read oid 6 snap -1 2024-03-20T08:05:11.888 INFO:tasks.rados.rados.0.smithi118.stdout:1121: expect deleted 2024-03-20T08:05:11.888 INFO:tasks.rados.rados.0.smithi118.stdout:1122: copy_from oid 50 from oid 30 current snap is 115 2024-03-20T08:05:11.888 INFO:tasks.rados.rados.0.smithi118.stdout:1123: rmattr oid 49 current snap is 115 2024-03-20T08:05:11.888 INFO:tasks.rados.rados.0.smithi118.stdout:1123: done (2 left) 2024-03-20T08:05:11.888 INFO:tasks.rados.rados.0.smithi118.stdout:1124: delete oid 20 current snap is 115 2024-03-20T08:05:11.895 INFO:tasks.rados.rados.0.smithi118.stdout:1121: done (2 left) 2024-03-20T08:05:11.895 INFO:tasks.rados.rados.0.smithi118.stdout:1124: done (1 left) 2024-03-20T08:05:11.895 INFO:tasks.rados.rados.0.smithi118.stdout:1125: delete oid 45 current snap is 115 2024-03-20T08:05:11.901 INFO:tasks.rados.rados.0.smithi118.stdout:1122: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:05:11.902 INFO:tasks.rados.rados.0.smithi118.stdout:1125: done (1 left) 2024-03-20T08:05:11.902 INFO:tasks.rados.rados.0.smithi118.stdout:1126: read oid 39 snap 97 2024-03-20T08:05:11.902 INFO:tasks.rados.rados.0.smithi118.stdout:1126: expect deleted 2024-03-20T08:05:11.902 INFO:tasks.rados.rados.0.smithi118.stdout:1127: copy_from oid 23 from oid 27 current snap is 115 2024-03-20T08:05:11.902 INFO:tasks.rados.rados.0.smithi118.stdout:1128: copy_from oid 44 from oid 31 current snap is 115 2024-03-20T08:05:11.902 INFO:tasks.rados.rados.0.smithi118.stdout:1129: snap_remove snap 69 2024-03-20T08:05:12.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all mon 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all crash 2024-03-20T08:05:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[93121]: Upgrade: osd.0 is safe to restart 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all mon 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-03-20T08:05:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all crash 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-20T08:05:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:11 smithi118 ceph-mon[95692]: Upgrade: osd.0 is safe to restart 2024-03-20T08:05:12.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:12.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all mon 2024-03-20T08:05:12.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]: dispatch 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.a"}]': finished 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]: dispatch 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.b"}]': finished 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]: dispatch 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.c"}]': finished 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all crash 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-20T08:05:12.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:11 smithi196 ceph-mon[87080]: Upgrade: osd.0 is safe to restart 2024-03-20T08:05:12.638 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:12.902 INFO:tasks.rados.rados.0.smithi118.stdout:1129: done (4 left) 2024-03-20T08:05:12.902 INFO:tasks.rados.rados.0.smithi118.stdout:1130: read oid 16 snap 106 2024-03-20T08:05:12.902 INFO:tasks.rados.rados.0.smithi118.stdout:1130: expect (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:05:12.902 INFO:tasks.rados.rados.0.smithi118.stdout:1131: snap_create 2024-03-20T08:05:12.902 INFO:tasks.rados.rados.0.smithi118.stdout:1122: finishing copy_from to smithi11881747-50 2024-03-20T08:05:12.902 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 289 (ObjNum 320 snap 104 seq_num 320) dirty exists 2024-03-20T08:05:12.902 INFO:tasks.rados.rados.0.smithi118.stdout:1127: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:05:12.903 INFO:tasks.rados.rados.0.smithi118.stdout:1127: finishing copy_from to smithi11881747-23 2024-03-20T08:05:12.903 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:05:12.903 INFO:tasks.rados.rados.0.smithi118.stdout:1128: finishing copy_from racing read to smithi11881747-44 2024-03-20T08:05:12.903 INFO:tasks.rados.rados.0.smithi118.stdout:1128: finishing copy_from to smithi11881747-44 2024-03-20T08:05:12.903 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 772 (ObjNum 207 snap 67 seq_num 207) dirty exists 2024-03-20T08:05:12.920 INFO:tasks.rados.rados.0.smithi118.stdout:1130: expect (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:05:13.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[93121]: osdmap e285: 8 total, 8 up, 8 in 2024-03-20T08:05:13.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[93121]: pgmap v104: 121 pgs: 121 active+clean; 322 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:05:13.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:13.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T08:05:13.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:13.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[95692]: osdmap e285: 8 total, 8 up, 8 in 2024-03-20T08:05:13.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[95692]: pgmap v104: 121 pgs: 121 active+clean; 322 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:05:13.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:13.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T08:05:13.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:12 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:12 smithi196 ceph-mon[87080]: osdmap e285: 8 total, 8 up, 8 in 2024-03-20T08:05:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:12 smithi196 ceph-mon[87080]: pgmap v104: 121 pgs: 121 active+clean; 322 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:05:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:12 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:12 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-20T08:05:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:12 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:13.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:13.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:13.899 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:05:13] "GET /metrics HTTP/1.1" 200 35998 "" "Prometheus/2.43.0" 2024-03-20T08:05:13.906 INFO:tasks.rados.rados.0.smithi118.stdout:1122: done (5 left) 2024-03-20T08:05:13.906 INFO:tasks.rados.rados.0.smithi118.stdout:1126: done (4 left) 2024-03-20T08:05:13.906 INFO:tasks.rados.rados.0.smithi118.stdout:1127: done (3 left) 2024-03-20T08:05:13.906 INFO:tasks.rados.rados.0.smithi118.stdout:1128: done (2 left) 2024-03-20T08:05:13.906 INFO:tasks.rados.rados.0.smithi118.stdout:1130: done (1 left) 2024-03-20T08:05:13.907 INFO:tasks.rados.rados.0.smithi118.stdout:1131: done (0 left) 2024-03-20T08:05:13.907 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 116 2024-03-20T08:05:13.907 INFO:tasks.rados.rados.0.smithi118.stdout:1132: seq_num 357 ranges {1458176=786432,2244608=401408} 2024-03-20T08:05:13.917 INFO:tasks.rados.rados.0.smithi118.stdout:1132: writing smithi11881747-13 from 1458176 to 2244608 tid 1 2024-03-20T08:05:13.920 INFO:tasks.rados.rados.0.smithi118.stdout:1132: writing smithi11881747-13 from 2244608 to 2646016 tid 2 2024-03-20T08:05:13.920 INFO:tasks.rados.rados.0.smithi118.stdout:1133: snap_create 2024-03-20T08:05:13.926 INFO:tasks.rados.rados.0.smithi118.stdout:1132: finishing write tid 1 to smithi11881747-13 2024-03-20T08:05:13.926 INFO:tasks.rados.rados.0.smithi118.stdout:1132: finishing write tid 2 to smithi11881747-13 2024-03-20T08:05:13.926 INFO:tasks.rados.rados.0.smithi118.stdout:1132: finishing write tid 3 to smithi11881747-13 2024-03-20T08:05:13.927 INFO:tasks.rados.rados.0.smithi118.stdout:1132: finishing write tid 4 to smithi11881747-13 2024-03-20T08:05:13.927 INFO:tasks.rados.rados.0.smithi118.stdout:1132: oid 13 updating version 0 to 705 2024-03-20T08:05:13.927 INFO:tasks.rados.rados.0.smithi118.stdout:1132: oid 13 updating version 705 to 706 2024-03-20T08:05:13.927 INFO:tasks.rados.rados.0.smithi118.stdout:1132: oid 13 version 706 is already newer than 704 2024-03-20T08:05:13.927 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 706 (ObjNum 357 snap 116 seq_num 357) dirty exists 2024-03-20T08:05:13.927 INFO:tasks.rados.rados.0.smithi118.stdout:1132: left oid 13 (ObjNum 357 snap 116 seq_num 357) 2024-03-20T08:05:14.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-mon[93121]: Upgrade: Updating osd.0 2024-03-20T08:05:14.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-mon[93121]: Deploying daemon osd.0 on smithi118 2024-03-20T08:05:14.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-mon[93121]: osdmap e286: 8 total, 8 up, 8 in 2024-03-20T08:05:14.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:14.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-mon[95692]: Upgrade: Updating osd.0 2024-03-20T08:05:14.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-mon[95692]: Deploying daemon osd.0 on smithi118 2024-03-20T08:05:14.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-mon[95692]: osdmap e286: 8 total, 8 up, 8 in 2024-03-20T08:05:14.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:13 smithi196 ceph-mon[87080]: Upgrade: Updating osd.0 2024-03-20T08:05:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:13 smithi196 ceph-mon[87080]: Deploying daemon osd.0 on smithi118 2024-03-20T08:05:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:13 smithi196 ceph-mon[87080]: osdmap e286: 8 total, 8 up, 8 in 2024-03-20T08:05:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:14.911 INFO:tasks.rados.rados.0.smithi118.stdout:1132: done (1 left) 2024-03-20T08:05:14.911 INFO:tasks.rados.rados.0.smithi118.stdout:1133: done (0 left) 2024-03-20T08:05:14.911 INFO:tasks.rados.rados.0.smithi118.stdout:1134: rollback oid 26 current snap is 117 2024-03-20T08:05:14.911 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 108 2024-03-20T08:05:14.911 INFO:tasks.rados.rados.0.smithi118.stdout:1135: copy_from oid 48 from oid 14 current snap is 117 2024-03-20T08:05:14.911 INFO:tasks.rados.rados.0.smithi118.stdout:1136: snap_create 2024-03-20T08:05:14.918 INFO:tasks.rados.rados.0.smithi118.stdout:1134: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:05:14.918 INFO:tasks.rados.rados.0.smithi118.stdout:1134: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:05:14.918 INFO:tasks.rados.rados.0.smithi118.stdout:1134: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:05:14.918 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 775 (ObjNum 266 snap 92 seq_num 266) dirty exists 2024-03-20T08:05:14.926 INFO:tasks.rados.rados.0.smithi118.stdout:1135: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:05:14.926 INFO:tasks.rados.rados.0.smithi118.stdout:1135: finishing copy_from to smithi11881747-48 2024-03-20T08:05:14.926 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 773 (ObjNum 104 snap 23 seq_num 104) dirty exists 2024-03-20T08:05:15.069 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:14 smithi118 ceph-mon[93121]: osdmap e287: 8 total, 8 up, 8 in 2024-03-20T08:05:15.069 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:14 smithi118 ceph-mon[93121]: pgmap v107: 121 pgs: 121 active+clean; 331 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 1 op/s 2024-03-20T08:05:15.070 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:14 smithi118 ceph-mon[95692]: osdmap e287: 8 total, 8 up, 8 in 2024-03-20T08:05:15.070 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:14 smithi118 ceph-mon[95692]: pgmap v107: 121 pgs: 121 active+clean; 331 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 1 op/s 2024-03-20T08:05:15.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:14 smithi196 ceph-mon[87080]: osdmap e287: 8 total, 8 up, 8 in 2024-03-20T08:05:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:14 smithi196 ceph-mon[87080]: pgmap v107: 121 pgs: 121 active+clean; 331 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 1 op/s 2024-03-20T08:05:15.913 INFO:tasks.rados.rados.0.smithi118.stdout:1134: done (2 left) 2024-03-20T08:05:15.913 INFO:tasks.rados.rados.0.smithi118.stdout:1135: done (1 left) 2024-03-20T08:05:15.913 INFO:tasks.rados.rados.0.smithi118.stdout:1136: done (0 left) 2024-03-20T08:05:15.913 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 118 2024-03-20T08:05:15.913 INFO:tasks.rados.rados.0.smithi118.stdout:1137: seq_num 358 ranges {3391488=802816,4194304=401408} 2024-03-20T08:05:15.937 INFO:tasks.rados.rados.0.smithi118.stdout:1137: writing smithi11881747-23 from 3391488 to 4194304 tid 1 2024-03-20T08:05:15.939 INFO:tasks.rados.rados.0.smithi118.stdout:1137: writing smithi11881747-23 from 4194304 to 4595712 tid 2 2024-03-20T08:05:15.940 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 118 2024-03-20T08:05:15.940 INFO:tasks.rados.rados.0.smithi118.stdout:1138: seq_num 359 ranges {1073152=417792,1490944=417792,1908736=385024} 2024-03-20T08:05:15.949 INFO:tasks.rados.rados.0.smithi118.stdout:1138: writing smithi11881747-36 from 1073152 to 1490944 tid 1 2024-03-20T08:05:15.951 INFO:tasks.rados.rados.0.smithi118.stdout:1138: writing smithi11881747-36 from 1490944 to 1908736 tid 2 2024-03-20T08:05:15.954 INFO:tasks.rados.rados.0.smithi118.stdout:1138: writing smithi11881747-36 from 1908736 to 2293760 tid 3 2024-03-20T08:05:15.954 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 118 2024-03-20T08:05:15.954 INFO:tasks.rados.rados.0.smithi118.stdout:1137: finishing write tid 1 to smithi11881747-23 2024-03-20T08:05:15.954 INFO:tasks.rados.rados.0.smithi118.stdout:1139: seq_num 360 ranges {0=434176,434176=704512,1138688=98304} 2024-03-20T08:05:15.956 INFO:tasks.rados.rados.0.smithi118.stdout:1139: writing smithi11881747-12 from 0 to 434176 tid 1 2024-03-20T08:05:15.961 INFO:tasks.rados.rados.0.smithi118.stdout:1139: writing smithi11881747-12 from 434176 to 1138688 tid 2 2024-03-20T08:05:15.962 INFO:tasks.rados.rados.0.smithi118.stdout:1139: writing smithi11881747-12 from 1138688 to 1236992 tid 3 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1140: snap_create 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1137: finishing write tid 2 to smithi11881747-23 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1137: finishing write tid 3 to smithi11881747-23 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1137: finishing write tid 4 to smithi11881747-23 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1137: oid 23 updating version 0 to 905 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1137: oid 23 version 905 is already newer than 903 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1137: oid 23 version 905 is already newer than 904 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 905 (ObjNum 358 snap 118 seq_num 358) dirty exists 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1137: left oid 23 (ObjNum 358 snap 118 seq_num 358) 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1138: finishing write tid 1 to smithi11881747-36 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1138: finishing write tid 2 to smithi11881747-36 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1138: finishing write tid 3 to smithi11881747-36 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1138: finishing write tid 4 to smithi11881747-36 2024-03-20T08:05:15.963 INFO:tasks.rados.rados.0.smithi118.stdout:1138: finishing write tid 5 to smithi11881747-36 2024-03-20T08:05:15.964 INFO:tasks.rados.rados.0.smithi118.stdout:1138: oid 36 updating version 0 to 264 2024-03-20T08:05:15.964 INFO:tasks.rados.rados.0.smithi118.stdout:1138: oid 36 version 264 is already newer than 263 2024-03-20T08:05:15.964 INFO:tasks.rados.rados.0.smithi118.stdout:1138: oid 36 updating version 264 to 266 2024-03-20T08:05:15.964 INFO:tasks.rados.rados.0.smithi118.stdout:1138: oid 36 version 266 is already newer than 265 2024-03-20T08:05:15.964 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 266 (ObjNum 359 snap 118 seq_num 359) dirty exists 2024-03-20T08:05:15.964 INFO:tasks.rados.rados.0.smithi118.stdout:1138: left oid 36 (ObjNum 359 snap 118 seq_num 359) 2024-03-20T08:05:15.964 INFO:tasks.rados.rados.0.smithi118.stdout:1139: finishing write tid 1 to smithi11881747-12 2024-03-20T08:05:15.965 INFO:tasks.rados.rados.0.smithi118.stdout:1139: finishing write tid 2 to smithi11881747-12 2024-03-20T08:05:15.965 INFO:tasks.rados.rados.0.smithi118.stdout:1139: finishing write tid 3 to smithi11881747-12 2024-03-20T08:05:15.965 INFO:tasks.rados.rados.0.smithi118.stdout:1139: finishing write tid 4 to smithi11881747-12 2024-03-20T08:05:15.966 INFO:tasks.rados.rados.0.smithi118.stdout:1139: finishing write tid 5 to smithi11881747-12 2024-03-20T08:05:15.966 INFO:tasks.rados.rados.0.smithi118.stdout:1139: oid 12 updating version 0 to 215 2024-03-20T08:05:15.966 INFO:tasks.rados.rados.0.smithi118.stdout:1139: oid 12 version 215 is already newer than 213 2024-03-20T08:05:15.966 INFO:tasks.rados.rados.0.smithi118.stdout:1139: oid 12 version 215 is already newer than 214 2024-03-20T08:05:15.966 INFO:tasks.rados.rados.0.smithi118.stdout:1139: oid 12 version 215 is already newer than 212 2024-03-20T08:05:15.966 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 215 (ObjNum 360 snap 118 seq_num 360) dirty exists 2024-03-20T08:05:15.966 INFO:tasks.rados.rados.0.smithi118.stdout:1139: left oid 12 (ObjNum 360 snap 118 seq_num 360) 2024-03-20T08:05:16.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:15 smithi118 ceph-mon[93121]: osdmap e288: 8 total, 8 up, 8 in 2024-03-20T08:05:16.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:15 smithi118 ceph-mon[95692]: osdmap e288: 8 total, 8 up, 8 in 2024-03-20T08:05:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:15 smithi196 ceph-mon[87080]: osdmap e288: 8 total, 8 up, 8 in 2024-03-20T08:05:16.522 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:16 smithi118 systemd[1]: Stopping Ceph osd.0 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:05:16.865 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[43417]: 2024-03-20T08:05:16.522+0000 7f7914ab2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:05:16.865 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[43417]: 2024-03-20T08:05:16.522+0000 7f7914ab2700 -1 osd.0 289 *** Got signal Terminated *** 2024-03-20T08:05:16.865 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[43417]: 2024-03-20T08:05:16.522+0000 7f7914ab2700 -1 osd.0 289 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:05:16.925 INFO:tasks.rados.rados.0.smithi118.stdout:1137: done (3 left) 2024-03-20T08:05:16.925 INFO:tasks.rados.rados.0.smithi118.stdout:1138: done (2 left) 2024-03-20T08:05:16.925 INFO:tasks.rados.rados.0.smithi118.stdout:1139: done (1 left) 2024-03-20T08:05:16.925 INFO:tasks.rados.rados.0.smithi118.stdout:1140: done (0 left) 2024-03-20T08:05:16.925 INFO:tasks.rados.rados.0.smithi118.stdout:1141: read oid 16 snap -1 2024-03-20T08:05:16.925 INFO:tasks.rados.rados.0.smithi118.stdout:1141: expect (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:05:16.925 INFO:tasks.rados.rados.0.smithi118.stdout:1142: snap_create 2024-03-20T08:05:16.938 INFO:tasks.rados.rados.0.smithi118.stdout:1141: expect (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:05:17.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-mon[93121]: osdmap e289: 8 total, 8 up, 8 in 2024-03-20T08:05:17.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-mon[93121]: pgmap v110: 121 pgs: 121 active+clean; 331 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 5 op/s 2024-03-20T08:05:17.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-mon[93121]: osd.0 marked itself down and dead 2024-03-20T08:05:17.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-mon[95692]: osdmap e289: 8 total, 8 up, 8 in 2024-03-20T08:05:17.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-mon[95692]: pgmap v110: 121 pgs: 121 active+clean; 331 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 5 op/s 2024-03-20T08:05:17.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:16 smithi118 ceph-mon[95692]: osd.0 marked itself down and dead 2024-03-20T08:05:17.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:16 smithi196 ceph-mon[87080]: osdmap e289: 8 total, 8 up, 8 in 2024-03-20T08:05:17.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:16 smithi196 ceph-mon[87080]: pgmap v110: 121 pgs: 121 active+clean; 331 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 5 op/s 2024-03-20T08:05:17.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:16 smithi196 ceph-mon[87080]: osd.0 marked itself down and dead 2024-03-20T08:05:17.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:17 smithi118 podman[101279]: 2024-03-20 08:05:17.473322006 +0000 UTC m=+1.063228419 container died 481bba84ab315a02b97d55bc40e3560a0d420e3c0c0c9c6039fc579e4daefa83 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, ceph=True, org.label-schema.build-date=20240319, RELEASE=reef-78dbc53, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-20T08:05:17.930 INFO:tasks.rados.rados.0.smithi118.stdout:1141: done (1 left) 2024-03-20T08:05:17.930 INFO:tasks.rados.rados.0.smithi118.stdout:1142: done (0 left) 2024-03-20T08:05:17.930 INFO:tasks.rados.rados.0.smithi118.stdout:1143: rmattr oid 40 current snap is 120 2024-03-20T08:05:17.930 INFO:tasks.rados.rados.0.smithi118.stdout:1143: done (0 left) 2024-03-20T08:05:17.931 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 120 2024-03-20T08:05:17.931 INFO:tasks.rados.rados.0.smithi118.stdout:1144: seq_num 361 ranges {3391488=450560,3842048=581632,4423680=221184} 2024-03-20T08:05:17.951 INFO:tasks.rados.rados.0.smithi118.stdout:1144: writing smithi11881747-19 from 3391488 to 3842048 tid 1 2024-03-20T08:05:17.954 INFO:tasks.rados.rados.0.smithi118.stdout:1144: writing smithi11881747-19 from 3842048 to 4423680 tid 2 2024-03-20T08:05:17.956 INFO:tasks.rados.rados.0.smithi118.stdout:1144: writing smithi11881747-19 from 4423680 to 4644864 tid 3 2024-03-20T08:05:17.956 INFO:tasks.rados.rados.0.smithi118.stdout:1145: snap_create 2024-03-20T08:05:17.956 INFO:tasks.rados.rados.0.smithi118.stdout:1144: finishing write tid 1 to smithi11881747-19 2024-03-20T08:05:17.958 INFO:tasks.rados.rados.0.smithi118.stdout:1144: finishing write tid 2 to smithi11881747-19 2024-03-20T08:05:17.958 INFO:tasks.rados.rados.0.smithi118.stdout:1144: finishing write tid 3 to smithi11881747-19 2024-03-20T08:05:17.958 INFO:tasks.rados.rados.0.smithi118.stdout:1144: finishing write tid 4 to smithi11881747-19 2024-03-20T08:05:17.960 INFO:tasks.rados.rados.0.smithi118.stdout:1144: finishing write tid 5 to smithi11881747-19 2024-03-20T08:05:17.960 INFO:tasks.rados.rados.0.smithi118.stdout:1144: oid 19 updating version 0 to 214 2024-03-20T08:05:17.960 INFO:tasks.rados.rados.0.smithi118.stdout:1144: oid 19 updating version 214 to 215 2024-03-20T08:05:17.960 INFO:tasks.rados.rados.0.smithi118.stdout:1144: oid 19 version 215 is already newer than 213 2024-03-20T08:05:17.960 INFO:tasks.rados.rados.0.smithi118.stdout:1144: oid 19 version 215 is already newer than 212 2024-03-20T08:05:17.960 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 215 (ObjNum 361 snap 120 seq_num 361) dirty exists 2024-03-20T08:05:17.960 INFO:tasks.rados.rados.0.smithi118.stdout:1144: left oid 19 (ObjNum 361 snap 120 seq_num 361) 2024-03-20T08:05:18.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:17 smithi118 ceph-mon[93121]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:05:18.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:17 smithi118 ceph-mon[93121]: osdmap e290: 8 total, 7 up, 8 in 2024-03-20T08:05:18.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:17 smithi118 ceph-mon[95692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:05:18.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:17 smithi118 ceph-mon[95692]: osdmap e290: 8 total, 7 up, 8 in 2024-03-20T08:05:18.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:17 smithi118 podman[101279]: 2024-03-20 08:05:17.978653165 +0000 UTC m=+1.568559623 container cleanup 481bba84ab315a02b97d55bc40e3560a0d420e3c0c0c9c6039fc579e4daefa83 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, RELEASE=reef-78dbc53, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:05:18.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:17 smithi118 bash[101279]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0 2024-03-20T08:05:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:17 smithi196 ceph-mon[87080]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:05:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:17 smithi196 ceph-mon[87080]: osdmap e290: 8 total, 7 up, 8 in 2024-03-20T08:05:18.581 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:18 smithi118 podman[101302]: 2024-03-20 08:05:18.238540821 +0000 UTC m=+0.759564100 container remove 481bba84ab315a02b97d55bc40e3560a0d420e3c0c0c9c6039fc579e4daefa83 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=reef-78dbc53, ceph=True, GIT_BRANCH=HEAD) 2024-03-20T08:05:18.915 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:18 smithi118 podman[101399]: 2024-03-20 08:05:18.486006642 +0000 UTC m=+0.017853460 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:05:18.915 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:18 smithi118 podman[101399]: 2024-03-20 08:05:18.589318653 +0000 UTC m=+0.121165495 container create 3392421cdfe57e7dcf825e5b3391981bc5c62f6ca2c47336b50c0c572a262886 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-03-20T08:05:18.915 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:18 smithi118 podman[101399]: 2024-03-20 08:05:18.825526715 +0000 UTC m=+0.357373528 container init 3392421cdfe57e7dcf825e5b3391981bc5c62f6ca2c47336b50c0c572a262886 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-20T08:05:18.916 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:18 smithi118 podman[101399]: 2024-03-20 08:05:18.831337421 +0000 UTC m=+0.363184224 container start 3392421cdfe57e7dcf825e5b3391981bc5c62f6ca2c47336b50c0c572a262886 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-20T08:05:18.916 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:18 smithi118 podman[101399]: 2024-03-20 08:05:18.916911529 +0000 UTC m=+0.448758348 container attach 3392421cdfe57e7dcf825e5b3391981bc5c62f6ca2c47336b50c0c572a262886 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:05:18.934 INFO:tasks.rados.rados.0.smithi118.stdout:1144: done (1 left) 2024-03-20T08:05:18.934 INFO:tasks.rados.rados.0.smithi118.stdout:1145: done (0 left) 2024-03-20T08:05:18.934 INFO:tasks.rados.rados.0.smithi118.stdout:1146: snap_remove snap 109 2024-03-20T08:05:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:18 smithi118 ceph-mon[93121]: osdmap e291: 8 total, 7 up, 8 in 2024-03-20T08:05:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:18 smithi118 ceph-mon[93121]: pgmap v113: 121 pgs: 15 stale+active+clean, 106 active+clean; 331 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 3 op/s 2024-03-20T08:05:19.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:18 smithi118 ceph-mon[95692]: osdmap e291: 8 total, 7 up, 8 in 2024-03-20T08:05:19.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:18 smithi118 ceph-mon[95692]: pgmap v113: 121 pgs: 15 stale+active+clean, 106 active+clean; 331 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 3 op/s 2024-03-20T08:05:19.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:19 smithi118 podman[101399]: 2024-03-20 08:05:19.130027133 +0000 UTC m=+0.661873933 container died 3392421cdfe57e7dcf825e5b3391981bc5c62f6ca2c47336b50c0c572a262886 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-20T08:05:19.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:18 smithi196 ceph-mon[87080]: osdmap e291: 8 total, 7 up, 8 in 2024-03-20T08:05:19.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:18 smithi196 ceph-mon[87080]: pgmap v113: 121 pgs: 15 stale+active+clean, 106 active+clean; 331 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 3 op/s 2024-03-20T08:05:19.767 INFO:tasks.rados.rados.0.smithi118.stdout:1146: done (0 left) 2024-03-20T08:05:19.768 INFO:tasks.rados.rados.0.smithi118.stdout:1147: copy_from oid 33 from oid 1 current snap is 121 2024-03-20T08:05:19.768 INFO:tasks.rados.rados.0.smithi118.stdout:1148: read oid 46 snap -1 2024-03-20T08:05:19.768 INFO:tasks.rados.rados.0.smithi118.stdout:1148: expect (ObjNum 346 snap 112 seq_num 346) 2024-03-20T08:05:19.768 INFO:tasks.rados.rados.0.smithi118.stdout:1149: snap_remove snap 64 2024-03-20T08:05:20.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:19 smithi118 ceph-mon[93121]: osdmap e292: 8 total, 7 up, 8 in 2024-03-20T08:05:20.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:19 smithi118 ceph-mon[93121]: osdmap e293: 8 total, 7 up, 8 in 2024-03-20T08:05:20.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:19 smithi118 ceph-mon[95692]: osdmap e292: 8 total, 7 up, 8 in 2024-03-20T08:05:20.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:19 smithi118 ceph-mon[95692]: osdmap e293: 8 total, 7 up, 8 in 2024-03-20T08:05:20.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:19 smithi118 podman[101445]: 2024-03-20 08:05:19.745817971 +0000 UTC m=+0.610661851 container remove 3392421cdfe57e7dcf825e5b3391981bc5c62f6ca2c47336b50c0c572a262886 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-20T08:05:20.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:19 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-20T08:05:20.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:19 smithi118 systemd[1]: Stopped Ceph osd.0 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:05:20.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:19 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.0.service: Consumed 10.477s CPU time. 2024-03-20T08:05:20.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:19 smithi118 systemd[1]: Starting Ceph osd.0 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:05:20.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:19 smithi196 ceph-mon[87080]: osdmap e292: 8 total, 7 up, 8 in 2024-03-20T08:05:20.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:19 smithi196 ceph-mon[87080]: osdmap e293: 8 total, 7 up, 8 in 2024-03-20T08:05:20.456 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 podman[101551]: 2024-03-20 08:05:20.088369481 +0000 UTC m=+0.017954064 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:05:20.456 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 podman[101551]: 2024-03-20 08:05:20.230194571 +0000 UTC m=+0.159779129 container create cdeb6a511ef08756e08f93f802530b764d50ecdb69611326a0b694c1131cdb29 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-03-20T08:05:20.768 INFO:tasks.rados.rados.0.smithi118.stdout:1149: done (2 left) 2024-03-20T08:05:20.768 INFO:tasks.rados.rados.0.smithi118.stdout:1150: snap_remove snap 103 2024-03-20T08:05:20.906 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 podman[101551]: 2024-03-20 08:05:20.458020731 +0000 UTC m=+0.387605299 container init cdeb6a511ef08756e08f93f802530b764d50ecdb69611326a0b694c1131cdb29 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-20T08:05:20.907 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 podman[101551]: 2024-03-20 08:05:20.463673432 +0000 UTC m=+0.393257991 container start cdeb6a511ef08756e08f93f802530b764d50ecdb69611326a0b694c1131cdb29 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-20T08:05:20.907 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 podman[101551]: 2024-03-20 08:05:20.549568942 +0000 UTC m=+0.479153501 container attach cdeb6a511ef08756e08f93f802530b764d50ecdb69611326a0b694c1131cdb29 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-20T08:05:21.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-mon[95692]: pgmap v116: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 337 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 6 op/s; 149/975 objects degraded (15.282%) 2024-03-20T08:05:21.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-mon[95692]: osdmap e294: 8 total, 7 up, 8 in 2024-03-20T08:05:21.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-mon[93121]: pgmap v116: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 337 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 6 op/s; 149/975 objects degraded (15.282%) 2024-03-20T08:05:21.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-mon[93121]: osdmap e294: 8 total, 7 up, 8 in 2024-03-20T08:05:21.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate[101565]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-20T08:05:21.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 bash[101551]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-20T08:05:21.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 bash[101551]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-20T08:05:21.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 bash[101551]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-20T08:05:21.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 bash[101551]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 bash[101551]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 bash[101551]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 bash[101551]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate[101565]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate[101565]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate[101565]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate[101565]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate[101565]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate[101565]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-20T08:05:21.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:20 smithi118 podman[101551]: 2024-03-20 08:05:20.932173987 +0000 UTC m=+0.861758546 container died cdeb6a511ef08756e08f93f802530b764d50ecdb69611326a0b694c1131cdb29 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-20T08:05:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:20 smithi196 ceph-mon[87080]: pgmap v116: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 337 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 6 op/s; 149/975 objects degraded (15.282%) 2024-03-20T08:05:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:20 smithi196 ceph-mon[87080]: osdmap e294: 8 total, 7 up, 8 in 2024-03-20T08:05:21.768 INFO:tasks.rados.rados.0.smithi118.stdout:1150: done (2 left) 2024-03-20T08:05:21.768 INFO:tasks.rados.rados.0.smithi118.stdout:1151: snap_create 2024-03-20T08:05:21.768 INFO:tasks.rados.rados.0.smithi118.stdout:1147: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:05:21.768 INFO:tasks.rados.rados.0.smithi118.stdout:1147: finishing copy_from to smithi11881747-33 2024-03-20T08:05:21.768 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 780 (ObjNum 343 snap 112 seq_num 343) dirty exists 2024-03-20T08:05:21.769 INFO:tasks.rados.rados.0.smithi118.stdout:1148: expect (ObjNum 346 snap 112 seq_num 346) 2024-03-20T08:05:21.827 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:21 smithi118 podman[101711]: 2024-03-20 08:05:21.478070367 +0000 UTC m=+0.540739592 container remove cdeb6a511ef08756e08f93f802530b764d50ecdb69611326a0b694c1131cdb29 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-20T08:05:21.828 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:21 smithi118 podman[101775]: 2024-03-20 08:05:21.687350038 +0000 UTC m=+0.109624223 container create 56e63ece2c91bc6bea1705d74f32caeb83ac15ef8a118c7add0b66a92cbf79b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-20T08:05:21.828 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:21 smithi118 podman[101775]: 2024-03-20 08:05:21.595550607 +0000 UTC m=+0.017824760 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:05:22.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:21 smithi118 ceph-mon[93121]: Health check failed: Degraded data redundancy: 149/975 objects degraded (15.282%), 14 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:22.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:21 smithi118 ceph-mon[93121]: osdmap e295: 8 total, 7 up, 8 in 2024-03-20T08:05:22.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:21 smithi118 ceph-mon[95692]: Health check failed: Degraded data redundancy: 149/975 objects degraded (15.282%), 14 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:22.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:21 smithi118 ceph-mon[95692]: osdmap e295: 8 total, 7 up, 8 in 2024-03-20T08:05:22.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:21 smithi118 podman[101775]: 2024-03-20 08:05:21.922850182 +0000 UTC m=+0.345124336 container init 56e63ece2c91bc6bea1705d74f32caeb83ac15ef8a118c7add0b66a92cbf79b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-03-20T08:05:22.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:21 smithi118 podman[101775]: 2024-03-20 08:05:21.928227027 +0000 UTC m=+0.350501174 container start 56e63ece2c91bc6bea1705d74f32caeb83ac15ef8a118c7add0b66a92cbf79b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:05:22.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:21 smithi118 ceph-osd[101814]: -- 172.21.15.118:0/1287038818 <== mon.2 v2:172.21.15.118:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d6cd6df680 con 0x55d6cd6a9000 2024-03-20T08:05:22.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:22 smithi118 bash[101775]: 56e63ece2c91bc6bea1705d74f32caeb83ac15ef8a118c7add0b66a92cbf79b7 2024-03-20T08:05:22.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:22 smithi118 systemd[1]: Started Ceph osd.0 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:05:22.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:21 smithi196 ceph-mon[87080]: Health check failed: Degraded data redundancy: 149/975 objects degraded (15.282%), 14 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:22.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:21 smithi196 ceph-mon[87080]: osdmap e295: 8 total, 7 up, 8 in 2024-03-20T08:05:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:22.769 INFO:tasks.rados.rados.0.smithi118.stdout:1147: done (2 left) 2024-03-20T08:05:22.769 INFO:tasks.rados.rados.0.smithi118.stdout:1148: done (1 left) 2024-03-20T08:05:22.769 INFO:tasks.rados.rados.0.smithi118.stdout:1151: done (0 left) 2024-03-20T08:05:22.769 INFO:tasks.rados.rados.0.smithi118.stdout:1152: rmattr oid 26 current snap is 122 2024-03-20T08:05:22.770 INFO:tasks.rados.rados.0.smithi118.stdout:1153: rmattr oid 42 current snap is 122 2024-03-20T08:05:22.770 INFO:tasks.rados.rados.0.smithi118.stdout:1153: done (1 left) 2024-03-20T08:05:22.770 INFO:tasks.rados.rados.0.smithi118.stdout:1154: rmattr oid 11 current snap is 122 2024-03-20T08:05:22.770 INFO:tasks.rados.rados.0.smithi118.stdout:1154: done (1 left) 2024-03-20T08:05:22.770 INFO:tasks.rados.rados.0.smithi118.stdout:1155: snap_remove snap 117 2024-03-20T08:05:22.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:22.475+0000 7f0d1883b7c0 -1 Falling back to public interface 2024-03-20T08:05:23.358 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:23.358 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:23.358 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:23.358 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[95692]: pgmap v119: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 337 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 6 op/s; 149/975 objects degraded (15.282%) 2024-03-20T08:05:23.358 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[95692]: osdmap e296: 8 total, 7 up, 8 in 2024-03-20T08:05:23.359 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:23.359 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:23.359 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:23.359 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[93121]: pgmap v119: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 337 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 6 op/s; 149/975 objects degraded (15.282%) 2024-03-20T08:05:23.359 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-mon[93121]: osdmap e296: 8 total, 7 up, 8 in 2024-03-20T08:05:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:23 smithi196 ceph-mon[87080]: pgmap v119: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 337 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 6 op/s; 149/975 objects degraded (15.282%) 2024-03-20T08:05:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:23 smithi196 ceph-mon[87080]: osdmap e296: 8 total, 7 up, 8 in 2024-03-20T08:05:23.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:05:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:05:23] "GET /metrics HTTP/1.1" 200 35998 "" "Prometheus/2.43.0" 2024-03-20T08:05:23.771 INFO:tasks.rados.rados.0.smithi118.stdout:1155: done (1 left) 2024-03-20T08:05:23.771 INFO:tasks.rados.rados.0.smithi118.stdout:1156: delete oid 37 current snap is 122 2024-03-20T08:05:23.771 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 776 (ObjNum 266 snap 92 seq_num 266) dirty exists 2024-03-20T08:05:23.775 INFO:tasks.rados.rados.0.smithi118.stdout:1152: done (1 left) 2024-03-20T08:05:23.776 INFO:tasks.rados.rados.0.smithi118.stdout:1156: done (0 left) 2024-03-20T08:05:23.776 INFO:tasks.rados.rados.0.smithi118.stdout:1157: rollback oid 3 current snap is 122 2024-03-20T08:05:23.776 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 3 to 119 2024-03-20T08:05:23.776 INFO:tasks.rados.rados.0.smithi118.stdout:1158: setattr oid 37 current snap is 122 2024-03-20T08:05:23.776 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 122 2024-03-20T08:05:23.776 INFO:tasks.rados.rados.0.smithi118.stdout:1159: seq_num 363 ranges {614400=483328,1097728=753664,1851392=49152} 2024-03-20T08:05:23.782 INFO:tasks.rados.rados.0.smithi118.stdout:1159: writing smithi11881747-18 from 614400 to 1097728 tid 1 2024-03-20T08:05:23.787 INFO:tasks.rados.rados.0.smithi118.stdout:1159: writing smithi11881747-18 from 1097728 to 1851392 tid 2 2024-03-20T08:05:23.787 INFO:tasks.rados.rados.0.smithi118.stdout:1159: writing smithi11881747-18 from 1851392 to 1900544 tid 3 2024-03-20T08:05:23.787 INFO:tasks.rados.rados.0.smithi118.stdout:1160: snap_create 2024-03-20T08:05:23.787 INFO:tasks.rados.rados.0.smithi118.stdout:1157: finishing rollback tid 1 to smithi11881747-3 2024-03-20T08:05:23.788 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 250 (ObjNum 65 snap 0 seq_num 1733909839) dirty dne 2024-03-20T08:05:23.788 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 283 (ObjNum 2401 snap 0 seq_num 1864348192) dirty exists 2024-03-20T08:05:23.788 INFO:tasks.rados.rados.0.smithi118.stdout:1159: finishing write tid 1 to smithi11881747-18 2024-03-20T08:05:23.790 INFO:tasks.rados.rados.0.smithi118.stdout:1159: finishing write tid 2 to smithi11881747-18 2024-03-20T08:05:23.790 INFO:tasks.rados.rados.0.smithi118.stdout:1159: finishing write tid 3 to smithi11881747-18 2024-03-20T08:05:23.791 INFO:tasks.rados.rados.0.smithi118.stdout:1159: finishing write tid 4 to smithi11881747-18 2024-03-20T08:05:23.791 INFO:tasks.rados.rados.0.smithi118.stdout:1159: finishing write tid 5 to smithi11881747-18 2024-03-20T08:05:23.792 INFO:tasks.rados.rados.0.smithi118.stdout:1159: oid 18 updating version 0 to 668 2024-03-20T08:05:23.792 INFO:tasks.rados.rados.0.smithi118.stdout:1159: oid 18 version 668 is already newer than 667 2024-03-20T08:05:23.792 INFO:tasks.rados.rados.0.smithi118.stdout:1159: oid 18 updating version 668 to 669 2024-03-20T08:05:23.792 INFO:tasks.rados.rados.0.smithi118.stdout:1159: oid 18 updating version 669 to 670 2024-03-20T08:05:23.792 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 670 (ObjNum 363 snap 122 seq_num 363) dirty exists 2024-03-20T08:05:23.792 INFO:tasks.rados.rados.0.smithi118.stdout:1159: left oid 18 (ObjNum 363 snap 122 seq_num 363) 2024-03-20T08:05:23.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:23.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:24.776 INFO:tasks.rados.rados.0.smithi118.stdout:1157: done (3 left) 2024-03-20T08:05:24.776 INFO:tasks.rados.rados.0.smithi118.stdout:1158: done (2 left) 2024-03-20T08:05:24.776 INFO:tasks.rados.rados.0.smithi118.stdout:1159: done (1 left) 2024-03-20T08:05:24.776 INFO:tasks.rados.rados.0.smithi118.stdout:1160: done (0 left) 2024-03-20T08:05:24.776 INFO:tasks.rados.rados.0.smithi118.stdout:1161: read oid 7 snap 78 2024-03-20T08:05:24.776 INFO:tasks.rados.rados.0.smithi118.stdout:1161: expect (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:05:24.777 INFO:tasks.rados.rados.0.smithi118.stdout:1162: copy_from oid 16 from oid 4 current snap is 123 2024-03-20T08:05:24.777 INFO:tasks.rados.rados.0.smithi118.stdout:1163: snap_remove snap 29 2024-03-20T08:05:25.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:24 smithi118 ceph-mon[93121]: osdmap e297: 8 total, 7 up, 8 in 2024-03-20T08:05:25.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:24 smithi118 ceph-mon[93121]: pgmap v122: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 336 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 1 op/s; 150/975 objects degraded (15.385%) 2024-03-20T08:05:25.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:24 smithi118 ceph-mon[95692]: osdmap e297: 8 total, 7 up, 8 in 2024-03-20T08:05:25.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:24 smithi118 ceph-mon[95692]: pgmap v122: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 336 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 1 op/s; 150/975 objects degraded (15.385%) 2024-03-20T08:05:25.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:24 smithi196 ceph-mon[87080]: osdmap e297: 8 total, 7 up, 8 in 2024-03-20T08:05:25.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:24 smithi196 ceph-mon[87080]: pgmap v122: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 336 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 1 op/s; 150/975 objects degraded (15.385%) 2024-03-20T08:05:25.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:25.375+0000 7f0d1883b7c0 -1 osd.0 289 log_to_monitors true 2024-03-20T08:05:25.780 INFO:tasks.rados.rados.0.smithi118.stdout:1163: done (2 left) 2024-03-20T08:05:25.780 INFO:tasks.rados.rados.0.smithi118.stdout:1164: delete oid 44 current snap is 123 2024-03-20T08:05:25.780 INFO:tasks.rados.rados.0.smithi118.stdout:1161: expect (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:05:25.787 INFO:tasks.rados.rados.0.smithi118.stdout:1162: finishing copy_from racing read to smithi11881747-16 2024-03-20T08:05:25.787 INFO:tasks.rados.rados.0.smithi118.stdout:1162: finishing copy_from to smithi11881747-16 2024-03-20T08:05:25.787 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:05:25.790 INFO:tasks.rados.rados.0.smithi118.stdout:1161: done (2 left) 2024-03-20T08:05:25.790 INFO:tasks.rados.rados.0.smithi118.stdout:1162: done (1 left) 2024-03-20T08:05:25.790 INFO:tasks.rados.rados.0.smithi118.stdout:1164: done (0 left) 2024-03-20T08:05:25.790 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 123 2024-03-20T08:05:25.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: seq_num 365 ranges {2383872=516096,2899968=573440,3473408=229376} 2024-03-20T08:05:25.807 INFO:tasks.rados.rados.0.smithi118.stdout:1165: writing smithi11881747-10 from 2383872 to 2899968 tid 1 2024-03-20T08:05:25.811 INFO:tasks.rados.rados.0.smithi118.stdout:1165: writing smithi11881747-10 from 2899968 to 3473408 tid 2 2024-03-20T08:05:25.814 INFO:tasks.rados.rados.0.smithi118.stdout:1165: writing smithi11881747-10 from 3473408 to 3702784 tid 3 2024-03-20T08:05:25.814 INFO:tasks.rados.rados.0.smithi118.stdout:1165: finishing write tid 1 to smithi11881747-10 2024-03-20T08:05:25.814 INFO:tasks.rados.rados.0.smithi118.stdout:1166: rmattr oid 1 current snap is 123 2024-03-20T08:05:25.814 INFO:tasks.rados.rados.0.smithi118.stdout:1166: done (1 left) 2024-03-20T08:05:25.814 INFO:tasks.rados.rados.0.smithi118.stdout:1167: read oid 7 snap 47 2024-03-20T08:05:25.814 INFO:tasks.rados.rados.0.smithi118.stdout:1167: expect (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:05:25.814 INFO:tasks.rados.rados.0.smithi118.stdout:1168: snap_remove snap 73 2024-03-20T08:05:26.052 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:25 smithi118 ceph-mon[93121]: osdmap e298: 8 total, 7 up, 8 in 2024-03-20T08:05:26.053 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:25 smithi118 ceph-mon[93121]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-20T08:05:26.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:25 smithi118 ceph-mon[95692]: osdmap e298: 8 total, 7 up, 8 in 2024-03-20T08:05:26.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:25 smithi118 ceph-mon[95692]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-20T08:05:26.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:25 smithi196 ceph-mon[87080]: osdmap e298: 8 total, 7 up, 8 in 2024-03-20T08:05:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:25 smithi196 ceph-mon[87080]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-20T08:05:26.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:26.067+0000 7f0d05c8b700 -1 osd.0 289 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T08:05:26.789 INFO:tasks.rados.rados.0.smithi118.stdout:1168: done (2 left) 2024-03-20T08:05:26.789 INFO:tasks.rados.rados.0.smithi118.stdout:1169: read oid 13 snap -1 2024-03-20T08:05:26.789 INFO:tasks.rados.rados.0.smithi118.stdout:1169: expect (ObjNum 357 snap 116 seq_num 357) 2024-03-20T08:05:26.790 INFO:tasks.rados.rados.0.smithi118.stdout:1170: snap_create 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: finishing write tid 2 to smithi11881747-10 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1167: expect (ObjNum 145 snap 41 seq_num 145) 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: finishing write tid 3 to smithi11881747-10 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: finishing write tid 4 to smithi11881747-10 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: finishing write tid 5 to smithi11881747-10 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: oid 10 updating version 0 to 784 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: oid 10 version 784 is already newer than 781 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: oid 10 version 784 is already newer than 783 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: oid 10 version 784 is already newer than 782 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 784 (ObjNum 365 snap 123 seq_num 365) dirty exists 2024-03-20T08:05:26.791 INFO:tasks.rados.rados.0.smithi118.stdout:1165: left oid 10 (ObjNum 365 snap 123 seq_num 365) 2024-03-20T08:05:27.040 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:26 smithi196 ceph-mon[87080]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-20T08:05:27.040 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:26 smithi196 ceph-mon[87080]: osdmap e299: 8 total, 7 up, 8 in 2024-03-20T08:05:27.041 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:26 smithi196 ceph-mon[87080]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:05:27.041 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:26 smithi196 ceph-mon[87080]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' 2024-03-20T08:05:27.041 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:26 smithi196 ceph-mon[87080]: pgmap v125: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 340 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 10 op/s; 151/969 objects degraded (15.583%) 2024-03-20T08:05:27.041 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:27.041 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:27.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[93121]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-20T08:05:27.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[93121]: osdmap e299: 8 total, 7 up, 8 in 2024-03-20T08:05:27.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[93121]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:05:27.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[93121]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' 2024-03-20T08:05:27.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[93121]: pgmap v125: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 340 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 10 op/s; 151/969 objects degraded (15.583%) 2024-03-20T08:05:27.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:27.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[95692]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-20T08:05:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[95692]: osdmap e299: 8 total, 7 up, 8 in 2024-03-20T08:05:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[95692]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:05:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[95692]: from='osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881]' entity='osd.0' 2024-03-20T08:05:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[95692]: pgmap v125: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 340 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 10 op/s; 151/969 objects degraded (15.583%) 2024-03-20T08:05:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:27.795 INFO:tasks.rados.rados.0.smithi118.stdout:1165: done (3 left) 2024-03-20T08:05:27.795 INFO:tasks.rados.rados.0.smithi118.stdout:1167: done (2 left) 2024-03-20T08:05:27.795 INFO:tasks.rados.rados.0.smithi118.stdout:1170: done (1 left) 2024-03-20T08:05:27.795 INFO:tasks.rados.rados.0.smithi118.stdout:1171: read oid 37 snap -1 2024-03-20T08:05:27.795 INFO:tasks.rados.rados.0.smithi118.stdout:1171: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:05:27.796 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 124 2024-03-20T08:05:27.796 INFO:tasks.rados.rados.0.smithi118.stdout:1172: seq_num 366 ranges {3391488=532480,3923968=458752,4382720=344064} 2024-03-20T08:05:27.818 INFO:tasks.rados.rados.0.smithi118.stdout:1172: writing smithi11881747-27 from 3391488 to 3923968 tid 1 2024-03-20T08:05:27.821 INFO:tasks.rados.rados.0.smithi118.stdout:1172: writing smithi11881747-27 from 3923968 to 4382720 tid 2 2024-03-20T08:05:27.824 INFO:tasks.rados.rados.0.smithi118.stdout:1172: writing smithi11881747-27 from 4382720 to 4726784 tid 3 2024-03-20T08:05:27.825 INFO:tasks.rados.rados.0.smithi118.stdout:1173: rollback oid 22 current snap is 124 2024-03-20T08:05:27.825 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 22 to 120 2024-03-20T08:05:27.825 INFO:tasks.rados.rados.0.smithi118.stdout:1174: rollback oid 39 current snap is 124 2024-03-20T08:05:27.825 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 123 2024-03-20T08:05:27.825 INFO:tasks.rados.rados.0.smithi118.stdout:1175: snap_remove snap 121 2024-03-20T08:05:28.032 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[93121]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:05:28.032 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[93121]: osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881] boot 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[93121]: osdmap e300: 8 total, 8 up, 8 in 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 151/969 objects degraded (15.583%), 14 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[95692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[95692]: osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881] boot 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[95692]: osdmap e300: 8 total, 8 up, 8 in 2024-03-20T08:05:28.033 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:05:28.034 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 151/969 objects degraded (15.583%), 14 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:28.034 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:28.034 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:28.034 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.805+0000 7f0ceec5d700 -1 osd.0 pg_epoch: 301 pg[5.2s1( v 288'535 (0'0,288'535] local-lis/les=300/301 n=26 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [6,0,5]p6(0) r=1 lpr=300 luod=0'0 crt=288'535 lcod 0'0 mlcod 0'0 active mbc={}] update_snap_map remove_oid 5:44a43f74:::smithi11881747-38:3b failed with -2 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.816+0000 7f0cedc5b700 -1 osd.0 pg_epoch: 301 pg[5.cs2( v 301'366 (0'0,301'366] local-lis/les=300/301 n=15 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [1,4,0]p1(0) r=2 lpr=300 luod=0'0 crt=286'364 lcod 0'0 mlcod 0'0 active mbc={}] update_snap_map remove_oid 5:332bf1c7:::smithi11881747-32:3f failed with -2 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.840+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.845+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.852+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.855+0000 7f0cf1462700 -1 osd.0 pg_epoch: 301 pg[5.5s0( v 301'722 (0'0,301'722] local-lis/les=300/301 n=32 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [0,1,4]p0(0) r=0 lpr=300 luod=301'720 crt=301'720 lcod 301'718 mlcod 301'718 active+clean+snaptrim trimq=[6b~1,83~1,a3~1,a9~1,b1~1] ps=[32~1]] update_snap_map remove_oid 5:ab9e6c19:::smithi11881747-11:6f failed with -2 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.861+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.863+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.865+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.869+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.873+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.877+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.035 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.881+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.885+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.888+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.892+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.895+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.898+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.902+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.905+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.909+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.912+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.916+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.919+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.922+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.036 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.926+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.928+0000 7f0cecc59700 -1 osd.0 pg_epoch: 301 pg[5.es2( v 301'449 (0'0,301'449] local-lis/les=300/301 n=22 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [4,5,0]p4(0) r=2 lpr=300 luod=0'0 crt=301'449 mlcod 301'449 active mbc={} ps=[32~1]] update_snap_map remove_oid 5:79a2dcb6:::smithi11881747-26:6f failed with -2 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.930+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.937+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.942+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.946+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.949+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.952+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.955+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.958+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.960+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.964+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.967+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.970+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.972+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.975+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.978+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.980+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.983+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.985+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.988+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.991+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.993+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.996+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:27.999+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.001+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.004+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.007+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.009+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.013+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.015+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.018+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.021+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.024+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.027+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.030+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.033+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:27 smithi196 ceph-mon[87080]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:05:28.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:27 smithi196 ceph-mon[87080]: osd.0 [v2:172.21.15.118:6802/3093978881,v1:172.21.15.118:6803/3093978881] boot 2024-03-20T08:05:28.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:27 smithi196 ceph-mon[87080]: osdmap e300: 8 total, 8 up, 8 in 2024-03-20T08:05:28.060 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-20T08:05:28.061 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:27 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 151/969 objects degraded (15.583%), 14 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:28.061 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:28.061 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.036+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.039+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.041+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.044+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.047+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.050+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.053+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.055+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.058+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.061+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.064+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.286 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.066+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.287 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.069+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.287 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.072+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.287 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.075+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.287 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.077+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.287 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.080+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.083+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.086+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.089+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.092+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.095+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.097+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.101+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.104+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.107+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.110+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.288 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.113+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.115+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.118+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.121+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.124+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.126+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.129+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.132+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.134+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.138+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.140+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.143+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.146+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.149+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.289 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.152+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.155+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.157+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.160+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.163+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.166+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.169+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.172+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.175+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.178+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.181+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.184+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.187+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.190+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.192+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.195+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.198+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.201+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.204+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.207+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.210+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.214+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.218+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.220+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.223+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.226+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.230+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.233+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.237+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.241+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.244+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.248+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.252+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.255+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.259+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.262+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.265+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.268+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.271+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.275+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.279+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.282+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.285+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.289+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.293+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.296+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.299+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.303+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.305+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.308+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.311+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.315+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.318+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.536 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.321+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.537 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.324+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.537 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.327+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.537 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.330+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.537 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.333+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.537 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.336+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.537 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.339+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.342+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.345+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.348+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.351+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.354+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.357+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.361+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.364+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.367+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.538 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.370+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.372+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.375+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.379+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.382+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.384+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.387+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.390+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.393+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.396+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.399+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.402+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.407+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.411+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.414+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.417+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.539 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.420+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.423+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.426+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.429+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.432+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.435+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.438+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.441+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.444+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.447+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.451+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.455+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.459+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.462+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.466+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.540 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.469+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.473+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.477+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.481+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.484+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.488+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.491+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.494+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.497+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.500+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.502+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.505+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.509+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.511+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.514+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.542 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.518+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.542 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.520+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.542 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.524+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.542 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.527+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.542 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.530+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.542 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.533+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.542 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.536+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.788 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.539+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.788 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.542+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.788 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.545+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.549+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.552+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.555+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.558+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.561+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.564+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.567+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.570+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.573+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.577+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.580+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.583+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.587+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.789 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.590+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.790 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.593+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.790 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.596+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.790 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.599+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.790 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.602+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.605+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.608+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.611+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.614+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.618+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.621+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.624+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.627+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.630+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.633+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.636+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.639+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.791 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.642+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.645+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.648+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.652+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.655+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.658+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.661+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.665+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.668+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.671+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.674+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.677+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.681+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.684+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.688+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.792 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.691+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.694+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.697+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.701+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.704+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.707+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.710+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.713+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.717+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.720+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.723+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.726+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.729+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.733+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.793 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.736+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.739+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.743+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.746+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.749+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.752+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.756+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.759+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.762+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.765+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.769+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.772+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.775+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.778+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.794 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.782+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.785+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.788+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:28.795 INFO:tasks.rados.rados.0.smithi118.stdout:1175: done (5 left) 2024-03-20T08:05:28.795 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 40 current snap is 124 2024-03-20T08:05:28.795 INFO:tasks.rados.rados.0.smithi118.stdout:1176: seq_num 367 ranges {1925120=548864,2473984=737280,3211264=65536} 2024-03-20T08:05:28.809 INFO:tasks.rados.rados.0.smithi118.stdout:1176: writing smithi11881747-40 from 1925120 to 2473984 tid 1 2024-03-20T08:05:28.814 INFO:tasks.rados.rados.0.smithi118.stdout:1176: writing smithi11881747-40 from 2473984 to 3211264 tid 2 2024-03-20T08:05:28.814 INFO:tasks.rados.rados.0.smithi118.stdout:1176: writing smithi11881747-40 from 3211264 to 3276800 tid 3 2024-03-20T08:05:28.815 INFO:tasks.rados.rados.0.smithi118.stdout:1171: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:05:28.816 INFO:tasks.rados.rados.0.smithi118.stdout:1172: finishing write tid 1 to smithi11881747-27 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1173: finishing rollback tid 0 to smithi11881747-22 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1172: finishing write tid 2 to smithi11881747-27 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1173: finishing rollback tid 1 to smithi11881747-22 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1172: finishing write tid 3 to smithi11881747-27 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1172: finishing write tid 4 to smithi11881747-27 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1171: done (5 left) 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 124 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1174: finishing rollback tid 0 to smithi11881747-39 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1174: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1173: finishing rollback tid 2 to smithi11881747-22 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 776 (ObjNum 260 snap 90 seq_num 260) dirty exists 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1172: finishing write tid 5 to smithi11881747-27 2024-03-20T08:05:28.817 INFO:tasks.rados.rados.0.smithi118.stdout:1172: oid 27 updating version 0 to 690 2024-03-20T08:05:28.818 INFO:tasks.rados.rados.0.smithi118.stdout:1172: oid 27 updating version 690 to 691 2024-03-20T08:05:28.818 INFO:tasks.rados.rados.0.smithi118.stdout:1172: oid 27 updating version 691 to 693 2024-03-20T08:05:28.818 INFO:tasks.rados.rados.0.smithi118.stdout:1172: oid 27 version 693 is already newer than 692 2024-03-20T08:05:28.818 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 693 (ObjNum 366 snap 124 seq_num 366) dirty exists 2024-03-20T08:05:28.818 INFO:tasks.rados.rados.0.smithi118.stdout:1172: left oid 27 (ObjNum 366 snap 124 seq_num 366) 2024-03-20T08:05:28.818 INFO:tasks.rados.rados.0.smithi118.stdout:1169: expect (ObjNum 357 snap 116 seq_num 357) 2024-03-20T08:05:28.959 INFO:tasks.rados.rados.0.smithi118.stdout:1174: finishing rollback tid 2 to smithi11881747-39 2024-03-20T08:05:28.960 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 697 (ObjNum 340 snap 111 seq_num 340) dirty exists 2024-03-20T08:05:28.960 INFO:tasks.rados.rados.0.smithi118.stdout:1176: finishing write tid 1 to smithi11881747-40 2024-03-20T08:05:28.960 INFO:tasks.rados.rados.0.smithi118.stdout:1177: seq_num 368 ranges {2293760=573440,2867200=622592,3489792=180224} 2024-03-20T08:05:28.975 INFO:tasks.rados.rados.0.smithi118.stdout:1177: writing smithi11881747-36 from 2293760 to 2867200 tid 1 2024-03-20T08:05:28.979 INFO:tasks.rados.rados.0.smithi118.stdout:1177: writing smithi11881747-36 from 2867200 to 3489792 tid 2 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1177: writing smithi11881747-36 from 3489792 to 3670016 tid 3 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1169: done (5 left) 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1172: done (4 left) 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1173: done (3 left) 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1174: done (2 left) 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1178: read oid 28 snap -1 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1178: expect deleted 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1179: copy_from oid 18 from oid 4 current snap is 124 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1176: finishing write tid 2 to smithi11881747-40 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1176: finishing write tid 3 to smithi11881747-40 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1176: finishing write tid 4 to smithi11881747-40 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1176: finishing write tid 5 to smithi11881747-40 2024-03-20T08:05:28.982 INFO:tasks.rados.rados.0.smithi118.stdout:1176: oid 40 updating version 0 to 293 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1176: oid 40 updating version 293 to 298 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1176: oid 40 version 298 is already newer than 296 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1176: oid 40 version 298 is already newer than 297 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 298 (ObjNum 367 snap 124 seq_num 367) dirty exists 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1176: left oid 40 (ObjNum 367 snap 124 seq_num 367) 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1177: finishing write tid 1 to smithi11881747-36 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1176: done (3 left) 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1180: read oid 44 snap -1 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1180: expect deleted 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1181: rollback oid 47 current snap is 124 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 104 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1182: rmattr oid 2 current snap is 124 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1183: read oid 10 snap -1 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1183: expect (ObjNum 365 snap 123 seq_num 365) 2024-03-20T08:05:28.983 INFO:tasks.rados.rados.0.smithi118.stdout:1184: rmattr oid 33 current snap is 124 2024-03-20T08:05:28.984 INFO:tasks.rados.rados.0.smithi118.stdout:1184: done (7 left) 2024-03-20T08:05:28.984 INFO:tasks.rados.rados.0.smithi118.stdout:1185: rmattr oid 40 current snap is 124 2024-03-20T08:05:28.984 INFO:tasks.rados.rados.0.smithi118.stdout:1185: done (7 left) 2024-03-20T08:05:28.984 INFO:tasks.rados.rados.0.smithi118.stdout:1186: copy_from oid 49 from oid 46 current snap is 124 2024-03-20T08:05:28.984 INFO:tasks.rados.rados.0.smithi118.stdout:1187: snap_create 2024-03-20T08:05:28.985 INFO:tasks.rados.rados.0.smithi118.stdout:1181: finishing rollback tid 0 to smithi11881747-47 2024-03-20T08:05:28.985 INFO:tasks.rados.rados.0.smithi118.stdout:1181: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:05:28.986 INFO:tasks.rados.rados.0.smithi118.stdout:1181: finishing rollback tid 2 to smithi11881747-47 2024-03-20T08:05:28.986 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 955 (ObjNum 8209 snap 0 seq_num 2687505) dirty exists 2024-03-20T08:05:28.986 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 459 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:05:28.987 INFO:tasks.rados.rados.0.smithi118.stdout:1177: finishing write tid 2 to smithi11881747-36 2024-03-20T08:05:28.989 INFO:tasks.rados.rados.0.smithi118.stdout:1177: finishing write tid 3 to smithi11881747-36 2024-03-20T08:05:28.989 INFO:tasks.rados.rados.0.smithi118.stdout:1177: finishing write tid 4 to smithi11881747-36 2024-03-20T08:05:28.991 INFO:tasks.rados.rados.0.smithi118.stdout:1177: finishing write tid 5 to smithi11881747-36 2024-03-20T08:05:28.991 INFO:tasks.rados.rados.0.smithi118.stdout:1177: oid 36 updating version 0 to 1656 2024-03-20T08:05:28.992 INFO:tasks.rados.rados.0.smithi118.stdout:1177: oid 36 updating version 1656 to 1661 2024-03-20T08:05:28.992 INFO:tasks.rados.rados.0.smithi118.stdout:1177: oid 36 version 1661 is already newer than 1659 2024-03-20T08:05:28.992 INFO:tasks.rados.rados.0.smithi118.stdout:1177: oid 36 version 1661 is already newer than 1660 2024-03-20T08:05:28.992 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 1661 (ObjNum 368 snap 124 seq_num 368) dirty exists 2024-03-20T08:05:28.992 INFO:tasks.rados.rados.0.smithi118.stdout:1177: left oid 36 (ObjNum 368 snap 124 seq_num 368) 2024-03-20T08:05:29.003 INFO:tasks.rados.rados.0.smithi118.stdout:1183: expect (ObjNum 365 snap 123 seq_num 365) 2024-03-20T08:05:29.040 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[93121]: osdmap e301: 8 total, 8 up, 8 in 2024-03-20T08:05:29.040 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:29.040 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[93121]: pgmap v128: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 340 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 8 op/s; 151/969 objects degraded (15.583%) 2024-03-20T08:05:29.040 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:29.040 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:29.040 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[95692]: osdmap e301: 8 total, 8 up, 8 in 2024-03-20T08:05:29.041 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:29.041 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[95692]: pgmap v128: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 340 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 8 op/s; 151/969 objects degraded (15.583%) 2024-03-20T08:05:29.041 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:29.041 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:29.041 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.795+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.800+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.802+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.810+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.817+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.825+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.829+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.832+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.836+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.839+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.843+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.847+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.042 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.850+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.854+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.857+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.860+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.864+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.867+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.871+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.874+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.878+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.882+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.886+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.889+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.893+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.896+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.900+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.043 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.903+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.907+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.911+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.914+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.918+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.921+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.925+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.928+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.933+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.937+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.941+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.944+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.947+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.951+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.954+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.044 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.958+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.961+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.966+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.969+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.973+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.977+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.987+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:28.993+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.000+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.008+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.012+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.017+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.021+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.025+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.028+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.046 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.032+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.046 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.036+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.046 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.040+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:28 smithi196 ceph-mon[87080]: osdmap e301: 8 total, 8 up, 8 in 2024-03-20T08:05:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:28 smithi196 ceph-mon[87080]: pgmap v128: 121 pgs: 14 active+undersized+degraded, 39 active+undersized, 68 active+clean; 340 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 8 op/s; 151/969 objects degraded (15.583%) 2024-03-20T08:05:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:29.223 INFO:tasks.rados.rados.0.smithi118.stdout:1179: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:05:29.224 INFO:tasks.rados.rados.0.smithi118.stdout:1186: finishing copy_from to smithi11881747-49 2024-03-20T08:05:29.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 406 (ObjNum 346 snap 112 seq_num 346) dirty exists 2024-03-20T08:05:29.224 INFO:tasks.rados.rados.0.smithi118.stdout:1186: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:05:29.224 INFO:tasks.rados.rados.0.smithi118.stdout:1179: finishing copy_from to smithi11881747-18 2024-03-20T08:05:29.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:05:29.290 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.044+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.048+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.052+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.055+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.059+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.063+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.067+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.071+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.075+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.078+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.082+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.291 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.086+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.090+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.094+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.097+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.101+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.105+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.109+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.112+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.116+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.120+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.124+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.127+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.131+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.135+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.139+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.142+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.146+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.150+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.293 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.154+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.157+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.160+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.165+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.169+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.173+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.177+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.181+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.185+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.189+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.192+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.197+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.294 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.200+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.205+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.209+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.213+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.217+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.220+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.224+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.228+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.232+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.236+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.239+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.243+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.246+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.250+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.295 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.253+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.296 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.257+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.296 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.261+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.296 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.264+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.296 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.268+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.296 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.271+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.296 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.281+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.296 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.286+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.296 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.290+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.543 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.295+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.299+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.303+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.307+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.311+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.315+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.318+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.322+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.326+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.329+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.333+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.337+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.342+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.347+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.351+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.356+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.361+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.365+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.370+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.375+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.380+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.384+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.389+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.545 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.393+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.396+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.400+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.404+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.408+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.412+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.415+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.419+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.423+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.426+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.430+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.546 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.433+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.437+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.441+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.444+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.448+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.452+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.456+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.460+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.463+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.547 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.467+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.471+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.475+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.479+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.483+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.486+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.490+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.494+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.498+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.502+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.507+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.511+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.515+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.520+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.548 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.524+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.549 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.528+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.549 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.531+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.549 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.535+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.549 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.540+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.549 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.543+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.794 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-mon[93121]: osdmap e302: 8 total, 8 up, 8 in 2024-03-20T08:05:29.794 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:29.794 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-mon[93121]: mgrmap e39: y(active, since 91s), standbys: x 2024-03-20T08:05:29.794 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-mon[95692]: osdmap e302: 8 total, 8 up, 8 in 2024-03-20T08:05:29.794 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:29.794 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-mon[95692]: mgrmap e39: y(active, since 91s), standbys: x 2024-03-20T08:05:29.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.547+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.552+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.555+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.559+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.563+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.566+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.570+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.795 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.574+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.578+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.582+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.586+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.590+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.594+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.599+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.602+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.606+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.610+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.615+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.621+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.625+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.629+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.633+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.796 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.638+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.644+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.649+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.653+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.658+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.663+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.668+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.673+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.676+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.680+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.685+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.689+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.797 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.693+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.697+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.701+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.705+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.709+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.713+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.717+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.798 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.721+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.725+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.729+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.733+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.737+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.741+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.745+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.749+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.753+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.758+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.762+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.766+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.770+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.799 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.773+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.800 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.778+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.800 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.782+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.800 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.786+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.800 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.790+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.800 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.795+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1177: done (8 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1178: done (7 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1179: done (6 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1180: done (5 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1181: done (4 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1182: done (3 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1183: done (2 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1186: done (1 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:1187: done (0 left) 2024-03-20T08:05:29.801 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 125 2024-03-20T08:05:29.802 INFO:tasks.rados.rados.0.smithi118.stdout:1188: seq_num 369 ranges {0=589824,589824=507904,1097728=294912} 2024-03-20T08:05:29.804 INFO:tasks.rados.rados.0.smithi118.stdout:1188: writing smithi11881747-28 from 0 to 589824 tid 1 2024-03-20T08:05:29.808 INFO:tasks.rados.rados.0.smithi118.stdout:1188: writing smithi11881747-28 from 589824 to 1097728 tid 2 2024-03-20T08:05:29.810 INFO:tasks.rados.rados.0.smithi118.stdout:1188: writing smithi11881747-28 from 1097728 to 1392640 tid 3 2024-03-20T08:05:29.810 INFO:tasks.rados.rados.0.smithi118.stdout:1189: rmattr oid 39 current snap is 125 2024-03-20T08:05:29.810 INFO:tasks.rados.rados.0.smithi118.stdout:1189: done (1 left) 2024-03-20T08:05:29.810 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 125 2024-03-20T08:05:29.810 INFO:tasks.rados.rados.0.smithi118.stdout:1190: seq_num 370 ranges {5775360=606208,6381568=794624,7176192=8192} 2024-03-20T08:05:29.846 INFO:tasks.rados.rados.0.smithi118.stdout:1190: writing smithi11881747-34 from 5775360 to 6381568 tid 1 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1190: writing smithi11881747-34 from 6381568 to 7176192 tid 2 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1190: writing smithi11881747-34 from 7176192 to 7184384 tid 3 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1191: copy_from oid 3 from oid 13 current snap is 125 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1188: finishing write tid 1 to smithi11881747-28 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1188: finishing write tid 2 to smithi11881747-28 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1188: finishing write tid 3 to smithi11881747-28 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1188: finishing write tid 4 to smithi11881747-28 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1188: finishing write tid 5 to smithi11881747-28 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1188: oid 28 updating version 0 to 958 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1188: oid 28 updating version 958 to 959 2024-03-20T08:05:29.852 INFO:tasks.rados.rados.0.smithi118.stdout:1188: oid 28 version 959 is already newer than 957 2024-03-20T08:05:29.853 INFO:tasks.rados.rados.0.smithi118.stdout:1188: oid 28 version 959 is already newer than 956 2024-03-20T08:05:29.853 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 959 (ObjNum 369 snap 125 seq_num 369) dirty exists 2024-03-20T08:05:29.853 INFO:tasks.rados.rados.0.smithi118.stdout:1188: left oid 28 (ObjNum 369 snap 125 seq_num 369) 2024-03-20T08:05:29.853 INFO:tasks.rados.rados.0.smithi118.stdout:1188: done (2 left) 2024-03-20T08:05:29.853 INFO:tasks.rados.rados.0.smithi118.stdout:1192: copy_from oid 5 from oid 45 current snap is 125 2024-03-20T08:05:29.853 INFO:tasks.rados.rados.0.smithi118.stdout:1193: snap_create 2024-03-20T08:05:29.853 INFO:tasks.rados.rados.0.smithi118.stdout:1192: finishing copy_from to smithi11881747-5 2024-03-20T08:05:29.853 INFO:tasks.rados.rados.0.smithi118.stdout:1192: got expected ENOENT (src dne) 2024-03-20T08:05:29.854 INFO:tasks.rados.rados.0.smithi118.stdout:1192: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:05:29.858 INFO:tasks.rados.rados.0.smithi118.stdout:1190: finishing write tid 1 to smithi11881747-34 2024-03-20T08:05:29.860 INFO:tasks.rados.rados.0.smithi118.stdout:1190: finishing write tid 2 to smithi11881747-34 2024-03-20T08:05:29.860 INFO:tasks.rados.rados.0.smithi118.stdout:1190: finishing write tid 3 to smithi11881747-34 2024-03-20T08:05:29.861 INFO:tasks.rados.rados.0.smithi118.stdout:1190: finishing write tid 4 to smithi11881747-34 2024-03-20T08:05:29.862 INFO:tasks.rados.rados.0.smithi118.stdout:1191: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:05:29.864 INFO:tasks.rados.rados.0.smithi118.stdout:1191: finishing copy_from to smithi11881747-3 2024-03-20T08:05:29.864 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 706 (ObjNum 357 snap 116 seq_num 357) dirty exists 2024-03-20T08:05:29.866 INFO:tasks.rados.rados.0.smithi118.stdout:1190: finishing write tid 5 to smithi11881747-34 2024-03-20T08:05:29.866 INFO:tasks.rados.rados.0.smithi118.stdout:1190: oid 34 updating version 0 to 2488 2024-03-20T08:05:29.866 INFO:tasks.rados.rados.0.smithi118.stdout:1190: oid 34 version 2488 is already newer than 2485 2024-03-20T08:05:29.866 INFO:tasks.rados.rados.0.smithi118.stdout:1190: oid 34 updating version 2488 to 2490 2024-03-20T08:05:29.866 INFO:tasks.rados.rados.0.smithi118.stdout:1190: oid 34 version 2490 is already newer than 2489 2024-03-20T08:05:29.867 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 2490 (ObjNum 370 snap 125 seq_num 370) dirty exists 2024-03-20T08:05:29.867 INFO:tasks.rados.rados.0.smithi118.stdout:1190: left oid 34 (ObjNum 370 snap 125 seq_num 370) 2024-03-20T08:05:30.049 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.805+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.049 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.815+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.049 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.816+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:30.049 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.822+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.049 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.827+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.049 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.832+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.049 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.837+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.049 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.842+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.847+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.864+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.870+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.874+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.878+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.882+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.886+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.891+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.050 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.895+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.899+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.903+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.908+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.912+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.917+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.921+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.926+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.931+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.935+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.051 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.939+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.944+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.948+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.952+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.957+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.961+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.965+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.970+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.974+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.978+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.983+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.987+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.991+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:29.996+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.001+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.052 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.006+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.011+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.016+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.021+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.025+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.030+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.034+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.039+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.044+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.053 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.049+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:29 smithi196 ceph-mon[87080]: osdmap e302: 8 total, 8 up, 8 in 2024-03-20T08:05:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:29 smithi196 ceph-mon[87080]: mgrmap e39: y(active, since 91s), standbys: x 2024-03-20T08:05:30.307 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.055+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.307 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.060+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.307 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.064+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.307 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.069+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.307 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.074+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.307 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.079+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.307 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.084+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.307 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.089+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.094+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.099+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.103+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.108+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.112+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.118+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.123+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.127+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.308 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.132+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.137+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.142+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.147+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.152+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.157+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.161+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.166+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.170+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.173+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.178+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.309 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.182+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.310 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.187+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.310 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.192+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.310 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.197+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.310 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.203+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.310 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.207+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.310 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.212+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.310 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.217+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.222+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.227+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.232+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.237+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.241+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.246+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.251+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.256+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.262+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.267+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.272+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.277+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.311 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.282+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.312 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.287+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.312 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.292+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.312 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.297+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.312 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.302+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.553 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.308+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.313+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.318+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.323+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.327+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.333+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.338+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.343+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.348+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.353+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.359+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.364+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.554 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.369+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.555 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.373+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.555 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.378+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.555 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.383+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.555 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.388+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.555 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.393+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.555 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.397+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.555 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.402+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.407+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.412+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.417+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.422+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.427+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.432+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.437+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.442+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.447+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.452+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.458+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.463+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.468+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.473+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.556 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.478+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.483+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.488+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.493+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.498+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.503+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.508+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.513+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.518+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.523+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.528+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.534+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.539+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.544+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.549+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.557 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.554+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.805 INFO:tasks.rados.rados.0.smithi118.stdout:1190: done (3 left) 2024-03-20T08:05:30.806 INFO:tasks.rados.rados.0.smithi118.stdout:1191: done (2 left) 2024-03-20T08:05:30.806 INFO:tasks.rados.rados.0.smithi118.stdout:1192: done (1 left) 2024-03-20T08:05:30.806 INFO:tasks.rados.rados.0.smithi118.stdout:1193: done (0 left) 2024-03-20T08:05:30.806 INFO:tasks.rados.rados.0.smithi118.stdout:1194: snap_create 2024-03-20T08:05:30.808 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-mon[93121]: osdmap e303: 8 total, 8 up, 8 in 2024-03-20T08:05:30.808 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:30.809 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-mon[93121]: pgmap v131: 121 pgs: 4 active+undersized+degraded, 11 active+undersized, 106 active+clean; 318 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 13 op/s; 51/888 objects degraded (5.743%); 5.1 MiB/s, 1 objects/s recovering 2024-03-20T08:05:30.809 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-mon[95692]: osdmap e303: 8 total, 8 up, 8 in 2024-03-20T08:05:30.809 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:30.809 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-mon[95692]: pgmap v131: 121 pgs: 4 active+undersized+degraded, 11 active+undersized, 106 active+clean; 318 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 13 op/s; 51/888 objects degraded (5.743%); 5.1 MiB/s, 1 objects/s recovering 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.559+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.566+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.571+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.576+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.581+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.587+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.592+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.598+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.810 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.603+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.608+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.613+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.618+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.623+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.629+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.634+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.639+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.644+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.650+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.657+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.663+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.669+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.675+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.681+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.688+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.695+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.701+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.707+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.712+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.716+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.813 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.720+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.813 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.725+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.813 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.729+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.813 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.733+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.813 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.738+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.813 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.742+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.814 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.747+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.814 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.751+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.756+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.760+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.764+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.769+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.773+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.777+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.782+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.786+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.791+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.815 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.795+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.816 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.801+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:30.816 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.809+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.815+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.820+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.824+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.829+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.833+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.837+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.842+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.846+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.851+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.064 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.855+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.860+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.864+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.868+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.873+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.878+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.882+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.887+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.892+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.899+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.899+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.905+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.065 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.909+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.914+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.919+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.924+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.929+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.933+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.938+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.943+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.947+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.952+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.956+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.066 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.961+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.067 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.966+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.067 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.971+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.067 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.976+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.067 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.981+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.985+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.990+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.994+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:30.999+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.003+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.008+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.013+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.017+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.022+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.026+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.031+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.068 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.036+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.043+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.049+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.055+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.059+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.064+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:30 smithi196 ceph-mon[87080]: osdmap e303: 8 total, 8 up, 8 in 2024-03-20T08:05:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:30 smithi196 ceph-mon[87080]: pgmap v131: 121 pgs: 4 active+undersized+degraded, 11 active+undersized, 106 active+clean; 318 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 13 op/s; 51/888 objects degraded (5.743%); 5.1 MiB/s, 1 objects/s recovering 2024-03-20T08:05:31.318 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.069+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.073+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.078+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.083+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.088+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.092+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.097+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.102+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.107+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.111+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.116+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.121+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.319 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.125+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.320 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.130+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.320 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.135+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.320 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.139+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.320 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.143+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.320 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.148+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.320 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.152+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.320 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.157+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.320 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.163+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.168+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.173+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.178+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.182+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.187+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.192+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.196+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.201+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.206+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.211+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.215+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.220+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.225+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.321 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.230+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.235+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.239+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.244+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.250+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.255+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.260+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.265+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.269+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.274+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.279+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.284+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.289+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.293+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.298+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.322 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.303+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.323 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.308+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.323 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.313+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.323 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.319+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.576 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.324+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.330+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.335+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.340+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.345+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.351+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.357+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.362+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.367+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.372+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.377+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.382+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.578 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.387+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.578 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.391+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.578 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.396+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.578 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.401+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.578 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.406+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.411+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.416+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.420+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.426+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.431+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.436+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.442+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.447+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.452+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.457+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.462+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.467+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.472+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.477+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.482+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.488+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.493+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.498+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.502+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.508+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.512+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.518+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.523+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.580 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.529+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.581 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.533+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.581 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.538+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.581 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.546+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.581 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.553+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.581 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.561+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.581 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.569+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.581 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.576+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.811 INFO:tasks.rados.rados.0.smithi118.stdout:1194: done (0 left) 2024-03-20T08:05:31.811 INFO:tasks.rados.rados.0.smithi118.stdout:1195: snap_remove snap 104 2024-03-20T08:05:31.830 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:05:31.775+0000 7ff9af1ba700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-20T08:05:31.831 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: osdmap e304: 8 total, 8 up, 8 in 2024-03-20T08:05:31.831 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:31.831 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:31.831 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:31.831 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:31.831 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:05:31.831 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: osdmap e304: 8 total, 8 up, 8 in 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:31.832 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:31.833 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:31.833 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:05:31.833 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:31.833 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:31.833 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:31.833 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:31.833 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:31.833 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:31.833 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.583+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.588+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.593+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.598+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.604+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.609+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.613+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.621+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.629+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.643+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.652+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.670+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.684+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.834 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.690+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.695+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.700+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.706+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.711+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.716+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.721+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.726+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.731+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.737+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.741+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.747+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.751+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.756+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.761+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.767+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.772+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.779+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.784+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.789+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.794+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.799+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.805+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.815+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.821+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.826+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:31.836 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.831+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.086 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.838+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.086 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.843+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.086 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.848+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.086 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.853+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.858+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.863+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.868+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.873+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.878+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.883+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.888+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.893+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.898+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.903+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.908+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.914+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.919+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.924+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.929+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.935+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.940+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.945+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.950+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.956+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.961+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.966+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.972+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.976+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.982+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.987+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.993+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:31.998+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.003+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.088 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.008+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.013+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.018+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.023+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.028+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.033+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.038+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.043+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.048+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.053+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.059+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.064+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.070+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.075+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.080+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.089 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.086+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: osdmap e304: 8 total, 8 up, 8 in 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:32.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:32.336 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.092+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.336 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.097+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.336 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.102+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.336 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.107+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.112+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.117+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.123+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.128+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.134+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.139+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.144+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.149+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.155+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.161+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.166+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.171+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.176+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.337 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.181+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.186+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.191+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.196+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.202+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.207+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.212+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.218+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.224+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.229+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.235+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.240+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.245+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.250+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.338 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.255+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.261+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.266+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.272+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.278+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.283+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.288+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.293+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.299+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.304+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.339 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.309+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.340 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.314+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.340 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.320+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.340 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.325+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.340 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.330+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.340 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.336+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.586 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.343+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.349+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.354+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.360+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.365+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.370+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.376+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.381+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.385+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.390+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.396+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.401+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.587 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.407+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.588 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.412+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.588 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.418+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.588 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.424+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.588 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.429+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.588 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.435+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.588 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.440+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.588 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.445+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.588 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.451+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.457+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.462+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.469+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.475+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.480+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.486+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.491+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.497+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.502+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.508+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.514+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.520+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.589 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.525+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.531+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.536+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.541+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.547+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.553+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.558+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.564+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.570+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.575+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.581+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.590 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.587+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:32.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:32.816 INFO:tasks.rados.rados.0.smithi118.stdout:1195: done (0 left) 2024-03-20T08:05:32.817 INFO:tasks.rados.rados.0.smithi118.stdout:1196: rollback oid 5 current snap is 127 2024-03-20T08:05:32.817 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 124 2024-03-20T08:05:32.817 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 127 2024-03-20T08:05:32.817 INFO:tasks.rados.rados.0.smithi118.stdout:1197: seq_num 371 ranges {4767744=622592,5390336=679936,6070272=122880} 2024-03-20T08:05:32.841 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:05:32.808+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 1 pg snaptrim_error (PG_DAMAGED) 2024-03-20T08:05:32.841 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:32.841 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-mon[93121]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-20T08:05:32.841 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-mon[93121]: osdmap e305: 8 total, 8 up, 8 in 2024-03-20T08:05:32.841 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-mon[93121]: pgmap v134: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 317 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 16 op/s; 5.1 MiB/s, 1 objects/s recovering 2024-03-20T08:05:32.841 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:32.841 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-mon[95692]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-20T08:05:32.841 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-mon[95692]: osdmap e305: 8 total, 8 up, 8 in 2024-03-20T08:05:32.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-mon[95692]: pgmap v134: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 317 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 16 op/s; 5.1 MiB/s, 1 objects/s recovering 2024-03-20T08:05:32.842 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.593+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.842 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.599+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.842 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.605+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.842 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.610+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.842 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.616+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.842 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.622+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.628+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.635+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.641+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.646+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.652+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.657+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.663+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.668+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.674+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.680+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.685+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.691+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.843 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.697+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.703+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.709+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.715+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.720+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.726+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.732+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.737+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.743+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.749+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.754+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.760+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.766+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.772+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.779+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.844 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.785+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.791+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.797+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.803+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.809+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.821+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.830+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:32.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.834+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.845 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.841+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:32.847 INFO:tasks.rados.rados.0.smithi118.stdout:1197: writing smithi11881747-29 from 4767744 to 5390336 tid 1 2024-03-20T08:05:32.852 INFO:tasks.rados.rados.0.smithi118.stdout:1197: writing smithi11881747-29 from 5390336 to 6070272 tid 2 2024-03-20T08:05:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:1197: writing smithi11881747-29 from 6070272 to 6193152 tid 3 2024-03-20T08:05:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 127 2024-03-20T08:05:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:1196: finishing rollback tid 0 to smithi11881747-5 2024-03-20T08:05:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:1196: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:05:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:1198: seq_num 372 ranges {3702784=638976,4341760=565248,4907008=237568} 2024-03-20T08:05:32.878 INFO:tasks.rados.rados.0.smithi118.stdout:1198: writing smithi11881747-10 from 3702784 to 4341760 tid 1 2024-03-20T08:05:32.881 INFO:tasks.rados.rados.0.smithi118.stdout:1198: writing smithi11881747-10 from 4341760 to 4907008 tid 2 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:1198: writing smithi11881747-10 from 4907008 to 5144576 tid 3 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:1199: delete oid 26 current snap is 127 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:1196: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 738 (ObjNum 333 snap 109 seq_num 333) dirty exists 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:1197: finishing write tid 1 to smithi11881747-29 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:1197: finishing write tid 2 to smithi11881747-29 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:1197: finishing write tid 3 to smithi11881747-29 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:1197: finishing write tid 4 to smithi11881747-29 2024-03-20T08:05:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:1197: finishing write tid 5 to smithi11881747-29 2024-03-20T08:05:32.884 INFO:tasks.rados.rados.0.smithi118.stdout:1197: oid 29 updating version 0 to 961 2024-03-20T08:05:32.884 INFO:tasks.rados.rados.0.smithi118.stdout:1197: oid 29 updating version 961 to 966 2024-03-20T08:05:32.884 INFO:tasks.rados.rados.0.smithi118.stdout:1197: oid 29 updating version 966 to 967 2024-03-20T08:05:32.884 INFO:tasks.rados.rados.0.smithi118.stdout:1197: oid 29 updating version 967 to 968 2024-03-20T08:05:32.884 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 968 (ObjNum 371 snap 127 seq_num 371) dirty exists 2024-03-20T08:05:32.884 INFO:tasks.rados.rados.0.smithi118.stdout:1197: left oid 29 (ObjNum 371 snap 127 seq_num 371) 2024-03-20T08:05:32.884 INFO:tasks.rados.rados.0.smithi118.stdout:1198: finishing write tid 1 to smithi11881747-10 2024-03-20T08:05:32.885 INFO:tasks.rados.rados.0.smithi118.stdout:1196: done (3 left) 2024-03-20T08:05:32.885 INFO:tasks.rados.rados.0.smithi118.stdout:1197: done (2 left) 2024-03-20T08:05:32.885 INFO:tasks.rados.rados.0.smithi118.stdout:1199: done (1 left) 2024-03-20T08:05:32.885 INFO:tasks.rados.rados.0.smithi118.stdout:1200: read oid 39 snap -1 2024-03-20T08:05:32.885 INFO:tasks.rados.rados.0.smithi118.stdout:1200: expect (ObjNum 340 snap 111 seq_num 340) 2024-03-20T08:05:32.885 INFO:tasks.rados.rados.0.smithi118.stdout:1201: snap_remove snap 111 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.849+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.862+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.870+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.877+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.884+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.893+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.899+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.904+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.910+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.916+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.922+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.927+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.096 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.933+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.096 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.939+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.096 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.945+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.096 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.950+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.096 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.956+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.096 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.964+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.096 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.970+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.975+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.981+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.988+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:32.994+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.000+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.006+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.012+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.018+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.024+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.029+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.035+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.040+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.046+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.052+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.097 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.058+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.098 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.063+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.098 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.071+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.098 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.077+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.098 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.083+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.098 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.089+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.098 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.095+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:32 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:33.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:32 smithi196 ceph-mon[87080]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-20T08:05:33.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:32 smithi196 ceph-mon[87080]: osdmap e305: 8 total, 8 up, 8 in 2024-03-20T08:05:33.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:32 smithi196 ceph-mon[87080]: pgmap v134: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 317 MiB data, 1.4 GiB used, 714 GiB / 715 GiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 16 op/s; 5.1 MiB/s, 1 objects/s recovering 2024-03-20T08:05:33.345 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.101+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.345 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.107+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.113+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.119+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.125+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.131+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.136+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.142+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.149+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.155+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.161+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.167+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.173+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.179+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.346 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.185+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.191+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.199+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.205+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.211+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.217+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.223+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.229+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.235+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.241+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.247+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.253+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.259+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.266+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.347 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.273+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.279+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.285+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.291+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.298+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.304+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.310+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.315+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.321+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.328+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.334+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.340+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.348 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.345+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:05:33] "GET /metrics HTTP/1.1" 200 35986 "" "Prometheus/2.43.0" 2024-03-20T08:05:33.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.352+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.359+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.366+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.372+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.378+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.383+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.390+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.396+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.402+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.408+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.414+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.419+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.426+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.432+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.438+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.444+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.603 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.450+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.457+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.463+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.469+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.475+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.481+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.487+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.493+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.499+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.505+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.511+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.517+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.524+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.531+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.604 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.538+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.544+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.550+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.556+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.562+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.569+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.575+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.581+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.588+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.595+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.605 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.602+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.813 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:33.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:1201: done (2 left) 2024-03-20T08:05:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:1202: snap_remove snap 125 2024-03-20T08:05:33.855 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:33.855 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[93121]: Health check failed: Possible data damage: 1 pg snaptrim_error (PG_DAMAGED) 2024-03-20T08:05:33.856 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[93121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 51/888 objects degraded (5.743%), 4 pgs degraded) 2024-03-20T08:05:33.856 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[93121]: osdmap e306: 8 total, 8 up, 8 in 2024-03-20T08:05:33.856 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:33.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:33.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[95692]: Health check failed: Possible data damage: 1 pg snaptrim_error (PG_DAMAGED) 2024-03-20T08:05:33.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[95692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 51/888 objects degraded (5.743%), 4 pgs degraded) 2024-03-20T08:05:33.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[95692]: osdmap e306: 8 total, 8 up, 8 in 2024-03-20T08:05:33.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.609+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.616+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.622+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.629+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.636+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.643+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.649+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.655+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.661+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.667+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.674+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.680+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.686+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.692+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.699+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.705+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.711+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.718+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.724+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.730+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.737+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.743+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.749+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.755+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.762+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.768+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.779+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.785+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.791+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.798+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.804+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.811+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.820+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.826+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.836+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.846+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:33.860 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.856+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.110 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.863+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.110 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.870+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.110 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.877+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.110 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.884+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.110 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.890+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.897+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.903+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.910+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.916+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.923+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.929+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.936+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.944+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.950+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.957+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.964+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.970+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.977+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.983+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.111 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.989+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:33.996+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.003+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.009+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.016+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.023+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.030+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.036+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.043+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.049+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.056+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.063+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.070+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.076+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.084+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.112 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.090+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.113 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.097+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.113 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.104+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.113 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.110+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:34.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:33 smithi196 ceph-mon[87080]: Health check failed: Possible data damage: 1 pg snaptrim_error (PG_DAMAGED) 2024-03-20T08:05:34.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:33 smithi196 ceph-mon[87080]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 51/888 objects degraded (5.743%), 4 pgs degraded) 2024-03-20T08:05:34.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:33 smithi196 ceph-mon[87080]: osdmap e306: 8 total, 8 up, 8 in 2024-03-20T08:05:34.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:34.364 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.118+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.364 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.125+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.364 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.131+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.138+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.144+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.151+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.157+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.164+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.171+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.178+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.184+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.191+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.198+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.205+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.213+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.220+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.365 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.228+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.366 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.234+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.366 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.241+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.366 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.248+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.366 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.255+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.366 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.262+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.366 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.269+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.366 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.275+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.283+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.289+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.296+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.303+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.309+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.316+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.323+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.329+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.336+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.344+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.350+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.358+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.367 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.365+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.620 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.373+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.380+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.387+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.393+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.400+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.406+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.413+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.420+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.426+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.433+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.441+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.448+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.455+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.621 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.461+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.469+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.476+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.483+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.491+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.499+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.505+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.512+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.518+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.525+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.532+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.538+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.545+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.551+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.558+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.565+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.572+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.579+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.586+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.593+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.600+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.607+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.613+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.623 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.620+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.825 INFO:tasks.rados.rados.0.smithi118.stdout:1202: done (2 left) 2024-03-20T08:05:34.826 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 127 2024-03-20T08:05:34.826 INFO:tasks.rados.rados.0.smithi118.stdout:1198: finishing write tid 2 to smithi11881747-10 2024-03-20T08:05:34.826 INFO:tasks.rados.rados.0.smithi118.stdout:1203: seq_num 374 ranges {0=671744,671744=778240,1449984=24576} 2024-03-20T08:05:34.828 INFO:tasks.rados.rados.0.smithi118.stdout:1203: writing smithi11881747-47 from 0 to 671744 tid 1 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1203: writing smithi11881747-47 from 671744 to 1449984 tid 2 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1203: writing smithi11881747-47 from 1449984 to 1474560 tid 3 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1204: copy_from oid 46 from oid 44 current snap is 127 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1198: finishing write tid 3 to smithi11881747-10 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1198: finishing write tid 4 to smithi11881747-10 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1198: finishing write tid 5 to smithi11881747-10 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1198: oid 10 updating version 0 to 785 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1198: oid 10 updating version 785 to 788 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1198: oid 10 version 788 is already newer than 786 2024-03-20T08:05:34.834 INFO:tasks.rados.rados.0.smithi118.stdout:1198: oid 10 version 788 is already newer than 787 2024-03-20T08:05:34.835 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 788 (ObjNum 372 snap 127 seq_num 372) dirty exists 2024-03-20T08:05:34.835 INFO:tasks.rados.rados.0.smithi118.stdout:1198: left oid 10 (ObjNum 372 snap 127 seq_num 372) 2024-03-20T08:05:34.835 INFO:tasks.rados.rados.0.smithi118.stdout:1200: expect (ObjNum 340 snap 111 seq_num 340) 2024-03-20T08:05:34.867 INFO:tasks.rados.rados.0.smithi118.stdout:1204: finishing copy_from to smithi11881747-46 2024-03-20T08:05:34.867 INFO:tasks.rados.rados.0.smithi118.stdout:1204: got expected ENOENT (src dne) 2024-03-20T08:05:34.867 INFO:tasks.rados.rados.0.smithi118.stdout:1204: finishing copy_from racing read to smithi11881747-46 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: finishing write tid 1 to smithi11881747-47 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: finishing write tid 2 to smithi11881747-47 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1198: done (3 left) 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1200: done (2 left) 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1204: done (1 left) 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1205: snap_create 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: finishing write tid 3 to smithi11881747-47 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: finishing write tid 4 to smithi11881747-47 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: finishing write tid 5 to smithi11881747-47 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: oid 47 updating version 0 to 983 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: oid 47 version 983 is already newer than 982 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: oid 47 version 983 is already newer than 978 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:1203: oid 47 version 983 is already newer than 981 2024-03-20T08:05:34.868 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 983 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:05:34.869 INFO:tasks.rados.rados.0.smithi118.stdout:1203: left oid 47 (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-mon[93121]: osdmap e307: 8 total, 8 up, 8 in 2024-03-20T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:34.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-mon[93121]: pgmap v137: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 317 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.9 MiB/s rd, 680 KiB/s wr, 4 op/s 2024-03-20T08:05:34.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-mon[95692]: osdmap e307: 8 total, 8 up, 8 in 2024-03-20T08:05:34.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:34.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-mon[95692]: pgmap v137: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 317 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.9 MiB/s rd, 680 KiB/s wr, 4 op/s 2024-03-20T08:05:34.871 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.628+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.871 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.635+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.871 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.642+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.871 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.648+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.871 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.655+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.871 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.663+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.669+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.676+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.683+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.690+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.696+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.703+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.710+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.718+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.724+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.732+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.872 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.739+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.746+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.754+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.760+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.767+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.774+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.780+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.787+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.794+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.801+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.807+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.815+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.873 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.825+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.874 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.841+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:34.874 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.844+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.874 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.855+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.874 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.863+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:34.874 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.871+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.121 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.879+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.886+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.893+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.899+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.907+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.915+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.922+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.928+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.936+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.942+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.949+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.956+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.963+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.122 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.970+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.977+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.984+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.990+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:34.997+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.004+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.011+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.018+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.025+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.123 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.032+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.039+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.046+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.053+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.060+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.067+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.074+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.080+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.087+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.094+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.101+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.108+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.115+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.124 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.121+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:34 smithi196 ceph-mon[87080]: osdmap e307: 8 total, 8 up, 8 in 2024-03-20T08:05:35.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:35.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:34 smithi196 ceph-mon[87080]: pgmap v137: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 317 MiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.9 MiB/s rd, 680 KiB/s wr, 4 op/s 2024-03-20T08:05:35.376 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.129+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.135+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.142+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.149+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.156+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.162+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.169+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.177+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.185+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.191+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.199+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.206+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.213+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.377 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.220+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.227+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.234+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.242+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.248+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.256+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.264+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.270+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.278+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.378 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.285+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.293+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.300+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.307+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.314+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.321+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.327+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.335+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.342+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.349+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.356+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.363+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.370+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.379 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.377+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.628 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.385+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.628 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.391+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.628 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.398+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.628 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.405+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.628 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.412+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.628 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.419+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.628 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.426+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.433+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.440+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.447+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.453+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.460+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.467+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.474+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.481+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.488+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.495+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.502+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.509+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.516+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.629 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.522+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.530+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.536+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.544+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.550+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.558+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.564+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.572+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.579+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.586+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.594+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.601+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.607+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.614+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.630 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.621+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.631 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.628+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.830 INFO:tasks.rados.rados.0.smithi118.stdout:1203: done (1 left) 2024-03-20T08:05:35.830 INFO:tasks.rados.rados.0.smithi118.stdout:1205: done (0 left) 2024-03-20T08:05:35.830 INFO:tasks.rados.rados.0.smithi118.stdout:1206: delete oid 3 current snap is 128 2024-03-20T08:05:35.835 INFO:tasks.rados.rados.0.smithi118.stdout:1206: done (0 left) 2024-03-20T08:05:35.835 INFO:tasks.rados.rados.0.smithi118.stdout:1207: read oid 6 snap -1 2024-03-20T08:05:35.835 INFO:tasks.rados.rados.0.smithi118.stdout:1207: expect deleted 2024-03-20T08:05:35.835 INFO:tasks.rados.rados.0.smithi118.stdout:1208: delete oid 30 current snap is 128 2024-03-20T08:05:35.838 INFO:tasks.rados.rados.0.smithi118.stdout:1207: done (1 left) 2024-03-20T08:05:35.839 INFO:tasks.rados.rados.0.smithi118.stdout:1208: done (0 left) 2024-03-20T08:05:35.839 INFO:tasks.rados.rados.0.smithi118.stdout:1209: snap_remove snap 47 2024-03-20T08:05:35.884 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-mon[93121]: osdmap e308: 8 total, 8 up, 8 in 2024-03-20T08:05:35.884 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:35.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-mon[95692]: osdmap e308: 8 total, 8 up, 8 in 2024-03-20T08:05:35.884 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.636+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.644+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.650+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.657+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.664+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.671+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.678+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.685+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.692+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.698+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.705+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.885 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.712+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.886 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.718+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.886 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.725+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.886 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.732+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.886 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.739+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.886 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.746+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.753+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.760+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.766+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.773+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.780+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.787+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.794+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.801+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.810+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.819+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.828+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.839+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.843+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:35.888 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.849+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.888 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.857+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.888 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.864+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.888 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.870+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.888 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.877+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:35.888 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.885+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:35 smithi196 ceph-mon[87080]: osdmap e308: 8 total, 8 up, 8 in 2024-03-20T08:05:36.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:36.139 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.892+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.139 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.899+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.907+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.914+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.921+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.928+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.935+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.944+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.954+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.965+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.973+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.981+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.990+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:35.997+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.005+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.140 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.013+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.021+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.029+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.039+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.046+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.056+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.066+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.074+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.084+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.094+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.141 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.102+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.142 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.110+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.142 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.120+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.142 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.129+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.142 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.140+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.364 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:05:36.397 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.150+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.397 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.157+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.397 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.164+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.171+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.179+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.186+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.193+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.200+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.208+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.216+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.225+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.234+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.243+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.252+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.262+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.269+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.398 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.276+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.284+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.291+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.298+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.306+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.313+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.320+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.328+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.336+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.343+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.399 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.351+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.400 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.360+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.400 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.370+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.400 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.379+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.400 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.388+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.400 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.397+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.653 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.406+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.653 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.415+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.653 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.422+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.653 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.431+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.653 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.441+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.451+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.459+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.466+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.475+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.485+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.493+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.501+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.508+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.517+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.524+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.532+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.540+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.550+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.654 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.559+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.567+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.575+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.582+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.590+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.599+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.607+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.616+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.623+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.631+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.638+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.645+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.655 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.653+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (5m) 10s ago 6m 28.5M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (5m) 56s ago 5m 86.9M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (7m) 10s ago 7m 14.8M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (7m) 56s ago 7m 19.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (2m) 56s ago 10m 449M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (2m) 10s ago 12m 502M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (107s) 10s ago 12m 47.0M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (63s) 56s ago 11m 26.6M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (82s) 10s ago 11m 41.3M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:05:36.754 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (5m) 10s ago 5m 14.5M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (5m) 56s ago 5m 15.8M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (15s) 10s ago 10m 11.9M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (10m) 10s ago 10m 202M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 d99b1c22204a 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (9m) 10s ago 9m 150M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (9m) 10s ago 9m 157M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (8m) 56s ago 8m 181M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (8m) 56s ago 8m 133M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (8m) 56s ago 8m 117M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (7m) 56s ago 7m 115M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:05:36.755 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (2m) 56s ago 6m 56.2M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:05:36.835 INFO:tasks.rados.rados.0.smithi118.stdout:1209: done (0 left) 2024-03-20T08:05:36.835 INFO:tasks.rados.rados.0.smithi118.stdout:1210: read oid 40 snap -1 2024-03-20T08:05:36.835 INFO:tasks.rados.rados.0.smithi118.stdout:1210: expect (ObjNum 367 snap 124 seq_num 367) 2024-03-20T08:05:36.835 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 128 2024-03-20T08:05:36.835 INFO:tasks.rados.rados.0.smithi118.stdout:1211: seq_num 377 ranges {1753088=720896,2473984=434176,2908160=368640} 2024-03-20T08:05:36.849 INFO:tasks.rados.rados.0.smithi118.stdout:1211: writing smithi11881747-14 from 1753088 to 2473984 tid 1 2024-03-20T08:05:36.853 INFO:tasks.rados.rados.0.smithi118.stdout:1211: writing smithi11881747-14 from 2473984 to 2908160 tid 2 2024-03-20T08:05:36.855 INFO:tasks.rados.rados.0.smithi118.stdout:1211: writing smithi11881747-14 from 2908160 to 3276800 tid 3 2024-03-20T08:05:36.855 INFO:tasks.rados.rados.0.smithi118.stdout:1212: snap_remove snap 62 2024-03-20T08:05:36.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-mon[93121]: osdmap e309: 8 total, 8 up, 8 in 2024-03-20T08:05:36.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:36.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-mon[93121]: pgmap v140: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 323 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 661 KiB/s rd, 1.6 MiB/s wr, 6 op/s 2024-03-20T08:05:36.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-mon[95692]: osdmap e309: 8 total, 8 up, 8 in 2024-03-20T08:05:36.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:36.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-mon[95692]: pgmap v140: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 323 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 661 KiB/s rd, 1.6 MiB/s wr, 6 op/s 2024-03-20T08:05:36.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.661+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.669+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.676+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.683+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.690+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.698+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.707+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.715+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.723+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.731+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.738+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.748+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.758+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.768+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.778+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.787+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.796+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.805+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.815+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.825+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.841+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.925 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.845+0000 7f0cecc59700 -1 osd.0 pg_epoch: 310 pg[5.es2( v 308'491 (0'0,308'491] local-lis/les=300/301 n=22 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [4,5,0]p4(0) r=2 lpr=300 luod=0'0 crt=308'491 mlcod 308'491 active mbc={} ps=[32~1,6b~1,83~1,a3~2,a9~1,ab~1,b1~1,b5~1,b9~1]] update_snap_map remove_oid 5:79a2dcb6:::smithi11881747-26:5b failed with -2 2024-03-20T08:05:36.925 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.847+0000 7f0cec458700 -1 osd.0 pg_epoch: 309 pg[5.7s2( v 307'233 (0'0,307'233] local-lis/les=300/301 n=8 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [5,1,0]p5(0) r=2 lpr=300 luod=0'0 crt=307'233 mlcod 307'233 active mbc={} ps=[32~1,6b~1,83~1,a3~2,a9~1,ab~1,b1~1,b5~1,b9~1]] update_snap_map remove_oid 5:e4198694:::smithi11881747-19:5e failed with -2 2024-03-20T08:05:36.925 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.870+0000 7f0cf1462700 -1 osd.0 pg_epoch: 310 pg[5.5s0( v 310'752 (0'0,310'752] local-lis/les=300/301 n=32 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [0,1,4]p0(0) r=0 lpr=300 luod=310'750 crt=310'750 lcod 310'748 mlcod 310'748 active+clean+snaptrim trimq=[50~1,6b~1,83~1,a3~2,a9~1,ab~1,b1~1,b5~1,b9~1] ps=[32~1]] update_snap_map remove_oid 5:ab050921:::smithi11881747-42:5f failed with -2 2024-03-20T08:05:36.925 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.874+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset a9=[]:{68=[65],99=[95,91,8b,8a,89],a9=[a9,a8,a6,a5,a4,a3,a2,9f,9d]} for object 5:ab050921:::smithi11881747-42:5f 2024-03-20T08:05:36.925 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.874+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:36.925 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:36.874+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset b7=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:37.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:36 smithi196 ceph-mon[87080]: osdmap e309: 8 total, 8 up, 8 in 2024-03-20T08:05:37.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset ae=[]:{65=[65,50],9f=[9f,9d,95,91],a3=[a3,a2],a9=[a9,a8,a6,a5,a4],ae=[ac,ab]} for object 5:ab9e6c19:::smithi11881747-11:6f 2024-03-20T08:05:37.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:36 smithi196 ceph-mon[87080]: pgmap v140: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 323 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 661 KiB/s rd, 1.6 MiB/s wr, 6 op/s 2024-03-20T08:05:37.186 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:05:37.186 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 7, 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 1 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:05:37.187 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 9, 2024-03-20T08:05:37.188 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 6 2024-03-20T08:05:37.188 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:05:37.188 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:05:37.838 INFO:tasks.rados.rados.0.smithi118.stdout:1212: done (2 left) 2024-03-20T08:05:37.838 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 128 2024-03-20T08:05:37.839 INFO:tasks.rados.rados.0.smithi118.stdout:1213: seq_num 378 ranges {3670016=737280,4407296=720896,5128192=81920} 2024-03-20T08:05:37.863 INFO:tasks.rados.rados.0.smithi118.stdout:1213: writing smithi11881747-36 from 3670016 to 4407296 tid 1 2024-03-20T08:05:37.867 INFO:tasks.rados.rados.0.smithi118.stdout:1213: writing smithi11881747-36 from 4407296 to 5128192 tid 2 2024-03-20T08:05:37.868 INFO:tasks.rados.rados.0.smithi118.stdout:1213: writing smithi11881747-36 from 5128192 to 5210112 tid 3 2024-03-20T08:05:37.868 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 128 2024-03-20T08:05:37.868 INFO:tasks.rados.rados.0.smithi118.stdout:1214: seq_num 379 ranges {2826240=753664,3579904=606208,4186112=196608} 2024-03-20T08:05:37.888 INFO:tasks.rados.rados.0.smithi118.stdout:1214: writing smithi11881747-25 from 2826240 to 3579904 tid 1 2024-03-20T08:05:37.891 INFO:tasks.rados.rados.0.smithi118.stdout:1214: writing smithi11881747-25 from 3579904 to 4186112 tid 2 2024-03-20T08:05:37.893 INFO:tasks.rados.rados.0.smithi118.stdout:1214: writing smithi11881747-25 from 4186112 to 4382720 tid 3 2024-03-20T08:05:37.893 INFO:tasks.rados.rados.0.smithi118.stdout:1215: snap_remove snap 115 2024-03-20T08:05:38.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:37 smithi196 ceph-mon[87080]: from='client.44129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:38.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:37 smithi196 ceph-mon[87080]: from='client.44135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:38.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:37 smithi196 ceph-mon[87080]: osdmap e310: 8 total, 8 up, 8 in 2024-03-20T08:05:38.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset a9=[]:{68=[65],99=[95,91,8b,8a,89],a9=[a9,a8,a6,a5,a4,a3,a2,9f,9d]} for object 5:ab050921:::smithi11881747-42:5f 2024-03-20T08:05:38.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset b7=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:38.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:37 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/4058145941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:38.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[93121]: from='client.44129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:38.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[93121]: from='client.44135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:38.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[93121]: osdmap e310: 8 total, 8 up, 8 in 2024-03-20T08:05:38.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset a9=[]:{68=[65],99=[95,91,8b,8a,89],a9=[a9,a8,a6,a5,a4,a3,a2,9f,9d]} for object 5:ab050921:::smithi11881747-42:5f 2024-03-20T08:05:38.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset b7=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:38.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/4058145941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:38.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:37.858+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset b7=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:38.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:37.859+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset a9=[]:{68=[65],99=[95,91,8b,8a,89],a9=[a9,a8,a6,a5,a4,a3,a2,9f,9d]} for object 5:ab050921:::smithi11881747-42:5f 2024-03-20T08:05:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[95692]: from='client.44129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[95692]: from='client.44135 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:05:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[95692]: osdmap e310: 8 total, 8 up, 8 in 2024-03-20T08:05:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset a9=[]:{68=[65],99=[95,91,8b,8a,89],a9=[a9,a8,a6,a5,a4,a3,a2,9f,9d]} for object 5:ab050921:::smithi11881747-42:5f 2024-03-20T08:05:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset b7=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:38.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:37 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/4058145941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:05:38.847 INFO:tasks.rados.rados.0.smithi118.stdout:1210: expect (ObjNum 367 snap 124 seq_num 367) 2024-03-20T08:05:38.991 INFO:tasks.rados.rados.0.smithi118.stdout:1211: finishing write tid 1 to smithi11881747-14 2024-03-20T08:05:38.991 INFO:tasks.rados.rados.0.smithi118.stdout:1211: finishing write tid 2 to smithi11881747-14 2024-03-20T08:05:38.991 INFO:tasks.rados.rados.0.smithi118.stdout:1211: finishing write tid 3 to smithi11881747-14 2024-03-20T08:05:38.991 INFO:tasks.rados.rados.0.smithi118.stdout:1211: finishing write tid 4 to smithi11881747-14 2024-03-20T08:05:38.991 INFO:tasks.rados.rados.0.smithi118.stdout:1211: finishing write tid 5 to smithi11881747-14 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1211: oid 14 updating version 0 to 776 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1211: oid 14 version 776 is already newer than 774 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1211: oid 14 version 776 is already newer than 775 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1211: oid 14 updating version 776 to 777 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 777 (ObjNum 377 snap 128 seq_num 377) dirty exists 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1211: left oid 14 (ObjNum 377 snap 128 seq_num 377) 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1210: done (4 left) 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1211: done (3 left) 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1215: done (2 left) 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1216: rollback oid 23 current snap is 128 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 23 to 122 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1213: finishing write tid 1 to smithi11881747-36 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1213: finishing write tid 2 to smithi11881747-36 2024-03-20T08:05:38.992 INFO:tasks.rados.rados.0.smithi118.stdout:1213: finishing write tid 3 to smithi11881747-36 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1213: finishing write tid 4 to smithi11881747-36 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1213: finishing write tid 5 to smithi11881747-36 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1213: oid 36 updating version 0 to 7007 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1213: oid 36 version 7007 is already newer than 7005 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1213: oid 36 version 7007 is already newer than 7006 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1213: oid 36 version 7007 is already newer than 7004 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 7007 (ObjNum 378 snap 128 seq_num 378) dirty exists 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1213: left oid 36 (ObjNum 378 snap 128 seq_num 378) 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1214: finishing write tid 1 to smithi11881747-25 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1214: finishing write tid 2 to smithi11881747-25 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1214: finishing write tid 3 to smithi11881747-25 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1214: finishing write tid 4 to smithi11881747-25 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1214: finishing write tid 5 to smithi11881747-25 2024-03-20T08:05:38.993 INFO:tasks.rados.rados.0.smithi118.stdout:1214: oid 25 updating version 0 to 763 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1214: oid 25 version 763 is already newer than 761 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1214: oid 25 version 763 is already newer than 760 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1214: oid 25 version 763 is already newer than 762 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 763 (ObjNum 379 snap 128 seq_num 379) dirty exists 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1214: left oid 25 (ObjNum 379 snap 128 seq_num 379) 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1213: done (2 left) 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1214: done (1 left) 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1217: read oid 9 snap -1 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1217: expect deleted 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1218: read oid 29 snap -1 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1218: expect (ObjNum 371 snap 127 seq_num 371) 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1219: rollback oid 39 current snap is 128 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 79 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1220: read oid 12 snap 89 2024-03-20T08:05:38.994 INFO:tasks.rados.rados.0.smithi118.stdout:1220: expect (ObjNum 242 snap 85 seq_num 242) 2024-03-20T08:05:38.995 INFO:tasks.rados.rados.0.smithi118.stdout:1221: setattr oid 27 current snap is 128 2024-03-20T08:05:38.995 INFO:tasks.rados.rados.0.smithi118.stdout:1217: done (5 left) 2024-03-20T08:05:38.995 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 128 2024-03-20T08:05:38.995 INFO:tasks.rados.rados.0.smithi118.stdout:1222: seq_num 380 ranges {1908736=770048,2678784=483328,3162112=319488} 2024-03-20T08:05:39.009 INFO:tasks.rados.rados.0.smithi118.stdout:1222: writing smithi11881747-24 from 1908736 to 2678784 tid 1 2024-03-20T08:05:39.011 INFO:tasks.rados.rados.0.smithi118.stdout:1222: writing smithi11881747-24 from 2678784 to 3162112 tid 2 2024-03-20T08:05:39.014 INFO:tasks.rados.rados.0.smithi118.stdout:1222: writing smithi11881747-24 from 3162112 to 3481600 tid 3 2024-03-20T08:05:39.014 INFO:tasks.rados.rados.0.smithi118.stdout:1223: snap_create 2024-03-20T08:05:39.014 INFO:tasks.rados.rados.0.smithi118.stdout:1219: finishing rollback tid 0 to smithi11881747-39 2024-03-20T08:05:39.014 INFO:tasks.rados.rados.0.smithi118.stdout:1219: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:05:39.014 INFO:tasks.rados.rados.0.smithi118.stdout:1219: finishing rollback tid 2 to smithi11881747-39 2024-03-20T08:05:39.014 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 749 (ObjNum 191 snap 61 seq_num 191) dirty exists 2024-03-20T08:05:39.015 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 751 (ObjNum 366 snap 124 seq_num 366) dirty exists 2024-03-20T08:05:39.015 INFO:tasks.rados.rados.0.smithi118.stdout:1216: finishing rollback tid 0 to smithi11881747-23 2024-03-20T08:05:39.015 INFO:tasks.rados.rados.0.smithi118.stdout:1220: expect (ObjNum 242 snap 85 seq_num 242) 2024-03-20T08:05:39.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:38 smithi196 ceph-mon[87080]: osdmap e311: 8 total, 8 up, 8 in 2024-03-20T08:05:39.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset b7=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:39.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset a9=[]:{68=[65],99=[95,91,8b,8a,89],a9=[a9,a8,a6,a5,a4,a3,a2,9f,9d]} for object 5:ab050921:::smithi11881747-42:5f 2024-03-20T08:05:39.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:38 smithi196 ceph-mon[87080]: pgmap v143: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 323 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 661 KiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:05:39.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:38 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:39.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:38.847+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:39.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:38.848+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:39.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:38.848+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:39.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[93121]: osdmap e311: 8 total, 8 up, 8 in 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset b7=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset a9=[]:{68=[65],99=[95,91,8b,8a,89],a9=[a9,a8,a6,a5,a4,a3,a2,9f,9d]} for object 5:ab050921:::smithi11881747-42:5f 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[93121]: pgmap v143: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 323 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 661 KiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[95692]: osdmap e311: 8 total, 8 up, 8 in 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset b7=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset a9=[]:{68=[65],99=[95,91,8b,8a,89],a9=[a9,a8,a6,a5,a4,a3,a2,9f,9d]} for object 5:ab050921:::smithi11881747-42:5f 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[95692]: pgmap v143: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 323 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 661 KiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:05:39.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:38 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:39.224 INFO:tasks.rados.rados.0.smithi118.stdout:1222: finishing write tid 1 to smithi11881747-24 2024-03-20T08:05:39.224 INFO:tasks.rados.rados.0.smithi118.stdout:1216: finishing rollback tid 1 to smithi11881747-23 2024-03-20T08:05:39.224 INFO:tasks.rados.rados.0.smithi118.stdout:1216: finishing rollback tid 2 to smithi11881747-23 2024-03-20T08:05:39.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 1013 (ObjNum 358 snap 118 seq_num 358) dirty exists 2024-03-20T08:05:39.224 INFO:tasks.rados.rados.0.smithi118.stdout:1222: finishing write tid 2 to smithi11881747-24 2024-03-20T08:05:39.224 INFO:tasks.rados.rados.0.smithi118.stdout:1222: finishing write tid 3 to smithi11881747-24 2024-03-20T08:05:39.224 INFO:tasks.rados.rados.0.smithi118.stdout:1222: finishing write tid 4 to smithi11881747-24 2024-03-20T08:05:39.225 INFO:tasks.rados.rados.0.smithi118.stdout:1222: finishing write tid 5 to smithi11881747-24 2024-03-20T08:05:39.225 INFO:tasks.rados.rados.0.smithi118.stdout:1222: oid 24 updating version 0 to 779 2024-03-20T08:05:39.225 INFO:tasks.rados.rados.0.smithi118.stdout:1222: oid 24 updating version 779 to 781 2024-03-20T08:05:39.225 INFO:tasks.rados.rados.0.smithi118.stdout:1222: oid 24 updating version 781 to 782 2024-03-20T08:05:39.225 INFO:tasks.rados.rados.0.smithi118.stdout:1222: oid 24 version 782 is already newer than 780 2024-03-20T08:05:39.225 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 782 (ObjNum 380 snap 128 seq_num 380) dirty exists 2024-03-20T08:05:39.225 INFO:tasks.rados.rados.0.smithi118.stdout:1222: left oid 24 (ObjNum 380 snap 128 seq_num 380) 2024-03-20T08:05:39.225 INFO:tasks.rados.rados.0.smithi118.stdout:1218: expect (ObjNum 371 snap 127 seq_num 371) 2024-03-20T08:05:39.843 INFO:tasks.rados.rados.0.smithi118.stdout:1216: done (6 left) 2024-03-20T08:05:39.843 INFO:tasks.rados.rados.0.smithi118.stdout:1218: done (5 left) 2024-03-20T08:05:39.843 INFO:tasks.rados.rados.0.smithi118.stdout:1219: done (4 left) 2024-03-20T08:05:39.844 INFO:tasks.rados.rados.0.smithi118.stdout:1220: done (3 left) 2024-03-20T08:05:39.844 INFO:tasks.rados.rados.0.smithi118.stdout:1221: done (2 left) 2024-03-20T08:05:39.844 INFO:tasks.rados.rados.0.smithi118.stdout:1222: done (1 left) 2024-03-20T08:05:39.845 INFO:tasks.rados.rados.0.smithi118.stdout:1223: done (0 left) 2024-03-20T08:05:39.845 INFO:tasks.rados.rados.0.smithi118.stdout:1224: copy_from oid 6 from oid 49 current snap is 129 2024-03-20T08:05:39.845 INFO:tasks.rados.rados.0.smithi118.stdout:1225: rollback oid 33 current snap is 129 2024-03-20T08:05:39.845 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 122 2024-03-20T08:05:39.845 INFO:tasks.rados.rados.0.smithi118.stdout:1226: copy_from oid 38 from oid 1 current snap is 129 2024-03-20T08:05:39.845 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 129 2024-03-20T08:05:39.845 INFO:tasks.rados.rados.0.smithi118.stdout:1227: seq_num 381 ranges {3391488=786432,4177920=417792,4595712=385024} 2024-03-20T08:05:39.868 INFO:tasks.rados.rados.0.smithi118.stdout:1227: writing smithi11881747-16 from 3391488 to 4177920 tid 1 2024-03-20T08:05:39.870 INFO:tasks.rados.rados.0.smithi118.stdout:1227: writing smithi11881747-16 from 4177920 to 4595712 tid 2 2024-03-20T08:05:39.873 INFO:tasks.rados.rados.0.smithi118.stdout:1227: writing smithi11881747-16 from 4595712 to 4980736 tid 3 2024-03-20T08:05:39.873 INFO:tasks.rados.rados.0.smithi118.stdout:1228: snap_create 2024-03-20T08:05:39.873 INFO:tasks.rados.rados.0.smithi118.stdout:1225: finishing rollback tid 0 to smithi11881747-33 2024-03-20T08:05:39.873 INFO:tasks.rados.rados.0.smithi118.stdout:1225: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:05:39.873 INFO:tasks.rados.rados.0.smithi118.stdout:1225: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 784 (ObjNum 343 snap 112 seq_num 343) dirty exists 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:1226: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:1224: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:1226: finishing copy_from to smithi11881747-38 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 780 (ObjNum 343 snap 112 seq_num 343) dirty exists 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:1224: finishing copy_from to smithi11881747-6 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 406 (ObjNum 346 snap 112 seq_num 346) dirty exists 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:1227: finishing write tid 1 to smithi11881747-16 2024-03-20T08:05:39.874 INFO:tasks.rados.rados.0.smithi118.stdout:1227: finishing write tid 2 to smithi11881747-16 2024-03-20T08:05:39.875 INFO:tasks.rados.rados.0.smithi118.stdout:1227: finishing write tid 3 to smithi11881747-16 2024-03-20T08:05:39.876 INFO:tasks.rados.rados.0.smithi118.stdout:1227: finishing write tid 4 to smithi11881747-16 2024-03-20T08:05:39.877 INFO:tasks.rados.rados.0.smithi118.stdout:1227: finishing write tid 5 to smithi11881747-16 2024-03-20T08:05:39.877 INFO:tasks.rados.rados.0.smithi118.stdout:1227: oid 16 updating version 0 to 1016 2024-03-20T08:05:39.877 INFO:tasks.rados.rados.0.smithi118.stdout:1227: oid 16 updating version 1016 to 1018 2024-03-20T08:05:39.877 INFO:tasks.rados.rados.0.smithi118.stdout:1227: oid 16 version 1018 is already newer than 1017 2024-03-20T08:05:39.878 INFO:tasks.rados.rados.0.smithi118.stdout:1227: oid 16 version 1018 is already newer than 1015 2024-03-20T08:05:39.878 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 1018 (ObjNum 381 snap 129 seq_num 381) dirty exists 2024-03-20T08:05:39.878 INFO:tasks.rados.rados.0.smithi118.stdout:1227: left oid 16 (ObjNum 381 snap 129 seq_num 381) 2024-03-20T08:05:40.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:39 smithi196 ceph-mon[87080]: osdmap e312: 8 total, 8 up, 8 in 2024-03-20T08:05:40.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:40.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:40.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:39.865+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:40.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:39.866+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:40.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:39 smithi118 ceph-mon[93121]: osdmap e312: 8 total, 8 up, 8 in 2024-03-20T08:05:40.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:40.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:39 smithi118 ceph-mon[95692]: osdmap e312: 8 total, 8 up, 8 in 2024-03-20T08:05:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:40.856 INFO:tasks.rados.rados.0.smithi118.stdout:1224: done (4 left) 2024-03-20T08:05:40.856 INFO:tasks.rados.rados.0.smithi118.stdout:1225: done (3 left) 2024-03-20T08:05:40.856 INFO:tasks.rados.rados.0.smithi118.stdout:1226: done (2 left) 2024-03-20T08:05:40.856 INFO:tasks.rados.rados.0.smithi118.stdout:1227: done (1 left) 2024-03-20T08:05:40.856 INFO:tasks.rados.rados.0.smithi118.stdout:1228: done (0 left) 2024-03-20T08:05:40.856 INFO:tasks.rados.rados.0.smithi118.stdout:1229: copy_from oid 31 from oid 47 current snap is 130 2024-03-20T08:05:40.856 INFO:tasks.rados.rados.0.smithi118.stdout:1230: read oid 12 snap 77 2024-03-20T08:05:40.856 INFO:tasks.rados.rados.0.smithi118.stdout:1230: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:40.857 INFO:tasks.rados.rados.0.smithi118.stdout:1231: delete oid 32 current snap is 130 2024-03-20T08:05:40.863 INFO:tasks.rados.rados.0.smithi118.stdout:1231: done (2 left) 2024-03-20T08:05:40.863 INFO:tasks.rados.rados.0.smithi118.stdout:1232: snap_create 2024-03-20T08:05:40.870 INFO:tasks.rados.rados.0.smithi118.stdout:1230: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:40.979 INFO:tasks.rados.rados.0.smithi118.stdout:1229: finishing copy_from racing read to smithi11881747-31 2024-03-20T08:05:40.979 INFO:tasks.rados.rados.0.smithi118.stdout:1229: finishing copy_from to smithi11881747-31 2024-03-20T08:05:40.979 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 983 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:05:41.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:40 smithi196 ceph-mon[87080]: osdmap e313: 8 total, 8 up, 8 in 2024-03-20T08:05:41.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:41.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:41.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:40 smithi196 ceph-mon[87080]: pgmap v146: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 288 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 13 op/s 2024-03-20T08:05:41.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:40.864+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:41.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:40.864+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:41.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:40.864+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-mon[93121]: osdmap e313: 8 total, 8 up, 8 in 2024-03-20T08:05:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-mon[93121]: pgmap v146: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 288 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 13 op/s 2024-03-20T08:05:41.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-mon[95692]: osdmap e313: 8 total, 8 up, 8 in 2024-03-20T08:05:41.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:41.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:41.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:40 smithi118 ceph-mon[95692]: pgmap v146: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim, 119 active+clean; 288 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 13 op/s 2024-03-20T08:05:41.861 INFO:tasks.rados.rados.0.smithi118.stdout:1229: done (2 left) 2024-03-20T08:05:41.873 INFO:tasks.rados.rados.0.smithi118.stdout:1230: done (1 left) 2024-03-20T08:05:41.874 INFO:tasks.rados.rados.0.smithi118.stdout:1232: done (0 left) 2024-03-20T08:05:41.874 INFO:tasks.rados.rados.0.smithi118.stdout:1233: copy_from oid 38 from oid 8 current snap is 131 2024-03-20T08:05:41.874 INFO:tasks.rados.rados.0.smithi118.stdout:1234: delete oid 42 current snap is 131 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:1234: done (1 left) 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:1235: delete oid 29 current snap is 131 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:1233: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:1233: finishing copy_from to smithi11881747-38 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 466 (ObjNum 33 snap 0 seq_num 322383616) dirty exists 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:1233: done (1 left) 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:1235: done (0 left) 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:1236: rollback oid 34 current snap is 131 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 34 to 105 2024-03-20T08:05:41.875 INFO:tasks.rados.rados.0.smithi118.stdout:1237: snap_remove snap 124 2024-03-20T08:05:42.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:41 smithi196 ceph-mon[87080]: osdmap e314: 8 total, 8 up, 8 in 2024-03-20T08:05:42.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:42.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:42.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:41 smithi118 ceph-mon[93121]: osdmap e314: 8 total, 8 up, 8 in 2024-03-20T08:05:42.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:42.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:42.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:41 smithi118 ceph-mon[95692]: osdmap e314: 8 total, 8 up, 8 in 2024-03-20T08:05:42.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:42.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:42.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:41.865+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:42.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:41.866+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:42.868 INFO:tasks.rados.rados.0.smithi118.stdout:1237: done (1 left) 2024-03-20T08:05:42.868 INFO:tasks.rados.rados.0.smithi118.stdout:1238: copy_from oid 15 from oid 41 current snap is 131 2024-03-20T08:05:42.868 INFO:tasks.rados.rados.0.smithi118.stdout:1236: finishing rollback tid 0 to smithi11881747-34 2024-03-20T08:05:42.868 INFO:tasks.rados.rados.0.smithi118.stdout:1236: finishing rollback tid 1 to smithi11881747-34 2024-03-20T08:05:42.869 INFO:tasks.rados.rados.0.smithi118.stdout:1236: finishing rollback tid 2 to smithi11881747-34 2024-03-20T08:05:42.869 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 7011 (ObjNum 285 snap 96 seq_num 285) dirty exists 2024-03-20T08:05:42.869 INFO:tasks.rados.rados.0.smithi118.stdout:1236: done (1 left) 2024-03-20T08:05:42.869 INFO:tasks.rados.rados.0.smithi118.stdout:1239: rollback oid 24 current snap is 131 2024-03-20T08:05:42.869 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 78 2024-03-20T08:05:42.869 INFO:tasks.rados.rados.0.smithi118.stdout:1240: rollback oid 49 current snap is 131 2024-03-20T08:05:42.869 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 49 to 97 2024-03-20T08:05:42.869 INFO:tasks.rados.rados.0.smithi118.stdout:1241: snap_remove snap 85 2024-03-20T08:05:43.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:42 smithi196 ceph-mon[87080]: osdmap e315: 8 total, 8 up, 8 in 2024-03-20T08:05:43.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:43.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:43.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:42 smithi196 ceph-mon[87080]: pgmap v149: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 288 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 1.8 MiB/s wr, 16 op/s 2024-03-20T08:05:43.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:42.881+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:43.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:42.881+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:43.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:42.881+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-mon[95692]: osdmap e315: 8 total, 8 up, 8 in 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-mon[95692]: pgmap v149: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 288 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 1.8 MiB/s wr, 16 op/s 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:05:42.858+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 2 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-mon[93121]: osdmap e315: 8 total, 8 up, 8 in 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:42 smithi118 ceph-mon[93121]: pgmap v149: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 288 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 10 MiB/s rd, 1.8 MiB/s wr, 16 op/s 2024-03-20T08:05:43.864 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:05:43] "GET /metrics HTTP/1.1" 200 36187 "" "Prometheus/2.43.0" 2024-03-20T08:05:43.865 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:43.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:43.873 INFO:tasks.rados.rados.0.smithi118.stdout:1241: done (3 left) 2024-03-20T08:05:43.873 INFO:tasks.rados.rados.0.smithi118.stdout:1242: copy_from oid 2 from oid 11 current snap is 131 2024-03-20T08:05:43.873 INFO:tasks.rados.rados.0.smithi118.stdout:1239: finishing rollback tid 0 to smithi11881747-24 2024-03-20T08:05:43.873 INFO:tasks.rados.rados.0.smithi118.stdout:1239: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:05:43.873 INFO:tasks.rados.rados.0.smithi118.stdout:1239: finishing rollback tid 2 to smithi11881747-24 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 785 (ObjNum 216 snap 69 seq_num 216) dirty exists 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1240: finishing rollback tid 0 to smithi11881747-49 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1240: finishing rollback tid 1 to smithi11881747-49 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1240: finishing rollback tid 2 to smithi11881747-49 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 755 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1238: finishing copy_from racing read to smithi11881747-15 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1238: finishing copy_from to smithi11881747-15 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 101 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1238: done (3 left) 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1239: done (2 left) 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1240: done (1 left) 2024-03-20T08:05:43.874 INFO:tasks.rados.rados.0.smithi118.stdout:1243: snap_create 2024-03-20T08:05:43.886 INFO:tasks.rados.rados.0.smithi118.stdout:1242: finishing copy_from to smithi11881747-2 2024-03-20T08:05:43.886 INFO:tasks.rados.rados.0.smithi118.stdout:1242: got expected ENOENT (src dne) 2024-03-20T08:05:43.886 INFO:tasks.rados.rados.0.smithi118.stdout:1242: finishing copy_from racing read to smithi11881747-2 2024-03-20T08:05:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:43 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 2 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:05:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:43 smithi196 ceph-mon[87080]: osdmap e316: 8 total, 8 up, 8 in 2024-03-20T08:05:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:44.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:43.873+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:44.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:43.873+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:44.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 2 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:05:44.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[93121]: osdmap e316: 8 total, 8 up, 8 in 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 2 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[95692]: osdmap e316: 8 total, 8 up, 8 in 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:44.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:44.878 INFO:tasks.rados.rados.0.smithi118.stdout:1242: done (1 left) 2024-03-20T08:05:44.878 INFO:tasks.rados.rados.0.smithi118.stdout:1243: done (0 left) 2024-03-20T08:05:44.878 INFO:tasks.rados.rados.0.smithi118.stdout:1244: copy_from oid 35 from oid 22 current snap is 132 2024-03-20T08:05:44.878 INFO:tasks.rados.rados.0.smithi118.stdout:1245: rollback oid 20 current snap is 132 2024-03-20T08:05:44.878 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 20 to 129 2024-03-20T08:05:44.879 INFO:tasks.rados.rados.0.smithi118.stdout:1246: delete oid 27 current snap is 132 2024-03-20T08:05:44.883 INFO:tasks.rados.rados.0.smithi118.stdout:1245: finishing rollback tid 1 to smithi11881747-20 2024-03-20T08:05:44.883 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 624 (ObjNum 177 snap 0 seq_num 322076064) dirty dne 2024-03-20T08:05:44.884 INFO:tasks.rados.rados.0.smithi118.stdout:1245: done (2 left) 2024-03-20T08:05:44.884 INFO:tasks.rados.rados.0.smithi118.stdout:1246: done (1 left) 2024-03-20T08:05:44.884 INFO:tasks.rados.rados.0.smithi118.stdout:1247: rollback oid 39 current snap is 132 2024-03-20T08:05:44.884 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 89 2024-03-20T08:05:44.884 INFO:tasks.rados.rados.0.smithi118.stdout:1248: delete oid 11 current snap is 132 2024-03-20T08:05:44.889 INFO:tasks.rados.rados.0.smithi118.stdout:1248: done (2 left) 2024-03-20T08:05:44.889 INFO:tasks.rados.rados.0.smithi118.stdout:1249: copy_from oid 20 from oid 34 current snap is 132 2024-03-20T08:05:44.889 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 132 2024-03-20T08:05:44.889 INFO:tasks.rados.rados.0.smithi118.stdout:1250: seq_num 387 ranges {0=483328,483328=524288,1007616=679936} 2024-03-20T08:05:44.891 INFO:tasks.rados.rados.0.smithi118.stdout:1250: writing smithi11881747-7 from 0 to 483328 tid 1 2024-03-20T08:05:44.895 INFO:tasks.rados.rados.0.smithi118.stdout:1250: writing smithi11881747-7 from 483328 to 1007616 tid 2 2024-03-20T08:05:44.899 INFO:tasks.rados.rados.0.smithi118.stdout:1250: writing smithi11881747-7 from 1007616 to 1687552 tid 3 2024-03-20T08:05:44.899 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 132 2024-03-20T08:05:44.899 INFO:tasks.rados.rados.0.smithi118.stdout:1247: finishing rollback tid 0 to smithi11881747-39 2024-03-20T08:05:44.899 INFO:tasks.rados.rados.0.smithi118.stdout:1251: seq_num 388 ranges {0=507904,507904=458752,966656=606208,1572864=139264} 2024-03-20T08:05:44.902 INFO:tasks.rados.rados.0.smithi118.stdout:1251: writing smithi11881747-3 from 0 to 507904 tid 1 2024-03-20T08:05:44.905 INFO:tasks.rados.rados.0.smithi118.stdout:1251: writing smithi11881747-3 from 507904 to 966656 tid 2 2024-03-20T08:05:44.908 INFO:tasks.rados.rados.0.smithi118.stdout:1251: writing smithi11881747-3 from 966656 to 1572864 tid 3 2024-03-20T08:05:44.911 INFO:tasks.rados.rados.0.smithi118.stdout:1251: writing smithi11881747-3 from 1572864 to 1712128 tid 4 2024-03-20T08:05:44.911 INFO:tasks.rados.rados.0.smithi118.stdout:1252: rollback oid 6 current snap is 132 2024-03-20T08:05:44.911 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 79 2024-03-20T08:05:44.911 INFO:tasks.rados.rados.0.smithi118.stdout:1247: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:05:44.911 INFO:tasks.rados.rados.0.smithi118.stdout:1247: finishing rollback tid 2 to smithi11881747-39 2024-03-20T08:05:44.911 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 774 (ObjNum 191 snap 61 seq_num 191) dirty exists 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1244: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1244: finishing copy_from to smithi11881747-35 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 776 (ObjNum 260 snap 90 seq_num 260) dirty exists 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: finishing write tid 1 to smithi11881747-7 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: finishing write tid 2 to smithi11881747-7 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: finishing write tid 3 to smithi11881747-7 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: finishing write tid 4 to smithi11881747-7 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1251: finishing write tid 1 to smithi11881747-3 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: finishing write tid 5 to smithi11881747-7 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: oid 7 updating version 0 to 244 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: oid 7 updating version 244 to 245 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: oid 7 updating version 245 to 246 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:1250: oid 7 version 246 is already newer than 243 2024-03-20T08:05:44.912 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 246 (ObjNum 387 snap 132 seq_num 387) dirty exists 2024-03-20T08:05:44.913 INFO:tasks.rados.rados.0.smithi118.stdout:1250: left oid 7 (ObjNum 387 snap 132 seq_num 387) 2024-03-20T08:05:44.913 INFO:tasks.rados.rados.0.smithi118.stdout:1251: finishing write tid 2 to smithi11881747-3 2024-03-20T08:05:44.913 INFO:tasks.rados.rados.0.smithi118.stdout:1244: done (5 left) 2024-03-20T08:05:44.913 INFO:tasks.rados.rados.0.smithi118.stdout:1247: done (4 left) 2024-03-20T08:05:44.913 INFO:tasks.rados.rados.0.smithi118.stdout:1250: done (3 left) 2024-03-20T08:05:44.913 INFO:tasks.rados.rados.0.smithi118.stdout:1253: read oid 14 snap -1 2024-03-20T08:05:44.913 INFO:tasks.rados.rados.0.smithi118.stdout:1253: expect (ObjNum 377 snap 128 seq_num 377) 2024-03-20T08:05:44.913 INFO:tasks.rados.rados.0.smithi118.stdout:1254: snap_remove snap 78 2024-03-20T08:05:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:44 smithi196 ceph-mon[87080]: osdmap e317: 8 total, 8 up, 8 in 2024-03-20T08:05:45.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:44 smithi196 ceph-mon[87080]: pgmap v152: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 292 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 1.4 MiB/s wr, 4 op/s 2024-03-20T08:05:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-mon[93121]: osdmap e317: 8 total, 8 up, 8 in 2024-03-20T08:05:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-mon[93121]: pgmap v152: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 292 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 1.4 MiB/s wr, 4 op/s 2024-03-20T08:05:45.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-mon[95692]: osdmap e317: 8 total, 8 up, 8 in 2024-03-20T08:05:45.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-mon[95692]: pgmap v152: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 292 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 1.4 MiB/s wr, 4 op/s 2024-03-20T08:05:45.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:44.887+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:45.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:44.887+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:45.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:44.889+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:45.886 INFO:tasks.rados.rados.0.smithi118.stdout:1251: finishing write tid 3 to smithi11881747-3 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1249: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1252: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1252: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1251: finishing write tid 4 to smithi11881747-3 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1252: finishing rollback tid 2 to smithi11881747-6 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 779 (ObjNum 219 snap 71 seq_num 219) dirty exists 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1251: finishing write tid 5 to smithi11881747-3 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1249: finishing copy_from to smithi11881747-20 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 7011 (ObjNum 285 snap 96 seq_num 285) dirty exists 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1251: finishing write tid 6 to smithi11881747-3 2024-03-20T08:05:45.887 INFO:tasks.rados.rados.0.smithi118.stdout:1251: oid 3 updating version 0 to 788 2024-03-20T08:05:45.888 INFO:tasks.rados.rados.0.smithi118.stdout:1251: oid 3 version 788 is already newer than 785 2024-03-20T08:05:45.888 INFO:tasks.rados.rados.0.smithi118.stdout:1251: oid 3 version 788 is already newer than 787 2024-03-20T08:05:45.888 INFO:tasks.rados.rados.0.smithi118.stdout:1251: oid 3 version 788 is already newer than 786 2024-03-20T08:05:45.888 INFO:tasks.rados.rados.0.smithi118.stdout:1251: oid 3 updating version 788 to 789 2024-03-20T08:05:45.888 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 789 (ObjNum 388 snap 132 seq_num 388) dirty exists 2024-03-20T08:05:45.888 INFO:tasks.rados.rados.0.smithi118.stdout:1251: left oid 3 (ObjNum 388 snap 132 seq_num 388) 2024-03-20T08:05:45.888 INFO:tasks.rados.rados.0.smithi118.stdout:1253: expect (ObjNum 377 snap 128 seq_num 377) 2024-03-20T08:05:46.007 INFO:tasks.rados.rados.0.smithi118.stdout:1249: done (4 left) 2024-03-20T08:05:46.007 INFO:tasks.rados.rados.0.smithi118.stdout:1251: done (3 left) 2024-03-20T08:05:46.007 INFO:tasks.rados.rados.0.smithi118.stdout:1252: done (2 left) 2024-03-20T08:05:46.007 INFO:tasks.rados.rados.0.smithi118.stdout:1253: done (1 left) 2024-03-20T08:05:46.007 INFO:tasks.rados.rados.0.smithi118.stdout:1254: done (0 left) 2024-03-20T08:05:46.007 INFO:tasks.rados.rados.0.smithi118.stdout:1255: delete oid 37 current snap is 132 2024-03-20T08:05:46.009 INFO:tasks.rados.rados.0.smithi118.stdout:1255: done (0 left) 2024-03-20T08:05:46.009 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 132 2024-03-20T08:05:46.009 INFO:tasks.rados.rados.0.smithi118.stdout:1256: seq_num 390 ranges {4644864=540672,5185536=630784,5816320=573440} 2024-03-20T08:05:46.036 INFO:tasks.rados.rados.0.smithi118.stdout:1256: writing smithi11881747-19 from 4644864 to 5185536 tid 1 2024-03-20T08:05:46.039 INFO:tasks.rados.rados.0.smithi118.stdout:1256: writing smithi11881747-19 from 5185536 to 5816320 tid 2 2024-03-20T08:05:46.043 INFO:tasks.rados.rados.0.smithi118.stdout:1256: writing smithi11881747-19 from 5816320 to 6389760 tid 3 2024-03-20T08:05:46.043 INFO:tasks.rados.rados.0.smithi118.stdout:1257: snap_create 2024-03-20T08:05:46.043 INFO:tasks.rados.rados.0.smithi118.stdout:1256: finishing write tid 1 to smithi11881747-19 2024-03-20T08:05:46.043 INFO:tasks.rados.rados.0.smithi118.stdout:1256: finishing write tid 2 to smithi11881747-19 2024-03-20T08:05:46.046 INFO:tasks.rados.rados.0.smithi118.stdout:1256: finishing write tid 3 to smithi11881747-19 2024-03-20T08:05:46.046 INFO:tasks.rados.rados.0.smithi118.stdout:1256: finishing write tid 4 to smithi11881747-19 2024-03-20T08:05:46.048 INFO:tasks.rados.rados.0.smithi118.stdout:1256: finishing write tid 5 to smithi11881747-19 2024-03-20T08:05:46.048 INFO:tasks.rados.rados.0.smithi118.stdout:1256: oid 19 updating version 0 to 249 2024-03-20T08:05:46.048 INFO:tasks.rados.rados.0.smithi118.stdout:1256: oid 19 updating version 249 to 252 2024-03-20T08:05:46.048 INFO:tasks.rados.rados.0.smithi118.stdout:1256: oid 19 version 252 is already newer than 250 2024-03-20T08:05:46.048 INFO:tasks.rados.rados.0.smithi118.stdout:1256: oid 19 version 252 is already newer than 251 2024-03-20T08:05:46.048 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 252 (ObjNum 390 snap 132 seq_num 390) dirty exists 2024-03-20T08:05:46.048 INFO:tasks.rados.rados.0.smithi118.stdout:1256: left oid 19 (ObjNum 390 snap 132 seq_num 390) 2024-03-20T08:05:46.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:45 smithi196 ceph-mon[87080]: osdmap e318: 8 total, 8 up, 8 in 2024-03-20T08:05:46.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:46.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:45 smithi118 ceph-mon[93121]: osdmap e318: 8 total, 8 up, 8 in 2024-03-20T08:05:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:46.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:45 smithi118 ceph-mon[95692]: osdmap e318: 8 total, 8 up, 8 in 2024-03-20T08:05:46.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:46.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:46.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:45.890+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:46.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:45.890+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:46.888 INFO:tasks.rados.rados.0.smithi118.stdout:1256: done (1 left) 2024-03-20T08:05:46.889 INFO:tasks.rados.rados.0.smithi118.stdout:1257: done (0 left) 2024-03-20T08:05:46.889 INFO:tasks.rados.rados.0.smithi118.stdout:1258: snap_create 2024-03-20T08:05:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:46 smithi196 ceph-mon[87080]: osdmap e319: 8 total, 8 up, 8 in 2024-03-20T08:05:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:46 smithi196 ceph-mon[87080]: pgmap v155: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 299 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 7 op/s 2024-03-20T08:05:47.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:47.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[93121]: osdmap e319: 8 total, 8 up, 8 in 2024-03-20T08:05:47.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:47.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:47.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[93121]: pgmap v155: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 299 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 7 op/s 2024-03-20T08:05:47.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[95692]: osdmap e319: 8 total, 8 up, 8 in 2024-03-20T08:05:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[95692]: pgmap v155: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 299 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 7 op/s 2024-03-20T08:05:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:47.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:46.983+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:47.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:46.983+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:47.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:46.983+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:47.889 INFO:tasks.rados.rados.0.smithi118.stdout:1258: done (0 left) 2024-03-20T08:05:47.889 INFO:tasks.rados.rados.0.smithi118.stdout:1259: rollback oid 44 current snap is 134 2024-03-20T08:05:47.889 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 44 to 112 2024-03-20T08:05:47.889 INFO:tasks.rados.rados.0.smithi118.stdout:1260: read oid 36 snap -1 2024-03-20T08:05:47.889 INFO:tasks.rados.rados.0.smithi118.stdout:1260: expect (ObjNum 378 snap 128 seq_num 378) 2024-03-20T08:05:47.889 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 134 2024-03-20T08:05:47.889 INFO:tasks.rados.rados.0.smithi118.stdout:1261: seq_num 391 ranges {2621440=557056,3178496=516096,3694592=688128} 2024-03-20T08:05:47.906 INFO:tasks.rados.rados.0.smithi118.stdout:1261: writing smithi11881747-1 from 2621440 to 3178496 tid 1 2024-03-20T08:05:47.909 INFO:tasks.rados.rados.0.smithi118.stdout:1261: writing smithi11881747-1 from 3178496 to 3694592 tid 2 2024-03-20T08:05:47.914 INFO:tasks.rados.rados.0.smithi118.stdout:1261: writing smithi11881747-1 from 3694592 to 4382720 tid 3 2024-03-20T08:05:47.914 INFO:tasks.rados.rados.0.smithi118.stdout:1262: delete oid 50 current snap is 134 2024-03-20T08:05:47.914 INFO:tasks.rados.rados.0.smithi118.stdout:1259: finishing rollback tid 1 to smithi11881747-44 2024-03-20T08:05:47.914 INFO:tasks.rados.rados.0.smithi118.stdout:1259: finishing rollback tid 2 to smithi11881747-44 2024-03-20T08:05:47.914 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 787 (ObjNum 332 snap 107 seq_num 332) dirty exists 2024-03-20T08:05:47.914 INFO:tasks.rados.rados.0.smithi118.stdout:1260: expect (ObjNum 378 snap 128 seq_num 378) 2024-03-20T08:05:48.081 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:47.889+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:48.082 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:47.889+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:48.082 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:48.082 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[93121]: Upgrade: osd.1 is safe to restart 2024-03-20T08:05:48.082 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[93121]: osdmap e320: 8 total, 8 up, 8 in 2024-03-20T08:05:48.082 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:48.082 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:48.082 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:48.083 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[95692]: Upgrade: osd.1 is safe to restart 2024-03-20T08:05:48.083 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[95692]: osdmap e320: 8 total, 8 up, 8 in 2024-03-20T08:05:48.083 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:48.083 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: finishing write tid 1 to smithi11881747-1 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: finishing write tid 2 to smithi11881747-1 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: finishing write tid 3 to smithi11881747-1 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: finishing write tid 4 to smithi11881747-1 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: finishing write tid 5 to smithi11881747-1 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: oid 1 updating version 0 to 987 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: oid 1 version 987 is already newer than 986 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: oid 1 version 987 is already newer than 985 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: oid 1 version 987 is already newer than 984 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 987 (ObjNum 391 snap 134 seq_num 391) dirty exists 2024-03-20T08:05:48.162 INFO:tasks.rados.rados.0.smithi118.stdout:1261: left oid 1 (ObjNum 391 snap 134 seq_num 391) 2024-03-20T08:05:48.163 INFO:tasks.rados.rados.0.smithi118.stdout:1259: done (3 left) 2024-03-20T08:05:48.163 INFO:tasks.rados.rados.0.smithi118.stdout:1260: done (2 left) 2024-03-20T08:05:48.163 INFO:tasks.rados.rados.0.smithi118.stdout:1261: done (1 left) 2024-03-20T08:05:48.163 INFO:tasks.rados.rados.0.smithi118.stdout:1262: done (0 left) 2024-03-20T08:05:48.163 INFO:tasks.rados.rados.0.smithi118.stdout:1263: read oid 15 snap -1 2024-03-20T08:05:48.163 INFO:tasks.rados.rados.0.smithi118.stdout:1263: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:48.163 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 134 2024-03-20T08:05:48.163 INFO:tasks.rados.rados.0.smithi118.stdout:1264: seq_num 393 ranges {770048=589824,1359872=679936,2039808=524288} 2024-03-20T08:05:48.169 INFO:tasks.rados.rados.0.smithi118.stdout:1264: writing smithi11881747-44 from 770048 to 1359872 tid 1 2024-03-20T08:05:48.172 INFO:tasks.rados.rados.0.smithi118.stdout:1264: writing smithi11881747-44 from 1359872 to 2039808 tid 2 2024-03-20T08:05:48.175 INFO:tasks.rados.rados.0.smithi118.stdout:1264: writing smithi11881747-44 from 2039808 to 2564096 tid 3 2024-03-20T08:05:48.175 INFO:tasks.rados.rados.0.smithi118.stdout:1265: snap_create 2024-03-20T08:05:48.176 INFO:tasks.rados.rados.0.smithi118.stdout:1264: finishing write tid 1 to smithi11881747-44 2024-03-20T08:05:48.179 INFO:tasks.rados.rados.0.smithi118.stdout:1264: finishing write tid 2 to smithi11881747-44 2024-03-20T08:05:48.179 INFO:tasks.rados.rados.0.smithi118.stdout:1264: finishing write tid 3 to smithi11881747-44 2024-03-20T08:05:48.180 INFO:tasks.rados.rados.0.smithi118.stdout:1264: finishing write tid 4 to smithi11881747-44 2024-03-20T08:05:48.181 INFO:tasks.rados.rados.0.smithi118.stdout:1263: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:48.291 INFO:tasks.rados.rados.0.smithi118.stdout:1264: finishing write tid 5 to smithi11881747-44 2024-03-20T08:05:48.291 INFO:tasks.rados.rados.0.smithi118.stdout:1264: oid 44 updating version 0 to 792 2024-03-20T08:05:48.291 INFO:tasks.rados.rados.0.smithi118.stdout:1264: oid 44 version 792 is already newer than 790 2024-03-20T08:05:48.292 INFO:tasks.rados.rados.0.smithi118.stdout:1264: oid 44 version 792 is already newer than 789 2024-03-20T08:05:48.292 INFO:tasks.rados.rados.0.smithi118.stdout:1264: oid 44 version 792 is already newer than 791 2024-03-20T08:05:48.292 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 792 (ObjNum 393 snap 134 seq_num 393) dirty exists 2024-03-20T08:05:48.292 INFO:tasks.rados.rados.0.smithi118.stdout:1264: left oid 44 (ObjNum 393 snap 134 seq_num 393) 2024-03-20T08:05:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:47 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-20T08:05:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:47 smithi196 ceph-mon[87080]: Upgrade: osd.1 is safe to restart 2024-03-20T08:05:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:47 smithi196 ceph-mon[87080]: osdmap e320: 8 total, 8 up, 8 in 2024-03-20T08:05:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:48.899 INFO:tasks.rados.rados.0.smithi118.stdout:1263: done (2 left) 2024-03-20T08:05:48.899 INFO:tasks.rados.rados.0.smithi118.stdout:1264: done (1 left) 2024-03-20T08:05:48.899 INFO:tasks.rados.rados.0.smithi118.stdout:1265: done (0 left) 2024-03-20T08:05:48.899 INFO:tasks.rados.rados.0.smithi118.stdout:1266: read oid 49 snap -1 2024-03-20T08:05:48.899 INFO:tasks.rados.rados.0.smithi118.stdout:1266: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:48.899 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 135 2024-03-20T08:05:48.900 INFO:tasks.rados.rados.0.smithi118.stdout:1267: seq_num 394 ranges {0=606208,606208=565248,1171456=557056,1728512=81920} 2024-03-20T08:05:48.903 INFO:tasks.rados.rados.0.smithi118.stdout:1267: writing smithi11881747-45 from 0 to 606208 tid 1 2024-03-20T08:05:48.906 INFO:tasks.rados.rados.0.smithi118.stdout:1267: writing smithi11881747-45 from 606208 to 1171456 tid 2 2024-03-20T08:05:48.911 INFO:tasks.rados.rados.0.smithi118.stdout:1267: writing smithi11881747-45 from 1171456 to 1728512 tid 3 2024-03-20T08:05:48.912 INFO:tasks.rados.rados.0.smithi118.stdout:1267: writing smithi11881747-45 from 1728512 to 1810432 tid 4 2024-03-20T08:05:48.912 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 135 2024-03-20T08:05:48.912 INFO:tasks.rados.rados.0.smithi118.stdout:1268: seq_num 395 ranges {3391488=622592,4014080=450560,4464640=753664} 2024-03-20T08:05:48.934 INFO:tasks.rados.rados.0.smithi118.stdout:1268: writing smithi11881747-18 from 3391488 to 4014080 tid 1 2024-03-20T08:05:48.937 INFO:tasks.rados.rados.0.smithi118.stdout:1268: writing smithi11881747-18 from 4014080 to 4464640 tid 2 2024-03-20T08:05:48.942 INFO:tasks.rados.rados.0.smithi118.stdout:1268: writing smithi11881747-18 from 4464640 to 5218304 tid 3 2024-03-20T08:05:48.942 INFO:tasks.rados.rados.0.smithi118.stdout:1269: rollback oid 27 current snap is 135 2024-03-20T08:05:48.942 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 89 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:1267: finishing write tid 1 to smithi11881747-45 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:1270: read oid 23 snap -1 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:1270: expect (ObjNum 358 snap 118 seq_num 358) 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:1271: read oid 35 snap -1 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:1271: expect (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:1272: read oid 1 snap 134 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:1272: expect (ObjNum 391 snap 134 seq_num 391) 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 135 2024-03-20T08:05:48.943 INFO:tasks.rados.rados.0.smithi118.stdout:1273: seq_num 396 ranges {0=638976,638976=786432,1425408=417792} 2024-03-20T08:05:48.946 INFO:tasks.rados.rados.0.smithi118.stdout:1273: writing smithi11881747-11 from 0 to 638976 tid 1 2024-03-20T08:05:48.950 INFO:tasks.rados.rados.0.smithi118.stdout:1273: writing smithi11881747-11 from 638976 to 1425408 tid 2 2024-03-20T08:05:48.953 INFO:tasks.rados.rados.0.smithi118.stdout:1273: writing smithi11881747-11 from 1425408 to 1843200 tid 3 2024-03-20T08:05:48.954 INFO:tasks.rados.rados.0.smithi118.stdout:1267: finishing write tid 2 to smithi11881747-45 2024-03-20T08:05:48.954 INFO:tasks.rados.rados.0.smithi118.stdout:1266: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:49.069 INFO:tasks.rados.rados.0.smithi118.stdout:1267: finishing write tid 3 to smithi11881747-45 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: finishing write tid 4 to smithi11881747-45 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: finishing write tid 5 to smithi11881747-45 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: finishing write tid 6 to smithi11881747-45 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: oid 45 updating version 0 to 332 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: oid 45 version 332 is already newer than 330 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: oid 45 version 332 is already newer than 329 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: oid 45 updating version 332 to 333 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: oid 45 version 333 is already newer than 331 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 333 (ObjNum 394 snap 135 seq_num 394) dirty exists 2024-03-20T08:05:49.070 INFO:tasks.rados.rados.0.smithi118.stdout:1267: left oid 45 (ObjNum 394 snap 135 seq_num 394) 2024-03-20T08:05:49.071 INFO:tasks.rados.rados.0.smithi118.stdout:1268: finishing write tid 1 to smithi11881747-18 2024-03-20T08:05:49.071 INFO:tasks.rados.rados.0.smithi118.stdout:1266: done (7 left) 2024-03-20T08:05:49.071 INFO:tasks.rados.rados.0.smithi118.stdout:1267: done (6 left) 2024-03-20T08:05:49.071 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 135 2024-03-20T08:05:49.071 INFO:tasks.rados.rados.0.smithi118.stdout:1268: finishing write tid 2 to smithi11881747-18 2024-03-20T08:05:49.071 INFO:tasks.rados.rados.0.smithi118.stdout:1268: finishing write tid 3 to smithi11881747-18 2024-03-20T08:05:49.071 INFO:tasks.rados.rados.0.smithi118.stdout:1268: finishing write tid 4 to smithi11881747-18 2024-03-20T08:05:49.071 INFO:tasks.rados.rados.0.smithi118.stdout:1274: seq_num 397 ranges {2146304=655360,2801664=671744,3473408=532480} 2024-03-20T08:05:49.084 INFO:tasks.rados.rados.0.smithi118.stdout:1274: writing smithi11881747-39 from 2146304 to 2801664 tid 1 2024-03-20T08:05:49.088 INFO:tasks.rados.rados.0.smithi118.stdout:1274: writing smithi11881747-39 from 2801664 to 3473408 tid 2 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1274: writing smithi11881747-39 from 3473408 to 4005888 tid 3 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1275: read oid 42 snap -1 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1275: expect deleted 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 135 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1268: finishing write tid 5 to smithi11881747-18 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1268: oid 18 updating version 0 to 7015 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1268: oid 18 version 7015 is already newer than 7013 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1268: oid 18 version 7015 is already newer than 7012 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1268: oid 18 version 7015 is already newer than 7014 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 7015 (ObjNum 395 snap 135 seq_num 395) dirty exists 2024-03-20T08:05:49.092 INFO:tasks.rados.rados.0.smithi118.stdout:1268: left oid 18 (ObjNum 395 snap 135 seq_num 395) 2024-03-20T08:05:49.093 INFO:tasks.rados.rados.0.smithi118.stdout:1269: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:05:49.093 INFO:tasks.rados.rados.0.smithi118.stdout:1269: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:05:49.093 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7017 (ObjNum 250 snap 88 seq_num 250) dirty exists 2024-03-20T08:05:49.093 INFO:tasks.rados.rados.0.smithi118.stdout:1273: finishing write tid 1 to smithi11881747-11 2024-03-20T08:05:49.093 INFO:tasks.rados.rados.0.smithi118.stdout:1276: seq_num 398 ranges {0=671744,671744=557056,1228800=532480,1761280=114688} 2024-03-20T08:05:49.095 INFO:tasks.rados.rados.0.smithi118.stdout:1276: writing smithi11881747-9 from 0 to 671744 tid 1 2024-03-20T08:05:49.098 INFO:tasks.rados.rados.0.smithi118.stdout:1276: writing smithi11881747-9 from 671744 to 1228800 tid 2 2024-03-20T08:05:49.102 INFO:tasks.rados.rados.0.smithi118.stdout:1276: writing smithi11881747-9 from 1228800 to 1761280 tid 3 2024-03-20T08:05:49.103 INFO:tasks.rados.rados.0.smithi118.stdout:1276: writing smithi11881747-9 from 1761280 to 1875968 tid 4 2024-03-20T08:05:49.103 INFO:tasks.rados.rados.0.smithi118.stdout:1268: done (8 left) 2024-03-20T08:05:49.103 INFO:tasks.rados.rados.0.smithi118.stdout:1269: done (7 left) 2024-03-20T08:05:49.103 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 135 2024-03-20T08:05:49.103 INFO:tasks.rados.rados.0.smithi118.stdout:1277: seq_num 399 ranges {5218304=688128,5906432=434176,6340608=745472,7086080=24576} 2024-03-20T08:05:49.134 INFO:tasks.rados.rados.0.smithi118.stdout:1277: writing smithi11881747-18 from 5218304 to 5906432 tid 1 2024-03-20T08:05:49.137 INFO:tasks.rados.rados.0.smithi118.stdout:1277: writing smithi11881747-18 from 5906432 to 6340608 tid 2 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1277: writing smithi11881747-18 from 6340608 to 7086080 tid 3 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1277: writing smithi11881747-18 from 7086080 to 7110656 tid 4 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1278: rmattr oid 19 current snap is 135 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1278: done (8 left) 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1279: delete oid 37 current snap is 135 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1273: finishing write tid 2 to smithi11881747-11 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1273: finishing write tid 3 to smithi11881747-11 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1273: finishing write tid 4 to smithi11881747-11 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1273: finishing write tid 5 to smithi11881747-11 2024-03-20T08:05:49.142 INFO:tasks.rados.rados.0.smithi118.stdout:1273: oid 11 updating version 0 to 767 2024-03-20T08:05:49.143 INFO:tasks.rados.rados.0.smithi118.stdout:1273: oid 11 updating version 767 to 769 2024-03-20T08:05:49.143 INFO:tasks.rados.rados.0.smithi118.stdout:1273: oid 11 updating version 769 to 770 2024-03-20T08:05:49.143 INFO:tasks.rados.rados.0.smithi118.stdout:1273: oid 11 version 770 is already newer than 768 2024-03-20T08:05:49.143 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 770 (ObjNum 396 snap 135 seq_num 396) dirty exists 2024-03-20T08:05:49.143 INFO:tasks.rados.rados.0.smithi118.stdout:1273: left oid 11 (ObjNum 396 snap 135 seq_num 396) 2024-03-20T08:05:49.143 INFO:tasks.rados.rados.0.smithi118.stdout:1270: expect (ObjNum 358 snap 118 seq_num 358) 2024-03-20T08:05:49.171 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:48.911+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:49.171 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:48.911+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:49.171 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:48.912+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:49.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[93121]: osdmap e321: 8 total, 8 up, 8 in 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[93121]: pgmap v158: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 299 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[95692]: osdmap e321: 8 total, 8 up, 8 in 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[95692]: pgmap v158: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 299 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T08:05:49.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:49.341 INFO:tasks.rados.rados.0.smithi118.stdout:1271: expect (ObjNum 260 snap 90 seq_num 260) 2024-03-20T08:05:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:48 smithi196 ceph-mon[87080]: osdmap e321: 8 total, 8 up, 8 in 2024-03-20T08:05:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:48 smithi196 ceph-mon[87080]: pgmap v158: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 299 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 5 op/s 2024-03-20T08:05:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-20T08:05:49.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:05:49.392 INFO:tasks.rados.rados.0.smithi118.stdout:1272: expect (ObjNum 391 snap 134 seq_num 391) 2024-03-20T08:05:49.594 INFO:tasks.rados.rados.0.smithi118.stdout:1274: finishing write tid 1 to smithi11881747-39 2024-03-20T08:05:49.594 INFO:tasks.rados.rados.0.smithi118.stdout:1270: done (8 left) 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1271: done (7 left) 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1272: done (6 left) 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1273: done (5 left) 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1279: done (4 left) 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1280: snap_create 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1274: finishing write tid 2 to smithi11881747-39 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1274: finishing write tid 3 to smithi11881747-39 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1274: finishing write tid 4 to smithi11881747-39 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1274: finishing write tid 5 to smithi11881747-39 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1274: oid 39 updating version 0 to 7020 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1274: oid 39 version 7020 is already newer than 7018 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1274: oid 39 version 7020 is already newer than 7019 2024-03-20T08:05:49.595 INFO:tasks.rados.rados.0.smithi118.stdout:1274: oid 39 updating version 7020 to 7021 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7021 (ObjNum 397 snap 135 seq_num 397) dirty exists 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1274: left oid 39 (ObjNum 397 snap 135 seq_num 397) 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: finishing write tid 1 to smithi11881747-9 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: finishing write tid 2 to smithi11881747-9 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: finishing write tid 3 to smithi11881747-9 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: finishing write tid 4 to smithi11881747-9 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: finishing write tid 5 to smithi11881747-9 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: finishing write tid 6 to smithi11881747-9 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: oid 9 updating version 0 to 775 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: oid 9 version 775 is already newer than 772 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: oid 9 version 775 is already newer than 774 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: oid 9 version 775 is already newer than 773 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: oid 9 version 775 is already newer than 771 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 775 (ObjNum 398 snap 135 seq_num 398) dirty exists 2024-03-20T08:05:49.596 INFO:tasks.rados.rados.0.smithi118.stdout:1276: left oid 9 (ObjNum 398 snap 135 seq_num 398) 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: finishing write tid 1 to smithi11881747-18 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: finishing write tid 2 to smithi11881747-18 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: finishing write tid 3 to smithi11881747-18 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: finishing write tid 4 to smithi11881747-18 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: finishing write tid 5 to smithi11881747-18 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: finishing write tid 6 to smithi11881747-18 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: oid 18 updating version 0 to 7023 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: oid 18 updating version 7023 to 7026 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: oid 18 version 7026 is already newer than 7022 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: oid 18 version 7026 is already newer than 7025 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: oid 18 version 7026 is already newer than 7024 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 7026 (ObjNum 399 snap 135 seq_num 399) dirty exists 2024-03-20T08:05:49.597 INFO:tasks.rados.rados.0.smithi118.stdout:1277: left oid 18 (ObjNum 399 snap 135 seq_num 399) 2024-03-20T08:05:49.905 INFO:tasks.rados.rados.0.smithi118.stdout:1274: done (4 left) 2024-03-20T08:05:49.905 INFO:tasks.rados.rados.0.smithi118.stdout:1275: done (3 left) 2024-03-20T08:05:49.905 INFO:tasks.rados.rados.0.smithi118.stdout:1276: done (2 left) 2024-03-20T08:05:49.905 INFO:tasks.rados.rados.0.smithi118.stdout:1277: done (1 left) 2024-03-20T08:05:49.905 INFO:tasks.rados.rados.0.smithi118.stdout:1280: done (0 left) 2024-03-20T08:05:49.905 INFO:tasks.rados.rados.0.smithi118.stdout:1281: read oid 10 snap -1 2024-03-20T08:05:49.905 INFO:tasks.rados.rados.0.smithi118.stdout:1281: expect (ObjNum 372 snap 127 seq_num 372) 2024-03-20T08:05:49.906 INFO:tasks.rados.rados.0.smithi118.stdout:1282: delete oid 19 current snap is 136 2024-03-20T08:05:49.934 INFO:tasks.rados.rados.0.smithi118.stdout:1281: expect (ObjNum 372 snap 127 seq_num 372) 2024-03-20T08:05:50.030 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[95692]: Upgrade: Updating osd.1 2024-03-20T08:05:50.031 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[95692]: Deploying daemon osd.1 on smithi118 2024-03-20T08:05:50.031 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[95692]: osdmap e322: 8 total, 8 up, 8 in 2024-03-20T08:05:50.031 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:50.031 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:50.031 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:49.924+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:50.031 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:49.924+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:50.031 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[93121]: Upgrade: Updating osd.1 2024-03-20T08:05:50.032 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[93121]: Deploying daemon osd.1 on smithi118 2024-03-20T08:05:50.032 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[93121]: osdmap e322: 8 total, 8 up, 8 in 2024-03-20T08:05:50.032 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:50.032 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:50.239 INFO:tasks.rados.rados.0.smithi118.stdout:1281: done (1 left) 2024-03-20T08:05:50.239 INFO:tasks.rados.rados.0.smithi118.stdout:1282: done (0 left) 2024-03-20T08:05:50.240 INFO:tasks.rados.rados.0.smithi118.stdout:1283: delete oid 11 current snap is 136 2024-03-20T08:05:50.242 INFO:tasks.rados.rados.0.smithi118.stdout:1283: done (0 left) 2024-03-20T08:05:50.242 INFO:tasks.rados.rados.0.smithi118.stdout:1284: read oid 3 snap -1 2024-03-20T08:05:50.242 INFO:tasks.rados.rados.0.smithi118.stdout:1284: expect (ObjNum 388 snap 132 seq_num 388) 2024-03-20T08:05:50.242 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 136 2024-03-20T08:05:50.242 INFO:tasks.rados.rados.0.smithi118.stdout:1285: seq_num 403 ranges {4595712=753664,5349376=425984,5775360=720896,6496256=57344} 2024-03-20T08:05:50.270 INFO:tasks.rados.rados.0.smithi118.stdout:1285: writing smithi11881747-23 from 4595712 to 5349376 tid 1 2024-03-20T08:05:50.273 INFO:tasks.rados.rados.0.smithi118.stdout:1285: writing smithi11881747-23 from 5349376 to 5775360 tid 2 2024-03-20T08:05:50.277 INFO:tasks.rados.rados.0.smithi118.stdout:1285: writing smithi11881747-23 from 5775360 to 6496256 tid 3 2024-03-20T08:05:50.279 INFO:tasks.rados.rados.0.smithi118.stdout:1285: writing smithi11881747-23 from 6496256 to 6553600 tid 4 2024-03-20T08:05:50.279 INFO:tasks.rados.rados.0.smithi118.stdout:1286: rmattr oid 20 current snap is 136 2024-03-20T08:05:50.279 INFO:tasks.rados.rados.0.smithi118.stdout:1286: done (2 left) 2024-03-20T08:05:50.279 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 136 2024-03-20T08:05:50.279 INFO:tasks.rados.rados.0.smithi118.stdout:1287: seq_num 404 ranges {3391488=770048,4161536=712704,4874240=491520} 2024-03-20T08:05:50.300 INFO:tasks.rados.rados.0.smithi118.stdout:1287: writing smithi11881747-41 from 3391488 to 4161536 tid 1 2024-03-20T08:05:50.305 INFO:tasks.rados.rados.0.smithi118.stdout:1287: writing smithi11881747-41 from 4161536 to 4874240 tid 2 2024-03-20T08:05:50.308 INFO:tasks.rados.rados.0.smithi118.stdout:1287: writing smithi11881747-41 from 4874240 to 5365760 tid 3 2024-03-20T08:05:50.308 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 136 2024-03-20T08:05:50.308 INFO:tasks.rados.rados.0.smithi118.stdout:1288: seq_num 405 ranges {0=786432,786432=589824,1376256=614400} 2024-03-20T08:05:50.313 INFO:tasks.rados.rados.0.smithi118.stdout:1288: writing smithi11881747-11 from 0 to 786432 tid 1 2024-03-20T08:05:50.317 INFO:tasks.rados.rados.0.smithi118.stdout:1288: writing smithi11881747-11 from 786432 to 1376256 tid 2 2024-03-20T08:05:50.321 INFO:tasks.rados.rados.0.smithi118.stdout:1288: writing smithi11881747-11 from 1376256 to 1990656 tid 3 2024-03-20T08:05:50.321 INFO:tasks.rados.rados.0.smithi118.stdout:1289: snap_remove snap 79 2024-03-20T08:05:50.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:49 smithi196 ceph-mon[87080]: Upgrade: Updating osd.1 2024-03-20T08:05:50.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:49 smithi196 ceph-mon[87080]: Deploying daemon osd.1 on smithi118 2024-03-20T08:05:50.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:49 smithi196 ceph-mon[87080]: osdmap e322: 8 total, 8 up, 8 in 2024-03-20T08:05:50.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:50.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:50.912 INFO:tasks.rados.rados.0.smithi118.stdout:1289: done (4 left) 2024-03-20T08:05:50.913 INFO:tasks.rados.rados.0.smithi118.stdout:1290: snap_remove snap 77 2024-03-20T08:05:51.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-mon[93121]: osdmap e323: 8 total, 8 up, 8 in 2024-03-20T08:05:51.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:51.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:51.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-mon[93121]: pgmap v161: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 330 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 21 op/s 2024-03-20T08:05:51.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-mon[95692]: osdmap e323: 8 total, 8 up, 8 in 2024-03-20T08:05:51.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:51.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:51.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-mon[95692]: pgmap v161: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 330 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 21 op/s 2024-03-20T08:05:51.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:50.914+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:51.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:50.914+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:51.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:50.915+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:50 smithi196 ceph-mon[87080]: osdmap e323: 8 total, 8 up, 8 in 2024-03-20T08:05:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:51.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:50 smithi196 ceph-mon[87080]: pgmap v161: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 330 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 21 op/s 2024-03-20T08:05:51.912 INFO:tasks.rados.rados.0.smithi118.stdout:1290: done (4 left) 2024-03-20T08:05:51.912 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 136 2024-03-20T08:05:51.912 INFO:tasks.rados.rados.0.smithi118.stdout:1291: seq_num 406 ranges {0=802816,802816=475136,1277952=499712,1777664=229376} 2024-03-20T08:05:51.917 INFO:tasks.rados.rados.0.smithi118.stdout:1291: writing smithi11881747-17 from 0 to 802816 tid 1 2024-03-20T08:05:51.920 INFO:tasks.rados.rados.0.smithi118.stdout:1291: writing smithi11881747-17 from 802816 to 1277952 tid 2 2024-03-20T08:05:51.923 INFO:tasks.rados.rados.0.smithi118.stdout:1291: writing smithi11881747-17 from 1277952 to 1777664 tid 3 2024-03-20T08:05:51.926 INFO:tasks.rados.rados.0.smithi118.stdout:1291: writing smithi11881747-17 from 1777664 to 2007040 tid 4 2024-03-20T08:05:51.926 INFO:tasks.rados.rados.0.smithi118.stdout:1292: read oid 31 snap 116 2024-03-20T08:05:51.926 INFO:tasks.rados.rados.0.smithi118.stdout:1292: expect (ObjNum 207 snap 67 seq_num 207) 2024-03-20T08:05:51.926 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 136 2024-03-20T08:05:51.926 INFO:tasks.rados.rados.0.smithi118.stdout:1284: expect (ObjNum 388 snap 132 seq_num 388) 2024-03-20T08:05:51.978 INFO:tasks.rados.rados.0.smithi118.stdout:1285: finishing write tid 1 to smithi11881747-23 2024-03-20T08:05:51.978 INFO:tasks.rados.rados.0.smithi118.stdout:1285: finishing write tid 2 to smithi11881747-23 2024-03-20T08:05:51.979 INFO:tasks.rados.rados.0.smithi118.stdout:1285: finishing write tid 3 to smithi11881747-23 2024-03-20T08:05:51.979 INFO:tasks.rados.rados.0.smithi118.stdout:1285: finishing write tid 4 to smithi11881747-23 2024-03-20T08:05:51.979 INFO:tasks.rados.rados.0.smithi118.stdout:1285: finishing write tid 5 to smithi11881747-23 2024-03-20T08:05:51.979 INFO:tasks.rados.rados.0.smithi118.stdout:1293: seq_num 407 ranges {0=425984,425984=761856,1187840=704512,1892352=139264} 2024-03-20T08:05:51.980 INFO:tasks.rados.rados.0.smithi118.stdout:1293: writing smithi11881747-19 from 0 to 425984 tid 1 2024-03-20T08:05:51.984 INFO:tasks.rados.rados.0.smithi118.stdout:1293: writing smithi11881747-19 from 425984 to 1187840 tid 2 2024-03-20T08:05:51.988 INFO:tasks.rados.rados.0.smithi118.stdout:1293: writing smithi11881747-19 from 1187840 to 1892352 tid 3 2024-03-20T08:05:51.990 INFO:tasks.rados.rados.0.smithi118.stdout:1293: writing smithi11881747-19 from 1892352 to 2031616 tid 4 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1284: done (6 left) 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1294: delete oid 6 current snap is 136 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1285: finishing write tid 6 to smithi11881747-23 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1285: oid 23 updating version 0 to 1053 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1285: oid 23 version 1053 is already newer than 1050 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1285: oid 23 version 1053 is already newer than 1052 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1285: oid 23 updating version 1053 to 1054 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1285: oid 23 version 1054 is already newer than 1051 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 1054 (ObjNum 403 snap 136 seq_num 403) dirty exists 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1285: left oid 23 (ObjNum 403 snap 136 seq_num 403) 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1287: finishing write tid 1 to smithi11881747-41 2024-03-20T08:05:51.991 INFO:tasks.rados.rados.0.smithi118.stdout:1287: finishing write tid 2 to smithi11881747-41 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1287: finishing write tid 3 to smithi11881747-41 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1287: finishing write tid 4 to smithi11881747-41 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1287: finishing write tid 5 to smithi11881747-41 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1287: oid 41 updating version 0 to 1056 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1287: oid 41 updating version 1056 to 1059 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1287: oid 41 version 1059 is already newer than 1058 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1287: oid 41 version 1059 is already newer than 1057 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1059 (ObjNum 404 snap 136 seq_num 404) dirty exists 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1287: left oid 41 (ObjNum 404 snap 136 seq_num 404) 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1288: finishing write tid 1 to smithi11881747-11 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1288: finishing write tid 2 to smithi11881747-11 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1288: finishing write tid 3 to smithi11881747-11 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1288: finishing write tid 4 to smithi11881747-11 2024-03-20T08:05:51.992 INFO:tasks.rados.rados.0.smithi118.stdout:1288: finishing write tid 5 to smithi11881747-11 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1288: oid 11 updating version 0 to 778 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1288: oid 11 updating version 778 to 779 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1288: oid 11 updating version 779 to 781 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1288: oid 11 version 781 is already newer than 780 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 781 (ObjNum 405 snap 136 seq_num 405) dirty exists 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1288: left oid 11 (ObjNum 405 snap 136 seq_num 405) 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: finishing write tid 1 to smithi11881747-17 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: finishing write tid 2 to smithi11881747-17 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: finishing write tid 3 to smithi11881747-17 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: finishing write tid 4 to smithi11881747-17 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: finishing write tid 5 to smithi11881747-17 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: finishing write tid 6 to smithi11881747-17 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: oid 17 updating version 0 to 192 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: oid 17 updating version 192 to 193 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: oid 17 updating version 193 to 194 2024-03-20T08:05:51.993 INFO:tasks.rados.rados.0.smithi118.stdout:1291: oid 17 updating version 194 to 195 2024-03-20T08:05:51.994 INFO:tasks.rados.rados.0.smithi118.stdout:1291: oid 17 updating version 195 to 196 2024-03-20T08:05:51.994 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 196 (ObjNum 406 snap 136 seq_num 406) dirty exists 2024-03-20T08:05:51.994 INFO:tasks.rados.rados.0.smithi118.stdout:1291: left oid 17 (ObjNum 406 snap 136 seq_num 406) 2024-03-20T08:05:51.994 INFO:tasks.rados.rados.0.smithi118.stdout:1292: expect (ObjNum 207 snap 67 seq_num 207) 2024-03-20T08:05:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:51 smithi118 ceph-mon[93121]: osdmap e324: 8 total, 8 up, 8 in 2024-03-20T08:05:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:51 smithi118 ceph-mon[95692]: osdmap e324: 8 total, 8 up, 8 in 2024-03-20T08:05:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:52.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:51.929+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:52.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:51.932+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:52.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:51 smithi118 systemd[1]: Stopping Ceph osd.1 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:05:52.312 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:51 smithi196 ceph-mon[87080]: osdmap e324: 8 total, 8 up, 8 in 2024-03-20T08:05:52.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:52.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: finishing write tid 1 to smithi11881747-19 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: finishing write tid 2 to smithi11881747-19 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: finishing write tid 3 to smithi11881747-19 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: finishing write tid 4 to smithi11881747-19 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: finishing write tid 5 to smithi11881747-19 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: finishing write tid 6 to smithi11881747-19 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: oid 19 updating version 0 to 262 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: oid 19 version 262 is already newer than 260 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: oid 19 version 262 is already newer than 259 2024-03-20T08:05:52.594 INFO:tasks.rados.rados.0.smithi118.stdout:1293: oid 19 updating version 262 to 263 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1293: oid 19 version 263 is already newer than 261 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 263 (ObjNum 407 snap 136 seq_num 407) dirty exists 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1293: left oid 19 (ObjNum 407 snap 136 seq_num 407) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1285: done (6 left) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1287: done (5 left) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1288: done (4 left) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1291: done (3 left) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1292: done (2 left) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1293: done (1 left) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1294: done (0 left) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1295: read oid 41 snap 89 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1295: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 136 2024-03-20T08:05:52.595 INFO:tasks.rados.rados.0.smithi118.stdout:1296: seq_num 409 ranges {5210112=458752,5668864=532480,6201344=720896,6922240=352256} 2024-03-20T08:05:52.624 INFO:tasks.rados.rados.0.smithi118.stdout:1296: writing smithi11881747-36 from 5210112 to 5668864 tid 1 2024-03-20T08:05:52.627 INFO:tasks.rados.rados.0.smithi118.stdout:1296: writing smithi11881747-36 from 5668864 to 6201344 tid 2 2024-03-20T08:05:52.631 INFO:tasks.rados.rados.0.smithi118.stdout:1296: writing smithi11881747-36 from 6201344 to 6922240 tid 3 2024-03-20T08:05:52.634 INFO:tasks.rados.rados.0.smithi118.stdout:1296: writing smithi11881747-36 from 6922240 to 7274496 tid 4 2024-03-20T08:05:52.634 INFO:tasks.rados.rados.0.smithi118.stdout:1297: rollback oid 4 current snap is 136 2024-03-20T08:05:52.634 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 4 to 127 2024-03-20T08:05:52.634 INFO:tasks.rados.rados.0.smithi118.stdout:1295: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:05:52.638 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:52.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[49360]: 2024-03-20T08:05:52.221+0000 7f48aeb31700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:05:52.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[49360]: 2024-03-20T08:05:52.221+0000 7f48aeb31700 -1 osd.1 325 *** Got signal Terminated *** 2024-03-20T08:05:52.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[49360]: 2024-03-20T08:05:52.221+0000 7f48aeb31700 -1 osd.1 325 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:05:52.745 INFO:tasks.rados.rados.0.smithi118.stdout:1296: finishing write tid 1 to smithi11881747-36 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1296: finishing write tid 2 to smithi11881747-36 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1295: done (2 left) 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1298: rmattr oid 44 current snap is 136 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1297: finishing rollback tid 0 to smithi11881747-4 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1297: finishing rollback tid 1 to smithi11881747-4 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1297: finishing rollback tid 2 to smithi11881747-4 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 293 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1296: finishing write tid 3 to smithi11881747-36 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1296: finishing write tid 4 to smithi11881747-36 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1296: finishing write tid 5 to smithi11881747-36 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1296: finishing write tid 6 to smithi11881747-36 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1296: oid 36 updating version 0 to 7017 2024-03-20T08:05:52.746 INFO:tasks.rados.rados.0.smithi118.stdout:1296: oid 36 version 7017 is already newer than 7014 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:1296: oid 36 version 7017 is already newer than 7015 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:1296: oid 36 version 7017 is already newer than 7013 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:1296: oid 36 version 7017 is already newer than 7016 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 7017 (ObjNum 409 snap 136 seq_num 409) dirty exists 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:1296: left oid 36 (ObjNum 409 snap 136 seq_num 409) 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:1296: done (2 left) 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:1297: done (1 left) 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:1299: setattr oid 26 current snap is 136 2024-03-20T08:05:52.747 INFO:tasks.rados.rados.0.smithi118.stdout:1300: snap_remove snap 105 2024-03-20T08:05:52.920 INFO:tasks.rados.rados.0.smithi118.stdout:1300: done (2 left) 2024-03-20T08:05:52.920 INFO:tasks.rados.rados.0.smithi118.stdout:1301: rmattr oid 50 current snap is 136 2024-03-20T08:05:52.920 INFO:tasks.rados.rados.0.smithi118.stdout:1301: done (2 left) 2024-03-20T08:05:52.920 INFO:tasks.rados.rados.0.smithi118.stdout:1302: snap_remove snap 116 2024-03-20T08:05:53.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[93121]: osdmap e325: 8 total, 8 up, 8 in 2024-03-20T08:05:53.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:53.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:53.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[93121]: pgmap v164: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 336 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 7.0 MiB/s wr, 27 op/s 2024-03-20T08:05:53.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[93121]: osd.1 marked itself down and dead 2024-03-20T08:05:53.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[95692]: osdmap e325: 8 total, 8 up, 8 in 2024-03-20T08:05:53.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:53.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:53.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[95692]: pgmap v164: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 336 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 7.0 MiB/s wr, 27 op/s 2024-03-20T08:05:53.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-mon[95692]: osd.1 marked itself down and dead 2024-03-20T08:05:53.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:52.934+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:53.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:52.934+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:53.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:52.944+0000 7f0ceec5d700 -1 osd.0 pg_epoch: 326 pg[5.2s1( v 326'628 (0'0,326'628] local-lis/les=300/301 n=21 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [6,0,5]p6(0) r=1 lpr=300 luod=0'0 crt=326'626 mlcod 326'626 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~2,a9~1,ab~1,af~1,b1~1,b5~1,b8~2]] update_snap_map remove_oid 5:43274c5b:::smithi11881747-21:a5 failed with -2 2024-03-20T08:05:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:52 smithi196 ceph-mon[87080]: osdmap e325: 8 total, 8 up, 8 in 2024-03-20T08:05:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset bb=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset ba=[]:{9b=[95,91,8b,8a,89],9f=[9f,9d],a5=[a5,a4,a3],a8=[a8,a6],ba=[ba,b9,b8,b7,b6,b4,b3,b2,b0,af,ac,ab]} for object 5:a551e079:::smithi11881747-5:6f 2024-03-20T08:05:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:52 smithi196 ceph-mon[87080]: pgmap v164: 121 pgs: 2 active+clean+snaptrim_error, 119 active+clean; 336 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 19 MiB/s rd, 7.0 MiB/s wr, 27 op/s 2024-03-20T08:05:53.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:52 smithi196 ceph-mon[87080]: osd.1 marked itself down and dead 2024-03-20T08:05:53.577 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:53 smithi118 podman[106073]: 2024-03-20 08:05:53.290062566 +0000 UTC m=+1.379326666 container died d99b1c22204af15f986b1a3b9845798a36a12d8b5d162407b546ff8483a02b66 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=reef-78dbc53, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-03-20T08:05:53.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:05:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:05:53.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:05:53.902 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:05:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:05:53] "GET /metrics HTTP/1.1" 200 36187 "" "Prometheus/2.43.0" 2024-03-20T08:05:53.902 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:53 smithi118 podman[106073]: 2024-03-20 08:05:53.677286893 +0000 UTC m=+1.766551001 container cleanup d99b1c22204af15f986b1a3b9845798a36a12d8b5d162407b546ff8483a02b66 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:05:53.902 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:53 smithi118 bash[106073]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1 2024-03-20T08:05:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 793 (ObjNum 393 snap 134 seq_num 393) dirty exists 2024-03-20T08:05:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 794 (ObjNum 577 snap 0 seq_num 322388480) dirty exists 2024-03-20T08:05:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:1298: done (2 left) 2024-03-20T08:05:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:1299: done (1 left) 2024-03-20T08:05:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:1302: done (0 left) 2024-03-20T08:05:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 136 2024-03-20T08:05:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:1303: seq_num 410 ranges {1990656=475136,2465792=417792,2883584=475136,3358720=425984,3784704=286720} 2024-03-20T08:05:53.941 INFO:tasks.rados.rados.0.smithi118.stdout:1303: writing smithi11881747-11 from 1990656 to 2465792 tid 1 2024-03-20T08:05:53.944 INFO:tasks.rados.rados.0.smithi118.stdout:1303: writing smithi11881747-11 from 2465792 to 2883584 tid 2 2024-03-20T08:05:53.947 INFO:tasks.rados.rados.0.smithi118.stdout:1303: writing smithi11881747-11 from 2883584 to 3358720 tid 3 2024-03-20T08:05:53.950 INFO:tasks.rados.rados.0.smithi118.stdout:1303: writing smithi11881747-11 from 3358720 to 3784704 tid 4 2024-03-20T08:05:53.952 INFO:tasks.rados.rados.0.smithi118.stdout:1303: writing smithi11881747-11 from 3784704 to 4071424 tid 5 2024-03-20T08:05:53.952 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 136 2024-03-20T08:05:53.952 INFO:tasks.rados.rados.0.smithi118.stdout:1303: finishing write tid 1 to smithi11881747-11 2024-03-20T08:05:53.952 INFO:tasks.rados.rados.0.smithi118.stdout:1303: finishing write tid 2 to smithi11881747-11 2024-03-20T08:05:53.953 INFO:tasks.rados.rados.0.smithi118.stdout:1304: seq_num 411 ranges {1712128=491520,2203648=753664,2957312=688128,3645440=163840} 2024-03-20T08:05:53.964 INFO:tasks.rados.rados.0.smithi118.stdout:1304: writing smithi11881747-3 from 1712128 to 2203648 tid 1 2024-03-20T08:05:53.968 INFO:tasks.rados.rados.0.smithi118.stdout:1304: writing smithi11881747-3 from 2203648 to 2957312 tid 2 2024-03-20T08:05:53.973 INFO:tasks.rados.rados.0.smithi118.stdout:1304: writing smithi11881747-3 from 2957312 to 3645440 tid 3 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1304: writing smithi11881747-3 from 3645440 to 3809280 tid 4 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1305: rollback oid 13 current snap is 136 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 13 to 102 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1303: finishing write tid 3 to smithi11881747-11 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1303: finishing write tid 4 to smithi11881747-11 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1303: finishing write tid 5 to smithi11881747-11 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1303: finishing write tid 6 to smithi11881747-11 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1303: finishing write tid 7 to smithi11881747-11 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1303: oid 11 updating version 0 to 786 2024-03-20T08:05:53.975 INFO:tasks.rados.rados.0.smithi118.stdout:1303: oid 11 version 786 is already newer than 785 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1303: oid 11 version 786 is already newer than 783 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1303: oid 11 updating version 786 to 787 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1303: oid 11 version 787 is already newer than 784 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1303: oid 11 version 787 is already newer than 782 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 787 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1303: left oid 11 (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1304: finishing write tid 1 to smithi11881747-3 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1304: finishing write tid 2 to smithi11881747-3 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1303: done (2 left) 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1306: rollback oid 25 current snap is 136 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 25 to 106 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1307: rmattr oid 46 current snap is 136 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1308: rmattr oid 18 current snap is 136 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1308: done (4 left) 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:1309: delete oid 12 current snap is 136 2024-03-20T08:05:53.976 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 0 (ObjNum 346 snap 112 seq_num 346) dirty exists 2024-03-20T08:05:53.977 INFO:tasks.rados.rados.0.smithi118.stdout:1307: done (4 left) 2024-03-20T08:05:53.977 INFO:tasks.rados.rados.0.smithi118.stdout:1309: done (3 left) 2024-03-20T08:05:53.977 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 136 2024-03-20T08:05:53.977 INFO:tasks.rados.rados.0.smithi118.stdout:1310: seq_num 413 ranges {2621440=524288,3145728=516096,3661824=696320,4358144=393216} 2024-03-20T08:05:53.994 INFO:tasks.rados.rados.0.smithi118.stdout:1310: writing smithi11881747-33 from 2621440 to 3145728 tid 1 2024-03-20T08:05:53.997 INFO:tasks.rados.rados.0.smithi118.stdout:1310: writing smithi11881747-33 from 3145728 to 3661824 tid 2 2024-03-20T08:05:54.001 INFO:tasks.rados.rados.0.smithi118.stdout:1310: writing smithi11881747-33 from 3661824 to 4358144 tid 3 2024-03-20T08:05:54.003 INFO:tasks.rados.rados.0.smithi118.stdout:1310: writing smithi11881747-33 from 4358144 to 4751360 tid 4 2024-03-20T08:05:54.003 INFO:tasks.rados.rados.0.smithi118.stdout:1311: snap_remove snap 132 2024-03-20T08:05:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:53.937+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:54.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:53 smithi118 ceph-mon[93121]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:05:54.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:53 smithi118 ceph-mon[93121]: osdmap e326: 8 total, 7 up, 8 in 2024-03-20T08:05:54.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:54.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:53 smithi118 ceph-mon[95692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:05:54.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:53 smithi118 ceph-mon[95692]: osdmap e326: 8 total, 7 up, 8 in 2024-03-20T08:05:54.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:54.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:53 smithi118 podman[106095]: 2024-03-20 08:05:53.903770552 +0000 UTC m=+0.618535723 container remove d99b1c22204af15f986b1a3b9845798a36a12d8b5d162407b546ff8483a02b66 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=reef-78dbc53, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-20T08:05:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:53 smithi196 ceph-mon[87080]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:05:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:53 smithi196 ceph-mon[87080]: osdmap e326: 8 total, 7 up, 8 in 2024-03-20T08:05:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:54.531 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:54 smithi118 podman[106191]: 2024-03-20 08:05:54.161508816 +0000 UTC m=+0.018910899 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:05:54.532 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:54 smithi118 podman[106191]: 2024-03-20 08:05:54.305295358 +0000 UTC m=+0.162697427 container create 68a94f99dfb3dec0c14c238f9cf539c45eb08e96ed4edd4e8b96b539a7a3cf76 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-20T08:05:54.532 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:54 smithi118 podman[106191]: 2024-03-20 08:05:54.533047383 +0000 UTC m=+0.390449461 container init 68a94f99dfb3dec0c14c238f9cf539c45eb08e96ed4edd4e8b96b539a7a3cf76 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-20T08:05:54.833 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:54 smithi118 podman[106191]: 2024-03-20 08:05:54.538830997 +0000 UTC m=+0.396233069 container start 68a94f99dfb3dec0c14c238f9cf539c45eb08e96ed4edd4e8b96b539a7a3cf76 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-20T08:05:54.833 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:54 smithi118 podman[106191]: 2024-03-20 08:05:54.624280789 +0000 UTC m=+0.481682860 container attach 68a94f99dfb3dec0c14c238f9cf539c45eb08e96ed4edd4e8b96b539a7a3cf76 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-20T08:05:54.931 INFO:tasks.rados.rados.0.smithi118.stdout:1311: done (4 left) 2024-03-20T08:05:54.931 INFO:tasks.rados.rados.0.smithi118.stdout:1312: read oid 31 snap -1 2024-03-20T08:05:54.931 INFO:tasks.rados.rados.0.smithi118.stdout:1312: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:05:54.931 INFO:tasks.rados.rados.0.smithi118.stdout:1304: finishing write tid 3 to smithi11881747-3 2024-03-20T08:05:54.931 INFO:tasks.rados.rados.0.smithi118.stdout:1313: read oid 14 snap -1 2024-03-20T08:05:54.931 INFO:tasks.rados.rados.0.smithi118.stdout:1313: expect (ObjNum 377 snap 128 seq_num 377) 2024-03-20T08:05:54.931 INFO:tasks.rados.rados.0.smithi118.stdout:1314: read oid 42 snap -1 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1314: expect deleted 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1315: read oid 27 snap 131 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1315: expect (ObjNum 366 snap 124 seq_num 366) 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1316: delete oid 47 current snap is 136 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: finishing write tid 4 to smithi11881747-3 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: finishing write tid 5 to smithi11881747-3 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: finishing write tid 6 to smithi11881747-3 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: oid 3 updating version 0 to 792 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: oid 3 updating version 792 to 794 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: oid 3 version 794 is already newer than 790 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: oid 3 version 794 is already newer than 791 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: oid 3 version 794 is already newer than 793 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 794 (ObjNum 411 snap 136 seq_num 411) dirty exists 2024-03-20T08:05:54.932 INFO:tasks.rados.rados.0.smithi118.stdout:1304: left oid 3 (ObjNum 411 snap 136 seq_num 411) 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1305: finishing rollback tid 0 to smithi11881747-13 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1305: finishing rollback tid 1 to smithi11881747-13 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1305: finishing rollback tid 2 to smithi11881747-13 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 791 (ObjNum 304 snap 100 seq_num 304) dirty exists 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1306: finishing rollback tid 0 to smithi11881747-25 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1306: finishing rollback tid 1 to smithi11881747-25 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1306: finishing rollback tid 2 to smithi11881747-25 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 795 (ObjNum 326 snap 106 seq_num 326) dirty exists 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1310: finishing write tid 1 to smithi11881747-33 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1310: finishing write tid 2 to smithi11881747-33 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1310: finishing write tid 3 to smithi11881747-33 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1310: finishing write tid 4 to smithi11881747-33 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1310: finishing write tid 5 to smithi11881747-33 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1310: finishing write tid 6 to smithi11881747-33 2024-03-20T08:05:54.933 INFO:tasks.rados.rados.0.smithi118.stdout:1310: oid 33 updating version 0 to 798 2024-03-20T08:05:54.934 INFO:tasks.rados.rados.0.smithi118.stdout:1310: oid 33 updating version 798 to 799 2024-03-20T08:05:54.934 INFO:tasks.rados.rados.0.smithi118.stdout:1310: oid 33 version 799 is already newer than 795 2024-03-20T08:05:54.934 INFO:tasks.rados.rados.0.smithi118.stdout:1310: oid 33 version 799 is already newer than 797 2024-03-20T08:05:54.934 INFO:tasks.rados.rados.0.smithi118.stdout:1310: oid 33 version 799 is already newer than 796 2024-03-20T08:05:54.934 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 799 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:05:54.934 INFO:tasks.rados.rados.0.smithi118.stdout:1310: left oid 33 (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:05:54.940 INFO:tasks.rados.rados.0.smithi118.stdout:1304: done (8 left) 2024-03-20T08:05:54.940 INFO:tasks.rados.rados.0.smithi118.stdout:1305: done (7 left) 2024-03-20T08:05:54.940 INFO:tasks.rados.rados.0.smithi118.stdout:1306: done (6 left) 2024-03-20T08:05:54.941 INFO:tasks.rados.rados.0.smithi118.stdout:1310: done (5 left) 2024-03-20T08:05:54.941 INFO:tasks.rados.rados.0.smithi118.stdout:1314: done (4 left) 2024-03-20T08:05:54.941 INFO:tasks.rados.rados.0.smithi118.stdout:1316: done (3 left) 2024-03-20T08:05:54.941 INFO:tasks.rados.rados.0.smithi118.stdout:1317: copy_from oid 38 from oid 22 current snap is 136 2024-03-20T08:05:54.941 INFO:tasks.rados.rados.0.smithi118.stdout:1318: snap_create 2024-03-20T08:05:54.951 INFO:tasks.rados.rados.0.smithi118.stdout:1312: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:05:54.991 INFO:tasks.rados.rados.0.smithi118.stdout:1317: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:05:54.991 INFO:tasks.rados.rados.0.smithi118.stdout:1317: finishing copy_from to smithi11881747-38 2024-03-20T08:05:54.991 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 776 (ObjNum 260 snap 90 seq_num 260) dirty exists 2024-03-20T08:05:54.991 INFO:tasks.rados.rados.0.smithi118.stdout:1315: expect (ObjNum 366 snap 124 seq_num 366) 2024-03-20T08:05:55.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:54 smithi118 ceph-mon[93121]: osdmap e327: 8 total, 7 up, 8 in 2024-03-20T08:05:55.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:55.126 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:54 smithi118 ceph-mon[93121]: pgmap v167: 121 pgs: 3 active+undersized+degraded, 12 stale+active+clean, 2 active+clean+snaptrim_error, 5 active+undersized, 99 active+clean; 332 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 12 op/s; 11/816 objects degraded (1.348%) 2024-03-20T08:05:55.127 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:54.936+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:55.127 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:54.936+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:55.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:54 smithi118 ceph-mon[95692]: osdmap e327: 8 total, 7 up, 8 in 2024-03-20T08:05:55.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:55.127 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:54 smithi118 ceph-mon[95692]: pgmap v167: 121 pgs: 3 active+undersized+degraded, 12 stale+active+clean, 2 active+clean+snaptrim_error, 5 active+undersized, 99 active+clean; 332 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 12 op/s; 11/816 objects degraded (1.348%) 2024-03-20T08:05:55.128 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:54 smithi118 podman[106191]: 2024-03-20 08:05:54.835317925 +0000 UTC m=+0.692720007 container died 68a94f99dfb3dec0c14c238f9cf539c45eb08e96ed4edd4e8b96b539a7a3cf76 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-20T08:05:55.192 INFO:tasks.rados.rados.0.smithi118.stdout:1313: expect (ObjNum 377 snap 128 seq_num 377) 2024-03-20T08:05:55.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:54 smithi196 ceph-mon[87080]: osdmap e327: 8 total, 7 up, 8 in 2024-03-20T08:05:55.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:55.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:54 smithi196 ceph-mon[87080]: pgmap v167: 121 pgs: 3 active+undersized+degraded, 12 stale+active+clean, 2 active+clean+snaptrim_error, 5 active+undersized, 99 active+clean; 332 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 12 op/s; 11/816 objects degraded (1.348%) 2024-03-20T08:05:55.887 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:55 smithi118 podman[106238]: 2024-03-20 08:05:55.444752917 +0000 UTC m=+0.604176542 container remove 68a94f99dfb3dec0c14c238f9cf539c45eb08e96ed4edd4e8b96b539a7a3cf76 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-20T08:05:55.887 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:55 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-20T08:05:55.887 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:55 smithi118 systemd[1]: Stopped Ceph osd.1 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:05:55.887 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:55 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.1.service: Consumed 10.941s CPU time. 2024-03-20T08:05:55.887 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:55 smithi118 systemd[1]: Starting Ceph osd.1 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:05:55.935 INFO:tasks.rados.rados.0.smithi118.stdout:1312: done (4 left) 2024-03-20T08:05:55.935 INFO:tasks.rados.rados.0.smithi118.stdout:1313: done (3 left) 2024-03-20T08:05:55.936 INFO:tasks.rados.rados.0.smithi118.stdout:1315: done (2 left) 2024-03-20T08:05:55.936 INFO:tasks.rados.rados.0.smithi118.stdout:1317: done (1 left) 2024-03-20T08:05:55.936 INFO:tasks.rados.rados.0.smithi118.stdout:1318: done (0 left) 2024-03-20T08:05:55.936 INFO:tasks.rados.rados.0.smithi118.stdout:1319: copy_from oid 48 from oid 36 current snap is 137 2024-03-20T08:05:55.936 INFO:tasks.rados.rados.0.smithi118.stdout:1320: read oid 13 snap -1 2024-03-20T08:05:55.936 INFO:tasks.rados.rados.0.smithi118.stdout:1320: expect (ObjNum 304 snap 100 seq_num 304) 2024-03-20T08:05:55.936 INFO:tasks.rados.rados.0.smithi118.stdout:1321: setattr oid 25 current snap is 137 2024-03-20T08:05:55.937 INFO:tasks.rados.rados.0.smithi118.stdout:1322: read oid 14 snap -1 2024-03-20T08:05:55.937 INFO:tasks.rados.rados.0.smithi118.stdout:1322: expect (ObjNum 377 snap 128 seq_num 377) 2024-03-20T08:05:55.937 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 137 2024-03-20T08:05:55.937 INFO:tasks.rados.rados.0.smithi118.stdout:1323: seq_num 415 ranges {0=557056,557056=688128,1245184=663552,1908736=253952} 2024-03-20T08:05:55.940 INFO:tasks.rados.rados.0.smithi118.stdout:1323: writing smithi11881747-26 from 0 to 557056 tid 1 2024-03-20T08:05:55.945 INFO:tasks.rados.rados.0.smithi118.stdout:1323: writing smithi11881747-26 from 557056 to 1245184 tid 2 2024-03-20T08:05:55.948 INFO:tasks.rados.rados.0.smithi118.stdout:1323: writing smithi11881747-26 from 1245184 to 1908736 tid 3 2024-03-20T08:05:55.950 INFO:tasks.rados.rados.0.smithi118.stdout:1323: writing smithi11881747-26 from 1908736 to 2162688 tid 4 2024-03-20T08:05:55.950 INFO:tasks.rados.rados.0.smithi118.stdout:1324: read oid 1 snap -1 2024-03-20T08:05:55.951 INFO:tasks.rados.rados.0.smithi118.stdout:1324: expect (ObjNum 391 snap 134 seq_num 391) 2024-03-20T08:05:55.951 INFO:tasks.rados.rados.0.smithi118.stdout:1325: setattr oid 42 current snap is 137 2024-03-20T08:05:55.951 INFO:tasks.rados.rados.0.smithi118.stdout:1320: expect (ObjNum 304 snap 100 seq_num 304) 2024-03-20T08:05:55.974 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 797 (ObjNum 326 snap 106 seq_num 326) dirty exists 2024-03-20T08:05:55.974 INFO:tasks.rados.rados.0.smithi118.stdout:1323: finishing write tid 1 to smithi11881747-26 2024-03-20T08:05:55.974 INFO:tasks.rados.rados.0.smithi118.stdout:1322: expect (ObjNum 377 snap 128 seq_num 377) 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1323: finishing write tid 2 to smithi11881747-26 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1323: finishing write tid 3 to smithi11881747-26 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1323: finishing write tid 4 to smithi11881747-26 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1320: done (6 left) 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1321: done (5 left) 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1322: done (4 left) 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1326: setattr oid 44 current snap is 137 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1323: finishing write tid 5 to smithi11881747-26 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1323: finishing write tid 6 to smithi11881747-26 2024-03-20T08:05:56.093 INFO:tasks.rados.rados.0.smithi118.stdout:1323: oid 26 updating version 0 to 798 2024-03-20T08:05:56.094 INFO:tasks.rados.rados.0.smithi118.stdout:1323: oid 26 updating version 798 to 799 2024-03-20T08:05:56.094 INFO:tasks.rados.rados.0.smithi118.stdout:1323: oid 26 version 799 is already newer than 797 2024-03-20T08:05:56.094 INFO:tasks.rados.rados.0.smithi118.stdout:1323: oid 26 version 799 is already newer than 795 2024-03-20T08:05:56.094 INFO:tasks.rados.rados.0.smithi118.stdout:1323: oid 26 version 799 is already newer than 796 2024-03-20T08:05:56.094 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 799 (ObjNum 415 snap 137 seq_num 415) dirty exists 2024-03-20T08:05:56.094 INFO:tasks.rados.rados.0.smithi118.stdout:1323: left oid 26 (ObjNum 415 snap 137 seq_num 415) 2024-03-20T08:05:56.094 INFO:tasks.rados.rados.0.smithi118.stdout:1319: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:05:56.094 INFO:tasks.rados.rados.0.smithi118.stdout:1324: expect (ObjNum 391 snap 134 seq_num 391) 2024-03-20T08:05:56.155 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:55.936+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:56.156 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:55 smithi118 ceph-mon[93121]: Health check failed: Degraded data redundancy: 11/816 objects degraded (1.348%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:56.156 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:55 smithi118 ceph-mon[93121]: osdmap e328: 8 total, 7 up, 8 in 2024-03-20T08:05:56.156 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:56.156 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:55 smithi118 podman[106347]: 2024-03-20 08:05:55.792109165 +0000 UTC m=+0.018312216 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:05:56.156 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:55 smithi118 podman[106347]: 2024-03-20 08:05:55.921114714 +0000 UTC m=+0.147317793 container create 51bc53468d778abbb7377b83c06685fd35149e72a1abbe1190c6c894eb4c5351 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-20T08:05:56.156 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:55 smithi118 ceph-mon[95692]: Health check failed: Degraded data redundancy: 11/816 objects degraded (1.348%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:56.156 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:55 smithi118 ceph-mon[95692]: osdmap e328: 8 total, 7 up, 8 in 2024-03-20T08:05:56.156 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:56.297 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 798 (ObjNum 49 snap 0 seq_num 1953066355) dirty exists 2024-03-20T08:05:56.297 INFO:tasks.rados.rados.0.smithi118.stdout:1319: finishing copy_from to smithi11881747-48 2024-03-20T08:05:56.298 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 7017 (ObjNum 409 snap 136 seq_num 409) dirty exists 2024-03-20T08:05:56.298 INFO:tasks.rados.rados.0.smithi118.stdout:1319: done (4 left) 2024-03-20T08:05:56.298 INFO:tasks.rados.rados.0.smithi118.stdout:1323: done (3 left) 2024-03-20T08:05:56.298 INFO:tasks.rados.rados.0.smithi118.stdout:1324: done (2 left) 2024-03-20T08:05:56.298 INFO:tasks.rados.rados.0.smithi118.stdout:1325: done (1 left) 2024-03-20T08:05:56.298 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 137 2024-03-20T08:05:56.298 INFO:tasks.rados.rados.0.smithi118.stdout:1327: seq_num 416 ranges {0=573440,573440=573440,1146880=475136,1622016=557056} 2024-03-20T08:05:56.300 INFO:tasks.rados.rados.0.smithi118.stdout:1327: writing smithi11881747-42 from 0 to 573440 tid 1 2024-03-20T08:05:56.303 INFO:tasks.rados.rados.0.smithi118.stdout:1327: writing smithi11881747-42 from 573440 to 1146880 tid 2 2024-03-20T08:05:56.306 INFO:tasks.rados.rados.0.smithi118.stdout:1327: writing smithi11881747-42 from 1146880 to 1622016 tid 3 2024-03-20T08:05:56.309 INFO:tasks.rados.rados.0.smithi118.stdout:1327: writing smithi11881747-42 from 1622016 to 2179072 tid 4 2024-03-20T08:05:56.309 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 137 2024-03-20T08:05:56.310 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 800 (ObjNum 393 snap 134 seq_num 393) dirty exists 2024-03-20T08:05:56.310 INFO:tasks.rados.rados.0.smithi118.stdout:1327: finishing write tid 1 to smithi11881747-42 2024-03-20T08:05:56.310 INFO:tasks.rados.rados.0.smithi118.stdout:1327: finishing write tid 2 to smithi11881747-42 2024-03-20T08:05:56.310 INFO:tasks.rados.rados.0.smithi118.stdout:1327: finishing write tid 3 to smithi11881747-42 2024-03-20T08:05:56.310 INFO:tasks.rados.rados.0.smithi118.stdout:1328: seq_num 417 ranges {5775360=589824,6365184=458752,6823936=679936,7503872=466944} 2024-03-20T08:05:56.344 INFO:tasks.rados.rados.0.smithi118.stdout:1328: writing smithi11881747-20 from 5775360 to 6365184 tid 1 2024-03-20T08:05:56.346 INFO:tasks.rados.rados.0.smithi118.stdout:1328: writing smithi11881747-20 from 6365184 to 6823936 tid 2 2024-03-20T08:05:56.351 INFO:tasks.rados.rados.0.smithi118.stdout:1328: writing smithi11881747-20 from 6823936 to 7503872 tid 3 2024-03-20T08:05:56.354 INFO:tasks.rados.rados.0.smithi118.stdout:1328: writing smithi11881747-20 from 7503872 to 7970816 tid 4 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1326: done (2 left) 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1329: rollback oid 30 current snap is 137 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 120 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: finishing write tid 4 to smithi11881747-42 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: finishing write tid 5 to smithi11881747-42 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: finishing write tid 6 to smithi11881747-42 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: oid 42 updating version 0 to 800 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: oid 42 updating version 800 to 801 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: oid 42 updating version 801 to 802 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: oid 42 version 802 is already newer than 799 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: oid 42 updating version 802 to 803 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 803 (ObjNum 416 snap 137 seq_num 416) dirty exists 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1327: left oid 42 (ObjNum 416 snap 137 seq_num 416) 2024-03-20T08:05:56.355 INFO:tasks.rados.rados.0.smithi118.stdout:1328: finishing write tid 1 to smithi11881747-20 2024-03-20T08:05:56.356 INFO:tasks.rados.rados.0.smithi118.stdout:1328: finishing write tid 2 to smithi11881747-20 2024-03-20T08:05:56.356 INFO:tasks.rados.rados.0.smithi118.stdout:1327: done (2 left) 2024-03-20T08:05:56.356 INFO:tasks.rados.rados.0.smithi118.stdout:1330: snap_remove snap 127 2024-03-20T08:05:56.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:55 smithi196 ceph-mon[87080]: Health check failed: Degraded data redundancy: 11/816 objects degraded (1.348%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:05:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:55 smithi196 ceph-mon[87080]: osdmap e328: 8 total, 7 up, 8 in 2024-03-20T08:05:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:56.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 podman[106347]: 2024-03-20 08:05:56.157215744 +0000 UTC m=+0.383418781 container init 51bc53468d778abbb7377b83c06685fd35149e72a1abbe1190c6c894eb4c5351 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:05:56.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 podman[106347]: 2024-03-20 08:05:56.162893442 +0000 UTC m=+0.389096479 container start 51bc53468d778abbb7377b83c06685fd35149e72a1abbe1190c6c894eb4c5351 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:05:56.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 podman[106347]: 2024-03-20 08:05:56.265669627 +0000 UTC m=+0.491872674 container attach 51bc53468d778abbb7377b83c06685fd35149e72a1abbe1190c6c894eb4c5351 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:05:56.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate[106363]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-20T08:05:56.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate[106363]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 bash[106347]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 bash[106347]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 bash[106347]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 bash[106347]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 bash[106347]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 bash[106347]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 bash[106347]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate[106363]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate[106363]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate[106363]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate[106363]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate[106363]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-20T08:05:56.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:56 smithi118 podman[106347]: 2024-03-20 08:05:56.631540145 +0000 UTC m=+0.857743184 container died 51bc53468d778abbb7377b83c06685fd35149e72a1abbe1190c6c894eb4c5351 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307) 2024-03-20T08:05:56.940 INFO:tasks.rados.rados.0.smithi118.stdout:1330: done (2 left) 2024-03-20T08:05:56.941 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 137 2024-03-20T08:05:56.941 INFO:tasks.rados.rados.0.smithi118.stdout:1328: finishing write tid 3 to smithi11881747-20 2024-03-20T08:05:56.941 INFO:tasks.rados.rados.0.smithi118.stdout:1331: seq_num 418 ranges {327680=606208,933888=737280,1671168=434176,2105344=434176} 2024-03-20T08:05:56.946 INFO:tasks.rados.rados.0.smithi118.stdout:1331: writing smithi11881747-13 from 327680 to 933888 tid 1 2024-03-20T08:05:56.950 INFO:tasks.rados.rados.0.smithi118.stdout:1331: writing smithi11881747-13 from 933888 to 1671168 tid 2 2024-03-20T08:05:56.953 INFO:tasks.rados.rados.0.smithi118.stdout:1331: writing smithi11881747-13 from 1671168 to 2105344 tid 3 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1331: writing smithi11881747-13 from 2105344 to 2539520 tid 4 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1332: snap_create 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1329: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1328: finishing write tid 4 to smithi11881747-20 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1329: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 295 (ObjNum 320 snap 104 seq_num 320) dirty exists 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1328: finishing write tid 5 to smithi11881747-20 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1328: finishing write tid 6 to smithi11881747-20 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1328: oid 20 updating version 0 to 7028 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1328: oid 20 version 7028 is already newer than 7027 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1328: oid 20 updating version 7028 to 7031 2024-03-20T08:05:56.956 INFO:tasks.rados.rados.0.smithi118.stdout:1328: oid 20 version 7031 is already newer than 7030 2024-03-20T08:05:56.957 INFO:tasks.rados.rados.0.smithi118.stdout:1328: oid 20 version 7031 is already newer than 7029 2024-03-20T08:05:56.957 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 7031 (ObjNum 417 snap 137 seq_num 417) dirty exists 2024-03-20T08:05:56.957 INFO:tasks.rados.rados.0.smithi118.stdout:1328: left oid 20 (ObjNum 417 snap 137 seq_num 417) 2024-03-20T08:05:56.957 INFO:tasks.rados.rados.0.smithi118.stdout:1331: finishing write tid 1 to smithi11881747-13 2024-03-20T08:05:56.957 INFO:tasks.rados.rados.0.smithi118.stdout:1331: finishing write tid 2 to smithi11881747-13 2024-03-20T08:05:56.957 INFO:tasks.rados.rados.0.smithi118.stdout:1331: finishing write tid 3 to smithi11881747-13 2024-03-20T08:05:56.958 INFO:tasks.rados.rados.0.smithi118.stdout:1331: finishing write tid 4 to smithi11881747-13 2024-03-20T08:05:56.958 INFO:tasks.rados.rados.0.smithi118.stdout:1331: finishing write tid 5 to smithi11881747-13 2024-03-20T08:05:56.960 INFO:tasks.rados.rados.0.smithi118.stdout:1331: finishing write tid 6 to smithi11881747-13 2024-03-20T08:05:56.960 INFO:tasks.rados.rados.0.smithi118.stdout:1331: oid 13 updating version 0 to 809 2024-03-20T08:05:56.960 INFO:tasks.rados.rados.0.smithi118.stdout:1331: oid 13 version 809 is already newer than 807 2024-03-20T08:05:56.960 INFO:tasks.rados.rados.0.smithi118.stdout:1331: oid 13 version 809 is already newer than 806 2024-03-20T08:05:56.960 INFO:tasks.rados.rados.0.smithi118.stdout:1331: oid 13 version 809 is already newer than 808 2024-03-20T08:05:56.960 INFO:tasks.rados.rados.0.smithi118.stdout:1331: oid 13 version 809 is already newer than 805 2024-03-20T08:05:56.960 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 809 (ObjNum 418 snap 137 seq_num 418) dirty exists 2024-03-20T08:05:56.960 INFO:tasks.rados.rados.0.smithi118.stdout:1331: left oid 13 (ObjNum 418 snap 137 seq_num 418) 2024-03-20T08:05:57.290 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:56 smithi196 ceph-mon[87080]: osdmap e329: 8 total, 7 up, 8 in 2024-03-20T08:05:57.290 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:57.290 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:56 smithi196 ceph-mon[87080]: pgmap v170: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 335 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 24 op/s; 123/777 objects degraded (15.830%) 2024-03-20T08:05:57.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-mon[93121]: osdmap e329: 8 total, 7 up, 8 in 2024-03-20T08:05:57.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:57.301 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-mon[93121]: pgmap v170: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 335 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 24 op/s; 123/777 objects degraded (15.830%) 2024-03-20T08:05:57.301 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:56.949+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:57.301 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:56.949+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:57.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-mon[95692]: osdmap e329: 8 total, 7 up, 8 in 2024-03-20T08:05:57.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:57.302 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:56 smithi118 ceph-mon[95692]: pgmap v170: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 335 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 24 op/s; 123/777 objects degraded (15.830%) 2024-03-20T08:05:57.594 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:57 smithi118 podman[106505]: 2024-03-20 08:05:57.302740653 +0000 UTC m=+0.662759957 container remove 51bc53468d778abbb7377b83c06685fd35149e72a1abbe1190c6c894eb4c5351 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-activate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:05:57.595 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:57 smithi118 podman[106574]: 2024-03-20 08:05:57.5037052 +0000 UTC m=+0.106881999 container create 3eae2fe0915aa02bdfb4c8617bf7ab488a2e8fb2911e5542967a421c871c3df7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:05:57.595 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:57 smithi118 podman[106574]: 2024-03-20 08:05:57.414282687 +0000 UTC m=+0.017459500 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:05:57.942 INFO:tasks.rados.rados.0.smithi118.stdout:1328: done (3 left) 2024-03-20T08:05:57.942 INFO:tasks.rados.rados.0.smithi118.stdout:1329: done (2 left) 2024-03-20T08:05:57.942 INFO:tasks.rados.rados.0.smithi118.stdout:1331: done (1 left) 2024-03-20T08:05:57.942 INFO:tasks.rados.rados.0.smithi118.stdout:1332: done (0 left) 2024-03-20T08:05:57.943 INFO:tasks.rados.rados.0.smithi118.stdout:1333: snap_create 2024-03-20T08:05:58.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:57 smithi118 ceph-mon[93121]: osdmap e330: 8 total, 7 up, 8 in 2024-03-20T08:05:58.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:58.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:57.984+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:57 smithi118 ceph-mon[95692]: osdmap e330: 8 total, 7 up, 8 in 2024-03-20T08:05:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:58.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:57 smithi118 podman[106574]: 2024-03-20 08:05:57.923769359 +0000 UTC m=+0.526946165 container init 3eae2fe0915aa02bdfb4c8617bf7ab488a2e8fb2911e5542967a421c871c3df7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-20T08:05:58.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:57 smithi118 podman[106574]: 2024-03-20 08:05:57.929505351 +0000 UTC m=+0.532682153 container start 3eae2fe0915aa02bdfb4c8617bf7ab488a2e8fb2911e5542967a421c871c3df7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-20T08:05:58.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:57 smithi118 ceph-osd[106612]: -- 172.21.15.118:0/3468439289 <== mon.2 v2:172.21.15.118:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8ce3dd680 con 0x55a8ce3a6c00 2024-03-20T08:05:58.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:58 smithi118 bash[106574]: 3eae2fe0915aa02bdfb4c8617bf7ab488a2e8fb2911e5542967a421c871c3df7 2024-03-20T08:05:58.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:58 smithi118 systemd[1]: Started Ceph osd.1 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:05:58.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:57 smithi196 ceph-mon[87080]: osdmap e330: 8 total, 7 up, 8 in 2024-03-20T08:05:58.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:58.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:05:58.472+0000 7fc2202117c0 -1 Falling back to public interface 2024-03-20T08:05:58.951 INFO:tasks.rados.rados.0.smithi118.stdout:1333: done (0 left) 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:1334: rollback oid 2 current snap is 139 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 136 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:1335: read oid 21 snap -1 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:1335: expect (ObjNum 335 snap 109 seq_num 335) 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:1336: rollback oid 16 current snap is 139 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 118 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:1337: read oid 19 snap -1 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:1337: expect (ObjNum 407 snap 136 seq_num 407) 2024-03-20T08:05:58.952 INFO:tasks.rados.rados.0.smithi118.stdout:1338: delete oid 38 current snap is 139 2024-03-20T08:05:58.958 INFO:tasks.rados.rados.0.smithi118.stdout:1336: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:05:58.959 INFO:tasks.rados.rados.0.smithi118.stdout:1336: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:05:58.959 INFO:tasks.rados.rados.0.smithi118.stdout:1336: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:05:58.959 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 1117 (ObjNum 315 snap 101 seq_num 315) dirty exists 2024-03-20T08:05:58.960 INFO:tasks.rados.rados.0.smithi118.stdout:1334: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:05:58.961 INFO:tasks.rados.rados.0.smithi118.stdout:1334: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:05:58.961 INFO:tasks.rados.rados.0.smithi118.stdout:1334: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:05:58.961 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 783 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:05:58.964 INFO:tasks.rados.rados.0.smithi118.stdout:1337: expect (ObjNum 407 snap 136 seq_num 407) 2024-03-20T08:05:59.019 INFO:tasks.rados.rados.0.smithi118.stdout:1335: expect (ObjNum 335 snap 109 seq_num 335) 2024-03-20T08:05:59.095 INFO:tasks.rados.rados.0.smithi118.stdout:1334: done (4 left) 2024-03-20T08:05:59.095 INFO:tasks.rados.rados.0.smithi118.stdout:1335: done (3 left) 2024-03-20T08:05:59.095 INFO:tasks.rados.rados.0.smithi118.stdout:1336: done (2 left) 2024-03-20T08:05:59.095 INFO:tasks.rados.rados.0.smithi118.stdout:1337: done (1 left) 2024-03-20T08:05:59.095 INFO:tasks.rados.rados.0.smithi118.stdout:1338: done (0 left) 2024-03-20T08:05:59.095 INFO:tasks.rados.rados.0.smithi118.stdout:1339: rollback oid 26 current snap is 139 2024-03-20T08:05:59.096 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 138 2024-03-20T08:05:59.096 INFO:tasks.rados.rados.0.smithi118.stdout:1340: read oid 44 snap 123 2024-03-20T08:05:59.096 INFO:tasks.rados.rados.0.smithi118.stdout:1340: expect deleted 2024-03-20T08:05:59.096 INFO:tasks.rados.rados.0.smithi118.stdout:1341: read oid 16 snap -1 2024-03-20T08:05:59.096 INFO:tasks.rados.rados.0.smithi118.stdout:1341: expect (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:05:59.096 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 139 2024-03-20T08:05:59.096 INFO:tasks.rados.rados.0.smithi118.stdout:1342: seq_num 420 ranges {5775360=638976,6414336=557056,6971392=450560,7421952=598016} 2024-03-20T08:05:59.128 INFO:tasks.rados.rados.0.smithi118.stdout:1342: writing smithi11881747-34 from 5775360 to 6414336 tid 1 2024-03-20T08:05:59.132 INFO:tasks.rados.rados.0.smithi118.stdout:1342: writing smithi11881747-34 from 6414336 to 6971392 tid 2 2024-03-20T08:05:59.135 INFO:tasks.rados.rados.0.smithi118.stdout:1342: writing smithi11881747-34 from 6971392 to 7421952 tid 3 2024-03-20T08:05:59.139 INFO:tasks.rados.rados.0.smithi118.stdout:1342: writing smithi11881747-34 from 7421952 to 8019968 tid 4 2024-03-20T08:05:59.139 INFO:tasks.rados.rados.0.smithi118.stdout:1343: copy_from oid 22 from oid 4 current snap is 139 2024-03-20T08:05:59.139 INFO:tasks.rados.rados.0.smithi118.stdout:1339: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:05:59.139 INFO:tasks.rados.rados.0.smithi118.stdout:1339: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:05:59.140 INFO:tasks.rados.rados.0.smithi118.stdout:1339: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:05:59.140 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 803 (ObjNum 415 snap 137 seq_num 415) dirty exists 2024-03-20T08:05:59.140 INFO:tasks.rados.rados.0.smithi118.stdout:1339: done (4 left) 2024-03-20T08:05:59.140 INFO:tasks.rados.rados.0.smithi118.stdout:1340: done (3 left) 2024-03-20T08:05:59.140 INFO:tasks.rados.rados.0.smithi118.stdout:1344: read oid 50 snap -1 2024-03-20T08:05:59.140 INFO:tasks.rados.rados.0.smithi118.stdout:1344: expect deleted 2024-03-20T08:05:59.140 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 139 2024-03-20T08:05:59.140 INFO:tasks.rados.rados.0.smithi118.stdout:1341: expect (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:05:59.210 INFO:tasks.rados.rados.0.smithi118.stdout:1342: finishing write tid 1 to smithi11881747-34 2024-03-20T08:05:59.210 INFO:tasks.rados.rados.0.smithi118.stdout:1345: seq_num 421 ranges {565248=655360,1220608=442368,1662976=663552,2326528=499712} 2024-03-20T08:05:59.216 INFO:tasks.rados.rados.0.smithi118.stdout:1345: writing smithi11881747-30 from 565248 to 1220608 tid 1 2024-03-20T08:05:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:1345: writing smithi11881747-30 from 1220608 to 1662976 tid 2 2024-03-20T08:05:59.222 INFO:tasks.rados.rados.0.smithi118.stdout:1345: writing smithi11881747-30 from 1662976 to 2326528 tid 3 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1345: writing smithi11881747-30 from 2326528 to 2826240 tid 4 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1341: done (4 left) 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1346: snap_create 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: finishing write tid 2 to smithi11881747-34 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: finishing write tid 3 to smithi11881747-34 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: finishing write tid 4 to smithi11881747-34 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: finishing write tid 5 to smithi11881747-34 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: finishing write tid 6 to smithi11881747-34 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: oid 34 updating version 0 to 7021 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: oid 34 updating version 7021 to 7022 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: oid 34 updating version 7022 to 7023 2024-03-20T08:05:59.226 INFO:tasks.rados.rados.0.smithi118.stdout:1342: oid 34 version 7023 is already newer than 7019 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:1342: oid 34 version 7023 is already newer than 7020 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 7023 (ObjNum 420 snap 139 seq_num 420) dirty exists 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:1342: left oid 34 (ObjNum 420 snap 139 seq_num 420) 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:1343: finishing copy_from racing read to smithi11881747-22 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:1343: finishing copy_from to smithi11881747-22 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 293 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:1345: finishing write tid 1 to smithi11881747-30 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:1345: finishing write tid 2 to smithi11881747-30 2024-03-20T08:05:59.227 INFO:tasks.rados.rados.0.smithi118.stdout:1345: finishing write tid 3 to smithi11881747-30 2024-03-20T08:05:59.228 INFO:tasks.rados.rados.0.smithi118.stdout:1345: finishing write tid 4 to smithi11881747-30 2024-03-20T08:05:59.228 INFO:tasks.rados.rados.0.smithi118.stdout:1345: finishing write tid 5 to smithi11881747-30 2024-03-20T08:05:59.228 INFO:tasks.rados.rados.0.smithi118.stdout:1345: finishing write tid 6 to smithi11881747-30 2024-03-20T08:05:59.228 INFO:tasks.rados.rados.0.smithi118.stdout:1345: oid 30 updating version 0 to 296 2024-03-20T08:05:59.228 INFO:tasks.rados.rados.0.smithi118.stdout:1345: oid 30 updating version 296 to 299 2024-03-20T08:05:59.228 INFO:tasks.rados.rados.0.smithi118.stdout:1345: oid 30 version 299 is already newer than 298 2024-03-20T08:05:59.229 INFO:tasks.rados.rados.0.smithi118.stdout:1345: oid 30 version 299 is already newer than 297 2024-03-20T08:05:59.229 INFO:tasks.rados.rados.0.smithi118.stdout:1345: oid 30 updating version 299 to 300 2024-03-20T08:05:59.229 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 300 (ObjNum 421 snap 139 seq_num 421) dirty exists 2024-03-20T08:05:59.229 INFO:tasks.rados.rados.0.smithi118.stdout:1345: left oid 30 (ObjNum 421 snap 139 seq_num 421) 2024-03-20T08:05:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:58 smithi196 ceph-mon[87080]: osdmap e331: 8 total, 7 up, 8 in 2024-03-20T08:05:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:58 smithi196 ceph-mon[87080]: pgmap v173: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 335 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 17 op/s; 123/777 objects degraded (15.830%) 2024-03-20T08:05:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:59.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:58.957+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:05:59.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:05:58.957+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[93121]: osdmap e331: 8 total, 7 up, 8 in 2024-03-20T08:05:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[93121]: pgmap v173: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 335 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 17 op/s; 123/777 objects degraded (15.830%) 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[95692]: osdmap e331: 8 total, 7 up, 8 in 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:05:59.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[95692]: pgmap v173: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 335 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 17 op/s; 123/777 objects degraded (15.830%) 2024-03-20T08:05:59.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:05:59.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:00.090 INFO:tasks.rados.rados.0.smithi118.stdout:1342: done (4 left) 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:1343: done (3 left) 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:1344: done (2 left) 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:1345: done (1 left) 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:1346: done (0 left) 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:1347: copy_from oid 49 from oid 24 current snap is 140 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:1348: rollback oid 15 current snap is 140 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 15 to 134 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 140 2024-03-20T08:06:00.091 INFO:tasks.rados.rados.0.smithi118.stdout:1349: seq_num 422 ranges {1966080=671744,2637824=729088,3366912=417792,3784704=458752} 2024-03-20T08:06:00.106 INFO:tasks.rados.rados.0.smithi118.stdout:1349: writing smithi11881747-35 from 1966080 to 2637824 tid 1 2024-03-20T08:06:00.110 INFO:tasks.rados.rados.0.smithi118.stdout:1349: writing smithi11881747-35 from 2637824 to 3366912 tid 2 2024-03-20T08:06:00.112 INFO:tasks.rados.rados.0.smithi118.stdout:1349: writing smithi11881747-35 from 3366912 to 3784704 tid 3 2024-03-20T08:06:00.115 INFO:tasks.rados.rados.0.smithi118.stdout:1349: writing smithi11881747-35 from 3784704 to 4243456 tid 4 2024-03-20T08:06:00.115 INFO:tasks.rados.rados.0.smithi118.stdout:1350: snap_create 2024-03-20T08:06:00.115 INFO:tasks.rados.rados.0.smithi118.stdout:1348: finishing rollback tid 0 to smithi11881747-15 2024-03-20T08:06:00.115 INFO:tasks.rados.rados.0.smithi118.stdout:1348: finishing rollback tid 1 to smithi11881747-15 2024-03-20T08:06:00.115 INFO:tasks.rados.rados.0.smithi118.stdout:1348: finishing rollback tid 2 to smithi11881747-15 2024-03-20T08:06:00.115 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 267 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:06:00.116 INFO:tasks.rados.rados.0.smithi118.stdout:1347: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:06:00.116 INFO:tasks.rados.rados.0.smithi118.stdout:1347: finishing copy_from to smithi11881747-49 2024-03-20T08:06:00.116 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 785 (ObjNum 216 snap 69 seq_num 216) dirty exists 2024-03-20T08:06:00.116 INFO:tasks.rados.rados.0.smithi118.stdout:1349: finishing write tid 1 to smithi11881747-35 2024-03-20T08:06:00.116 INFO:tasks.rados.rados.0.smithi118.stdout:1349: finishing write tid 2 to smithi11881747-35 2024-03-20T08:06:00.118 INFO:tasks.rados.rados.0.smithi118.stdout:1349: finishing write tid 3 to smithi11881747-35 2024-03-20T08:06:00.118 INFO:tasks.rados.rados.0.smithi118.stdout:1349: finishing write tid 4 to smithi11881747-35 2024-03-20T08:06:00.118 INFO:tasks.rados.rados.0.smithi118.stdout:1349: finishing write tid 5 to smithi11881747-35 2024-03-20T08:06:00.120 INFO:tasks.rados.rados.0.smithi118.stdout:1349: finishing write tid 6 to smithi11881747-35 2024-03-20T08:06:00.120 INFO:tasks.rados.rados.0.smithi118.stdout:1349: oid 35 updating version 0 to 785 2024-03-20T08:06:00.120 INFO:tasks.rados.rados.0.smithi118.stdout:1349: oid 35 version 785 is already newer than 784 2024-03-20T08:06:00.120 INFO:tasks.rados.rados.0.smithi118.stdout:1349: oid 35 updating version 785 to 787 2024-03-20T08:06:00.120 INFO:tasks.rados.rados.0.smithi118.stdout:1349: oid 35 updating version 787 to 788 2024-03-20T08:06:00.120 INFO:tasks.rados.rados.0.smithi118.stdout:1349: oid 35 version 788 is already newer than 786 2024-03-20T08:06:00.120 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 788 (ObjNum 422 snap 140 seq_num 422) dirty exists 2024-03-20T08:06:00.120 INFO:tasks.rados.rados.0.smithi118.stdout:1349: left oid 35 (ObjNum 422 snap 140 seq_num 422) 2024-03-20T08:06:00.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:59 smithi118 ceph-mon[93121]: osdmap e332: 8 total, 7 up, 8 in 2024-03-20T08:06:00.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:05:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:59 smithi118 ceph-mon[95692]: osdmap e332: 8 total, 7 up, 8 in 2024-03-20T08:06:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:05:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:00.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:00.095+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:59 smithi196 ceph-mon[87080]: osdmap e332: 8 total, 7 up, 8 in 2024-03-20T08:06:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:05:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:01.094 INFO:tasks.rados.rados.0.smithi118.stdout:1347: done (3 left) 2024-03-20T08:06:01.095 INFO:tasks.rados.rados.0.smithi118.stdout:1348: done (2 left) 2024-03-20T08:06:01.095 INFO:tasks.rados.rados.0.smithi118.stdout:1349: done (1 left) 2024-03-20T08:06:01.095 INFO:tasks.rados.rados.0.smithi118.stdout:1350: done (0 left) 2024-03-20T08:06:01.095 INFO:tasks.rados.rados.0.smithi118.stdout:1351: rollback oid 1 current snap is 141 2024-03-20T08:06:01.095 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 1 to 129 2024-03-20T08:06:01.095 INFO:tasks.rados.rados.0.smithi118.stdout:1352: rollback oid 36 current snap is 141 2024-03-20T08:06:01.095 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 36 to 97 2024-03-20T08:06:01.095 INFO:tasks.rados.rados.0.smithi118.stdout:1353: snap_remove snap 139 2024-03-20T08:06:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:01 smithi196 ceph-mon[87080]: osdmap e333: 8 total, 7 up, 8 in 2024-03-20T08:06:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:01 smithi196 ceph-mon[87080]: pgmap v176: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 353 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 16 MiB/s rd, 4.1 MiB/s wr, 19 op/s; 125/810 objects degraded (15.432%) 2024-03-20T08:06:01.411 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:01.111+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:01.411 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:01.111+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:01.412 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-mon[93121]: osdmap e333: 8 total, 7 up, 8 in 2024-03-20T08:06:01.412 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:01.412 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-mon[93121]: pgmap v176: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 353 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 16 MiB/s rd, 4.1 MiB/s wr, 19 op/s; 125/810 objects degraded (15.432%) 2024-03-20T08:06:01.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-mon[95692]: osdmap e333: 8 total, 7 up, 8 in 2024-03-20T08:06:01.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset c3=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:01.412 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-mon[95692]: pgmap v176: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 10 active+undersized+degraded, 1 active+clean+snaptrim_error, 34 active+undersized, 75 active+clean; 353 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 16 MiB/s rd, 4.1 MiB/s wr, 19 op/s; 125/810 objects degraded (15.432%) 2024-03-20T08:06:01.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:01.412+0000 7fc2202117c0 -1 osd.1 325 log_to_monitors true 2024-03-20T08:06:02.099 INFO:tasks.rados.rados.0.smithi118.stdout:1353: done (2 left) 2024-03-20T08:06:02.099 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 141 2024-03-20T08:06:02.099 INFO:tasks.rados.rados.0.smithi118.stdout:1351: finishing rollback tid 0 to smithi11881747-1 2024-03-20T08:06:02.099 INFO:tasks.rados.rados.0.smithi118.stdout:1354: seq_num 423 ranges {0=688128,688128=614400,1302528=622592,1925120=368640} 2024-03-20T08:06:02.103 INFO:tasks.rados.rados.0.smithi118.stdout:1354: writing smithi11881747-29 from 0 to 688128 tid 1 2024-03-20T08:06:02.107 INFO:tasks.rados.rados.0.smithi118.stdout:1354: writing smithi11881747-29 from 688128 to 1302528 tid 2 2024-03-20T08:06:02.110 INFO:tasks.rados.rados.0.smithi118.stdout:1354: writing smithi11881747-29 from 1302528 to 1925120 tid 3 2024-03-20T08:06:02.113 INFO:tasks.rados.rados.0.smithi118.stdout:1354: writing smithi11881747-29 from 1925120 to 2293760 tid 4 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:1355: read oid 16 snap 120 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:1355: expect (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:1356: delete oid 26 current snap is 141 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:1351: finishing rollback tid 1 to smithi11881747-1 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:1351: finishing rollback tid 2 to smithi11881747-1 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 990 (ObjNum 343 snap 112 seq_num 343) dirty exists 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:1352: finishing rollback tid 0 to smithi11881747-36 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:1352: finishing rollback tid 1 to smithi11881747-36 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:1352: finishing rollback tid 2 to smithi11881747-36 2024-03-20T08:06:02.114 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 7027 (ObjNum 289 snap 97 seq_num 289) dirty exists 2024-03-20T08:06:02.120 INFO:tasks.rados.rados.0.smithi118.stdout:1351: done (4 left) 2024-03-20T08:06:02.120 INFO:tasks.rados.rados.0.smithi118.stdout:1352: done (3 left) 2024-03-20T08:06:02.120 INFO:tasks.rados.rados.0.smithi118.stdout:1356: done (2 left) 2024-03-20T08:06:02.120 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 141 2024-03-20T08:06:02.120 INFO:tasks.rados.rados.0.smithi118.stdout:1357: seq_num 425 ranges {5365760=720896,6086656=778240,6864896=638976,7503872=188416} 2024-03-20T08:06:02.153 INFO:tasks.rados.rados.0.smithi118.stdout:1357: writing smithi11881747-41 from 5365760 to 6086656 tid 1 2024-03-20T08:06:02.158 INFO:tasks.rados.rados.0.smithi118.stdout:1357: writing smithi11881747-41 from 6086656 to 6864896 tid 2 2024-03-20T08:06:02.162 INFO:tasks.rados.rados.0.smithi118.stdout:1357: writing smithi11881747-41 from 6864896 to 7503872 tid 3 2024-03-20T08:06:02.163 INFO:tasks.rados.rados.0.smithi118.stdout:1357: writing smithi11881747-41 from 7503872 to 7692288 tid 4 2024-03-20T08:06:02.163 INFO:tasks.rados.rados.0.smithi118.stdout:1358: delete oid 4 current snap is 141 2024-03-20T08:06:02.163 INFO:tasks.rados.rados.0.smithi118.stdout:1354: finishing write tid 1 to smithi11881747-29 2024-03-20T08:06:02.164 INFO:tasks.rados.rados.0.smithi118.stdout:1354: finishing write tid 2 to smithi11881747-29 2024-03-20T08:06:02.164 INFO:tasks.rados.rados.0.smithi118.stdout:1354: finishing write tid 3 to smithi11881747-29 2024-03-20T08:06:02.164 INFO:tasks.rados.rados.0.smithi118.stdout:1354: finishing write tid 4 to smithi11881747-29 2024-03-20T08:06:02.164 INFO:tasks.rados.rados.0.smithi118.stdout:1354: finishing write tid 5 to smithi11881747-29 2024-03-20T08:06:02.164 INFO:tasks.rados.rados.0.smithi118.stdout:1355: expect (ObjNum 315 snap 101 seq_num 315) 2024-03-20T08:06:02.233 INFO:tasks.rados.rados.0.smithi118.stdout:1354: finishing write tid 6 to smithi11881747-29 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1354: oid 29 updating version 0 to 1119 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1354: oid 29 updating version 1119 to 1120 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1354: oid 29 updating version 1120 to 1122 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1354: oid 29 version 1122 is already newer than 1118 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1354: oid 29 version 1122 is already newer than 1121 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1122 (ObjNum 423 snap 141 seq_num 423) dirty exists 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1354: left oid 29 (ObjNum 423 snap 141 seq_num 423) 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1357: finishing write tid 1 to smithi11881747-41 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1357: finishing write tid 2 to smithi11881747-41 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1357: finishing write tid 3 to smithi11881747-41 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1354: done (3 left) 2024-03-20T08:06:02.234 INFO:tasks.rados.rados.0.smithi118.stdout:1355: done (2 left) 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1358: done (1 left) 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1359: rollback oid 42 current snap is 141 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 112 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: finishing write tid 4 to smithi11881747-41 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: finishing write tid 5 to smithi11881747-41 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: finishing write tid 6 to smithi11881747-41 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: oid 41 updating version 0 to 1124 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: oid 41 updating version 1124 to 1126 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: oid 41 updating version 1126 to 1127 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: oid 41 version 1127 is already newer than 1125 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: oid 41 updating version 1127 to 1128 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1128 (ObjNum 425 snap 141 seq_num 425) dirty exists 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: left oid 41 (ObjNum 425 snap 141 seq_num 425) 2024-03-20T08:06:02.235 INFO:tasks.rados.rados.0.smithi118.stdout:1357: done (1 left) 2024-03-20T08:06:02.236 INFO:tasks.rados.rados.0.smithi118.stdout:1360: read oid 22 snap -1 2024-03-20T08:06:02.236 INFO:tasks.rados.rados.0.smithi118.stdout:1360: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:06:02.236 INFO:tasks.rados.rados.0.smithi118.stdout:1361: snap_create 2024-03-20T08:06:02.238 INFO:tasks.rados.rados.0.smithi118.stdout:1359: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:06:02.242 INFO:tasks.rados.rados.0.smithi118.stdout:1359: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:06:02.242 INFO:tasks.rados.rados.0.smithi118.stdout:1359: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:06:02.242 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 813 (ObjNum 339 snap 110 seq_num 339) dirty exists 2024-03-20T08:06:02.252 INFO:tasks.rados.rados.0.smithi118.stdout:1360: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:06:02.381 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:02.125+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:02.381 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 125/810 objects degraded (15.432%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[93121]: osdmap e334: 8 total, 7 up, 8 in 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[93121]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 125/810 objects degraded (15.432%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[95692]: osdmap e334: 8 total, 7 up, 8 in 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[95692]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-20T08:06:02.382 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:02.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:02.388 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:02 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 125/810 objects degraded (15.432%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:02 smithi196 ceph-mon[87080]: osdmap e334: 8 total, 7 up, 8 in 2024-03-20T08:06:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:02 smithi196 ceph-mon[87080]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-20T08:06:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:02.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:02.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:02.382+0000 7fc20d661700 -1 osd.1 325 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T08:06:03.110 INFO:tasks.rados.rados.0.smithi118.stdout:1359: done (2 left) 2024-03-20T08:06:03.110 INFO:tasks.rados.rados.0.smithi118.stdout:1360: done (1 left) 2024-03-20T08:06:03.110 INFO:tasks.rados.rados.0.smithi118.stdout:1361: done (0 left) 2024-03-20T08:06:03.110 INFO:tasks.rados.rados.0.smithi118.stdout:1362: copy_from oid 48 from oid 8 current snap is 142 2024-03-20T08:06:03.110 INFO:tasks.rados.rados.0.smithi118.stdout:1363: read oid 41 snap -1 2024-03-20T08:06:03.110 INFO:tasks.rados.rados.0.smithi118.stdout:1363: expect (ObjNum 425 snap 141 seq_num 425) 2024-03-20T08:06:03.110 INFO:tasks.rados.rados.0.smithi118.stdout:1364: snap_create 2024-03-20T08:06:03.148 INFO:tasks.rados.rados.0.smithi118.stdout:1363: expect (ObjNum 425 snap 141 seq_num 425) 2024-03-20T08:06:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:03 smithi196 ceph-mon[87080]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-20T08:06:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:03 smithi196 ceph-mon[87080]: osdmap e335: 8 total, 7 up, 8 in 2024-03-20T08:06:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:03 smithi196 ceph-mon[87080]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:06:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:03 smithi196 ceph-mon[87080]: pgmap v179: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 34 active+undersized, 75 active+clean; 360 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 17 MiB/s rd, 5.6 MiB/s wr, 35 op/s; 127/819 objects degraded (15.507%) 2024-03-20T08:06:03.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:03 smithi196 ceph-mon[87080]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' 2024-03-20T08:06:03.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:03.111+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:03.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:03.111+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:03.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[93121]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-20T08:06:03.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[93121]: osdmap e335: 8 total, 7 up, 8 in 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[93121]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[93121]: pgmap v179: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 34 active+undersized, 75 active+clean; 360 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 17 MiB/s rd, 5.6 MiB/s wr, 35 op/s; 127/819 objects degraded (15.507%) 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[93121]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[95692]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[95692]: osdmap e335: 8 total, 7 up, 8 in 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[95692]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[95692]: pgmap v179: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 34 active+undersized, 75 active+clean; 360 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 17 MiB/s rd, 5.6 MiB/s wr, 35 op/s; 127/819 objects degraded (15.507%) 2024-03-20T08:06:03.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-mon[95692]: from='osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489]' entity='osd.1' 2024-03-20T08:06:03.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:03.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:03.905 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:06:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:06:03] "GET /metrics HTTP/1.1" 200 36260 "" "Prometheus/2.43.0" 2024-03-20T08:06:04.106 INFO:tasks.rados.rados.0.smithi118.stdout:1363: done (2 left) 2024-03-20T08:06:04.106 INFO:tasks.rados.rados.0.smithi118.stdout:1364: done (1 left) 2024-03-20T08:06:04.106 INFO:tasks.rados.rados.0.smithi118.stdout:1365: snap_create 2024-03-20T08:06:04.131 INFO:tasks.rados.rados.0.smithi118.stdout:1362: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:06:04.132 INFO:tasks.rados.rados.0.smithi118.stdout:1362: finishing copy_from to smithi11881747-48 2024-03-20T08:06:04.132 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 466 (ObjNum 65 snap 0 seq_num 0) dirty exists 2024-03-20T08:06:04.164 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:04.111+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:04.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[93121]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:06:04.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[93121]: osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489] boot 2024-03-20T08:06:04.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[93121]: osdmap e336: 8 total, 8 up, 8 in 2024-03-20T08:06:04.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:06:04.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:04.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:04.164 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:04.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[95692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:06:04.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[95692]: osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489] boot 2024-03-20T08:06:04.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[95692]: osdmap e336: 8 total, 8 up, 8 in 2024-03-20T08:06:04.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:06:04.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:04.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:04.165 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:04 smithi196 ceph-mon[87080]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:06:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:04 smithi196 ceph-mon[87080]: osd.1 [v2:172.21.15.118:6810/406990489,v1:172.21.15.118:6811/406990489] boot 2024-03-20T08:06:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:04 smithi196 ceph-mon[87080]: osdmap e336: 8 total, 8 up, 8 in 2024-03-20T08:06:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-20T08:06:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:04.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:04.288+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:05.110 INFO:tasks.rados.rados.0.smithi118.stdout:1362: done (1 left) 2024-03-20T08:06:05.110 INFO:tasks.rados.rados.0.smithi118.stdout:1365: done (0 left) 2024-03-20T08:06:05.110 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 144 2024-03-20T08:06:05.110 INFO:tasks.rados.rados.0.smithi118.stdout:1366: seq_num 427 ranges {1687552=761856,2449408=598016,3047424=606208,3653632=401408} 2024-03-20T08:06:05.121 INFO:tasks.rados.rados.0.smithi118.stdout:1366: writing smithi11881747-7 from 1687552 to 2449408 tid 1 2024-03-20T08:06:05.124 INFO:tasks.rados.rados.0.smithi118.stdout:1366: writing smithi11881747-7 from 2449408 to 3047424 tid 2 2024-03-20T08:06:05.128 INFO:tasks.rados.rados.0.smithi118.stdout:1366: writing smithi11881747-7 from 3047424 to 3653632 tid 3 2024-03-20T08:06:05.131 INFO:tasks.rados.rados.0.smithi118.stdout:1366: writing smithi11881747-7 from 3653632 to 4055040 tid 4 2024-03-20T08:06:05.132 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 144 2024-03-20T08:06:05.132 INFO:tasks.rados.rados.0.smithi118.stdout:1366: finishing write tid 1 to smithi11881747-7 2024-03-20T08:06:05.132 INFO:tasks.rados.rados.0.smithi118.stdout:1367: seq_num 428 ranges {1810432=778240,2588672=483328,3072000=409600,3481600=417792,3899392=294912} 2024-03-20T08:06:05.146 INFO:tasks.rados.rados.0.smithi118.stdout:1367: writing smithi11881747-45 from 1810432 to 2588672 tid 1 2024-03-20T08:06:05.149 INFO:tasks.rados.rados.0.smithi118.stdout:1367: writing smithi11881747-45 from 2588672 to 3072000 tid 2 2024-03-20T08:06:05.152 INFO:tasks.rados.rados.0.smithi118.stdout:1367: writing smithi11881747-45 from 3072000 to 3481600 tid 3 2024-03-20T08:06:05.155 INFO:tasks.rados.rados.0.smithi118.stdout:1367: writing smithi11881747-45 from 3481600 to 3899392 tid 4 2024-03-20T08:06:05.157 INFO:tasks.rados.rados.0.smithi118.stdout:1367: writing smithi11881747-45 from 3899392 to 4194304 tid 5 2024-03-20T08:06:05.157 INFO:tasks.rados.rados.0.smithi118.stdout:1368: copy_from oid 38 from oid 44 current snap is 144 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: finishing write tid 2 to smithi11881747-7 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: finishing write tid 3 to smithi11881747-7 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: finishing write tid 4 to smithi11881747-7 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: finishing write tid 5 to smithi11881747-7 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: finishing write tid 6 to smithi11881747-7 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: oid 7 updating version 0 to 270 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: oid 7 updating version 270 to 272 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: oid 7 updating version 272 to 274 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: oid 7 version 274 is already newer than 273 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: oid 7 version 274 is already newer than 271 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 274 (ObjNum 427 snap 144 seq_num 427) dirty exists 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1366: left oid 7 (ObjNum 427 snap 144 seq_num 427) 2024-03-20T08:06:05.158 INFO:tasks.rados.rados.0.smithi118.stdout:1367: finishing write tid 1 to smithi11881747-45 2024-03-20T08:06:05.159 INFO:tasks.rados.rados.0.smithi118.stdout:1367: finishing write tid 2 to smithi11881747-45 2024-03-20T08:06:05.159 INFO:tasks.rados.rados.0.smithi118.stdout:1367: finishing write tid 3 to smithi11881747-45 2024-03-20T08:06:05.159 INFO:tasks.rados.rados.0.smithi118.stdout:1366: done (2 left) 2024-03-20T08:06:05.159 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 144 2024-03-20T08:06:05.159 INFO:tasks.rados.rados.0.smithi118.stdout:1369: seq_num 429 ranges {2220032=794624,3014656=770048,3784704=622592,4407296=212992} 2024-03-20T08:06:05.174 INFO:tasks.rados.rados.0.smithi118.stdout:1369: writing smithi11881747-27 from 2220032 to 3014656 tid 1 2024-03-20T08:06:05.179 INFO:tasks.rados.rados.0.smithi118.stdout:1369: writing smithi11881747-27 from 3014656 to 3784704 tid 2 2024-03-20T08:06:05.183 INFO:tasks.rados.rados.0.smithi118.stdout:1369: writing smithi11881747-27 from 3784704 to 4407296 tid 3 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1369: writing smithi11881747-27 from 4407296 to 4620288 tid 4 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: finishing write tid 4 to smithi11881747-45 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: finishing write tid 5 to smithi11881747-45 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: finishing write tid 6 to smithi11881747-45 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: finishing write tid 7 to smithi11881747-45 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: oid 45 updating version 0 to 353 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: oid 45 version 353 is already newer than 350 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: oid 45 version 353 is already newer than 352 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: oid 45 version 353 is already newer than 349 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: oid 45 version 353 is already newer than 348 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: oid 45 version 353 is already newer than 351 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 353 (ObjNum 428 snap 144 seq_num 428) dirty exists 2024-03-20T08:06:05.186 INFO:tasks.rados.rados.0.smithi118.stdout:1367: left oid 45 (ObjNum 428 snap 144 seq_num 428) 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:1368: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:1368: finishing copy_from to smithi11881747-38 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 800 (ObjNum 393 snap 134 seq_num 393) dirty exists 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:1369: finishing write tid 1 to smithi11881747-27 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:1369: finishing write tid 2 to smithi11881747-27 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:1367: done (2 left) 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:1368: done (1 left) 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 144 2024-03-20T08:06:05.187 INFO:tasks.rados.rados.0.smithi118.stdout:1370: seq_num 430 ranges {1671168=8192} 2024-03-20T08:06:05.195 INFO:tasks.rados.rados.0.smithi118.stdout:1370: writing smithi11881747-16 from 1671168 to 1679360 tid 1 2024-03-20T08:06:05.195 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 144 2024-03-20T08:06:05.195 INFO:tasks.rados.rados.0.smithi118.stdout:1371: seq_num 431 ranges {3276800=24576} 2024-03-20T08:06:05.213 INFO:tasks.rados.rados.0.smithi118.stdout:1371: writing smithi11881747-14 from 3276800 to 3301376 tid 1 2024-03-20T08:06:05.213 INFO:tasks.rados.rados.0.smithi118.stdout:1372: snap_remove snap 126 2024-03-20T08:06:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:05 smithi196 ceph-mon[87080]: osdmap e337: 8 total, 8 up, 8 in 2024-03-20T08:06:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:05 smithi196 ceph-mon[87080]: pgmap v182: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 30 active+undersized, 79 active+clean; 362 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 19 op/s; 127/831 objects degraded (15.283%) 2024-03-20T08:06:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:05 smithi196 ceph-mon[87080]: osdmap e338: 8 total, 8 up, 8 in 2024-03-20T08:06:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[93121]: osdmap e337: 8 total, 8 up, 8 in 2024-03-20T08:06:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[93121]: pgmap v182: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 30 active+undersized, 79 active+clean; 362 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 19 op/s; 127/831 objects degraded (15.283%) 2024-03-20T08:06:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:05.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[93121]: osdmap e338: 8 total, 8 up, 8 in 2024-03-20T08:06:05.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:05.152+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:05.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:05.152+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:05.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:05.154+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:05.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[95692]: osdmap e337: 8 total, 8 up, 8 in 2024-03-20T08:06:05.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:05.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[95692]: pgmap v182: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 30 active+undersized, 79 active+clean; 362 MiB data, 1.7 GiB used, 713 GiB / 715 GiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 19 op/s; 127/831 objects degraded (15.283%) 2024-03-20T08:06:05.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:05.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:05 smithi118 ceph-mon[95692]: osdmap e338: 8 total, 8 up, 8 in 2024-03-20T08:06:06.129 INFO:tasks.rados.rados.0.smithi118.stdout:1372: done (3 left) 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1373: snap_create 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: finishing write tid 3 to smithi11881747-27 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: finishing write tid 4 to smithi11881747-27 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: finishing write tid 5 to smithi11881747-27 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: finishing write tid 6 to smithi11881747-27 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: oid 27 updating version 0 to 7036 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: oid 27 version 7036 is already newer than 7035 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: oid 27 version 7036 is already newer than 7032 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: oid 27 version 7036 is already newer than 7033 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: oid 27 version 7036 is already newer than 7034 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7036 (ObjNum 429 snap 144 seq_num 429) dirty exists 2024-03-20T08:06:06.130 INFO:tasks.rados.rados.0.smithi118.stdout:1369: left oid 27 (ObjNum 429 snap 144 seq_num 429) 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1370: finishing write tid 1 to smithi11881747-16 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1370: finishing write tid 2 to smithi11881747-16 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1370: finishing write tid 3 to smithi11881747-16 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1370: oid 16 updating version 0 to 1130 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1370: oid 16 updating version 1130 to 1131 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 1131 (ObjNum 430 snap 144 seq_num 430) dirty exists 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1370: left oid 16 (ObjNum 430 snap 144 seq_num 430) 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1371: finishing write tid 1 to smithi11881747-14 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1371: finishing write tid 2 to smithi11881747-14 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1371: finishing write tid 3 to smithi11881747-14 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1371: oid 14 updating version 0 to 2013 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1371: oid 14 updating version 2013 to 2014 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 2014 (ObjNum 431 snap 144 seq_num 431) dirty exists 2024-03-20T08:06:06.131 INFO:tasks.rados.rados.0.smithi118.stdout:1371: left oid 14 (ObjNum 431 snap 144 seq_num 431) 2024-03-20T08:06:06.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:06.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:06.417 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:06.417 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:06.417 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:06.417 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:06.417 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:06.131+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:06.417 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:06.131+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:07.134 INFO:tasks.rados.rados.0.smithi118.stdout:1369: done (3 left) 2024-03-20T08:06:07.134 INFO:tasks.rados.rados.0.smithi118.stdout:1370: done (2 left) 2024-03-20T08:06:07.135 INFO:tasks.rados.rados.0.smithi118.stdout:1371: done (1 left) 2024-03-20T08:06:07.135 INFO:tasks.rados.rados.0.smithi118.stdout:1373: done (0 left) 2024-03-20T08:06:07.135 INFO:tasks.rados.rados.0.smithi118.stdout:1374: copy_from oid 34 from oid 38 current snap is 145 2024-03-20T08:06:07.135 INFO:tasks.rados.rados.0.smithi118.stdout:1375: rollback oid 46 current snap is 145 2024-03-20T08:06:07.135 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 128 2024-03-20T08:06:07.135 INFO:tasks.rados.rados.0.smithi118.stdout:1376: rollback oid 23 current snap is 145 2024-03-20T08:06:07.135 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 23 to 97 2024-03-20T08:06:07.135 INFO:tasks.rados.rados.0.smithi118.stdout:1377: delete oid 22 current snap is 145 2024-03-20T08:06:07.142 INFO:tasks.rados.rados.0.smithi118.stdout:1375: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:06:07.142 INFO:tasks.rados.rados.0.smithi118.stdout:1375: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:06:07.143 INFO:tasks.rados.rados.0.smithi118.stdout:1375: finishing rollback tid 2 to smithi11881747-46 2024-03-20T08:06:07.143 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 791 (ObjNum 346 snap 112 seq_num 346) dirty exists 2024-03-20T08:06:07.144 INFO:tasks.rados.rados.0.smithi118.stdout:1375: done (3 left) 2024-03-20T08:06:07.144 INFO:tasks.rados.rados.0.smithi118.stdout:1377: done (2 left) 2024-03-20T08:06:07.144 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 145 2024-03-20T08:06:07.144 INFO:tasks.rados.rados.0.smithi118.stdout:1378: seq_num 433 ranges {0=57344} 2024-03-20T08:06:07.144 INFO:tasks.rados.rados.0.smithi118.stdout:1378: writing smithi11881747-48 from 0 to 57344 tid 1 2024-03-20T08:06:07.144 INFO:tasks.rados.rados.0.smithi118.stdout:1379: copy_from oid 13 from oid 1 current snap is 145 2024-03-20T08:06:07.144 INFO:tasks.rados.rados.0.smithi118.stdout:1380: snap_remove snap 112 2024-03-20T08:06:07.371 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-mon[93121]: osdmap e339: 8 total, 8 up, 8 in 2024-03-20T08:06:07.372 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:07.372 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:07.372 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-mon[93121]: pgmap v185: 121 pgs: 2 active+clean+snaptrim_error, 4 active+undersized+degraded, 3 active+recovery_wait+degraded, 12 active+undersized, 100 active+clean; 372 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.5 MiB/s wr, 12 op/s; 23/852 objects degraded (2.700%); 8.6 MiB/s, 3 objects/s recovering 2024-03-20T08:06:07.372 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:07.137+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:07.372 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:07.137+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:07.372 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:07.140+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:07.372 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-mon[95692]: osdmap e339: 8 total, 8 up, 8 in 2024-03-20T08:06:07.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:07.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:07.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-mon[95692]: pgmap v185: 121 pgs: 2 active+clean+snaptrim_error, 4 active+undersized+degraded, 3 active+recovery_wait+degraded, 12 active+undersized, 100 active+clean; 372 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.5 MiB/s wr, 12 op/s; 23/852 objects degraded (2.700%); 8.6 MiB/s, 3 objects/s recovering 2024-03-20T08:06:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:07 smithi196 ceph-mon[87080]: osdmap e339: 8 total, 8 up, 8 in 2024-03-20T08:06:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:07.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:07 smithi196 ceph-mon[87080]: pgmap v185: 121 pgs: 2 active+clean+snaptrim_error, 4 active+undersized+degraded, 3 active+recovery_wait+degraded, 12 active+undersized, 100 active+clean; 372 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.5 MiB/s wr, 12 op/s; 23/852 objects degraded (2.700%); 8.6 MiB/s, 3 objects/s recovering 2024-03-20T08:06:07.645 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:06:07.646 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:06:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:06:07.370+0000 7ff9af1ba700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (4 PGs are or would become offline) 2024-03-20T08:06:08.099 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:06:08.099 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (5m) 6s ago 6m 26.8M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:06:08.099 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (6m) 88s ago 6m 86.9M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:06:08.099 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (7m) 6s ago 7m 15.0M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:06:08.099 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (7m) 88s ago 7m 19.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:06:08.099 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (2m) 88s ago 11m 449M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:06:08.099 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (3m) 6s ago 13m 504M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (2m) 6s ago 13m 49.4M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (94s) 88s ago 11m 26.6M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (114s) 6s ago 11m 43.5M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (6m) 6s ago 6m 14.4M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (6m) 88s ago 6m 15.8M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (46s) 6s ago 11m 168M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (10s) 6s ago 10m 14.9M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (10m) 6s ago 10m 160M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 209395cbab90 2024-03-20T08:06:08.100 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (9m) 6s ago 9m 219M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:06:08.101 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (9m) 88s ago 9m 181M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:06:08.101 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (8m) 88s ago 8m 133M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:06:08.101 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (8m) 88s ago 8m 117M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:06:08.101 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (8m) 88s ago 8m 115M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:06:08.101 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (2m) 88s ago 7m 56.2M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:06:08.140 INFO:tasks.rados.rados.0.smithi118.stdout:1376: finishing rollback tid 0 to smithi11881747-23 2024-03-20T08:06:08.140 INFO:tasks.rados.rados.0.smithi118.stdout:1376: finishing rollback tid 1 to smithi11881747-23 2024-03-20T08:06:08.140 INFO:tasks.rados.rados.0.smithi118.stdout:1376: finishing rollback tid 2 to smithi11881747-23 2024-03-20T08:06:08.140 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 1145 (ObjNum 230 snap 79 seq_num 230) dirty exists 2024-03-20T08:06:08.140 INFO:tasks.rados.rados.0.smithi118.stdout:1378: finishing write tid 1 to smithi11881747-48 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1378: finishing write tid 2 to smithi11881747-48 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1378: finishing write tid 3 to smithi11881747-48 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1378: oid 48 updating version 0 to 7021 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1378: oid 48 version 7021 is already newer than 7020 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 7021 (ObjNum 433 snap 145 seq_num 433) dirty exists 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1378: left oid 48 (ObjNum 433 snap 145 seq_num 433) 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1374: finishing copy_from racing read to smithi11881747-34 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1374: finishing copy_from to smithi11881747-34 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 800 (ObjNum 393 snap 134 seq_num 393) dirty exists 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1379: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1379: finishing copy_from to smithi11881747-13 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 990 (ObjNum 343 snap 112 seq_num 343) dirty exists 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1374: done (4 left) 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1376: done (3 left) 2024-03-20T08:06:08.141 INFO:tasks.rados.rados.0.smithi118.stdout:1378: done (2 left) 2024-03-20T08:06:08.142 INFO:tasks.rados.rados.0.smithi118.stdout:1379: done (1 left) 2024-03-20T08:06:08.142 INFO:tasks.rados.rados.0.smithi118.stdout:1380: done (0 left) 2024-03-20T08:06:08.142 INFO:tasks.rados.rados.0.smithi118.stdout:1381: copy_from oid 32 from oid 37 current snap is 145 2024-03-20T08:06:08.142 INFO:tasks.rados.rados.0.smithi118.stdout:1382: copy_from oid 7 from oid 48 current snap is 145 2024-03-20T08:06:08.142 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 145 2024-03-20T08:06:08.142 INFO:tasks.rados.rados.0.smithi118.stdout:1383: seq_num 434 ranges {4620288=73728} 2024-03-20T08:06:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:1383: writing smithi11881747-27 from 4620288 to 4694016 tid 1 2024-03-20T08:06:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 145 2024-03-20T08:06:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:1382: finishing copy_from racing read to smithi11881747-7 2024-03-20T08:06:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:1384: seq_num 435 ranges {1474560=90112} 2024-03-20T08:06:08.176 INFO:tasks.rados.rados.0.smithi118.stdout:1384: writing smithi11881747-31 from 1474560 to 1564672 tid 1 2024-03-20T08:06:08.176 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 145 2024-03-20T08:06:08.176 INFO:tasks.rados.rados.0.smithi118.stdout:1382: finishing copy_from to smithi11881747-7 2024-03-20T08:06:08.177 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 7021 (ObjNum 433 snap 145 seq_num 433) dirty exists 2024-03-20T08:06:08.177 INFO:tasks.rados.rados.0.smithi118.stdout:1381: finishing copy_from to smithi11881747-32 2024-03-20T08:06:08.177 INFO:tasks.rados.rados.0.smithi118.stdout:1381: got expected ENOENT (src dne) 2024-03-20T08:06:08.177 INFO:tasks.rados.rados.0.smithi118.stdout:1381: finishing copy_from racing read to smithi11881747-32 2024-03-20T08:06:08.177 INFO:tasks.rados.rados.0.smithi118.stdout:1383: finishing write tid 1 to smithi11881747-27 2024-03-20T08:06:08.177 INFO:tasks.rados.rados.0.smithi118.stdout:1383: finishing write tid 2 to smithi11881747-27 2024-03-20T08:06:08.177 INFO:tasks.rados.rados.0.smithi118.stdout:1385: seq_num 436 ranges {2826240=106496} 2024-03-20T08:06:08.193 INFO:tasks.rados.rados.0.smithi118.stdout:1385: writing smithi11881747-30 from 2826240 to 2932736 tid 1 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1381: done (4 left) 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1382: done (3 left) 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1386: rmattr oid 36 current snap is 145 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1383: finishing write tid 3 to smithi11881747-27 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1383: oid 27 updating version 0 to 7037 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1383: oid 27 updating version 7037 to 7038 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7038 (ObjNum 434 snap 145 seq_num 434) dirty exists 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1383: left oid 27 (ObjNum 434 snap 145 seq_num 434) 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1384: finishing write tid 1 to smithi11881747-31 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1384: finishing write tid 2 to smithi11881747-31 2024-03-20T08:06:08.194 INFO:tasks.rados.rados.0.smithi118.stdout:1383: done (3 left) 2024-03-20T08:06:08.195 INFO:tasks.rados.rados.0.smithi118.stdout:1387: delete oid 1 current snap is 145 2024-03-20T08:06:08.195 INFO:tasks.rados.rados.0.smithi118.stdout:1384: finishing write tid 3 to smithi11881747-31 2024-03-20T08:06:08.195 INFO:tasks.rados.rados.0.smithi118.stdout:1384: oid 31 updating version 0 to 991 2024-03-20T08:06:08.195 INFO:tasks.rados.rados.0.smithi118.stdout:1384: oid 31 updating version 991 to 992 2024-03-20T08:06:08.195 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 992 (ObjNum 435 snap 145 seq_num 435) dirty exists 2024-03-20T08:06:08.195 INFO:tasks.rados.rados.0.smithi118.stdout:1384: left oid 31 (ObjNum 435 snap 145 seq_num 435) 2024-03-20T08:06:08.196 INFO:tasks.rados.rados.0.smithi118.stdout:1384: done (3 left) 2024-03-20T08:06:08.196 INFO:tasks.rados.rados.0.smithi118.stdout:1387: done (2 left) 2024-03-20T08:06:08.196 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 145 2024-03-20T08:06:08.196 INFO:tasks.rados.rados.0.smithi118.stdout:1388: seq_num 438 ranges {1392640=139264} 2024-03-20T08:06:08.205 INFO:tasks.rados.rados.0.smithi118.stdout:1388: writing smithi11881747-28 from 1392640 to 1531904 tid 1 2024-03-20T08:06:08.205 INFO:tasks.rados.rados.0.smithi118.stdout:1389: delete oid 34 current snap is 145 2024-03-20T08:06:08.205 INFO:tasks.rados.rados.0.smithi118.stdout:1385: finishing write tid 1 to smithi11881747-30 2024-03-20T08:06:08.205 INFO:tasks.rados.rados.0.smithi118.stdout:1385: finishing write tid 2 to smithi11881747-30 2024-03-20T08:06:08.205 INFO:tasks.rados.rados.0.smithi118.stdout:1385: finishing write tid 3 to smithi11881747-30 2024-03-20T08:06:08.205 INFO:tasks.rados.rados.0.smithi118.stdout:1385: oid 30 updating version 0 to 302 2024-03-20T08:06:08.206 INFO:tasks.rados.rados.0.smithi118.stdout:1385: oid 30 updating version 302 to 303 2024-03-20T08:06:08.206 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 303 (ObjNum 436 snap 145 seq_num 436) dirty exists 2024-03-20T08:06:08.206 INFO:tasks.rados.rados.0.smithi118.stdout:1385: left oid 30 (ObjNum 436 snap 145 seq_num 436) 2024-03-20T08:06:08.206 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 7031 (ObjNum 289 snap 97 seq_num 289) dirty exists 2024-03-20T08:06:08.210 INFO:tasks.rados.rados.0.smithi118.stdout:1388: finishing write tid 1 to smithi11881747-28 2024-03-20T08:06:08.211 INFO:tasks.rados.rados.0.smithi118.stdout:1388: finishing write tid 2 to smithi11881747-28 2024-03-20T08:06:08.212 INFO:tasks.rados.rados.0.smithi118.stdout:1385: done (3 left) 2024-03-20T08:06:08.213 INFO:tasks.rados.rados.0.smithi118.stdout:1386: done (2 left) 2024-03-20T08:06:08.213 INFO:tasks.rados.rados.0.smithi118.stdout:1389: done (1 left) 2024-03-20T08:06:08.213 INFO:tasks.rados.rados.0.smithi118.stdout:1390: setattr oid 17 current snap is 145 2024-03-20T08:06:08.213 INFO:tasks.rados.rados.0.smithi118.stdout:1388: finishing write tid 3 to smithi11881747-28 2024-03-20T08:06:08.213 INFO:tasks.rados.rados.0.smithi118.stdout:1388: oid 28 updating version 0 to 1155 2024-03-20T08:06:08.213 INFO:tasks.rados.rados.0.smithi118.stdout:1388: oid 28 updating version 1155 to 1156 2024-03-20T08:06:08.213 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 1156 (ObjNum 438 snap 145 seq_num 438) dirty exists 2024-03-20T08:06:08.213 INFO:tasks.rados.rados.0.smithi118.stdout:1388: left oid 28 (ObjNum 438 snap 145 seq_num 438) 2024-03-20T08:06:08.217 INFO:tasks.rados.rados.0.smithi118.stdout:1388: done (1 left) 2024-03-20T08:06:08.217 INFO:tasks.rados.rados.0.smithi118.stdout:1391: read oid 1 snap -1 2024-03-20T08:06:08.217 INFO:tasks.rados.rados.0.smithi118.stdout:1391: expect deleted 2024-03-20T08:06:08.217 INFO:tasks.rados.rados.0.smithi118.stdout:1392: snap_remove snap 123 2024-03-20T08:06:08.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 23/852 objects degraded (2.700%), 7 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: osdmap e340: 8 total, 8 up, 8 in 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:08.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:08.144+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:08.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:08.146+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:08.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:08.165+0000 7fc1fae3c700 -1 osd.1 pg_epoch: 341 pg[5.9s2( v 341'548 (0'0,341'548] local-lis/les=336/337 n=18 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [7,6,1]p7(0) r=2 lpr=336 luod=0'0 crt=341'546 lcod 341'546 mlcod 341'546 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~3,a9~1,ab~1,af~3,b5~1,b8~4,c0~1,c7~1]] update_snap_map remove_oid 5:9d5f254b:::smithi11881747-6:ad failed with -2 2024-03-20T08:06:08.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:08.170+0000 7fc1f8637700 -1 osd.1 pg_epoch: 341 pg[5.6s2( v 339'350 (0'0,339'350] local-lis/les=336/337 n=6 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [2,5,1]p2(0) r=2 lpr=336 luod=0'0 crt=339'350 mlcod 339'350 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~3,a9~1,ab~1,af~3,b5~1,b8~4,c0~1,c7~1]] update_snap_map remove_oid 5:66a7f7bc:::smithi11881747-33:ad failed with -2 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 23/852 objects degraded (2.700%), 7 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: osdmap e340: 8 total, 8 up, 8 in 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 23/852 objects degraded (2.700%), 7 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: osdmap e340: 8 total, 8 up, 8 in 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:08.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:08.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:08.647 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:06:08.647 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:06:08.647 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 6, 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 8, 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 7 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:06:08.648 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:06:09.145 INFO:tasks.rados.rados.0.smithi118.stdout:1392: done (2 left) 2024-03-20T08:06:09.145 INFO:tasks.rados.rados.0.smithi118.stdout:1393: snap_remove snap 99 2024-03-20T08:06:09.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:09.145+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:09.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:09.145+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:09.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:09.145+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: Upgrade: unsafe to stop osd(s) at this time (4 PGs are or would become offline) 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: from='client.34233 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: from='client.34239 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: osdmap e341: 8 total, 8 up, 8 in 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: pgmap v188: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 3 active+recovery_wait+degraded, 115 active+clean; 372 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 7 op/s; 10/855 objects degraded (1.170%); 8.6 MiB/s, 3 objects/s recovering 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/95288150' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: Upgrade: unsafe to stop osd(s) at this time (4 PGs are or would become offline) 2024-03-20T08:06:09.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: from='client.34233 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:09.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: from='client.34239 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:09.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: osdmap e341: 8 total, 8 up, 8 in 2024-03-20T08:06:09.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:09.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:09.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: pgmap v188: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 3 active+recovery_wait+degraded, 115 active+clean; 372 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 7 op/s; 10/855 objects degraded (1.170%); 8.6 MiB/s, 3 objects/s recovering 2024-03-20T08:06:09.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:09 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/95288150' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:09.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: Upgrade: unsafe to stop osd(s) at this time (4 PGs are or would become offline) 2024-03-20T08:06:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: from='client.34233 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: from='client.34239 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: osdmap e341: 8 total, 8 up, 8 in 2024-03-20T08:06:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset c8=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: pgmap v188: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 3 active+recovery_wait+degraded, 115 active+clean; 372 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 7 op/s; 10/855 objects degraded (1.170%); 8.6 MiB/s, 3 objects/s recovering 2024-03-20T08:06:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:09 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/95288150' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:10.148 INFO:tasks.rados.rados.0.smithi118.stdout:1393: done (2 left) 2024-03-20T08:06:10.148 INFO:tasks.rados.rados.0.smithi118.stdout:1394: snap_create 2024-03-20T08:06:10.148 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 197 (ObjNum 406 snap 136 seq_num 406) dirty exists 2024-03-20T08:06:10.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-mon[93121]: osdmap e342: 8 total, 8 up, 8 in 2024-03-20T08:06:10.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:10.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:10.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-mon[95692]: osdmap e342: 8 total, 8 up, 8 in 2024-03-20T08:06:10.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:10.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:10.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:10.154+0000 7fc1fae3c700 -1 osd.1 pg_epoch: 343 pg[5.9s2( v 343'560 (0'0,343'560] local-lis/les=336/337 n=17 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [7,6,1]p7(0) r=2 lpr=336 luod=0'0 crt=343'560 mlcod 343'560 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~3,a9~1,ab~2,af~3,b5~1,b7~5,c0~1,c7~1]] update_snap_map remove_oid 5:95859c30:::smithi11881747-2:a0 failed with -2 2024-03-20T08:06:10.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:10.160+0000 7fc1f7635700 -1 osd.1 pg_epoch: 343 pg[5.0s1( v 343'883 (0'0,343'883] local-lis/les=336/337 n=30 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [3,1,4]p3(0) r=1 lpr=336 luod=0'0 crt=342'881 lcod 342'881 mlcod 342'881 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~3,a9~1,ab~2,af~3,b5~1,b7~5,c0~1,c7~1]] update_snap_map remove_oid 5:06c2fc64:::smithi11881747-39:9f failed with -2 2024-03-20T08:06:10.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:10.170+0000 7fc1f7635700 -1 osd.1 pg_epoch: 343 pg[5.0s1( v 343'889 (0'0,343'889] local-lis/les=336/337 n=29 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [3,1,4]p3(0) r=1 lpr=336 luod=0'0 crt=343'889 mlcod 343'889 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~3,a9~1,ab~2,af~3,b5~1,b7~5,c0~1,c7~1]] update_snap_map remove_oid 5:07779a33:::smithi11881747-18:a0 failed with -2 2024-03-20T08:06:10.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:10.158+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:10.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:10.159+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:10.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:10.160+0000 7f0cecc59700 -1 osd.0 pg_epoch: 343 pg[5.es2( v 342'599 (0'0,342'599] local-lis/les=300/301 n=25 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [4,5,0]p4(0) r=2 lpr=300 luod=0'0 crt=342'599 mlcod 342'599 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~3,a9~1,ab~2,af~3,b5~1,b7~5,c0~1,c7~1]] update_snap_map remove_oid 5:70e08578:::smithi11881747-24:9f failed with -2 2024-03-20T08:06:10.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:10.166+0000 7f0ceec5d700 -1 osd.0 pg_epoch: 343 pg[5.2s1( v 343'668 (0'0,343'668] local-lis/les=300/301 n=26 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [6,0,5]p6(0) r=1 lpr=300 luod=0'0 crt=343'666 lcod 343'666 mlcod 343'666 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~3,a9~1,ab~2,af~3,b5~1,b7~5,c0~1,c7~1]] update_snap_map remove_oid 5:43274c5b:::smithi11881747-21:a0 failed with -2 2024-03-20T08:06:10.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:10 smithi196 ceph-mon[87080]: osdmap e342: 8 total, 8 up, 8 in 2024-03-20T08:06:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:11.153 INFO:tasks.rados.rados.0.smithi118.stdout:1390: done (2 left) 2024-03-20T08:06:11.153 INFO:tasks.rados.rados.0.smithi118.stdout:1391: done (1 left) 2024-03-20T08:06:11.153 INFO:tasks.rados.rados.0.smithi118.stdout:1394: done (0 left) 2024-03-20T08:06:11.153 INFO:tasks.rados.rados.0.smithi118.stdout:1395: rollback oid 14 current snap is 146 2024-03-20T08:06:11.153 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 106 2024-03-20T08:06:11.153 INFO:tasks.rados.rados.0.smithi118.stdout:1396: snap_remove snap 128 2024-03-20T08:06:11.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-mon[93121]: osdmap e343: 8 total, 8 up, 8 in 2024-03-20T08:06:11.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:11.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:11.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-mon[93121]: pgmap v191: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 382 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 15 op/s; 9.5 MiB/s, 2 objects/s recovering 2024-03-20T08:06:11.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-mon[95692]: osdmap e343: 8 total, 8 up, 8 in 2024-03-20T08:06:11.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:11.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:11.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-mon[95692]: pgmap v191: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 382 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 15 op/s; 9.5 MiB/s, 2 objects/s recovering 2024-03-20T08:06:11.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:11.163+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:11.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:11.163+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:11.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:11.163+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:11.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:11 smithi196 ceph-mon[87080]: osdmap e343: 8 total, 8 up, 8 in 2024-03-20T08:06:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:11.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:11 smithi196 ceph-mon[87080]: pgmap v191: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 382 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 15 op/s; 9.5 MiB/s, 2 objects/s recovering 2024-03-20T08:06:12.153 INFO:tasks.rados.rados.0.smithi118.stdout:1396: done (1 left) 2024-03-20T08:06:12.153 INFO:tasks.rados.rados.0.smithi118.stdout:1397: snap_create 2024-03-20T08:06:12.154 INFO:tasks.rados.rados.0.smithi118.stdout:1395: finishing rollback tid 0 to smithi11881747-14 2024-03-20T08:06:12.154 INFO:tasks.rados.rados.0.smithi118.stdout:1395: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:06:12.154 INFO:tasks.rados.rados.0.smithi118.stdout:1395: finishing rollback tid 2 to smithi11881747-14 2024-03-20T08:06:12.154 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 6942 (ObjNum 315 snap 101 seq_num 315) dirty exists 2024-03-20T08:06:12.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-mon[95692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/855 objects degraded (1.170%), 3 pgs degraded) 2024-03-20T08:06:12.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-mon[95692]: osdmap e344: 8 total, 8 up, 8 in 2024-03-20T08:06:12.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:12.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:12.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:12.160+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:12.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:12.160+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-mon[93121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/855 objects degraded (1.170%), 3 pgs degraded) 2024-03-20T08:06:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-mon[93121]: osdmap e344: 8 total, 8 up, 8 in 2024-03-20T08:06:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:12.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:12.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:12 smithi196 ceph-mon[87080]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/855 objects degraded (1.170%), 3 pgs degraded) 2024-03-20T08:06:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:12 smithi196 ceph-mon[87080]: osdmap e344: 8 total, 8 up, 8 in 2024-03-20T08:06:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:12.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:13.162 INFO:tasks.rados.rados.0.smithi118.stdout:1395: done (1 left) 2024-03-20T08:06:13.163 INFO:tasks.rados.rados.0.smithi118.stdout:1397: done (0 left) 2024-03-20T08:06:13.163 INFO:tasks.rados.rados.0.smithi118.stdout:1398: rollback oid 16 current snap is 147 2024-03-20T08:06:13.163 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 102 2024-03-20T08:06:13.163 INFO:tasks.rados.rados.0.smithi118.stdout:1399: read oid 31 snap -1 2024-03-20T08:06:13.163 INFO:tasks.rados.rados.0.smithi118.stdout:1399: expect (ObjNum 435 snap 145 seq_num 435) 2024-03-20T08:06:13.163 INFO:tasks.rados.rados.0.smithi118.stdout:1400: snap_remove snap 129 2024-03-20T08:06:13.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-mon[93121]: osdmap e345: 8 total, 8 up, 8 in 2024-03-20T08:06:13.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:13.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:13.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-mon[93121]: pgmap v194: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 386 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 3.7 MiB/s wr, 21 op/s; 9.5 MiB/s, 2 objects/s recovering 2024-03-20T08:06:13.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-mon[95692]: osdmap e345: 8 total, 8 up, 8 in 2024-03-20T08:06:13.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:13.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:13.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-mon[95692]: pgmap v194: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 386 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 3.7 MiB/s wr, 21 op/s; 9.5 MiB/s, 2 objects/s recovering 2024-03-20T08:06:13.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:13.171+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:13.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:13.171+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:13.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:13.171+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:13.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:13 smithi196 ceph-mon[87080]: osdmap e345: 8 total, 8 up, 8 in 2024-03-20T08:06:13.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:13.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:13.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:13 smithi196 ceph-mon[87080]: pgmap v194: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 386 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 3.7 MiB/s wr, 21 op/s; 9.5 MiB/s, 2 objects/s recovering 2024-03-20T08:06:13.888 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:13.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:06:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:06:13] "GET /metrics HTTP/1.1" 200 36284 "" "Prometheus/2.43.0" 2024-03-20T08:06:14.166 INFO:tasks.rados.rados.0.smithi118.stdout:1400: done (2 left) 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:1401: read oid 20 snap -1 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:1401: expect (ObjNum 417 snap 137 seq_num 417) 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:1402: rollback oid 30 current snap is 147 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 144 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:1398: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:1398: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:1398: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 1190 (ObjNum 315 snap 101 seq_num 315) dirty exists 2024-03-20T08:06:14.167 INFO:tasks.rados.rados.0.smithi118.stdout:1399: expect (ObjNum 435 snap 145 seq_num 435) 2024-03-20T08:06:14.251 INFO:tasks.rados.rados.0.smithi118.stdout:1402: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:06:14.251 INFO:tasks.rados.rados.0.smithi118.stdout:1402: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:06:14.252 INFO:tasks.rados.rados.0.smithi118.stdout:1402: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:06:14.252 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 311 (ObjNum 421 snap 139 seq_num 421) dirty exists 2024-03-20T08:06:14.252 INFO:tasks.rados.rados.0.smithi118.stdout:1398: done (3 left) 2024-03-20T08:06:14.252 INFO:tasks.rados.rados.0.smithi118.stdout:1399: done (2 left) 2024-03-20T08:06:14.252 INFO:tasks.rados.rados.0.smithi118.stdout:1402: done (1 left) 2024-03-20T08:06:14.252 INFO:tasks.rados.rados.0.smithi118.stdout:1403: rollback oid 9 current snap is 147 2024-03-20T08:06:14.252 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 9 to 141 2024-03-20T08:06:14.252 INFO:tasks.rados.rados.0.smithi118.stdout:1401: expect (ObjNum 417 snap 137 seq_num 417) 2024-03-20T08:06:14.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:14.176+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:14.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:14.179+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[93121]: osdmap e346: 8 total, 8 up, 8 in 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[95692]: osdmap e346: 8 total, 8 up, 8 in 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:14.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:14 smithi196 ceph-mon[87080]: osdmap e346: 8 total, 8 up, 8 in 2024-03-20T08:06:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:14.680 INFO:tasks.rados.rados.0.smithi118.stdout:1403: finishing rollback tid 0 to smithi11881747-9 2024-03-20T08:06:14.680 INFO:tasks.rados.rados.0.smithi118.stdout:1403: finishing rollback tid 1 to smithi11881747-9 2024-03-20T08:06:14.680 INFO:tasks.rados.rados.0.smithi118.stdout:1403: finishing rollback tid 2 to smithi11881747-9 2024-03-20T08:06:14.680 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 993 (ObjNum 398 snap 135 seq_num 398) dirty exists 2024-03-20T08:06:14.680 INFO:tasks.rados.rados.0.smithi118.stdout:1401: done (1 left) 2024-03-20T08:06:14.680 INFO:tasks.rados.rados.0.smithi118.stdout:1403: done (0 left) 2024-03-20T08:06:14.680 INFO:tasks.rados.rados.0.smithi118.stdout:1404: snap_create 2024-03-20T08:06:15.172 INFO:tasks.rados.rados.0.smithi118.stdout:1404: done (0 left) 2024-03-20T08:06:15.172 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 5 current snap is 148 2024-03-20T08:06:15.172 INFO:tasks.rados.rados.0.smithi118.stdout:1405: seq_num 440 ranges {1982464=172032} 2024-03-20T08:06:15.184 INFO:tasks.rados.rados.0.smithi118.stdout:1405: writing smithi11881747-5 from 1982464 to 2154496 tid 1 2024-03-20T08:06:15.184 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 148 2024-03-20T08:06:15.184 INFO:tasks.rados.rados.0.smithi118.stdout:1406: seq_num 441 ranges {5144576=188416} 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1406: writing smithi11881747-10 from 5144576 to 5332992 tid 1 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1407: rmattr oid 30 current snap is 148 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1407: done (2 left) 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1408: copy_from oid 33 from oid 50 current snap is 148 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1405: finishing write tid 1 to smithi11881747-5 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1405: finishing write tid 2 to smithi11881747-5 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1405: finishing write tid 3 to smithi11881747-5 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1405: oid 5 updating version 0 to 995 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1405: oid 5 version 995 is already newer than 994 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 995 (ObjNum 440 snap 148 seq_num 440) dirty exists 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1405: left oid 5 (ObjNum 440 snap 148 seq_num 440) 2024-03-20T08:06:15.215 INFO:tasks.rados.rados.0.smithi118.stdout:1405: done (2 left) 2024-03-20T08:06:15.216 INFO:tasks.rados.rados.0.smithi118.stdout:1409: read oid 28 snap -1 2024-03-20T08:06:15.216 INFO:tasks.rados.rados.0.smithi118.stdout:1409: expect (ObjNum 438 snap 145 seq_num 438) 2024-03-20T08:06:15.216 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 148 2024-03-20T08:06:15.216 INFO:tasks.rados.rados.0.smithi118.stdout:1410: seq_num 442 ranges {7970816=204800} 2024-03-20T08:06:15.260 INFO:tasks.rados.rados.0.smithi118.stdout:1410: writing smithi11881747-20 from 7970816 to 8175616 tid 1 2024-03-20T08:06:15.260 INFO:tasks.rados.rados.0.smithi118.stdout:1411: copy_from oid 36 from oid 38 current snap is 148 2024-03-20T08:06:15.260 INFO:tasks.rados.rados.0.smithi118.stdout:1408: finishing copy_from to smithi11881747-33 2024-03-20T08:06:15.260 INFO:tasks.rados.rados.0.smithi118.stdout:1408: got expected ENOENT (src dne) 2024-03-20T08:06:15.260 INFO:tasks.rados.rados.0.smithi118.stdout:1408: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:06:15.261 INFO:tasks.rados.rados.0.smithi118.stdout:1406: finishing write tid 1 to smithi11881747-10 2024-03-20T08:06:15.261 INFO:tasks.rados.rados.0.smithi118.stdout:1406: finishing write tid 2 to smithi11881747-10 2024-03-20T08:06:15.261 INFO:tasks.rados.rados.0.smithi118.stdout:1406: finishing write tid 3 to smithi11881747-10 2024-03-20T08:06:15.261 INFO:tasks.rados.rados.0.smithi118.stdout:1406: oid 10 updating version 0 to 995 2024-03-20T08:06:15.261 INFO:tasks.rados.rados.0.smithi118.stdout:1406: oid 10 version 995 is already newer than 994 2024-03-20T08:06:15.261 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 995 (ObjNum 441 snap 148 seq_num 441) dirty exists 2024-03-20T08:06:15.261 INFO:tasks.rados.rados.0.smithi118.stdout:1406: left oid 10 (ObjNum 441 snap 148 seq_num 441) 2024-03-20T08:06:15.261 INFO:tasks.rados.rados.0.smithi118.stdout:1409: expect (ObjNum 438 snap 145 seq_num 438) 2024-03-20T08:06:15.333 INFO:tasks.rados.rados.0.smithi118.stdout:1410: finishing write tid 1 to smithi11881747-20 2024-03-20T08:06:15.333 INFO:tasks.rados.rados.0.smithi118.stdout:1410: finishing write tid 2 to smithi11881747-20 2024-03-20T08:06:15.333 INFO:tasks.rados.rados.0.smithi118.stdout:1410: finishing write tid 3 to smithi11881747-20 2024-03-20T08:06:15.333 INFO:tasks.rados.rados.0.smithi118.stdout:1410: oid 20 updating version 0 to 7040 2024-03-20T08:06:15.333 INFO:tasks.rados.rados.0.smithi118.stdout:1410: oid 20 version 7040 is already newer than 7039 2024-03-20T08:06:15.333 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 7040 (ObjNum 442 snap 148 seq_num 442) dirty exists 2024-03-20T08:06:15.334 INFO:tasks.rados.rados.0.smithi118.stdout:1410: left oid 20 (ObjNum 442 snap 148 seq_num 442) 2024-03-20T08:06:15.334 INFO:tasks.rados.rados.0.smithi118.stdout:1406: done (4 left) 2024-03-20T08:06:15.334 INFO:tasks.rados.rados.0.smithi118.stdout:1408: done (3 left) 2024-03-20T08:06:15.334 INFO:tasks.rados.rados.0.smithi118.stdout:1409: done (2 left) 2024-03-20T08:06:15.334 INFO:tasks.rados.rados.0.smithi118.stdout:1410: done (1 left) 2024-03-20T08:06:15.334 INFO:tasks.rados.rados.0.smithi118.stdout:1412: snap_remove snap 136 2024-03-20T08:06:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-mon[93121]: osdmap e347: 8 total, 8 up, 8 in 2024-03-20T08:06:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-mon[93121]: pgmap v197: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 384 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 5.7 MiB/s rd, 1.2 MiB/s wr, 6 op/s 2024-03-20T08:06:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-mon[95692]: osdmap e347: 8 total, 8 up, 8 in 2024-03-20T08:06:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-mon[95692]: pgmap v197: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 384 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 5.7 MiB/s rd, 1.2 MiB/s wr, 6 op/s 2024-03-20T08:06:15.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:15.187+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:15.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:15.187+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:15.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:15.192+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:15.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:15 smithi196 ceph-mon[87080]: osdmap e347: 8 total, 8 up, 8 in 2024-03-20T08:06:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:15 smithi196 ceph-mon[87080]: pgmap v197: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 384 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 5.7 MiB/s rd, 1.2 MiB/s wr, 6 op/s 2024-03-20T08:06:16.175 INFO:tasks.rados.rados.0.smithi118.stdout:1412: done (1 left) 2024-03-20T08:06:16.176 INFO:tasks.rados.rados.0.smithi118.stdout:1413: snap_remove snap 106 2024-03-20T08:06:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:16 smithi118 ceph-mon[93121]: osdmap e348: 8 total, 8 up, 8 in 2024-03-20T08:06:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:16.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:16.175+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:16.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:16.176+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:16 smithi118 ceph-mon[95692]: osdmap e348: 8 total, 8 up, 8 in 2024-03-20T08:06:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:16.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:16 smithi196 ceph-mon[87080]: osdmap e348: 8 total, 8 up, 8 in 2024-03-20T08:06:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset cc=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:17.180 INFO:tasks.rados.rados.0.smithi118.stdout:1413: done (1 left) 2024-03-20T08:06:17.180 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 148 2024-03-20T08:06:17.180 INFO:tasks.rados.rados.0.smithi118.stdout:1414: seq_num 443 ranges {1875968=221184} 2024-03-20T08:06:17.192 INFO:tasks.rados.rados.0.smithi118.stdout:1414: writing smithi11881747-21 from 1875968 to 2097152 tid 1 2024-03-20T08:06:17.192 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 148 2024-03-20T08:06:17.192 INFO:tasks.rados.rados.0.smithi118.stdout:1415: seq_num 444 ranges {4694016=237568} 2024-03-20T08:06:17.220 INFO:tasks.rados.rados.0.smithi118.stdout:1415: writing smithi11881747-27 from 4694016 to 4931584 tid 1 2024-03-20T08:06:17.220 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 148 2024-03-20T08:06:17.220 INFO:tasks.rados.rados.0.smithi118.stdout:1416: seq_num 445 ranges {4751360=253952} 2024-03-20T08:06:17.249 INFO:tasks.rados.rados.0.smithi118.stdout:1416: writing smithi11881747-33 from 4751360 to 5005312 tid 1 2024-03-20T08:06:17.249 INFO:tasks.rados.rados.0.smithi118.stdout:1417: copy_from oid 14 from oid 11 current snap is 148 2024-03-20T08:06:17.249 INFO:tasks.rados.rados.0.smithi118.stdout:1411: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1411: finishing copy_from to smithi11881747-36 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 800 (ObjNum 393 snap 134 seq_num 393) dirty exists 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1411: done (4 left) 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1418: copy_from oid 50 from oid 6 current snap is 148 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1414: finishing write tid 1 to smithi11881747-21 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1414: finishing write tid 2 to smithi11881747-21 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1414: finishing write tid 3 to smithi11881747-21 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1414: oid 21 updating version 0 to 7023 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1414: oid 21 version 7023 is already newer than 7022 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 7023 (ObjNum 443 snap 148 seq_num 443) dirty exists 2024-03-20T08:06:17.250 INFO:tasks.rados.rados.0.smithi118.stdout:1414: left oid 21 (ObjNum 443 snap 148 seq_num 443) 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1415: finishing write tid 1 to smithi11881747-27 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1415: finishing write tid 2 to smithi11881747-27 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1415: finishing write tid 3 to smithi11881747-27 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1415: oid 27 updating version 0 to 7042 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1415: oid 27 version 7042 is already newer than 7041 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7042 (ObjNum 444 snap 148 seq_num 444) dirty exists 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1415: left oid 27 (ObjNum 444 snap 148 seq_num 444) 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1414: done (4 left) 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1415: done (3 left) 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1419: rollback oid 37 current snap is 148 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 138 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1420: read oid 41 snap -1 2024-03-20T08:06:17.251 INFO:tasks.rados.rados.0.smithi118.stdout:1420: expect (ObjNum 425 snap 141 seq_num 425) 2024-03-20T08:06:17.252 INFO:tasks.rados.rados.0.smithi118.stdout:1421: snap_remove snap 118 2024-03-20T08:06:17.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:17 smithi196 ceph-mon[87080]: osdmap e349: 8 total, 8 up, 8 in 2024-03-20T08:06:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:17 smithi196 ceph-mon[87080]: pgmap v200: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 387 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 6.8 MiB/s rd, 668 KiB/s wr, 4 op/s 2024-03-20T08:06:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-mon[93121]: osdmap e349: 8 total, 8 up, 8 in 2024-03-20T08:06:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-mon[93121]: pgmap v200: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 387 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 6.8 MiB/s rd, 668 KiB/s wr, 4 op/s 2024-03-20T08:06:17.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:17.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-mon[95692]: osdmap e349: 8 total, 8 up, 8 in 2024-03-20T08:06:17.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:17.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-mon[95692]: pgmap v200: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 387 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 6.8 MiB/s rd, 668 KiB/s wr, 4 op/s 2024-03-20T08:06:17.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:17.184+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:17.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:17.184+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:17.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:17.186+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:18.185 INFO:tasks.rados.rados.0.smithi118.stdout:1421: done (5 left) 2024-03-20T08:06:18.185 INFO:tasks.rados.rados.0.smithi118.stdout:1422: copy_from oid 12 from oid 19 current snap is 148 2024-03-20T08:06:18.185 INFO:tasks.rados.rados.0.smithi118.stdout:1418: finishing copy_from to smithi11881747-50 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1418: got expected ENOENT (src dne) 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1418: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1419: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 328 (ObjNum 225 snap 0 seq_num 319762368) dirty dne 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1416: finishing write tid 1 to smithi11881747-33 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1416: finishing write tid 2 to smithi11881747-33 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1416: finishing write tid 3 to smithi11881747-33 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1416: oid 33 updating version 0 to 800 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1416: oid 33 updating version 800 to 801 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 801 (ObjNum 445 snap 148 seq_num 445) dirty exists 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1416: left oid 33 (ObjNum 445 snap 148 seq_num 445) 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1417: finishing copy_from racing read to smithi11881747-14 2024-03-20T08:06:18.186 INFO:tasks.rados.rados.0.smithi118.stdout:1420: expect (ObjNum 425 snap 141 seq_num 425) 2024-03-20T08:06:18.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:18 smithi118 ceph-mon[93121]: osdmap e350: 8 total, 8 up, 8 in 2024-03-20T08:06:18.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:18.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:18.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:18.188+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:18.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:18.189+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:18.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:18 smithi118 ceph-mon[95692]: osdmap e350: 8 total, 8 up, 8 in 2024-03-20T08:06:18.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:18.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:18.599 INFO:tasks.rados.rados.0.smithi118.stdout:1417: finishing copy_from to smithi11881747-14 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 787 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1422: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1416: done (5 left) 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1417: done (4 left) 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1418: done (3 left) 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1419: done (2 left) 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1420: done (1 left) 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1423: read oid 26 snap 102 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1423: expect (ObjNum 266 snap 92 seq_num 266) 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 148 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1422: finishing copy_from to smithi11881747-12 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 263 (ObjNum 407 snap 136 seq_num 407) dirty exists 2024-03-20T08:06:18.600 INFO:tasks.rados.rados.0.smithi118.stdout:1424: seq_num 446 ranges {1531904=278528} 2024-03-20T08:06:18.609 INFO:tasks.rados.rados.0.smithi118.stdout:1424: writing smithi11881747-28 from 1531904 to 1810432 tid 1 2024-03-20T08:06:18.609 INFO:tasks.rados.rados.0.smithi118.stdout:1422: done (2 left) 2024-03-20T08:06:18.609 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 148 2024-03-20T08:06:18.609 INFO:tasks.rados.rados.0.smithi118.stdout:1425: seq_num 447 ranges {2621440=294912} 2024-03-20T08:06:18.625 INFO:tasks.rados.rados.0.smithi118.stdout:1425: writing smithi11881747-13 from 2621440 to 2916352 tid 1 2024-03-20T08:06:18.626 INFO:tasks.rados.rados.0.smithi118.stdout:1426: setattr oid 33 current snap is 148 2024-03-20T08:06:18.626 INFO:tasks.rados.rados.0.smithi118.stdout:1423: expect (ObjNum 266 snap 92 seq_num 266) 2024-03-20T08:06:18.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:18 smithi196 ceph-mon[87080]: osdmap e350: 8 total, 8 up, 8 in 2024-03-20T08:06:18.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:18.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:1424: finishing write tid 1 to smithi11881747-28 2024-03-20T08:06:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:1424: finishing write tid 2 to smithi11881747-28 2024-03-20T08:06:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:1424: finishing write tid 3 to smithi11881747-28 2024-03-20T08:06:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:1424: oid 28 updating version 0 to 1230 2024-03-20T08:06:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:1424: oid 28 updating version 1230 to 1231 2024-03-20T08:06:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 1231 (ObjNum 446 snap 148 seq_num 446) dirty exists 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1424: left oid 28 (ObjNum 446 snap 148 seq_num 446) 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1425: finishing write tid 1 to smithi11881747-13 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1425: finishing write tid 2 to smithi11881747-13 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1425: finishing write tid 3 to smithi11881747-13 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1425: oid 13 updating version 0 to 996 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1425: oid 13 updating version 996 to 997 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 997 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1425: left oid 13 (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1423: done (3 left) 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1424: done (2 left) 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1425: done (1 left) 2024-03-20T08:06:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:1427: rollback oid 2 current snap is 148 2024-03-20T08:06:18.694 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 135 2024-03-20T08:06:18.694 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 148 2024-03-20T08:06:18.694 INFO:tasks.rados.rados.0.smithi118.stdout:1428: seq_num 448 ranges {57344=311296} 2024-03-20T08:06:18.695 INFO:tasks.rados.rados.0.smithi118.stdout:1428: writing smithi11881747-48 from 57344 to 368640 tid 1 2024-03-20T08:06:18.695 INFO:tasks.rados.rados.0.smithi118.stdout:1429: rmattr oid 16 current snap is 148 2024-03-20T08:06:18.695 INFO:tasks.rados.rados.0.smithi118.stdout:1429: done (3 left) 2024-03-20T08:06:18.695 INFO:tasks.rados.rados.0.smithi118.stdout:1430: read oid 9 snap -1 2024-03-20T08:06:18.695 INFO:tasks.rados.rados.0.smithi118.stdout:1430: expect (ObjNum 398 snap 135 seq_num 398) 2024-03-20T08:06:18.695 INFO:tasks.rados.rados.0.smithi118.stdout:1431: snap_create 2024-03-20T08:06:18.695 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 802 (ObjNum 445 snap 148 seq_num 445) dirty exists 2024-03-20T08:06:18.696 INFO:tasks.rados.rados.0.smithi118.stdout:1427: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:06:18.696 INFO:tasks.rados.rados.0.smithi118.stdout:1427: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:06:18.696 INFO:tasks.rados.rados.0.smithi118.stdout:1427: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:06:18.696 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 794 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:06:18.698 INFO:tasks.rados.rados.0.smithi118.stdout:1428: finishing write tid 1 to smithi11881747-48 2024-03-20T08:06:18.698 INFO:tasks.rados.rados.0.smithi118.stdout:1428: finishing write tid 2 to smithi11881747-48 2024-03-20T08:06:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:1428: finishing write tid 3 to smithi11881747-48 2024-03-20T08:06:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:1428: oid 48 updating version 0 to 7024 2024-03-20T08:06:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:1428: oid 48 updating version 7024 to 7025 2024-03-20T08:06:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 7025 (ObjNum 448 snap 148 seq_num 448) dirty exists 2024-03-20T08:06:18.700 INFO:tasks.rados.rados.0.smithi118.stdout:1428: left oid 48 (ObjNum 448 snap 148 seq_num 448) 2024-03-20T08:06:18.700 INFO:tasks.rados.rados.0.smithi118.stdout:1430: expect (ObjNum 398 snap 135 seq_num 398) 2024-03-20T08:06:19.301 INFO:tasks.rados.rados.0.smithi118.stdout:1426: done (4 left) 2024-03-20T08:06:19.302 INFO:tasks.rados.rados.0.smithi118.stdout:1427: done (3 left) 2024-03-20T08:06:19.302 INFO:tasks.rados.rados.0.smithi118.stdout:1428: done (2 left) 2024-03-20T08:06:19.302 INFO:tasks.rados.rados.0.smithi118.stdout:1430: done (1 left) 2024-03-20T08:06:19.302 INFO:tasks.rados.rados.0.smithi118.stdout:1431: done (0 left) 2024-03-20T08:06:19.302 INFO:tasks.rados.rados.0.smithi118.stdout:1432: read oid 49 snap -1 2024-03-20T08:06:19.302 INFO:tasks.rados.rados.0.smithi118.stdout:1432: expect (ObjNum 216 snap 69 seq_num 216) 2024-03-20T08:06:19.302 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 149 2024-03-20T08:06:19.302 INFO:tasks.rados.rados.0.smithi118.stdout:1433: seq_num 449 ranges {0=327680} 2024-03-20T08:06:19.303 INFO:tasks.rados.rados.0.smithi118.stdout:1433: writing smithi11881747-26 from 0 to 327680 tid 1 2024-03-20T08:06:19.303 INFO:tasks.rados.rados.0.smithi118.stdout:1434: snap_create 2024-03-20T08:06:19.307 INFO:tasks.rados.rados.0.smithi118.stdout:1433: finishing write tid 1 to smithi11881747-26 2024-03-20T08:06:19.307 INFO:tasks.rados.rados.0.smithi118.stdout:1433: finishing write tid 2 to smithi11881747-26 2024-03-20T08:06:19.308 INFO:tasks.rados.rados.0.smithi118.stdout:1433: finishing write tid 3 to smithi11881747-26 2024-03-20T08:06:19.308 INFO:tasks.rados.rados.0.smithi118.stdout:1433: oid 26 updating version 0 to 806 2024-03-20T08:06:19.308 INFO:tasks.rados.rados.0.smithi118.stdout:1433: oid 26 version 806 is already newer than 805 2024-03-20T08:06:19.308 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 806 (ObjNum 449 snap 149 seq_num 449) dirty exists 2024-03-20T08:06:19.308 INFO:tasks.rados.rados.0.smithi118.stdout:1433: left oid 26 (ObjNum 449 snap 149 seq_num 449) 2024-03-20T08:06:19.309 INFO:tasks.rados.rados.0.smithi118.stdout:1432: expect (ObjNum 216 snap 69 seq_num 216) 2024-03-20T08:06:19.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:19 smithi196 ceph-mon[87080]: osdmap e351: 8 total, 8 up, 8 in 2024-03-20T08:06:19.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:19.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:19.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:19 smithi196 ceph-mon[87080]: pgmap v203: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 387 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 1.1 MiB/s rd, 668 KiB/s wr, 2 op/s 2024-03-20T08:06:19.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:19 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[93121]: osdmap e351: 8 total, 8 up, 8 in 2024-03-20T08:06:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:19.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[93121]: pgmap v203: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 387 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 1.1 MiB/s rd, 668 KiB/s wr, 2 op/s 2024-03-20T08:06:19.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:19.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:19.306+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:19.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:19.306+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:19.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:19.306+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:19.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[95692]: osdmap e351: 8 total, 8 up, 8 in 2024-03-20T08:06:19.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:19.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:19.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[95692]: pgmap v203: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 387 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 1.1 MiB/s rd, 668 KiB/s wr, 2 op/s 2024-03-20T08:06:19.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:19 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:20.305 INFO:tasks.rados.rados.0.smithi118.stdout:1432: done (2 left) 2024-03-20T08:06:20.305 INFO:tasks.rados.rados.0.smithi118.stdout:1433: done (1 left) 2024-03-20T08:06:20.305 INFO:tasks.rados.rados.0.smithi118.stdout:1434: done (0 left) 2024-03-20T08:06:20.306 INFO:tasks.rados.rados.0.smithi118.stdout:1435: snap_create 2024-03-20T08:06:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:20 smithi196 ceph-mon[87080]: osdmap e352: 8 total, 8 up, 8 in 2024-03-20T08:06:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:20 smithi196 ceph-mon[87080]: pgmap v205: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 390 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 8.0 MiB/s rd, 901 KiB/s wr, 14 op/s 2024-03-20T08:06:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-mon[93121]: osdmap e352: 8 total, 8 up, 8 in 2024-03-20T08:06:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-mon[93121]: pgmap v205: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 390 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 8.0 MiB/s rd, 901 KiB/s wr, 14 op/s 2024-03-20T08:06:20.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:20.308+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:20.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:20.308+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-mon[95692]: osdmap e352: 8 total, 8 up, 8 in 2024-03-20T08:06:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:20 smithi118 ceph-mon[95692]: pgmap v205: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 390 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 8.0 MiB/s rd, 901 KiB/s wr, 14 op/s 2024-03-20T08:06:21.309 INFO:tasks.rados.rados.0.smithi118.stdout:1435: done (0 left) 2024-03-20T08:06:21.309 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 151 2024-03-20T08:06:21.309 INFO:tasks.rados.rados.0.smithi118.stdout:1436: seq_num 450 ranges {2564096=344064} 2024-03-20T08:06:21.326 INFO:tasks.rados.rados.0.smithi118.stdout:1436: writing smithi11881747-44 from 2564096 to 2908160 tid 1 2024-03-20T08:06:21.326 INFO:tasks.rados.rados.0.smithi118.stdout:1437: read oid 10 snap -1 2024-03-20T08:06:21.326 INFO:tasks.rados.rados.0.smithi118.stdout:1437: expect (ObjNum 441 snap 148 seq_num 441) 2024-03-20T08:06:21.326 INFO:tasks.rados.rados.0.smithi118.stdout:1438: read oid 1 snap -1 2024-03-20T08:06:21.326 INFO:tasks.rados.rados.0.smithi118.stdout:1438: expect deleted 2024-03-20T08:06:21.326 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 151 2024-03-20T08:06:21.327 INFO:tasks.rados.rados.0.smithi118.stdout:1439: seq_num 451 ranges {2031616=360448} 2024-03-20T08:06:21.340 INFO:tasks.rados.rados.0.smithi118.stdout:1439: writing smithi11881747-19 from 2031616 to 2392064 tid 1 2024-03-20T08:06:21.340 INFO:tasks.rados.rados.0.smithi118.stdout:1440: copy_from oid 17 from oid 18 current snap is 151 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:1436: finishing write tid 1 to smithi11881747-44 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:1436: finishing write tid 2 to smithi11881747-44 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:1436: finishing write tid 3 to smithi11881747-44 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:1436: oid 44 updating version 0 to 808 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:1436: oid 44 version 808 is already newer than 807 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 808 (ObjNum 450 snap 151 seq_num 450) dirty exists 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:1436: left oid 44 (ObjNum 450 snap 151 seq_num 450) 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:1436: done (4 left) 2024-03-20T08:06:21.341 INFO:tasks.rados.rados.0.smithi118.stdout:1441: snap_create 2024-03-20T08:06:21.353 INFO:tasks.rados.rados.0.smithi118.stdout:1437: expect (ObjNum 441 snap 148 seq_num 441) 2024-03-20T08:06:21.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:21 smithi196 ceph-mon[87080]: osdmap e353: 8 total, 8 up, 8 in 2024-03-20T08:06:21.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:21.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:21.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-mon[93121]: osdmap e353: 8 total, 8 up, 8 in 2024-03-20T08:06:21.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:21.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:21.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-mon[95692]: osdmap e353: 8 total, 8 up, 8 in 2024-03-20T08:06:21.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:21.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:21.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:21.316+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:21.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:21.316+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:21.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:21.316+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:21.765 INFO:tasks.rados.rados.0.smithi118.stdout:1439: finishing write tid 1 to smithi11881747-19 2024-03-20T08:06:21.765 INFO:tasks.rados.rados.0.smithi118.stdout:1439: finishing write tid 2 to smithi11881747-19 2024-03-20T08:06:21.765 INFO:tasks.rados.rados.0.smithi118.stdout:1439: finishing write tid 3 to smithi11881747-19 2024-03-20T08:06:21.765 INFO:tasks.rados.rados.0.smithi118.stdout:1439: oid 19 updating version 0 to 315 2024-03-20T08:06:21.765 INFO:tasks.rados.rados.0.smithi118.stdout:1439: oid 19 updating version 315 to 316 2024-03-20T08:06:21.765 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 316 (ObjNum 451 snap 151 seq_num 451) dirty exists 2024-03-20T08:06:21.766 INFO:tasks.rados.rados.0.smithi118.stdout:1439: left oid 19 (ObjNum 451 snap 151 seq_num 451) 2024-03-20T08:06:21.766 INFO:tasks.rados.rados.0.smithi118.stdout:1440: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:06:21.766 INFO:tasks.rados.rados.0.smithi118.stdout:1440: finishing copy_from to smithi11881747-17 2024-03-20T08:06:21.766 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7026 (ObjNum 399 snap 135 seq_num 399) dirty exists 2024-03-20T08:06:22.315 INFO:tasks.rados.rados.0.smithi118.stdout:1437: done (4 left) 2024-03-20T08:06:22.315 INFO:tasks.rados.rados.0.smithi118.stdout:1438: done (3 left) 2024-03-20T08:06:22.315 INFO:tasks.rados.rados.0.smithi118.stdout:1439: done (2 left) 2024-03-20T08:06:22.315 INFO:tasks.rados.rados.0.smithi118.stdout:1440: done (1 left) 2024-03-20T08:06:22.315 INFO:tasks.rados.rados.0.smithi118.stdout:1441: done (0 left) 2024-03-20T08:06:22.315 INFO:tasks.rados.rados.0.smithi118.stdout:1442: snap_create 2024-03-20T08:06:22.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:22 smithi196 ceph-mon[87080]: osdmap e354: 8 total, 8 up, 8 in 2024-03-20T08:06:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:22 smithi196 ceph-mon[87080]: pgmap v208: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 394 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 9.4 MiB/s rd, 1.9 MiB/s wr, 16 op/s 2024-03-20T08:06:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:22.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:22.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-mon[95692]: osdmap e354: 8 total, 8 up, 8 in 2024-03-20T08:06:22.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:22.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:22.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-mon[95692]: pgmap v208: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 394 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 9.4 MiB/s rd, 1.9 MiB/s wr, 16 op/s 2024-03-20T08:06:22.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-mon[93121]: osdmap e354: 8 total, 8 up, 8 in 2024-03-20T08:06:22.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:22.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:22.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-mon[93121]: pgmap v208: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 394 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 9.4 MiB/s rd, 1.9 MiB/s wr, 16 op/s 2024-03-20T08:06:23.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:22.761+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:23.039 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:22.761+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:23.316 INFO:tasks.rados.rados.0.smithi118.stdout:1442: done (0 left) 2024-03-20T08:06:23.316 INFO:tasks.rados.rados.0.smithi118.stdout:1443: snap_remove snap 150 2024-03-20T08:06:23.585 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-mon[93121]: osdmap e355: 8 total, 8 up, 8 in 2024-03-20T08:06:23.585 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:23.585 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:23.585 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-mon[93121]: Upgrade: osd.2 is safe to restart 2024-03-20T08:06:23.585 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-mon[95692]: osdmap e355: 8 total, 8 up, 8 in 2024-03-20T08:06:23.585 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:23.586 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:23.586 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-mon[95692]: Upgrade: osd.2 is safe to restart 2024-03-20T08:06:23.586 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:23.315+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:23.586 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:23.315+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:23.586 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:23.316+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:23.638 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:23.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:23 smithi196 ceph-mon[87080]: osdmap e355: 8 total, 8 up, 8 in 2024-03-20T08:06:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:23 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-20T08:06:23.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:23 smithi196 ceph-mon[87080]: Upgrade: osd.2 is safe to restart 2024-03-20T08:06:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:06:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:06:23] "GET /metrics HTTP/1.1" 200 36284 "" "Prometheus/2.43.0" 2024-03-20T08:06:24.325 INFO:tasks.rados.rados.0.smithi118.stdout:1443: done (0 left) 2024-03-20T08:06:24.325 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 153 2024-03-20T08:06:24.325 INFO:tasks.rados.rados.0.smithi118.stdout:1444: seq_num 452 ranges {5005312=376832} 2024-03-20T08:06:24.355 INFO:tasks.rados.rados.0.smithi118.stdout:1444: writing smithi11881747-33 from 5005312 to 5382144 tid 1 2024-03-20T08:06:24.356 INFO:tasks.rados.rados.0.smithi118.stdout:1445: copy_from oid 22 from oid 50 current snap is 153 2024-03-20T08:06:24.356 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 153 2024-03-20T08:06:24.356 INFO:tasks.rados.rados.0.smithi118.stdout:1446: seq_num 453 ranges {2564096=393216} 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1446: writing smithi11881747-38 from 2564096 to 2957312 tid 1 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1445: finishing copy_from to smithi11881747-22 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1445: got expected ENOENT (src dne) 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1445: finishing copy_from racing read to smithi11881747-22 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1444: finishing write tid 1 to smithi11881747-33 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1444: finishing write tid 2 to smithi11881747-33 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1444: finishing write tid 3 to smithi11881747-33 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1444: oid 33 updating version 0 to 803 2024-03-20T08:06:24.375 INFO:tasks.rados.rados.0.smithi118.stdout:1444: oid 33 updating version 803 to 804 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 804 (ObjNum 452 snap 153 seq_num 452) dirty exists 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1444: left oid 33 (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1444: done (2 left) 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1445: done (1 left) 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1447: read oid 2 snap -1 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1447: expect (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1448: rmattr oid 49 current snap is 153 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1449: read oid 8 snap -1 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1449: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:06:24.376 INFO:tasks.rados.rados.0.smithi118.stdout:1450: snap_create 2024-03-20T08:06:24.377 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 0 (ObjNum 216 snap 69 seq_num 216) dirty exists 2024-03-20T08:06:24.377 INFO:tasks.rados.rados.0.smithi118.stdout:1446: finishing write tid 1 to smithi11881747-38 2024-03-20T08:06:24.378 INFO:tasks.rados.rados.0.smithi118.stdout:1446: finishing write tid 2 to smithi11881747-38 2024-03-20T08:06:24.381 INFO:tasks.rados.rados.0.smithi118.stdout:1446: finishing write tid 3 to smithi11881747-38 2024-03-20T08:06:24.381 INFO:tasks.rados.rados.0.smithi118.stdout:1446: oid 38 updating version 0 to 7027 2024-03-20T08:06:24.381 INFO:tasks.rados.rados.0.smithi118.stdout:1446: oid 38 version 7027 is already newer than 7026 2024-03-20T08:06:24.381 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7027 (ObjNum 453 snap 153 seq_num 453) dirty exists 2024-03-20T08:06:24.381 INFO:tasks.rados.rados.0.smithi118.stdout:1446: left oid 38 (ObjNum 453 snap 153 seq_num 453) 2024-03-20T08:06:24.383 INFO:tasks.rados.rados.0.smithi118.stdout:1449: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:06:24.396 INFO:tasks.rados.rados.0.smithi118.stdout:1447: expect (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:06:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:24 smithi196 ceph-mon[87080]: osdmap e356: 8 total, 8 up, 8 in 2024-03-20T08:06:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:24 smithi196 ceph-mon[87080]: pgmap v211: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 395 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-03-20T08:06:24.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:24.328+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:24.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:24.328+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[93121]: osdmap e356: 8 total, 8 up, 8 in 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[93121]: pgmap v211: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 395 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[95692]: osdmap e356: 8 total, 8 up, 8 in 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:24 smithi118 ceph-mon[95692]: pgmap v211: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 395 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 4 op/s 2024-03-20T08:06:25.359 INFO:tasks.rados.rados.0.smithi118.stdout:1446: done (4 left) 2024-03-20T08:06:25.359 INFO:tasks.rados.rados.0.smithi118.stdout:1447: done (3 left) 2024-03-20T08:06:25.359 INFO:tasks.rados.rados.0.smithi118.stdout:1448: done (2 left) 2024-03-20T08:06:25.359 INFO:tasks.rados.rados.0.smithi118.stdout:1449: done (1 left) 2024-03-20T08:06:25.359 INFO:tasks.rados.rados.0.smithi118.stdout:1450: done (0 left) 2024-03-20T08:06:25.359 INFO:tasks.rados.rados.0.smithi118.stdout:1451: delete oid 40 current snap is 154 2024-03-20T08:06:25.450 INFO:tasks.rados.rados.0.smithi118.stdout:1451: done (0 left) 2024-03-20T08:06:25.450 INFO:tasks.rados.rados.0.smithi118.stdout:1452: snap_remove snap 122 2024-03-20T08:06:25.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:25 smithi196 ceph-mon[87080]: osdmap e357: 8 total, 8 up, 8 in 2024-03-20T08:06:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:25 smithi196 ceph-mon[87080]: Upgrade: Updating osd.2 2024-03-20T08:06:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T08:06:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:25 smithi196 ceph-mon[87080]: Deploying daemon osd.2 on smithi118 2024-03-20T08:06:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[93121]: osdmap e357: 8 total, 8 up, 8 in 2024-03-20T08:06:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[93121]: Upgrade: Updating osd.2 2024-03-20T08:06:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T08:06:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[93121]: Deploying daemon osd.2 on smithi118 2024-03-20T08:06:25.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:25.417+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:25.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:25.417+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:25.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:25.418+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:25.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[95692]: osdmap e357: 8 total, 8 up, 8 in 2024-03-20T08:06:25.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:25.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[95692]: Upgrade: Updating osd.2 2024-03-20T08:06:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-20T08:06:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:25 smithi118 ceph-mon[95692]: Deploying daemon osd.2 on smithi118 2024-03-20T08:06:26.365 INFO:tasks.rados.rados.0.smithi118.stdout:1452: done (0 left) 2024-03-20T08:06:26.365 INFO:tasks.rados.rados.0.smithi118.stdout:1453: snap_remove snap 148 2024-03-20T08:06:26.367 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:26 smithi118 ceph-mon[93121]: osdmap e358: 8 total, 8 up, 8 in 2024-03-20T08:06:26.368 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:26 smithi118 ceph-mon[93121]: pgmap v214: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-03-20T08:06:26.368 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:26 smithi118 ceph-mon[95692]: osdmap e358: 8 total, 8 up, 8 in 2024-03-20T08:06:26.368 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:26 smithi118 ceph-mon[95692]: pgmap v214: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-03-20T08:06:26.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:26 smithi196 ceph-mon[87080]: osdmap e358: 8 total, 8 up, 8 in 2024-03-20T08:06:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:26 smithi196 ceph-mon[87080]: pgmap v214: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 4 op/s 2024-03-20T08:06:26.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:26.368+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:26.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:26.368+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:27.365 INFO:tasks.rados.rados.0.smithi118.stdout:1453: done (0 left) 2024-03-20T08:06:27.365 INFO:tasks.rados.rados.0.smithi118.stdout:1454: snap_remove snap 153 2024-03-20T08:06:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:27 smithi196 ceph-mon[87080]: osdmap e359: 8 total, 8 up, 8 in 2024-03-20T08:06:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:27.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:27.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:27.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[93121]: osdmap e359: 8 total, 8 up, 8 in 2024-03-20T08:06:27.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:27.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:27.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:27.369+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:27.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:27.369+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:27.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:27.370+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[95692]: osdmap e359: 8 total, 8 up, 8 in 2024-03-20T08:06:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:28.375 INFO:tasks.rados.rados.0.smithi118.stdout:1454: done (0 left) 2024-03-20T08:06:28.375 INFO:tasks.rados.rados.0.smithi118.stdout:1455: delete oid 34 current snap is 154 2024-03-20T08:06:28.383 INFO:tasks.rados.rados.0.smithi118.stdout:1455: done (0 left) 2024-03-20T08:06:28.383 INFO:tasks.rados.rados.0.smithi118.stdout:1456: read oid 33 snap -1 2024-03-20T08:06:28.383 INFO:tasks.rados.rados.0.smithi118.stdout:1456: expect (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:28.383 INFO:tasks.rados.rados.0.smithi118.stdout:1457: snap_remove snap 102 2024-03-20T08:06:28.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:28.379+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:28.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:28.382+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[93121]: osdmap e360: 8 total, 8 up, 8 in 2024-03-20T08:06:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:28.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:28.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[93121]: pgmap v217: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 3 op/s 2024-03-20T08:06:28.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[95692]: osdmap e360: 8 total, 8 up, 8 in 2024-03-20T08:06:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[95692]: pgmap v217: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 3 op/s 2024-03-20T08:06:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:28.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:28 smithi196 ceph-mon[87080]: osdmap e360: 8 total, 8 up, 8 in 2024-03-20T08:06:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:28 smithi196 ceph-mon[87080]: pgmap v217: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 3.8 MiB/s rd, 1.8 MiB/s wr, 3 op/s 2024-03-20T08:06:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:29.379 INFO:tasks.rados.rados.0.smithi118.stdout:1457: done (1 left) 2024-03-20T08:06:29.379 INFO:tasks.rados.rados.0.smithi118.stdout:1458: setattr oid 2 current snap is 154 2024-03-20T08:06:29.379 INFO:tasks.rados.rados.0.smithi118.stdout:1456: expect (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:29.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:29 smithi196 ceph-mon[87080]: osdmap e361: 8 total, 8 up, 8 in 2024-03-20T08:06:29.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:29.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:29.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-mon[93121]: osdmap e361: 8 total, 8 up, 8 in 2024-03-20T08:06:29.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:29.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-mon[95692]: osdmap e361: 8 total, 8 up, 8 in 2024-03-20T08:06:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:29.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:29.383+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:29.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:29.383+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:29.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:29.383+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:29.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:29.388+0000 7fc1f7635700 -1 osd.1 pg_epoch: 362 pg[5.0s1( v 362'945 (0'0,362'945] local-lis/les=336/337 n=29 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [3,1,4]p3(0) r=1 lpr=336 luod=0'0 crt=359'943 mlcod 359'943 active mbc={} ps=36] update_snap_map remove_oid 5:06c2fc64:::smithi11881747-39:a3 failed with -2 2024-03-20T08:06:29.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:29.396+0000 7fc1f7635700 -1 osd.1 pg_epoch: 362 pg[5.0s1( v 362'947 (0'0,362'947] local-lis/les=336/337 n=28 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [3,1,4]p3(0) r=1 lpr=336 luod=0'0 crt=362'947 mlcod 362'947 active mbc={} ps=36] update_snap_map remove_oid 5:0e32d9f9:::smithi11881747-27:a3 failed with -2 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1456: done (1 left) 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1459: copy_from oid 12 from oid 18 current snap is 154 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1460: read oid 25 snap -1 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1460: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1461: read oid 33 snap -1 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1461: expect (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1462: rmattr oid 13 current snap is 154 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1462: done (4 left) 2024-03-20T08:06:29.815 INFO:tasks.rados.rados.0.smithi118.stdout:1463: read oid 8 snap -1 2024-03-20T08:06:29.816 INFO:tasks.rados.rados.0.smithi118.stdout:1463: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:06:29.816 INFO:tasks.rados.rados.0.smithi118.stdout:1464: snap_remove snap 144 2024-03-20T08:06:30.381 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:30 smithi118 systemd[1]: Stopping Ceph osd.2 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:06:30.385 INFO:tasks.rados.rados.0.smithi118.stdout:1464: done (5 left) 2024-03-20T08:06:30.385 INFO:tasks.rados.rados.0.smithi118.stdout:1465: snap_create 2024-03-20T08:06:30.385 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 795 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:06:30.385 INFO:tasks.rados.rados.0.smithi118.stdout:1463: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:06:30.389 INFO:tasks.rados.rados.0.smithi118.stdout:1460: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:06:30.558 INFO:tasks.rados.rados.0.smithi118.stdout:1461: expect (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:30.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:30 smithi196 ceph-mon[87080]: osdmap e362: 8 total, 8 up, 8 in 2024-03-20T08:06:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:30 smithi196 ceph-mon[87080]: pgmap v220: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 6.9 MiB/s rd, 188 KiB/s wr, 4 op/s 2024-03-20T08:06:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-mon[93121]: osdmap e362: 8 total, 8 up, 8 in 2024-03-20T08:06:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-mon[93121]: pgmap v220: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 6.9 MiB/s rd, 188 KiB/s wr, 4 op/s 2024-03-20T08:06:30.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:30.387+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:30.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:30.388+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:30.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[55341]: 2024-03-20T08:06:30.409+0000 7feeadc45700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:06:30.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[55341]: 2024-03-20T08:06:30.409+0000 7feeadc45700 -1 osd.2 363 *** Got signal Terminated *** 2024-03-20T08:06:30.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[55341]: 2024-03-20T08:06:30.409+0000 7feeadc45700 -1 osd.2 363 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:06:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-mon[95692]: osdmap e362: 8 total, 8 up, 8 in 2024-03-20T08:06:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:30 smithi118 ceph-mon[95692]: pgmap v220: 121 pgs: 1 active+clean+snaptrim, 2 active+clean+snaptrim_error, 118 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 6.9 MiB/s rd, 188 KiB/s wr, 4 op/s 2024-03-20T08:06:30.991 INFO:tasks.rados.rados.0.smithi118.stdout:1459: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:06:30.991 INFO:tasks.rados.rados.0.smithi118.stdout:1459: finishing copy_from to smithi11881747-12 2024-03-20T08:06:30.991 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 7026 (ObjNum 399 snap 135 seq_num 399) dirty exists 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1458: done (5 left) 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1459: done (4 left) 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1460: done (3 left) 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1461: done (2 left) 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1463: done (1 left) 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1465: done (0 left) 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1466: copy_from oid 36 from oid 33 current snap is 155 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1467: rollback oid 44 current snap is 155 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 44 to 133 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 155 2024-03-20T08:06:31.396 INFO:tasks.rados.rados.0.smithi118.stdout:1468: seq_num 456 ranges {0=442368} 2024-03-20T08:06:31.397 INFO:tasks.rados.rados.0.smithi118.stdout:1468: writing smithi11881747-8 from 0 to 442368 tid 1 2024-03-20T08:06:31.397 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 155 2024-03-20T08:06:31.397 INFO:tasks.rados.rados.0.smithi118.stdout:1469: seq_num 457 ranges {1875968=458752} 2024-03-20T08:06:31.404 INFO:tasks.rados.rados.0.smithi118.stdout:1469: writing smithi11881747-9 from 1875968 to 2334720 tid 1 2024-03-20T08:06:31.404 INFO:tasks.rados.rados.0.smithi118.stdout:1467: finishing rollback tid 0 to smithi11881747-44 2024-03-20T08:06:31.404 INFO:tasks.rados.rados.0.smithi118.stdout:1470: setattr oid 29 current snap is 155 2024-03-20T08:06:31.405 INFO:tasks.rados.rados.0.smithi118.stdout:1467: finishing rollback tid 1 to smithi11881747-44 2024-03-20T08:06:31.405 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 810 (ObjNum 33 snap 0 seq_num 324624432) dirty dne 2024-03-20T08:06:31.405 INFO:tasks.rados.rados.0.smithi118.stdout:1467: done (4 left) 2024-03-20T08:06:31.406 INFO:tasks.rados.rados.0.smithi118.stdout:1471: snap_remove snap 119 2024-03-20T08:06:31.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:31 smithi118 ceph-mon[93121]: osdmap e363: 8 total, 8 up, 8 in 2024-03-20T08:06:31.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:31 smithi118 ceph-mon[93121]: osd.2 marked itself down and dead 2024-03-20T08:06:31.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:31 smithi118 ceph-mon[95692]: osdmap e363: 8 total, 8 up, 8 in 2024-03-20T08:06:31.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:31 smithi118 ceph-mon[95692]: osd.2 marked itself down and dead 2024-03-20T08:06:31.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:31.398+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:31.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:31.398+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:31.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:31.401+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:31.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:31 smithi196 ceph-mon[87080]: osdmap e363: 8 total, 8 up, 8 in 2024-03-20T08:06:31.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:31 smithi196 ceph-mon[87080]: osd.2 marked itself down and dead 2024-03-20T08:06:32.270 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:31 smithi118 podman[111318]: 2024-03-20 08:06:31.931327151 +0000 UTC m=+1.645242476 container died 209395cbab90e18f73a3ca2b1f42d2311720e0f82eabe02bb3c19fdd2492a8e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:06:32.387 INFO:tasks.rados.rados.0.smithi118.stdout:1471: done (4 left) 2024-03-20T08:06:32.387 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 155 2024-03-20T08:06:32.388 INFO:tasks.rados.rados.0.smithi118.stdout:1472: seq_num 458 ranges {1581056=475136} 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1472: writing smithi11881747-42 from 1581056 to 2056192 tid 1 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1468: finishing write tid 1 to smithi11881747-8 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1469: finishing write tid 1 to smithi11881747-9 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1469: finishing write tid 2 to smithi11881747-9 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1468: finishing write tid 2 to smithi11881747-8 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1469: finishing write tid 3 to smithi11881747-9 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1469: oid 9 updating version 0 to 999 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1469: oid 9 version 999 is already newer than 998 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 999 (ObjNum 457 snap 155 seq_num 457) dirty exists 2024-03-20T08:06:32.400 INFO:tasks.rados.rados.0.smithi118.stdout:1469: left oid 9 (ObjNum 457 snap 155 seq_num 457) 2024-03-20T08:06:32.401 INFO:tasks.rados.rados.0.smithi118.stdout:1468: finishing write tid 3 to smithi11881747-8 2024-03-20T08:06:32.401 INFO:tasks.rados.rados.0.smithi118.stdout:1468: oid 8 updating version 0 to 19202 2024-03-20T08:06:32.401 INFO:tasks.rados.rados.0.smithi118.stdout:1468: oid 8 updating version 19202 to 19203 2024-03-20T08:06:32.401 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 19203 (ObjNum 456 snap 155 seq_num 456) dirty exists 2024-03-20T08:06:32.401 INFO:tasks.rados.rados.0.smithi118.stdout:1468: left oid 8 (ObjNum 456 snap 155 seq_num 456) 2024-03-20T08:06:32.401 INFO:tasks.rados.rados.0.smithi118.stdout:1468: done (4 left) 2024-03-20T08:06:32.401 INFO:tasks.rados.rados.0.smithi118.stdout:1469: done (3 left) 2024-03-20T08:06:32.401 INFO:tasks.rados.rados.0.smithi118.stdout:1473: setattr oid 28 current snap is 155 2024-03-20T08:06:32.402 INFO:tasks.rados.rados.0.smithi118.stdout:1474: snap_remove snap 146 2024-03-20T08:06:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-mon[87080]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:06:32.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-mon[87080]: osdmap e364: 8 total, 7 up, 8 in 2024-03-20T08:06:32.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:32.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:32.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-mon[87080]: pgmap v223: 121 pgs: 7 peering, 1 active+clean+snaptrim, 11 stale+active+clean, 2 active+clean+snaptrim_error, 100 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 8.9 MiB/s rd, 296 KiB/s wr, 6 op/s 2024-03-20T08:06:32.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:32 smithi196 ceph-mon[87080]: osdmap e365: 8 total, 7 up, 8 in 2024-03-20T08:06:32.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:32.390+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:32.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:32.390+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[93121]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[93121]: osdmap e364: 8 total, 7 up, 8 in 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[93121]: pgmap v223: 121 pgs: 7 peering, 1 active+clean+snaptrim, 11 stale+active+clean, 2 active+clean+snaptrim_error, 100 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 8.9 MiB/s rd, 296 KiB/s wr, 6 op/s 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[93121]: osdmap e365: 8 total, 7 up, 8 in 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[95692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[95692]: osdmap e364: 8 total, 7 up, 8 in 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[95692]: pgmap v223: 121 pgs: 7 peering, 1 active+clean+snaptrim, 11 stale+active+clean, 2 active+clean+snaptrim_error, 100 active+clean; 402 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 8.9 MiB/s rd, 296 KiB/s wr, 6 op/s 2024-03-20T08:06:32.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:32 smithi118 ceph-mon[95692]: osdmap e365: 8 total, 7 up, 8 in 2024-03-20T08:06:33.407 INFO:tasks.rados.rados.0.smithi118.stdout:1474: done (4 left) 2024-03-20T08:06:33.407 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 155 2024-03-20T08:06:33.407 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1259 (ObjNum 423 snap 141 seq_num 423) dirty exists 2024-03-20T08:06:33.407 INFO:tasks.rados.rados.0.smithi118.stdout:1475: seq_num 459 ranges {7692288=491520} 2024-03-20T08:06:33.453 INFO:tasks.rados.rados.0.smithi118.stdout:1475: writing smithi11881747-41 from 7692288 to 8183808 tid 1 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1470: done (4 left) 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1476: copy_from oid 14 from oid 4 current snap is 155 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1472: finishing write tid 1 to smithi11881747-42 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1472: finishing write tid 2 to smithi11881747-42 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1472: finishing write tid 3 to smithi11881747-42 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1472: oid 42 updating version 0 to 1001 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1472: oid 42 version 1001 is already newer than 1000 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 1001 (ObjNum 458 snap 155 seq_num 458) dirty exists 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1472: left oid 42 (ObjNum 458 snap 155 seq_num 458) 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 1261 (ObjNum 446 snap 148 seq_num 446) dirty exists 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1466: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:06:33.454 INFO:tasks.rados.rados.0.smithi118.stdout:1466: finishing copy_from to smithi11881747-36 2024-03-20T08:06:33.455 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 804 (ObjNum 452 snap 153 seq_num 452) dirty exists 2024-03-20T08:06:33.455 INFO:tasks.rados.rados.0.smithi118.stdout:1466: done (4 left) 2024-03-20T08:06:33.455 INFO:tasks.rados.rados.0.smithi118.stdout:1472: done (3 left) 2024-03-20T08:06:33.455 INFO:tasks.rados.rados.0.smithi118.stdout:1473: done (2 left) 2024-03-20T08:06:33.455 INFO:tasks.rados.rados.0.smithi118.stdout:1477: setattr oid 25 current snap is 155 2024-03-20T08:06:33.455 INFO:tasks.rados.rados.0.smithi118.stdout:1478: rollback oid 46 current snap is 155 2024-03-20T08:06:33.455 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 151 2024-03-20T08:06:33.456 INFO:tasks.rados.rados.0.smithi118.stdout:1479: rollback oid 6 current snap is 155 2024-03-20T08:06:33.456 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 154 2024-03-20T08:06:33.456 INFO:tasks.rados.rados.0.smithi118.stdout:1480: delete oid 7 current snap is 155 2024-03-20T08:06:33.457 INFO:tasks.rados.rados.0.smithi118.stdout:1480: done (5 left) 2024-03-20T08:06:33.458 INFO:tasks.rados.rados.0.smithi118.stdout:1481: read oid 33 snap -1 2024-03-20T08:06:33.458 INFO:tasks.rados.rados.0.smithi118.stdout:1481: expect (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:33.458 INFO:tasks.rados.rados.0.smithi118.stdout:1482: delete oid 3 current snap is 155 2024-03-20T08:06:33.458 INFO:tasks.rados.rados.0.smithi118.stdout:1475: finishing write tid 1 to smithi11881747-41 2024-03-20T08:06:33.458 INFO:tasks.rados.rados.0.smithi118.stdout:1478: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:06:33.458 INFO:tasks.rados.rados.0.smithi118.stdout:1475: finishing write tid 2 to smithi11881747-41 2024-03-20T08:06:33.459 INFO:tasks.rados.rados.0.smithi118.stdout:1478: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:06:33.459 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 1002 (ObjNum 326 snap 106 seq_num 326) dirty exists 2024-03-20T08:06:33.459 INFO:tasks.rados.rados.0.smithi118.stdout:1478: finishing rollback tid 2 to smithi11881747-46 2024-03-20T08:06:33.460 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 798 (ObjNum 346 snap 112 seq_num 346) dirty exists 2024-03-20T08:06:33.461 INFO:tasks.rados.rados.0.smithi118.stdout:1475: finishing write tid 3 to smithi11881747-41 2024-03-20T08:06:33.461 INFO:tasks.rados.rados.0.smithi118.stdout:1475: oid 41 updating version 0 to 1263 2024-03-20T08:06:33.461 INFO:tasks.rados.rados.0.smithi118.stdout:1475: oid 41 updating version 1263 to 1264 2024-03-20T08:06:33.461 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1264 (ObjNum 459 snap 155 seq_num 459) dirty exists 2024-03-20T08:06:33.461 INFO:tasks.rados.rados.0.smithi118.stdout:1475: left oid 41 (ObjNum 459 snap 155 seq_num 459) 2024-03-20T08:06:33.462 INFO:tasks.rados.rados.0.smithi118.stdout:1479: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:06:33.462 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 780 (ObjNum 129 snap 0 seq_num 322683904) dirty dne 2024-03-20T08:06:33.473 INFO:tasks.rados.rados.0.smithi118.stdout:1476: finishing copy_from to smithi11881747-14 2024-03-20T08:06:33.473 INFO:tasks.rados.rados.0.smithi118.stdout:1476: got expected ENOENT (src dne) 2024-03-20T08:06:33.473 INFO:tasks.rados.rados.0.smithi118.stdout:1476: finishing copy_from racing read to smithi11881747-14 2024-03-20T08:06:33.481 INFO:tasks.rados.rados.0.smithi118.stdout:1475: done (6 left) 2024-03-20T08:06:33.481 INFO:tasks.rados.rados.0.smithi118.stdout:1476: done (5 left) 2024-03-20T08:06:33.481 INFO:tasks.rados.rados.0.smithi118.stdout:1477: done (4 left) 2024-03-20T08:06:33.481 INFO:tasks.rados.rados.0.smithi118.stdout:1478: done (3 left) 2024-03-20T08:06:33.482 INFO:tasks.rados.rados.0.smithi118.stdout:1479: done (2 left) 2024-03-20T08:06:33.482 INFO:tasks.rados.rados.0.smithi118.stdout:1482: done (1 left) 2024-03-20T08:06:33.482 INFO:tasks.rados.rados.0.smithi118.stdout:1483: snap_remove snap 154 2024-03-20T08:06:33.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:33.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:33.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-mon[93121]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-20T08:06:33.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:06:33] "GET /metrics HTTP/1.1" 200 36269 "" "Prometheus/2.43.0" 2024-03-20T08:06:33.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:33 smithi118 podman[111318]: 2024-03-20 08:06:33.446708109 +0000 UTC m=+3.160623455 container cleanup 209395cbab90e18f73a3ca2b1f42d2311720e0f82eabe02bb3c19fdd2492a8e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-78dbc53, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-20T08:06:33.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:33 smithi118 bash[111318]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2 2024-03-20T08:06:33.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:33.412+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:33.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:33.412+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:33.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:33.412+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:33.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:33.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:33.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:33 smithi118 ceph-mon[95692]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-20T08:06:33.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:33.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset cf=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:33.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:33 smithi196 ceph-mon[87080]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-20T08:06:33.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:33.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:34.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:33 smithi118 podman[111340]: 2024-03-20 08:06:33.848741238 +0000 UTC m=+1.920700008 container remove 209395cbab90e18f73a3ca2b1f42d2311720e0f82eabe02bb3c19fdd2492a8e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, RELEASE=reef-78dbc53, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-20T08:06:34.413 INFO:tasks.rados.rados.0.smithi118.stdout:1483: done (1 left) 2024-03-20T08:06:34.413 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 155 2024-03-20T08:06:34.414 INFO:tasks.rados.rados.0.smithi118.stdout:1481: expect (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:34.660 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:34.412+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:34.660 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:34.413+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:34.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:34 smithi118 podman[111431]: 2024-03-20 08:06:34.265989531 +0000 UTC m=+0.020588714 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:06:34.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:34 smithi118 podman[111431]: 2024-03-20 08:06:34.399790925 +0000 UTC m=+0.154390092 container create 09c97038a28a7f125d8fb4bb94d6414651bc5e18634dfd57de5889e8c7ff5b51 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS) 2024-03-20T08:06:34.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-mon[93121]: osdmap e366: 8 total, 7 up, 8 in 2024-03-20T08:06:34.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:34.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:34.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-mon[93121]: pgmap v226: 121 pgs: 8 active+undersized+degraded, 84 active+clean, 17 active+undersized, 2 active+clean+snaptrim_error, 2 stale+active+clean, 1 active+clean+snaptrim, 7 peering; 401 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 1.9 MiB/s wr, 22 op/s; 59/894 objects degraded (6.600%) 2024-03-20T08:06:34.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-mon[95692]: osdmap e366: 8 total, 7 up, 8 in 2024-03-20T08:06:34.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:34.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:34.661 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:34 smithi118 ceph-mon[95692]: pgmap v226: 121 pgs: 8 active+undersized+degraded, 84 active+clean, 17 active+undersized, 2 active+clean+snaptrim_error, 2 stale+active+clean, 1 active+clean+snaptrim, 7 peering; 401 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 1.9 MiB/s wr, 22 op/s; 59/894 objects degraded (6.600%) 2024-03-20T08:06:34.854 INFO:tasks.rados.rados.0.smithi118.stdout:1484: seq_num 462 ranges {0=540672} 2024-03-20T08:06:34.857 INFO:tasks.rados.rados.0.smithi118.stdout:1484: writing smithi11881747-3 from 0 to 540672 tid 1 2024-03-20T08:06:34.858 INFO:tasks.rados.rados.0.smithi118.stdout:1481: done (1 left) 2024-03-20T08:06:34.858 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 155 2024-03-20T08:06:34.858 INFO:tasks.rados.rados.0.smithi118.stdout:1485: seq_num 463 ranges {3391488=557056} 2024-03-20T08:06:34.879 INFO:tasks.rados.rados.0.smithi118.stdout:1485: writing smithi11881747-15 from 3391488 to 3948544 tid 1 2024-03-20T08:06:34.879 INFO:tasks.rados.rados.0.smithi118.stdout:1486: snap_remove snap 152 2024-03-20T08:06:34.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:34 smithi196 ceph-mon[87080]: osdmap e366: 8 total, 7 up, 8 in 2024-03-20T08:06:34.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:34.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:34.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:34 smithi196 ceph-mon[87080]: pgmap v226: 121 pgs: 8 active+undersized+degraded, 84 active+clean, 17 active+undersized, 2 active+clean+snaptrim_error, 2 stale+active+clean, 1 active+clean+snaptrim, 7 peering; 401 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 9.8 MiB/s rd, 1.9 MiB/s wr, 22 op/s; 59/894 objects degraded (6.600%) 2024-03-20T08:06:34.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:34 smithi118 podman[111431]: 2024-03-20 08:06:34.661334331 +0000 UTC m=+0.415933502 container init 09c97038a28a7f125d8fb4bb94d6414651bc5e18634dfd57de5889e8c7ff5b51 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-20T08:06:34.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:34 smithi118 podman[111431]: 2024-03-20 08:06:34.669583801 +0000 UTC m=+0.424182966 container start 09c97038a28a7f125d8fb4bb94d6414651bc5e18634dfd57de5889e8c7ff5b51 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-20T08:06:34.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:34 smithi118 podman[111431]: 2024-03-20 08:06:34.769783505 +0000 UTC m=+0.524382670 container attach 09c97038a28a7f125d8fb4bb94d6414651bc5e18634dfd57de5889e8c7ff5b51 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS) 2024-03-20T08:06:35.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:35 smithi118 podman[111431]: 2024-03-20 08:06:35.033716318 +0000 UTC m=+0.788315484 container died 09c97038a28a7f125d8fb4bb94d6414651bc5e18634dfd57de5889e8c7ff5b51 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-20T08:06:35.418 INFO:tasks.rados.rados.0.smithi118.stdout:1484: finishing write tid 1 to smithi11881747-3 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1484: finishing write tid 2 to smithi11881747-3 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1484: finishing write tid 3 to smithi11881747-3 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1484: oid 3 updating version 0 to 807 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1484: oid 3 version 807 is already newer than 806 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 807 (ObjNum 462 snap 155 seq_num 462) dirty exists 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1484: left oid 3 (ObjNum 462 snap 155 seq_num 462) 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1485: finishing write tid 1 to smithi11881747-15 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1485: finishing write tid 2 to smithi11881747-15 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1485: finishing write tid 3 to smithi11881747-15 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1485: oid 15 updating version 0 to 339 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1485: oid 15 version 339 is already newer than 338 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 339 (ObjNum 463 snap 155 seq_num 463) dirty exists 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1485: left oid 15 (ObjNum 463 snap 155 seq_num 463) 2024-03-20T08:06:35.419 INFO:tasks.rados.rados.0.smithi118.stdout:1484: done (2 left) 2024-03-20T08:06:35.420 INFO:tasks.rados.rados.0.smithi118.stdout:1485: done (1 left) 2024-03-20T08:06:35.420 INFO:tasks.rados.rados.0.smithi118.stdout:1486: done (0 left) 2024-03-20T08:06:35.420 INFO:tasks.rados.rados.0.smithi118.stdout:1487: read oid 33 snap -1 2024-03-20T08:06:35.420 INFO:tasks.rados.rados.0.smithi118.stdout:1487: expect (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:35.420 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 155 2024-03-20T08:06:35.420 INFO:tasks.rados.rados.0.smithi118.stdout:1488: seq_num 464 ranges {3407872=573440} 2024-03-20T08:06:35.442 INFO:tasks.rados.rados.0.smithi118.stdout:1488: writing smithi11881747-23 from 3407872 to 3981312 tid 1 2024-03-20T08:06:35.442 INFO:tasks.rados.rados.0.smithi118.stdout:1489: rollback oid 41 current snap is 155 2024-03-20T08:06:35.442 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 41 to 149 2024-03-20T08:06:35.442 INFO:tasks.rados.rados.0.smithi118.stdout:1490: read oid 29 snap -1 2024-03-20T08:06:35.442 INFO:tasks.rados.rados.0.smithi118.stdout:1490: expect (ObjNum 423 snap 141 seq_num 423) 2024-03-20T08:06:35.443 INFO:tasks.rados.rados.0.smithi118.stdout:1491: copy_from oid 42 from oid 25 current snap is 155 2024-03-20T08:06:35.443 INFO:tasks.rados.rados.0.smithi118.stdout:1492: rollback oid 31 current snap is 155 2024-03-20T08:06:35.443 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 141 2024-03-20T08:06:35.443 INFO:tasks.rados.rados.0.smithi118.stdout:1493: snap_remove snap 131 2024-03-20T08:06:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-mon[93121]: Health check failed: Degraded data redundancy: 59/894 objects degraded (6.600%), 8 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-mon[93121]: osdmap e367: 8 total, 7 up, 8 in 2024-03-20T08:06:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-mon[95692]: Health check failed: Degraded data redundancy: 59/894 objects degraded (6.600%), 8 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-mon[95692]: osdmap e367: 8 total, 7 up, 8 in 2024-03-20T08:06:35.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:35.420+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:35.674 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:35.420+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:35.674 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:35.420+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:35.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:35 smithi196 ceph-mon[87080]: Health check failed: Degraded data redundancy: 59/894 objects degraded (6.600%), 8 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:35.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:35.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:35.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:35 smithi196 ceph-mon[87080]: osdmap e367: 8 total, 7 up, 8 in 2024-03-20T08:06:36.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:35 smithi118 podman[111477]: 2024-03-20 08:06:35.716179157 +0000 UTC m=+0.677626121 container remove 09c97038a28a7f125d8fb4bb94d6414651bc5e18634dfd57de5889e8c7ff5b51 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-20T08:06:36.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:35 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-20T08:06:36.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:35 smithi118 systemd[1]: Stopped Ceph osd.2 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:06:36.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:35 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.2.service: Consumed 12.805s CPU time. 2024-03-20T08:06:36.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:35 smithi118 systemd[1]: Starting Ceph osd.2 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:06:36.424 INFO:tasks.rados.rados.0.smithi118.stdout:1493: done (6 left) 2024-03-20T08:06:36.425 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 155 2024-03-20T08:06:36.425 INFO:tasks.rados.rados.0.smithi118.stdout:1487: expect (ObjNum 452 snap 153 seq_num 452) 2024-03-20T08:06:36.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-mon[95692]: osdmap e368: 8 total, 7 up, 8 in 2024-03-20T08:06:36.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:36.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:36.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-mon[95692]: pgmap v229: 121 pgs: 9 active+undersized+degraded, 81 active+clean, 21 active+undersized, 2 active+clean+snaptrim_error, 1 active+clean+snaptrim, 7 peering; 409 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.0 MiB/s wr, 27 op/s; 70/906 objects degraded (7.726%) 2024-03-20T08:06:36.428 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:36 smithi118 podman[111586]: 2024-03-20 08:06:36.092171166 +0000 UTC m=+0.022798592 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:06:36.428 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:36 smithi118 podman[111586]: 2024-03-20 08:06:36.217990701 +0000 UTC m=+0.148618124 container create 6e964a94541392241e07b2e95bf617538ca79a824021aaf29dd854322b49f22e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:06:36.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-mon[93121]: osdmap e368: 8 total, 7 up, 8 in 2024-03-20T08:06:36.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:36.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:36.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-mon[93121]: pgmap v229: 121 pgs: 9 active+undersized+degraded, 81 active+clean, 21 active+undersized, 2 active+clean+snaptrim_error, 1 active+clean+snaptrim, 7 peering; 409 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.0 MiB/s wr, 27 op/s; 70/906 objects degraded (7.726%) 2024-03-20T08:06:36.686 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:36.426+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:36.686 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:36.427+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:36.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:36 smithi118 podman[111586]: 2024-03-20 08:06:36.491391145 +0000 UTC m=+0.422018568 container init 6e964a94541392241e07b2e95bf617538ca79a824021aaf29dd854322b49f22e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-20T08:06:36.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:36 smithi118 podman[111586]: 2024-03-20 08:06:36.497966276 +0000 UTC m=+0.428593701 container start 6e964a94541392241e07b2e95bf617538ca79a824021aaf29dd854322b49f22e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True) 2024-03-20T08:06:36.859 INFO:tasks.rados.rados.0.smithi118.stdout:1492: finishing rollback tid 0 to smithi11881747-31 2024-03-20T08:06:36.859 INFO:tasks.rados.rados.0.smithi118.stdout:1492: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:06:36.859 INFO:tasks.rados.rados.0.smithi118.stdout:1492: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:06:36.859 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 998 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:06:36.859 INFO:tasks.rados.rados.0.smithi118.stdout:1488: finishing write tid 1 to smithi11881747-23 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1488: finishing write tid 2 to smithi11881747-23 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1489: finishing rollback tid 0 to smithi11881747-41 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1489: finishing rollback tid 1 to smithi11881747-41 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1489: finishing rollback tid 2 to smithi11881747-41 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1270 (ObjNum 425 snap 141 seq_num 425) dirty exists 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1488: finishing write tid 3 to smithi11881747-23 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1488: oid 23 updating version 0 to 1266 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1488: oid 23 updating version 1266 to 1267 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 1267 (ObjNum 464 snap 155 seq_num 464) dirty exists 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1488: left oid 23 (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:06:36.860 INFO:tasks.rados.rados.0.smithi118.stdout:1494: seq_num 465 ranges {7110656=598016} 2024-03-20T08:06:36.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:36 smithi196 ceph-mon[87080]: osdmap e368: 8 total, 7 up, 8 in 2024-03-20T08:06:36.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:36.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:36.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:36 smithi196 ceph-mon[87080]: pgmap v229: 121 pgs: 9 active+undersized+degraded, 81 active+clean, 21 active+undersized, 2 active+clean+snaptrim_error, 1 active+clean+snaptrim, 7 peering; 409 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 7.7 MiB/s rd, 4.0 MiB/s wr, 27 op/s; 70/906 objects degraded (7.726%) 2024-03-20T08:06:36.902 INFO:tasks.rados.rados.0.smithi118.stdout:1494: writing smithi11881747-17 from 7110656 to 7708672 tid 1 2024-03-20T08:06:36.902 INFO:tasks.rados.rados.0.smithi118.stdout:1491: finishing copy_from racing read to smithi11881747-42 2024-03-20T08:06:36.902 INFO:tasks.rados.rados.0.smithi118.stdout:1491: finishing copy_from to smithi11881747-42 2024-03-20T08:06:36.902 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 1002 (ObjNum 326 snap 106 seq_num 326) dirty exists 2024-03-20T08:06:36.903 INFO:tasks.rados.rados.0.smithi118.stdout:1487: done (6 left) 2024-03-20T08:06:36.903 INFO:tasks.rados.rados.0.smithi118.stdout:1488: done (5 left) 2024-03-20T08:06:36.903 INFO:tasks.rados.rados.0.smithi118.stdout:1489: done (4 left) 2024-03-20T08:06:36.903 INFO:tasks.rados.rados.0.smithi118.stdout:1491: done (3 left) 2024-03-20T08:06:36.903 INFO:tasks.rados.rados.0.smithi118.stdout:1492: done (2 left) 2024-03-20T08:06:36.903 INFO:tasks.rados.rados.0.smithi118.stdout:1495: delete oid 45 current snap is 155 2024-03-20T08:06:36.903 INFO:tasks.rados.rados.0.smithi118.stdout:1490: expect (ObjNum 423 snap 141 seq_num 423) 2024-03-20T08:06:36.987 INFO:tasks.rados.rados.0.smithi118.stdout:1494: finishing write tid 1 to smithi11881747-17 2024-03-20T08:06:36.987 INFO:tasks.rados.rados.0.smithi118.stdout:1494: finishing write tid 2 to smithi11881747-17 2024-03-20T08:06:36.987 INFO:tasks.rados.rados.0.smithi118.stdout:1490: done (2 left) 2024-03-20T08:06:36.987 INFO:tasks.rados.rados.0.smithi118.stdout:1495: done (1 left) 2024-03-20T08:06:36.988 INFO:tasks.rados.rados.0.smithi118.stdout:1496: snap_remove snap 142 2024-03-20T08:06:37.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:36 smithi118 podman[111586]: 2024-03-20 08:06:36.687974951 +0000 UTC m=+0.618602393 container attach 6e964a94541392241e07b2e95bf617538ca79a824021aaf29dd854322b49f22e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-20T08:06:37.357 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate[111600]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-20T08:06:37.357 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate[111600]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-20T08:06:37.357 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 bash[111586]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 bash[111586]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 bash[111586]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 bash[111586]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 bash[111586]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 bash[111586]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 bash[111586]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate[111600]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate[111600]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate[111600]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate[111600]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-20T08:06:37.358 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate[111600]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-20T08:06:37.359 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 podman[111586]: 2024-03-20 08:06:37.036114574 +0000 UTC m=+0.966742015 container died 6e964a94541392241e07b2e95bf617538ca79a824021aaf29dd854322b49f22e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-20T08:06:37.430 INFO:tasks.rados.rados.0.smithi118.stdout:1496: done (1 left) 2024-03-20T08:06:37.430 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 155 2024-03-20T08:06:37.430 INFO:tasks.rados.rados.0.smithi118.stdout:1494: finishing write tid 3 to smithi11881747-17 2024-03-20T08:06:37.430 INFO:tasks.rados.rados.0.smithi118.stdout:1494: oid 17 updating version 0 to 7028 2024-03-20T08:06:37.431 INFO:tasks.rados.rados.0.smithi118.stdout:1494: oid 17 version 7028 is already newer than 7027 2024-03-20T08:06:37.431 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7028 (ObjNum 465 snap 155 seq_num 465) dirty exists 2024-03-20T08:06:37.431 INFO:tasks.rados.rados.0.smithi118.stdout:1494: left oid 17 (ObjNum 465 snap 155 seq_num 465) 2024-03-20T08:06:37.431 INFO:tasks.rados.rados.0.smithi118.stdout:1497: seq_num 467 ranges {2826240=630784} 2024-03-20T08:06:37.451 INFO:tasks.rados.rados.0.smithi118.stdout:1497: writing smithi11881747-25 from 2826240 to 3457024 tid 1 2024-03-20T08:06:37.451 INFO:tasks.rados.rados.0.smithi118.stdout:1494: done (1 left) 2024-03-20T08:06:37.451 INFO:tasks.rados.rados.0.smithi118.stdout:1498: read oid 23 snap -1 2024-03-20T08:06:37.451 INFO:tasks.rados.rados.0.smithi118.stdout:1498: expect (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:06:37.451 INFO:tasks.rados.rados.0.smithi118.stdout:1499: delete oid 16 current snap is 155 2024-03-20T08:06:37.454 INFO:tasks.rados.rados.0.smithi118.stdout:1497: finishing write tid 1 to smithi11881747-25 2024-03-20T08:06:37.455 INFO:tasks.rados.rados.0.smithi118.stdout:1497: finishing write tid 2 to smithi11881747-25 2024-03-20T08:06:37.457 INFO:tasks.rados.rados.0.smithi118.stdout:1497: finishing write tid 3 to smithi11881747-25 2024-03-20T08:06:37.457 INFO:tasks.rados.rados.0.smithi118.stdout:1497: oid 25 updating version 0 to 1003 2024-03-20T08:06:37.457 INFO:tasks.rados.rados.0.smithi118.stdout:1497: oid 25 updating version 1003 to 1004 2024-03-20T08:06:37.457 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 1004 (ObjNum 467 snap 155 seq_num 467) dirty exists 2024-03-20T08:06:37.457 INFO:tasks.rados.rados.0.smithi118.stdout:1497: left oid 25 (ObjNum 467 snap 155 seq_num 467) 2024-03-20T08:06:37.471 INFO:tasks.rados.rados.0.smithi118.stdout:1498: expect (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:06:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-mon[93121]: osdmap e369: 8 total, 7 up, 8 in 2024-03-20T08:06:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:37.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:37.437+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:37.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:37.437+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:37.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:37.437+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:37.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-mon[95692]: osdmap e369: 8 total, 7 up, 8 in 2024-03-20T08:06:37.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:37.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1497: done (2 left) 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1498: done (1 left) 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1499: done (0 left) 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1500: read oid 5 snap -1 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1500: expect (ObjNum 440 snap 148 seq_num 440) 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1501: read oid 40 snap -1 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1501: expect deleted 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1502: rmattr oid 41 current snap is 155 2024-03-20T08:06:37.709 INFO:tasks.rados.rados.0.smithi118.stdout:1502: done (2 left) 2024-03-20T08:06:37.710 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 155 2024-03-20T08:06:37.710 INFO:tasks.rados.rados.0.smithi118.stdout:1503: seq_num 469 ranges {4243456=663552} 2024-03-20T08:06:37.738 INFO:tasks.rados.rados.0.smithi118.stdout:1503: writing smithi11881747-35 from 4243456 to 4907008 tid 1 2024-03-20T08:06:37.738 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 155 2024-03-20T08:06:37.738 INFO:tasks.rados.rados.0.smithi118.stdout:1504: seq_num 470 ranges {2981888=679936} 2024-03-20T08:06:37.758 INFO:tasks.rados.rados.0.smithi118.stdout:1504: writing smithi11881747-46 from 2981888 to 3661824 tid 1 2024-03-20T08:06:37.759 INFO:tasks.rados.rados.0.smithi118.stdout:1505: snap_create 2024-03-20T08:06:37.759 INFO:tasks.rados.rados.0.smithi118.stdout:1500: expect (ObjNum 440 snap 148 seq_num 440) 2024-03-20T08:06:37.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:37 smithi196 ceph-mon[87080]: osdmap e369: 8 total, 7 up, 8 in 2024-03-20T08:06:37.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:37.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1503: finishing write tid 1 to smithi11881747-35 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1503: finishing write tid 2 to smithi11881747-35 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1503: finishing write tid 3 to smithi11881747-35 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1503: oid 35 updating version 0 to 799 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1503: oid 35 updating version 799 to 800 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 800 (ObjNum 469 snap 155 seq_num 469) dirty exists 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1503: left oid 35 (ObjNum 469 snap 155 seq_num 469) 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1504: finishing write tid 1 to smithi11881747-46 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1504: finishing write tid 2 to smithi11881747-46 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1504: finishing write tid 3 to smithi11881747-46 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1504: oid 46 updating version 0 to 801 2024-03-20T08:06:37.894 INFO:tasks.rados.rados.0.smithi118.stdout:1504: oid 46 updating version 801 to 802 2024-03-20T08:06:37.895 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 802 (ObjNum 470 snap 155 seq_num 470) dirty exists 2024-03-20T08:06:37.895 INFO:tasks.rados.rados.0.smithi118.stdout:1504: left oid 46 (ObjNum 470 snap 155 seq_num 470) 2024-03-20T08:06:37.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 podman[111742]: 2024-03-20 08:06:37.709348315 +0000 UTC m=+0.668025629 container remove 6e964a94541392241e07b2e95bf617538ca79a824021aaf29dd854322b49f22e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-20T08:06:37.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 podman[111810]: 2024-03-20 08:06:37.861553203 +0000 UTC m=+0.027378674 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:06:38.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:37 smithi118 podman[111810]: 2024-03-20 08:06:37.994084083 +0000 UTC m=+0.159909551 container create 21c6276337212b3e3fad51869278169ddb764661de05663b44580832f4f49948 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-20T08:06:38.435 INFO:tasks.rados.rados.0.smithi118.stdout:1500: done (4 left) 2024-03-20T08:06:38.435 INFO:tasks.rados.rados.0.smithi118.stdout:1501: done (3 left) 2024-03-20T08:06:38.435 INFO:tasks.rados.rados.0.smithi118.stdout:1503: done (2 left) 2024-03-20T08:06:38.435 INFO:tasks.rados.rados.0.smithi118.stdout:1504: done (1 left) 2024-03-20T08:06:38.435 INFO:tasks.rados.rados.0.smithi118.stdout:1505: done (0 left) 2024-03-20T08:06:38.435 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 156 2024-03-20T08:06:38.436 INFO:tasks.rados.rados.0.smithi118.stdout:1506: seq_num 471 ranges {2334720=696320} 2024-03-20T08:06:38.453 INFO:tasks.rados.rados.0.smithi118.stdout:1506: writing smithi11881747-9 from 2334720 to 3031040 tid 1 2024-03-20T08:06:38.453 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 156 2024-03-20T08:06:38.453 INFO:tasks.rados.rados.0.smithi118.stdout:1507: seq_num 472 ranges {5332992=712704} 2024-03-20T08:06:38.487 INFO:tasks.rados.rados.0.smithi118.stdout:1507: writing smithi11881747-10 from 5332992 to 6045696 tid 1 2024-03-20T08:06:38.488 INFO:tasks.rados.rados.0.smithi118.stdout:1508: copy_from oid 6 from oid 7 current snap is 156 2024-03-20T08:06:38.488 INFO:tasks.rados.rados.0.smithi118.stdout:1506: finishing write tid 1 to smithi11881747-9 2024-03-20T08:06:38.488 INFO:tasks.rados.rados.0.smithi118.stdout:1506: finishing write tid 2 to smithi11881747-9 2024-03-20T08:06:38.488 INFO:tasks.rados.rados.0.smithi118.stdout:1506: finishing write tid 3 to smithi11881747-9 2024-03-20T08:06:38.488 INFO:tasks.rados.rados.0.smithi118.stdout:1506: oid 9 updating version 0 to 1005 2024-03-20T08:06:38.489 INFO:tasks.rados.rados.0.smithi118.stdout:1506: oid 9 updating version 1005 to 1006 2024-03-20T08:06:38.489 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 1006 (ObjNum 471 snap 156 seq_num 471) dirty exists 2024-03-20T08:06:38.489 INFO:tasks.rados.rados.0.smithi118.stdout:1506: left oid 9 (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:06:38.489 INFO:tasks.rados.rados.0.smithi118.stdout:1506: done (2 left) 2024-03-20T08:06:38.489 INFO:tasks.rados.rados.0.smithi118.stdout:1509: rmattr oid 50 current snap is 156 2024-03-20T08:06:38.489 INFO:tasks.rados.rados.0.smithi118.stdout:1509: done (2 left) 2024-03-20T08:06:38.489 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 156 2024-03-20T08:06:38.489 INFO:tasks.rados.rados.0.smithi118.stdout:1510: seq_num 473 ranges {2957312=729088} 2024-03-20T08:06:38.507 INFO:tasks.rados.rados.0.smithi118.stdout:1510: writing smithi11881747-38 from 2957312 to 3686400 tid 1 2024-03-20T08:06:38.508 INFO:tasks.rados.rados.0.smithi118.stdout:1511: delete oid 18 current snap is 156 2024-03-20T08:06:38.508 INFO:tasks.rados.rados.0.smithi118.stdout:1508: finishing copy_from to smithi11881747-6 2024-03-20T08:06:38.508 INFO:tasks.rados.rados.0.smithi118.stdout:1508: got expected ENOENT (src dne) 2024-03-20T08:06:38.508 INFO:tasks.rados.rados.0.smithi118.stdout:1508: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:06:38.508 INFO:tasks.rados.rados.0.smithi118.stdout:1507: finishing write tid 1 to smithi11881747-10 2024-03-20T08:06:38.508 INFO:tasks.rados.rados.0.smithi118.stdout:1507: finishing write tid 2 to smithi11881747-10 2024-03-20T08:06:38.508 INFO:tasks.rados.rados.0.smithi118.stdout:1507: finishing write tid 3 to smithi11881747-10 2024-03-20T08:06:38.508 INFO:tasks.rados.rados.0.smithi118.stdout:1507: oid 10 updating version 0 to 1000 2024-03-20T08:06:38.509 INFO:tasks.rados.rados.0.smithi118.stdout:1507: oid 10 version 1000 is already newer than 999 2024-03-20T08:06:38.509 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 1000 (ObjNum 472 snap 156 seq_num 472) dirty exists 2024-03-20T08:06:38.509 INFO:tasks.rados.rados.0.smithi118.stdout:1507: left oid 10 (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:06:38.512 INFO:tasks.rados.rados.0.smithi118.stdout:1510: finishing write tid 1 to smithi11881747-38 2024-03-20T08:06:38.512 INFO:tasks.rados.rados.0.smithi118.stdout:1510: finishing write tid 2 to smithi11881747-38 2024-03-20T08:06:38.513 INFO:tasks.rados.rados.0.smithi118.stdout:1507: done (3 left) 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1508: done (2 left) 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1511: done (1 left) 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1512: read oid 11 snap -1 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1512: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1513: snap_create 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1510: finishing write tid 3 to smithi11881747-38 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1510: oid 38 updating version 0 to 7029 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1510: oid 38 version 7029 is already newer than 7028 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7029 (ObjNum 473 snap 156 seq_num 473) dirty exists 2024-03-20T08:06:38.514 INFO:tasks.rados.rados.0.smithi118.stdout:1510: left oid 38 (ObjNum 473 snap 156 seq_num 473) 2024-03-20T08:06:38.528 INFO:tasks.rados.rados.0.smithi118.stdout:1512: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:06:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:38 smithi196 ceph-mon[87080]: osdmap e370: 8 total, 7 up, 8 in 2024-03-20T08:06:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:38 smithi196 ceph-mon[87080]: pgmap v232: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 409 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 2.2 MiB/s wr, 7 op/s; 74/906 objects degraded (8.168%) 2024-03-20T08:06:38.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-mon[93121]: osdmap e370: 8 total, 7 up, 8 in 2024-03-20T08:06:38.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:38.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:38.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-mon[93121]: pgmap v232: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 409 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 2.2 MiB/s wr, 7 op/s; 74/906 objects degraded (8.168%) 2024-03-20T08:06:38.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:38.458+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:38.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:38.466+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:38.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:38 smithi118 podman[111810]: 2024-03-20 08:06:38.439573246 +0000 UTC m=+0.605398713 container init 21c6276337212b3e3fad51869278169ddb764661de05663b44580832f4f49948 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-20T08:06:38.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:38 smithi118 podman[111810]: 2024-03-20 08:06:38.446990606 +0000 UTC m=+0.612816065 container start 21c6276337212b3e3fad51869278169ddb764661de05663b44580832f4f49948 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-20T08:06:38.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:38 smithi118 bash[111810]: 21c6276337212b3e3fad51869278169ddb764661de05663b44580832f4f49948 2024-03-20T08:06:38.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:38 smithi118 systemd[1]: Started Ceph osd.2 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:06:38.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-mon[95692]: osdmap e370: 8 total, 7 up, 8 in 2024-03-20T08:06:38.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:38.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:38.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-mon[95692]: pgmap v232: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 409 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 2.2 MiB/s wr, 7 op/s; 74/906 objects degraded (8.168%) 2024-03-20T08:06:39.284 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:06:39.284 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:38.995+0000 7f833e2787c0 -1 Falling back to public interface 2024-03-20T08:06:39.441 INFO:tasks.rados.rados.0.smithi118.stdout:1510: done (2 left) 2024-03-20T08:06:39.441 INFO:tasks.rados.rados.0.smithi118.stdout:1512: done (1 left) 2024-03-20T08:06:39.441 INFO:tasks.rados.rados.0.smithi118.stdout:1513: done (0 left) 2024-03-20T08:06:39.441 INFO:tasks.rados.rados.0.smithi118.stdout:1514: delete oid 24 current snap is 157 2024-03-20T08:06:39.445 INFO:tasks.rados.rados.0.smithi118.stdout:1514: done (0 left) 2024-03-20T08:06:39.445 INFO:tasks.rados.rados.0.smithi118.stdout:1515: snap_create 2024-03-20T08:06:39.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:39.450+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:39.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:39.450+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:39.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:39.450+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[93121]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[93121]: osdmap e371: 8 total, 7 up, 8 in 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[93121]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:39.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[95692]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-20T08:06:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[95692]: osdmap e371: 8 total, 7 up, 8 in 2024-03-20T08:06:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:39 smithi118 ceph-mon[95692]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (6m) 37s ago 7m 26.8M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (7m) 119s ago 7m 86.9M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (8m) 37s ago 8m 15.0M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (8m) 119s ago 8m 19.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (3m) 119s ago 11m 449M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (3m) 37s ago 13m 504M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (2m) 37s ago 13m 49.4M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:06:39.722 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (2m) 119s ago 12m 26.6M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:06:39.723 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (2m) 37s ago 12m 43.5M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:06:39.723 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (6m) 37s ago 6m 14.4M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:06:39.723 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (6m) 119s ago 6m 15.8M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:06:39.723 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (78s) 37s ago 11m 168M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:06:39.723 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (42s) 37s ago 11m 14.9M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:06:39.725 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 starting - - - 2480M 2024-03-20T08:06:39.725 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (10m) 37s ago 10m 219M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 79c58e60b2c5 2024-03-20T08:06:39.725 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (9m) 119s ago 9m 181M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:06:39.725 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (9m) 119s ago 9m 133M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:06:39.725 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (9m) 119s ago 9m 117M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:06:39.725 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (8m) 119s ago 8m 115M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:06:39.725 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (3m) 119s ago 7m 56.2M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:06:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:39 smithi196 ceph-mon[87080]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-20T08:06:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:39 smithi196 ceph-mon[87080]: osdmap e371: 8 total, 7 up, 8 in 2024-03-20T08:06:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:39 smithi196 ceph-mon[87080]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:40.224 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:06:40.224 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:06:40.224 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:06:40.224 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 5, 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 7, 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 7 2024-03-20T08:06:40.225 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:06:40.226 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:06:40.447 INFO:tasks.rados.rados.0.smithi118.stdout:1515: done (0 left) 2024-03-20T08:06:40.447 INFO:tasks.rados.rados.0.smithi118.stdout:1516: snap_remove snap 143 2024-03-20T08:06:40.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:40 smithi196 ceph-mon[87080]: osdmap e372: 8 total, 7 up, 8 in 2024-03-20T08:06:40.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:40.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:40.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:40 smithi196 ceph-mon[87080]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:40.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:40 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 74/906 objects degraded (8.168%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:40.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:40 smithi196 ceph-mon[87080]: pgmap v235: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 413 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 10 op/s; 77/933 objects degraded (8.253%) 2024-03-20T08:06:40.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:40 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/2377030006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[93121]: osdmap e372: 8 total, 7 up, 8 in 2024-03-20T08:06:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[93121]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 74/906 objects degraded (8.168%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[93121]: pgmap v235: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 413 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 10 op/s; 77/933 objects degraded (8.253%) 2024-03-20T08:06:40.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/2377030006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:40.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[95692]: osdmap e372: 8 total, 7 up, 8 in 2024-03-20T08:06:40.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:40.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:40.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[95692]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:06:40.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 74/906 objects degraded (8.168%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:40.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[95692]: pgmap v235: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 413 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 8.4 MiB/s rd, 1.2 MiB/s wr, 10 op/s; 77/933 objects degraded (8.253%) 2024-03-20T08:06:40.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:40 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/2377030006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:41.455 INFO:tasks.rados.rados.0.smithi118.stdout:1516: done (0 left) 2024-03-20T08:06:41.455 INFO:tasks.rados.rados.0.smithi118.stdout:1517: snap_create 2024-03-20T08:06:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-mon[93121]: osdmap e373: 8 total, 7 up, 8 in 2024-03-20T08:06:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-mon[95692]: osdmap e373: 8 total, 7 up, 8 in 2024-03-20T08:06:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:41.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:41.355+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:41.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:41.356+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:41.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:41.459+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:41.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:41.459+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:41.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:41.459+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:41.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:41 smithi196 ceph-mon[87080]: osdmap e373: 8 total, 7 up, 8 in 2024-03-20T08:06:41.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:41.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:42.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:41.832+0000 7f833e2787c0 -1 osd.2 363 log_to_monitors true 2024-03-20T08:06:42.457 INFO:tasks.rados.rados.0.smithi118.stdout:1517: done (0 left) 2024-03-20T08:06:42.458 INFO:tasks.rados.rados.0.smithi118.stdout:1518: read oid 11 snap -1 2024-03-20T08:06:42.458 INFO:tasks.rados.rados.0.smithi118.stdout:1518: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:06:42.458 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 159 2024-03-20T08:06:42.458 INFO:tasks.rados.rados.0.smithi118.stdout:1519: seq_num 476 ranges {0=778240} 2024-03-20T08:06:42.462 INFO:tasks.rados.rados.0.smithi118.stdout:1519: writing smithi11881747-24 from 0 to 778240 tid 1 2024-03-20T08:06:42.463 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 159 2024-03-20T08:06:42.463 INFO:tasks.rados.rados.0.smithi118.stdout:1520: seq_num 477 ranges {0=794624} 2024-03-20T08:06:42.468 INFO:tasks.rados.rados.0.smithi118.stdout:1520: writing smithi11881747-32 from 0 to 794624 tid 1 2024-03-20T08:06:42.468 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 159 2024-03-20T08:06:42.468 INFO:tasks.rados.rados.0.smithi118.stdout:1521: seq_num 478 ranges {5382144=409600,5791744=401408} 2024-03-20T08:06:42.501 INFO:tasks.rados.rados.0.smithi118.stdout:1521: writing smithi11881747-36 from 5382144 to 5791744 tid 1 2024-03-20T08:06:42.504 INFO:tasks.rados.rados.0.smithi118.stdout:1521: writing smithi11881747-36 from 5791744 to 6193152 tid 2 2024-03-20T08:06:42.504 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 159 2024-03-20T08:06:42.504 INFO:tasks.rados.rados.0.smithi118.stdout:1522: seq_num 479 ranges {0=425984,425984=401408} 2024-03-20T08:06:42.506 INFO:tasks.rados.rados.0.smithi118.stdout:1522: writing smithi11881747-22 from 0 to 425984 tid 1 2024-03-20T08:06:42.508 INFO:tasks.rados.rados.0.smithi118.stdout:1522: writing smithi11881747-22 from 425984 to 827392 tid 2 2024-03-20T08:06:42.508 INFO:tasks.rados.rados.0.smithi118.stdout:1523: snap_remove snap 97 2024-03-20T08:06:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:42 smithi196 ceph-mon[87080]: osdmap e374: 8 total, 7 up, 8 in 2024-03-20T08:06:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:42.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:42 smithi196 ceph-mon[87080]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T08:06:42.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:42 smithi196 ceph-mon[87080]: pgmap v238: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 415 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 14 op/s; 78/939 objects degraded (8.307%) 2024-03-20T08:06:42.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:42.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:42.754 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:42.491+0000 7f832f6d0700 -1 osd.2 363 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T08:06:42.755 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:42.456+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:42.755 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:42.456+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:42.755 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[93121]: osdmap e374: 8 total, 7 up, 8 in 2024-03-20T08:06:42.755 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:42.755 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:42.755 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[93121]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T08:06:42.755 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[93121]: pgmap v238: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 415 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 14 op/s; 78/939 objects degraded (8.307%) 2024-03-20T08:06:42.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[95692]: osdmap e374: 8 total, 7 up, 8 in 2024-03-20T08:06:42.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:42.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:42.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[95692]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-20T08:06:42.756 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:42 smithi118 ceph-mon[95692]: pgmap v238: 121 pgs: 1 active+clean+snaptrim, 10 active+undersized+degraded, 81 active+clean, 27 active+undersized, 2 active+clean+snaptrim_error; 415 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 13 MiB/s rd, 1.5 MiB/s wr, 14 op/s; 78/939 objects degraded (8.307%) 2024-03-20T08:06:43.459 INFO:tasks.rados.rados.0.smithi118.stdout:1523: done (5 left) 2024-03-20T08:06:43.459 INFO:tasks.rados.rados.0.smithi118.stdout:1524: rollback oid 34 current snap is 159 2024-03-20T08:06:43.460 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 34 to 141 2024-03-20T08:06:43.460 INFO:tasks.rados.rados.0.smithi118.stdout:1525: snap_remove snap 149 2024-03-20T08:06:43.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:43.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:43 smithi196 ceph-mon[87080]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-20T08:06:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:43 smithi196 ceph-mon[87080]: osdmap e375: 8 total, 7 up, 8 in 2024-03-20T08:06:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:43 smithi196 ceph-mon[87080]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:06:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:06:43] "GET /metrics HTTP/1.1" 200 36123 "" "Prometheus/2.43.0" 2024-03-20T08:06:43.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[93121]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-20T08:06:43.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:43.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:43.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[93121]: osdmap e375: 8 total, 7 up, 8 in 2024-03-20T08:06:43.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[93121]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:06:43.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:43.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:43.460+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:43.460+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:43.464+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:43.480+0000 7f0ceec5d700 -1 osd.0 pg_epoch: 376 pg[5.2s1( v 376'779 (0'0,376'779] local-lis/les=300/301 n=24 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [6,0,5]p6(0) r=1 lpr=300 luod=0'0 crt=376'777 lcod 376'777 mlcod 376'777 active mbc={} ps=45] update_snap_map remove_oid 5:43274c5b:::smithi11881747-21:9e failed with -2 2024-03-20T08:06:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:43.481+0000 7f0cecc59700 -1 osd.0 pg_epoch: 376 pg[5.es2( v 376'695 (0'0,376'695] local-lis/les=300/301 n=20 ec=73/73 lis/c=300/300 les/c/f=301/301/0 sis=300) [4,5,0]p4(0) r=2 lpr=300 luod=0'0 crt=376'695 mlcod 376'695 active mbc={} ps=45] update_snap_map remove_oid 5:7513f0ba:::smithi11881747-50:9e failed with -2 2024-03-20T08:06:43.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:06:43.480+0000 7fc1fae3c700 -1 osd.1 pg_epoch: 376 pg[5.9s2( v 376'668 (0'0,376'668] local-lis/les=336/337 n=20 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [7,6,1]p7(0) r=2 lpr=336 luod=0'0 crt=376'668 mlcod 376'668 active mbc={} ps=45] update_snap_map remove_oid 5:95859c30:::smithi11881747-2:9e failed with -2 2024-03-20T08:06:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[95692]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-20T08:06:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset d6=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[95692]: osdmap e375: 8 total, 7 up, 8 in 2024-03-20T08:06:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[95692]: from='osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:06:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:44.466 INFO:tasks.rados.rados.0.smithi118.stdout:1525: done (6 left) 2024-03-20T08:06:44.466 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 159 2024-03-20T08:06:44.466 INFO:tasks.rados.rados.0.smithi118.stdout:1526: seq_num 480 ranges {4931584=442368,5373952=401408} 2024-03-20T08:06:44.491 INFO:tasks.rados.rados.0.smithi118.stdout:1526: writing smithi11881747-27 from 4931584 to 5373952 tid 1 2024-03-20T08:06:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:1526: writing smithi11881747-27 from 5373952 to 5775360 tid 2 2024-03-20T08:06:44.498 INFO:tasks.rados.rados.0.smithi118.stdout:1519: finishing write tid 1 to smithi11881747-24 2024-03-20T08:06:44.498 INFO:tasks.rados.rados.0.smithi118.stdout:1519: finishing write tid 2 to smithi11881747-24 2024-03-20T08:06:44.499 INFO:tasks.rados.rados.0.smithi118.stdout:1519: finishing write tid 3 to smithi11881747-24 2024-03-20T08:06:44.499 INFO:tasks.rados.rados.0.smithi118.stdout:1519: oid 24 updating version 0 to 812 2024-03-20T08:06:44.499 INFO:tasks.rados.rados.0.smithi118.stdout:1519: oid 24 updating version 812 to 813 2024-03-20T08:06:44.499 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 813 (ObjNum 476 snap 159 seq_num 476) dirty exists 2024-03-20T08:06:44.499 INFO:tasks.rados.rados.0.smithi118.stdout:1519: left oid 24 (ObjNum 476 snap 159 seq_num 476) 2024-03-20T08:06:44.499 INFO:tasks.rados.rados.0.smithi118.stdout:1518: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:06:44.699 INFO:tasks.rados.rados.0.smithi118.stdout:1520: finishing write tid 1 to smithi11881747-32 2024-03-20T08:06:44.700 INFO:tasks.rados.rados.0.smithi118.stdout:1518: done (6 left) 2024-03-20T08:06:44.700 INFO:tasks.rados.rados.0.smithi118.stdout:1519: done (5 left) 2024-03-20T08:06:44.700 INFO:tasks.rados.rados.0.smithi118.stdout:1527: read oid 31 snap 157 2024-03-20T08:06:44.700 INFO:tasks.rados.rados.0.smithi118.stdout:1527: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:06:44.700 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 159 2024-03-20T08:06:44.700 INFO:tasks.rados.rados.0.smithi118.stdout:1528: seq_num 481 ranges {7708672=458752,8167424=401408} 2024-03-20T08:06:44.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:06:44.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555] boot 2024-03-20T08:06:44.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: osdmap e376: 8 total, 8 up, 8 in 2024-03-20T08:06:44.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:44.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:06:44.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:44.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:44.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: pgmap v241: 121 pgs: 16 peering, 1 active+clean+snaptrim, 7 active+undersized+degraded, 81 active+clean, 14 active+undersized, 2 active+clean+snaptrim_error; 408 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 4.5 MiB/s rd, 716 KiB/s wr, 6 op/s; 70/900 objects degraded (7.778%) 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[93121]: osdmap e377: 8 total, 8 up, 8 in 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555] boot 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: osdmap e376: 8 total, 8 up, 8 in 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:44.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: pgmap v241: 121 pgs: 16 peering, 1 active+clean+snaptrim, 7 active+undersized+degraded, 81 active+clean, 14 active+undersized, 2 active+clean+snaptrim_error; 408 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 4.5 MiB/s rd, 716 KiB/s wr, 6 op/s; 70/900 objects degraded (7.778%) 2024-03-20T08:06:44.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-mon[95692]: osdmap e377: 8 total, 8 up, 8 in 2024-03-20T08:06:44.732 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:44.481+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:44.732 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:44.481+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:44.732 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.561+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.732 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.566+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.732 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.570+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.574+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.576+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.580+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.583+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.585+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.590+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.594+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.598+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.599+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.602+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.605+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.608+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.610+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.614+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.616+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.620+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.623+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.627+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.628+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.634+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.634+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.640+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.642+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.647+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.649+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.654+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.656+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.660+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.662+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.666+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.669+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.675+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.676+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.680+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.684+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.686+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.693+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.694+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.699+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.702+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.707+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.712+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.713+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.719+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.720+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.726+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.728+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.746 INFO:tasks.rados.rados.0.smithi118.stdout:1528: writing smithi11881747-17 from 7708672 to 8167424 tid 1 2024-03-20T08:06:44.749 INFO:tasks.rados.rados.0.smithi118.stdout:1528: writing smithi11881747-17 from 8167424 to 8568832 tid 2 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1529: copy_from oid 39 from oid 14 current snap is 159 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1520: finishing write tid 2 to smithi11881747-32 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1520: finishing write tid 3 to smithi11881747-32 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1520: oid 32 updating version 0 to 24872 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1520: oid 32 updating version 24872 to 24873 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 24873 (ObjNum 477 snap 159 seq_num 477) dirty exists 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1520: left oid 32 (ObjNum 477 snap 159 seq_num 477) 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1522: finishing write tid 1 to smithi11881747-22 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1522: finishing write tid 2 to smithi11881747-22 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1521: finishing write tid 1 to smithi11881747-36 2024-03-20T08:06:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:1522: finishing write tid 3 to smithi11881747-22 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1521: finishing write tid 2 to smithi11881747-36 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1521: finishing write tid 3 to smithi11881747-36 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1522: finishing write tid 4 to smithi11881747-22 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1522: oid 22 updating version 0 to 7032 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1522: oid 22 version 7032 is already newer than 7030 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1522: oid 22 version 7032 is already newer than 7031 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 7032 (ObjNum 479 snap 159 seq_num 479) dirty exists 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1522: left oid 22 (ObjNum 479 snap 159 seq_num 479) 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1521: finishing write tid 4 to smithi11881747-36 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1521: oid 36 updating version 0 to 7041 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1521: oid 36 version 7041 is already newer than 7040 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1521: oid 36 version 7041 is already newer than 7039 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 7041 (ObjNum 478 snap 159 seq_num 478) dirty exists 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1521: left oid 36 (ObjNum 478 snap 159 seq_num 478) 2024-03-20T08:06:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:1524: finishing rollback tid 1 to smithi11881747-34 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1524: finishing rollback tid 2 to smithi11881747-34 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 7043 (ObjNum 420 snap 139 seq_num 420) dirty exists 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1526: finishing write tid 1 to smithi11881747-27 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1526: finishing write tid 2 to smithi11881747-27 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1526: finishing write tid 3 to smithi11881747-27 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1526: finishing write tid 4 to smithi11881747-27 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1526: oid 27 updating version 0 to 7046 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1526: oid 27 version 7046 is already newer than 7044 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1526: oid 27 version 7046 is already newer than 7045 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7046 (ObjNum 480 snap 159 seq_num 480) dirty exists 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1526: left oid 27 (ObjNum 480 snap 159 seq_num 480) 2024-03-20T08:06:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:1527: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:06:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:1528: finishing write tid 1 to smithi11881747-17 2024-03-20T08:06:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:1520: done (7 left) 2024-03-20T08:06:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:1521: done (6 left) 2024-03-20T08:06:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:1522: done (5 left) 2024-03-20T08:06:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:1524: done (4 left) 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1526: done (3 left) 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1527: done (2 left) 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1530: rmattr oid 35 current snap is 159 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1530: done (2 left) 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1531: setattr oid 49 current snap is 159 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1528: finishing write tid 2 to smithi11881747-17 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1528: finishing write tid 3 to smithi11881747-17 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1528: finishing write tid 4 to smithi11881747-17 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1528: oid 17 updating version 0 to 7029 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1528: oid 17 updating version 7029 to 7030 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1528: oid 17 updating version 7030 to 7031 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7031 (ObjNum 481 snap 159 seq_num 481) dirty exists 2024-03-20T08:06:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:1528: left oid 17 (ObjNum 481 snap 159 seq_num 481) 2024-03-20T08:06:44.822 INFO:tasks.rados.rados.0.smithi118.stdout:1529: finishing copy_from racing read to smithi11881747-39 2024-03-20T08:06:44.822 INFO:tasks.rados.rados.0.smithi118.stdout:1529: finishing copy_from to smithi11881747-39 2024-03-20T08:06:44.822 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 787 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:06:44.822 INFO:tasks.rados.rados.0.smithi118.stdout:1528: done (2 left) 2024-03-20T08:06:44.823 INFO:tasks.rados.rados.0.smithi118.stdout:1529: done (1 left) 2024-03-20T08:06:44.823 INFO:tasks.rados.rados.0.smithi118.stdout:1532: rollback oid 48 current snap is 159 2024-03-20T08:06:44.823 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 48 to 130 2024-03-20T08:06:44.823 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 159 2024-03-20T08:06:44.823 INFO:tasks.rados.rados.0.smithi118.stdout:1533: seq_num 482 ranges {4907008=475136,5382144=401408} 2024-03-20T08:06:44.854 INFO:tasks.rados.rados.0.smithi118.stdout:1533: writing smithi11881747-35 from 4907008 to 5382144 tid 1 2024-03-20T08:06:44.856 INFO:tasks.rados.rados.0.smithi118.stdout:1533: writing smithi11881747-35 from 5382144 to 5783552 tid 2 2024-03-20T08:06:44.857 INFO:tasks.rados.rados.0.smithi118.stdout:1534: snap_remove snap 151 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: osd.2 [v2:172.21.15.118:6818/3195797555,v1:172.21.15.118:6819/3195797555] boot 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: osdmap e376: 8 total, 8 up, 8 in 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: pgmap v241: 121 pgs: 16 peering, 1 active+clean+snaptrim, 7 active+undersized+degraded, 81 active+clean, 14 active+undersized, 2 active+clean+snaptrim_error; 408 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 4.5 MiB/s rd, 716 KiB/s wr, 6 op/s; 70/900 objects degraded (7.778%) 2024-03-20T08:06:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:44 smithi196 ceph-mon[87080]: osdmap e377: 8 total, 8 up, 8 in 2024-03-20T08:06:44.980 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.733+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.980 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.737+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.980 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.737+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.980 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.743+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.980 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.744+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.980 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.749+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.752+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.759+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.761+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.768+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.769+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.776+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.776+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.784+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.785+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.791+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.792+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.796+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.797+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.802+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.803+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.809+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.810+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.814+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.816+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.819+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.821+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.828+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.834+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.835+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.838+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.841+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.843+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.847+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.847+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.853+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.854+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.858+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.860+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.864+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.865+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.868+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.869+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.872+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.874+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.877+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.879+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.881+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.884+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.886+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.890+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.890+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.895+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.895+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.900+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.900+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.905+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.906+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.911+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.911+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.915+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.916+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.920+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.921+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.925+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.926+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.929+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.931+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.934+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.936+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.939+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.942+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.944+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.946+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.949+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.952+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.954+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.959+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.962+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.964+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.968+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.969+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.974+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.988 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.974+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:44.988 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.980+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:44.988 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.980+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.236 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.987+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.987+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.993+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.993+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:44.999+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.238 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.005+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.238 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.005+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.238 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.010+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.238 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.011+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.238 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.015+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.238 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.016+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.019+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.020+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.024+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.025+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.028+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.030+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.033+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.036+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.037+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.041+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.041+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.045+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.241 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.045+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.241 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.049+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.241 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.049+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.241 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.053+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.241 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.241 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.057+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.058+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.061+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.062+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.065+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.066+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.069+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.243 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.070+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.243 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.074+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.243 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.075+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.243 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.079+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.243 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.079+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.243 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.083+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.085+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.088+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.089+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.093+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.095+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.098+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.100+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.104+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.105+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.109+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.110+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.115+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.115+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.120+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.125+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.126+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.129+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.130+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.135+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.140+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.140+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.145+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.145+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.150+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.151+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.155+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.155+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.160+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.161+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.165+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.166+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.170+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.172+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.175+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.177+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.180+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.183+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.185+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.188+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.190+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.193+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.195+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.197+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.199+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.202+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.204+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.209+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.210+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.215+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.215+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.220+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.220+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.225+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.225+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.230+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.234+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.235+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.488 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:45.488 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:45.488 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 78/939 objects degraded (8.307%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:45.489 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:45.489 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:45.489 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 78/939 objects degraded (8.307%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.241+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.242+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.246+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.247+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.252+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.253+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.258+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.258+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.263+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.263+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.269+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.269+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.275+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.275+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.280+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.280+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.286+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.286+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.290+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.291+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.295+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.297+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.299+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.302+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.305+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.308+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.310+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.315+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.318+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.320+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.324+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.325+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.330+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.330+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.335+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.336+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.341+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.342+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.346+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.349+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.351+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.354+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.356+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.359+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.361+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.365+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.366+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.370+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.370+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.374+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.376+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.380+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.381+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.386+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.386+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.390+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.392+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.396+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.397+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.401+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.402+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.407+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.407+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.412+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.412+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.417+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.417+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.422+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.423+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.427+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.428+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.432+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.433+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.436+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.438+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.441+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.443+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.495 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.446+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.449+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.452+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.454+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.457+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.459+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.462+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.464+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.466+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.470+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.471+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.475+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.477+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.496 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.481+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.483+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.486+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.497 INFO:tasks.rados.rados.0.smithi118.stdout:1534: done (3 left) 2024-03-20T08:06:45.497 INFO:tasks.rados.rados.0.smithi118.stdout:1535: snap_remove snap 133 2024-03-20T08:06:45.737 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:45.500+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:45.737 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:45.500+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:45.738 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:45.500+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:45.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.498+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.500+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.507+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.508+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.512+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.514+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.518+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.518+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.523+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.524+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.528+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.528+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.533+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.534+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.538+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.538+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.543+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.544+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.548+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.550+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.554+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.555+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.558+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.560+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.563+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.566+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.569+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.575+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.576+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.580+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.581+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.585+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.587+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.590+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.592+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.595+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.597+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.601+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.603+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.606+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.608+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.612+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.613+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.617+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.619+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.623+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.624+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.628+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.630+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.633+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.635+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.639+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.639+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.644+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.645+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.649+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.652+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.654+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.658+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.659+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.663+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.664+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.669+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.669+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.674+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.675+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.678+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.680+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.684+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.684+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.689+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.689+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.695+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.696+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.701+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.744 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.701+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.706+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.707+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.711+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.712+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.717+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.718+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.722+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.723+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.727+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.728+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.732+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.735+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.738+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:45.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:45.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:45 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 78/939 objects degraded (8.307%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:06:45.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.742+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.748+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.748+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.753+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.754+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.759+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.761+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.765+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.767+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.771+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.773+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.779+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.782+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.784+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.789+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.790+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.794+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.796+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.799+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.802+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.804+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.808+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.810+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.814+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.815+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.820+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.821+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.826+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.831+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.831+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.836+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.839+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.845+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.846+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.854+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.854+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.861+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.861+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.867+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.871+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.874+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.876+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.880+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.881+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.885+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.887+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.891+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.897+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.898+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.903+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.904+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.909+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.909+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.914+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.915+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.920+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.921+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.926+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.926+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.932+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.933+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.939+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.939+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.944+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.944+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.949+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.949+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.955+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.955+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.961+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.997 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.961+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.998 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.966+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.998 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.966+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.998 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.971+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.998 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.971+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.998 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.976+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.998 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.977+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:45.998 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.982+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:45.998 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.983+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.989+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.989+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.996+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:45.996+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.002+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.002+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.007+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.013+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.013+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.018+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.019+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.023+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.025+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.028+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.031+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.033+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.036+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.039+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.043+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.045+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.048+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.050+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.053+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.055+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.059+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.061+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.064+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.067+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.069+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.073+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.250 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.076+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.078+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.084+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.087+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.090+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.093+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.095+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.099+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.100+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.105+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.107+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.112+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.251 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.113+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.117+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.119+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.122+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.125+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.129+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.130+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.134+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.136+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.140+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.142+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.145+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.148+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.252 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.151+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.153+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.157+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.159+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.162+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.164+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.168+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.170+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.174+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.176+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.182+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.253 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.187+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.188+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.193+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.194+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.199+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.200+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.206+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.211+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.212+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.217+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.218+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.224+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.224+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.230+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.236+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.236+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.242+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.242+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.247+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.249+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.490 INFO:tasks.rados.rados.0.smithi118.stdout:1535: done (3 left) 2024-03-20T08:06:46.490 INFO:tasks.rados.rados.0.smithi118.stdout:1536: delete oid 34 current snap is 159 2024-03-20T08:06:46.490 INFO:tasks.rados.rados.0.smithi118.stdout:1532: finishing rollback tid 0 to smithi11881747-48 2024-03-20T08:06:46.490 INFO:tasks.rados.rados.0.smithi118.stdout:1532: finishing rollback tid 1 to smithi11881747-48 2024-03-20T08:06:46.490 INFO:tasks.rados.rados.0.smithi118.stdout:1532: finishing rollback tid 2 to smithi11881747-48 2024-03-20T08:06:46.490 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 7035 (ObjNum 104 snap 23 seq_num 104) dirty exists 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 7047 (ObjNum 216 snap 69 seq_num 216) dirty exists 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:1533: finishing write tid 1 to smithi11881747-35 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:1533: finishing write tid 2 to smithi11881747-35 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:1533: finishing write tid 3 to smithi11881747-35 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:1533: finishing write tid 4 to smithi11881747-35 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:1533: oid 35 updating version 0 to 805 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:1533: oid 35 version 805 is already newer than 803 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:1533: oid 35 version 805 is already newer than 804 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 805 (ObjNum 482 snap 159 seq_num 482) dirty exists 2024-03-20T08:06:46.491 INFO:tasks.rados.rados.0.smithi118.stdout:1533: left oid 35 (ObjNum 482 snap 159 seq_num 482) 2024-03-20T08:06:46.501 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[95692]: osdmap e378: 8 total, 8 up, 8 in 2024-03-20T08:06:46.501 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:46.501 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:46.501 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:46.501 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:46.501 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[95692]: pgmap v244: 121 pgs: 16 peering, 1 active+clean+snaptrim, 7 active+undersized+degraded, 84 active+clean, 11 active+undersized, 2 active+clean+snaptrim_error; 416 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 2.9 MiB/s rd, 590 KiB/s wr, 4 op/s; 70/903 objects degraded (7.752%) 2024-03-20T08:06:46.501 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[93121]: osdmap e378: 8 total, 8 up, 8 in 2024-03-20T08:06:46.501 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:46.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:46.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:46.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:46.502 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-mon[93121]: pgmap v244: 121 pgs: 16 peering, 1 active+clean+snaptrim, 7 active+undersized+degraded, 84 active+clean, 11 active+undersized, 2 active+clean+snaptrim_error; 416 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 2.9 MiB/s rd, 590 KiB/s wr, 4 op/s; 70/903 objects degraded (7.752%) 2024-03-20T08:06:46.502 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.255+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.502 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.256+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.502 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.261+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.502 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.261+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.267+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.267+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.273+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.273+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.279+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.280+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.285+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.286+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.291+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.291+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.297+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.299+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.302+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.503 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.304+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.308+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.311+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.316+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.319+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.321+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.324+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.327+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.330+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.333+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.335+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.339+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.341+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.347+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.504 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.347+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.353+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.353+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.359+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.359+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.364+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.365+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.371+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.371+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.378+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.378+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.383+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.384+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.390+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.390+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.396+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.396+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.401+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.403+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.409+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.409+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.415+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.415+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.421+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.422+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.427+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.429+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.432+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.435+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.438+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.441+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.444+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.448+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.451+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.454+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.457+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.460+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.463+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.466+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.469+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.473+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.476+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.478+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.482+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.485+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.495+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.508 INFO:tasks.rados.rados.0.smithi118.stdout:1531: done (3 left) 2024-03-20T08:06:46.509 INFO:tasks.rados.rados.0.smithi118.stdout:1532: done (2 left) 2024-03-20T08:06:46.509 INFO:tasks.rados.rados.0.smithi118.stdout:1533: done (1 left) 2024-03-20T08:06:46.509 INFO:tasks.rados.rados.0.smithi118.stdout:1536: done (0 left) 2024-03-20T08:06:46.509 INFO:tasks.rados.rados.0.smithi118.stdout:1537: setattr oid 18 current snap is 159 2024-03-20T08:06:46.511 INFO:tasks.rados.rados.0.smithi118.stdout:1538: delete oid 22 current snap is 159 2024-03-20T08:06:46.518 INFO:tasks.rados.rados.0.smithi118.stdout:1538: done (1 left) 2024-03-20T08:06:46.518 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 159 2024-03-20T08:06:46.518 INFO:tasks.rados.rados.0.smithi118.stdout:1539: seq_num 485 ranges {0=532480,532480=401408} 2024-03-20T08:06:46.518 INFO:tasks.rados.rados.0.smithi118.stdout:1539: writing smithi11881747-6 from 0 to 532480 tid 1 2024-03-20T08:06:46.520 INFO:tasks.rados.rados.0.smithi118.stdout:1539: writing smithi11881747-6 from 532480 to 933888 tid 2 2024-03-20T08:06:46.521 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 159 2024-03-20T08:06:46.521 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 7048 (ObjNum 33 snap 0 seq_num 322299792) dirty exists 2024-03-20T08:06:46.521 INFO:tasks.rados.rados.0.smithi118.stdout:1540: seq_num 486 ranges {8175616=548864,8724480=401408} 2024-03-20T08:06:46.571 INFO:tasks.rados.rados.0.smithi118.stdout:1540: writing smithi11881747-20 from 8175616 to 8724480 tid 1 2024-03-20T08:06:46.574 INFO:tasks.rados.rados.0.smithi118.stdout:1540: writing smithi11881747-20 from 8724480 to 9125888 tid 2 2024-03-20T08:06:46.574 INFO:tasks.rados.rados.0.smithi118.stdout:1539: finishing write tid 1 to smithi11881747-6 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1539: finishing write tid 2 to smithi11881747-6 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1539: finishing write tid 3 to smithi11881747-6 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1539: finishing write tid 4 to smithi11881747-6 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1539: oid 6 updating version 0 to 806 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1539: oid 6 updating version 806 to 807 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1539: oid 6 updating version 807 to 808 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 808 (ObjNum 485 snap 159 seq_num 485) dirty exists 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1539: left oid 6 (ObjNum 485 snap 159 seq_num 485) 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1537: done (2 left) 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1539: done (1 left) 2024-03-20T08:06:46.575 INFO:tasks.rados.rados.0.smithi118.stdout:1541: snap_remove snap 138 2024-03-20T08:06:46.752 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:46.499+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:46.752 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:46.506+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.506+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.509+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.522+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.525+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.531+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.533+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.538+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.541+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.546+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.548+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.753 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.554+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.754 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.554+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.754 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.562+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.754 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.562+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.754 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.569+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.754 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.569+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.754 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.576+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.579+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.584+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.587+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.591+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.594+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.598+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.600+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.605+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.606+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.612+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.613+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.618+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.620+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.625+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.627+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.632+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.634+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.639+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.641+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.646+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.647+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.653+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.654+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.659+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.659+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.665+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.666+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.672+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.673+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.679+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.679+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.685+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.686+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.692+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.692+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.698+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.700+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.704+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.705+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.709+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.711+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.715+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.717+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.720+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.723+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.726+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.729+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.731+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.735+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.736+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.741+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.742+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.747+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.747+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.753+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:46.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.753+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:46.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:46 smithi196 ceph-mon[87080]: osdmap e378: 8 total, 8 up, 8 in 2024-03-20T08:06:46.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:46.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:46.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:46.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:46.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:46 smithi196 ceph-mon[87080]: pgmap v244: 121 pgs: 16 peering, 1 active+clean+snaptrim, 7 active+undersized+degraded, 84 active+clean, 11 active+undersized, 2 active+clean+snaptrim_error; 416 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 2.9 MiB/s rd, 590 KiB/s wr, 4 op/s; 70/903 objects degraded (7.752%) 2024-03-20T08:06:47.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.759+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.760+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.765+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.765+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.770+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.771+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.775+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.780+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.783+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.786+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.788+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.792+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.794+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.803+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.804+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.006 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.809+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.007 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.810+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.007 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.815+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.007 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.815+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.007 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.820+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.820+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.826+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.831+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.833+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.836+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.838+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.842+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.845+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.848+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.850+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.854+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.856+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.861+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.862+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.866+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.868+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.871+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.874+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.877+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.880+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.882+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.886+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.887+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.892+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.893+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.897+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.899+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.903+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.904+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.909+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.909+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.915+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.915+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.920+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.921+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.926+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.926+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.931+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.932+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.937+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.938+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.942+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.943+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.948+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.950+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.956+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.962+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.011 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.965+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.968+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.972+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.973+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.978+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.979+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.983+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.984+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.989+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.990+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.995+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.996+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:46.999+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.002+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.012 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.005+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.009+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.011+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.015+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.017+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.021+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.022+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.026+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.029+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.031+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.034+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.037+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.040+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.046+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.049+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.052+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.055+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.058+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.061+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.064+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.067+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.069+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.072+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.075+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.079+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.083+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.086+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.090+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.093+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.095+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.100+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.104+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.108+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.113+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.115+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.119+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.122+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.125+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.129+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.132+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.137+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.140+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.144+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.146+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.150+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.153+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.158+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.160+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.164+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.166+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.170+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.172+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.177+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.178+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.183+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.184+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.189+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.189+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.194+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.195+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.200+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.201+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.206+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.207+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.212+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.213+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.218+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.219+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.224+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.226+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.230+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.232+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.235+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.237+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.241+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.266 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.243+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.266 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.247+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.266 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.250+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.266 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.253+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.266 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.256+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.266 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.259+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.503 INFO:tasks.rados.rados.0.smithi118.stdout:1541: done (1 left) 2024-03-20T08:06:47.503 INFO:tasks.rados.rados.0.smithi118.stdout:1542: snap_remove snap 130 2024-03-20T08:06:47.511 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-mon[93121]: osdmap e379: 8 total, 8 up, 8 in 2024-03-20T08:06:47.511 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:47.511 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:47.511 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-mon[95692]: osdmap e379: 8 total, 8 up, 8 in 2024-03-20T08:06:47.511 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:47.511 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:47.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.263+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.266+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.269+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.273+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.274+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.279+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.281+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.285+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.287+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.291+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.294+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.297+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.300+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.303+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.306+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.309+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.313+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.316+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.319+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.322+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.325+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.328+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.331+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.333+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.336+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.339+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.342+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.345+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.352+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.353+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.358+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.359+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.364+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.366+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.370+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.372+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.377+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.379+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.383+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.387+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.391+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.396+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.398+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.404+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.406+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.410+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.414+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.418+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.420+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.424+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.426+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.431+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.431+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.437+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.438+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.443+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.445+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.449+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.451+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.455+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.457+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.460+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.464+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.467+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.470+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.473+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.476+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.479+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.483+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.487+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.489+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.493+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.496+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.501+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.507+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.762 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:47.509+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:47.762 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:47.509+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:47.762 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:47.510+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:47.763 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.517+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.763 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.522+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.763 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.526+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.763 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.530+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.763 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.535+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.763 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.536+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.541+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.543+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.548+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.549+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.554+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.556+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.561+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.564+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.567+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.570+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.765 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.573+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.577+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.579+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.583+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.586+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.589+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.591+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.596+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.598+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.602+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.604+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.608+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.610+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.615+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.616+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.622+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.623+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.628+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.629+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.633+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.635+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.640+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.642+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.647+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.647+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.653+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.655+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.660+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.661+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.667+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.668+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.673+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.675+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.679+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.681+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.685+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.688+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.692+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.694+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.699+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.700+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.705+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.708+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.711+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.714+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.718+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.720+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.724+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.727+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.731+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.733+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.737+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.740+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.745+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.747+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.752+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.755+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.759+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:47.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.762+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:47 smithi196 ceph-mon[87080]: osdmap e379: 8 total, 8 up, 8 in 2024-03-20T08:06:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:48.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.768+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.771+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.774+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.781+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.783+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.788+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.790+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.795+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.796+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.801+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.803+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.809+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.811+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.817+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.817+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.017 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.824+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.824+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.830+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.830+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.836+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.837+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.843+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.844+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.849+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.851+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.855+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.859+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.861+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.867+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.872+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.875+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.880+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.883+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.889+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.891+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.897+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.019 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.899+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.905+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.907+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.913+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.916+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.921+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.924+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.929+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.931+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.936+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.937+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.943+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.944+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.950+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.950+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.020 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.957+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.963+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.964+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.969+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.971+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.976+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.977+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.982+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.984+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.989+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.991+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.021 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.996+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.022 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:47.998+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.022 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.002+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.022 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.022 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.009+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.022 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.011+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.022 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.016+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.022 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.019+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.270 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.023+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.270 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.026+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.270 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.029+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.270 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.033+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.270 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.035+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.270 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.039+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.270 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.042+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.047+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.048+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.056+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.060+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.063+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.066+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.070+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.271 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.073+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.077+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.080+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.084+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.086+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.091+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.092+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.097+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.099+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.104+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.272 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.110+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.112+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.117+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.120+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.124+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.127+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.130+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.133+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.138+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.140+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.144+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.147+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.150+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.154+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.157+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.160+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.164+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.167+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.171+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.174+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.178+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.181+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.184+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.188+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.192+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.198+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.201+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.205+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.212+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.215+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.218+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.222+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.225+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.228+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.232+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.235+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.239+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.243+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.245+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.249+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.252+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.256+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.258+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.263+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.265+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.270+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.513 INFO:tasks.rados.rados.0.smithi118.stdout:1542: done (1 left) 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1543: snap_create 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1540: finishing write tid 1 to smithi11881747-20 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1540: finishing write tid 2 to smithi11881747-20 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1540: finishing write tid 3 to smithi11881747-20 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1540: finishing write tid 4 to smithi11881747-20 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1540: oid 20 updating version 0 to 7049 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1540: oid 20 updating version 7049 to 7050 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1540: oid 20 updating version 7050 to 7051 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 7051 (ObjNum 486 snap 159 seq_num 486) dirty exists 2024-03-20T08:06:48.514 INFO:tasks.rados.rados.0.smithi118.stdout:1540: left oid 20 (ObjNum 486 snap 159 seq_num 486) 2024-03-20T08:06:48.519 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:48.516+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:48.519 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-mon[93121]: osdmap e380: 8 total, 8 up, 8 in 2024-03-20T08:06:48.519 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:48.520 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:48.520 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-mon[93121]: pgmap v247: 121 pgs: 3 active+clean+snaptrim_wait, 16 peering, 3 active+clean+snaptrim, 97 active+clean, 2 active+clean+snaptrim_error; 417 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 392 KiB/s wr, 1 op/s; 9.8 MiB/s, 3 objects/s recovering 2024-03-20T08:06:48.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-mon[95692]: osdmap e380: 8 total, 8 up, 8 in 2024-03-20T08:06:48.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:48.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:48.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-mon[95692]: pgmap v247: 121 pgs: 3 active+clean+snaptrim_wait, 16 peering, 3 active+clean+snaptrim, 97 active+clean, 2 active+clean+snaptrim_error; 417 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 392 KiB/s wr, 1 op/s; 9.8 MiB/s, 3 objects/s recovering 2024-03-20T08:06:48.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.274+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.277+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.281+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.285+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.289+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.297+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.301+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.304+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.309+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.313+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.316+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.321+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.325+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.328+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.331+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.335+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.338+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.345+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.347+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.352+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.355+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.361+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.365+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.369+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.373+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.376+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.381+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.523 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.386+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.523 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.389+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.523 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.394+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.523 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.397+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.523 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.401+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.523 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.404+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.523 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.408+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.523 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.411+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.416+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.419+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.423+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.425+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.430+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.432+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.437+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.438+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.443+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.445+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.451+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.452+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.458+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.524 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.459+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.465+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.466+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.472+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.473+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.479+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.481+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.486+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.489+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.493+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.496+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.500+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.504+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.509+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.521+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.771 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:48.518+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:48.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.527+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.531+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.535+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.539+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.542+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.547+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.550+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.555+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.559+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.562+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.566+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.569+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.572+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.577+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.773 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.581+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.585+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.589+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.592+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.597+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.601+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.606+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.609+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.615+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.617+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.624+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.625+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.632+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.774 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.634+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.641+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.643+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.651+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.652+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.659+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.660+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.667+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.668+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.675+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.676+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.683+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.683+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.692+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.693+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.702+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.703+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.710+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.710+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.718+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.718+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.726+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.727+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.734+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.734+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.742+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.742+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.749+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.749+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.756+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.756+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.763+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:48.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.764+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:48.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:48 smithi196 ceph-mon[87080]: osdmap e380: 8 total, 8 up, 8 in 2024-03-20T08:06:48.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:48.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:48.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:48 smithi196 ceph-mon[87080]: pgmap v247: 121 pgs: 3 active+clean+snaptrim_wait, 16 peering, 3 active+clean+snaptrim, 97 active+clean, 2 active+clean+snaptrim_error; 417 MiB data, 2.5 GiB used, 713 GiB / 715 GiB avail; 392 KiB/s wr, 1 op/s; 9.8 MiB/s, 3 objects/s recovering 2024-03-20T08:06:49.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.771+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.773+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.779+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.780+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.787+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.788+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.795+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.797+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.801+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.805+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.809+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.813+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.817+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.821+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.825+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.830+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.832+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.837+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.839+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.845+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.845+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.852+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.852+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.859+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.860+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.870+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.871+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.877+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.879+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.884+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.885+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.891+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.899+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.900+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.906+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.909+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.915+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.918+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.925+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.927+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.933+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.935+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.942+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.943+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.951+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.951+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.959+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.960+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.967+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.971+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.976+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.980+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.986+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.987+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.994+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:48.996+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.002+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.010+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.014+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.017+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.023+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.026+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.034+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.034+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.042+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.050+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.277 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.051+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.058+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.059+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.065+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.068+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.074+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.076+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.083+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.085+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.091+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.096+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.100+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.278 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.104+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.109+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.112+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.117+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.121+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.126+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.131+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.135+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.140+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.144+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.148+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.153+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.156+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.162+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.165+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.279 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.170+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.175+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.178+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.184+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.187+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.192+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.195+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.201+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.205+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.209+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.214+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.218+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.223+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.225+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.232+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.233+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.241+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.242+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.249+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.251+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.257+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.259+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.282 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.265+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.282 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.268+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.282 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.276+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.512 INFO:tasks.rados.rados.0.smithi118.stdout:1540: done (1 left) 2024-03-20T08:06:49.512 INFO:tasks.rados.rados.0.smithi118.stdout:1543: done (0 left) 2024-03-20T08:06:49.512 INFO:tasks.rados.rados.0.smithi118.stdout:1544: read oid 50 snap -1 2024-03-20T08:06:49.512 INFO:tasks.rados.rados.0.smithi118.stdout:1544: expect deleted 2024-03-20T08:06:49.512 INFO:tasks.rados.rados.0.smithi118.stdout:1545: rollback oid 37 current snap is 160 2024-03-20T08:06:49.512 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 158 2024-03-20T08:06:49.512 INFO:tasks.rados.rados.0.smithi118.stdout:1546: delete oid 27 current snap is 160 2024-03-20T08:06:49.522 INFO:tasks.rados.rados.0.smithi118.stdout:1545: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:06:49.522 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 328 (ObjNum 129 snap 0 seq_num 322218400) dirty dne 2024-03-20T08:06:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:1544: done (2 left) 2024-03-20T08:06:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:1545: done (1 left) 2024-03-20T08:06:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:1546: done (0 left) 2024-03-20T08:06:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 160 2024-03-20T08:06:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:1547: seq_num 488 ranges {8568832=581632,9150464=401408} 2024-03-20T08:06:49.528 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:49.523+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:49.528 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:49.523+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:49.528 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:49.523+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:49.528 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.279+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.528 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.285+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.287+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.293+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.297+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.300+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.307+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.309+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.316+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.318+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.323+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.327+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.331+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.336+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.529 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.339+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.345+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.349+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.354+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.358+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.363+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.367+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.371+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.375+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.380+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.386+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.391+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.394+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.400+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.403+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.409+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.411+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.417+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.420+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.426+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.429+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.434+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.439+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.443+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.449+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.452+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.458+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.461+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.467+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.469+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.476+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.479+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.485+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.488+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.494+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.499+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.504+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.508+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.533 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-mon[93121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 70/903 objects degraded (7.752%), 7 pgs degraded) 2024-03-20T08:06:49.533 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-mon[93121]: osdmap e381: 8 total, 8 up, 8 in 2024-03-20T08:06:49.533 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:49.533 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:49.534 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.516+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.534 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.522+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.534 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.528+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.534 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-mon[95692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 70/903 objects degraded (7.752%), 7 pgs degraded) 2024-03-20T08:06:49.534 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-mon[95692]: osdmap e381: 8 total, 8 up, 8 in 2024-03-20T08:06:49.534 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:49.534 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:49.577 INFO:tasks.rados.rados.0.smithi118.stdout:1547: writing smithi11881747-17 from 8568832 to 9150464 tid 1 2024-03-20T08:06:49.582 INFO:tasks.rados.rados.0.smithi118.stdout:1547: writing smithi11881747-17 from 9150464 to 9551872 tid 2 2024-03-20T08:06:49.582 INFO:tasks.rados.rados.0.smithi118.stdout:1548: rollback oid 19 current snap is 160 2024-03-20T08:06:49.583 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 19 to 137 2024-03-20T08:06:49.583 INFO:tasks.rados.rados.0.smithi118.stdout:1549: copy_from oid 30 from oid 11 current snap is 160 2024-03-20T08:06:49.583 INFO:tasks.rados.rados.0.smithi118.stdout:1550: read oid 6 snap -1 2024-03-20T08:06:49.583 INFO:tasks.rados.rados.0.smithi118.stdout:1550: expect (ObjNum 485 snap 159 seq_num 485) 2024-03-20T08:06:49.583 INFO:tasks.rados.rados.0.smithi118.stdout:1551: rmattr oid 1 current snap is 160 2024-03-20T08:06:49.583 INFO:tasks.rados.rados.0.smithi118.stdout:1551: done (4 left) 2024-03-20T08:06:49.584 INFO:tasks.rados.rados.0.smithi118.stdout:1552: read oid 21 snap -1 2024-03-20T08:06:49.584 INFO:tasks.rados.rados.0.smithi118.stdout:1552: expect (ObjNum 443 snap 148 seq_num 443) 2024-03-20T08:06:49.584 INFO:tasks.rados.rados.0.smithi118.stdout:1553: rmattr oid 25 current snap is 160 2024-03-20T08:06:49.584 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 160 2024-03-20T08:06:49.584 INFO:tasks.rados.rados.0.smithi118.stdout:1554: seq_num 489 ranges {0=598016,598016=401408} 2024-03-20T08:06:49.585 INFO:tasks.rados.rados.0.smithi118.stdout:1554: writing smithi11881747-44 from 0 to 598016 tid 1 2024-03-20T08:06:49.590 INFO:tasks.rados.rados.0.smithi118.stdout:1554: writing smithi11881747-44 from 598016 to 999424 tid 2 2024-03-20T08:06:49.591 INFO:tasks.rados.rados.0.smithi118.stdout:1555: read oid 47 snap -1 2024-03-20T08:06:49.591 INFO:tasks.rados.rados.0.smithi118.stdout:1555: expect deleted 2024-03-20T08:06:49.591 INFO:tasks.rados.rados.0.smithi118.stdout:1556: rollback oid 41 current snap is 160 2024-03-20T08:06:49.591 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 41 to 159 2024-03-20T08:06:49.591 INFO:tasks.rados.rados.0.smithi118.stdout:1557: snap_remove snap 147 2024-03-20T08:06:49.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.536+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.538+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.545+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.549+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.556+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.560+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.565+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.569+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.574+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.579+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.585+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.593+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.605+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.607+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.614+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.617+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.623+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.626+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.632+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.635+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.641+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.644+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.650+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.654+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.659+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.664+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.669+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.673+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.678+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.682+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.687+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.691+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.696+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.701+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.706+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.709+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.786 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.715+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.719+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.723+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.728+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.732+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.737+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.741+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.746+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.750+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.756+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.761+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.765+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.771+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.787 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.773+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.788 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.780+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:49.788 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.783+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:49.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:49 smithi196 ceph-mon[87080]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 70/903 objects degraded (7.752%), 7 pgs degraded) 2024-03-20T08:06:49.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:49 smithi196 ceph-mon[87080]: osdmap e381: 8 total, 8 up, 8 in 2024-03-20T08:06:49.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:49.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:50.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.790+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.790+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.799+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.800+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.808+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.809+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.821+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.822+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.831+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.831+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.840+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.840+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.849+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.849+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.859+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.859+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.038 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.868+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.868+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.877+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.877+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.886+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.887+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.897+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.897+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.906+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.907+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.915+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.915+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.924+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.925+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.933+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.933+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.942+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.951+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.952+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.961+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.961+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.970+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.970+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.980+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.980+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.989+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.990+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.998+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:49.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.008+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.009+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.018+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.020+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.028+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.031+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.037+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.291 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.043+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.291 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.048+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.291 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.052+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.291 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.057+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.291 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.061+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.291 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.066+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.070+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.078+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.083+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.086+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.091+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.094+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.099+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.102+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.108+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.110+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.116+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.118+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.123+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.127+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.131+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.135+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.139+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.144+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.149+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.152+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.157+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.160+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.166+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.169+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.174+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.176+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.183+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.294 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.185+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.191+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.193+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.199+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.202+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.210+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.217+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.218+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.226+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.226+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.235+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.235+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.243+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.295 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.244+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.251+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.252+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.260+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.261+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.267+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.270+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.274+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.278+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.282+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.287+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.291+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.525 INFO:tasks.rados.rados.0.smithi118.stdout:1557: done (9 left) 2024-03-20T08:06:50.525 INFO:tasks.rados.rados.0.smithi118.stdout:1558: read oid 27 snap -1 2024-03-20T08:06:50.525 INFO:tasks.rados.rados.0.smithi118.stdout:1558: expect deleted 2024-03-20T08:06:50.525 INFO:tasks.rados.rados.0.smithi118.stdout:1559: delete oid 29 current snap is 160 2024-03-20T08:06:50.525 INFO:tasks.rados.rados.0.smithi118.stdout:1547: finishing write tid 1 to smithi11881747-17 2024-03-20T08:06:50.525 INFO:tasks.rados.rados.0.smithi118.stdout:1547: finishing write tid 2 to smithi11881747-17 2024-03-20T08:06:50.525 INFO:tasks.rados.rados.0.smithi118.stdout:1547: finishing write tid 3 to smithi11881747-17 2024-03-20T08:06:50.525 INFO:tasks.rados.rados.0.smithi118.stdout:1548: finishing rollback tid 0 to smithi11881747-19 2024-03-20T08:06:50.526 INFO:tasks.rados.rados.0.smithi118.stdout:1548: finishing rollback tid 1 to smithi11881747-19 2024-03-20T08:06:50.526 INFO:tasks.rados.rados.0.smithi118.stdout:1548: finishing rollback tid 2 to smithi11881747-19 2024-03-20T08:06:50.526 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 379 (ObjNum 407 snap 136 seq_num 407) dirty exists 2024-03-20T08:06:50.526 INFO:tasks.rados.rados.0.smithi118.stdout:1550: expect (ObjNum 485 snap 159 seq_num 485) 2024-03-20T08:06:50.543 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:50.521+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:50.543 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:50.522+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:50.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.299+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.304+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.308+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.312+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.316+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.321+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.325+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.330+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.335+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.339+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.344+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.347+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.352+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.357+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.360+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.365+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.368+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.374+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.377+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.383+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.387+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.393+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.397+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.402+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.405+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.410+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.414+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.419+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.423+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.427+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.431+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.437+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.441+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.445+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.449+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.453+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.457+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.462+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.466+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.470+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.474+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.479+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.482+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.488+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.491+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.498+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.499+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.507+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.547 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.508+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.548 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.515+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.548 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.519+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.548 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.531+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.548 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-mon[93121]: osdmap e382: 8 total, 8 up, 8 in 2024-03-20T08:06:50.548 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:50.548 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:50.548 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-mon[93121]: pgmap v250: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 413 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 15 op/s; 13 MiB/s, 4 objects/s recovering 2024-03-20T08:06:50.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-mon[95692]: osdmap e382: 8 total, 8 up, 8 in 2024-03-20T08:06:50.549 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:50.549 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:50.549 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-mon[95692]: pgmap v250: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 413 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 15 op/s; 13 MiB/s, 4 objects/s recovering 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1547: finishing write tid 4 to smithi11881747-17 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1547: oid 17 updating version 0 to 7032 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1547: oid 17 updating version 7032 to 7033 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1547: oid 17 updating version 7033 to 7034 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7034 (ObjNum 488 snap 160 seq_num 488) dirty exists 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1547: left oid 17 (ObjNum 488 snap 160 seq_num 488) 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1547: done (10 left) 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1548: done (9 left) 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1550: done (8 left) 2024-03-20T08:06:50.565 INFO:tasks.rados.rados.0.smithi118.stdout:1559: done (7 left) 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1560: snap_create 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 1007 (ObjNum 467 snap 155 seq_num 467) dirty exists 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1554: finishing write tid 1 to smithi11881747-44 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1554: finishing write tid 2 to smithi11881747-44 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1554: finishing write tid 3 to smithi11881747-44 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1554: finishing write tid 4 to smithi11881747-44 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1554: oid 44 updating version 0 to 815 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1554: oid 44 version 815 is already newer than 814 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1554: oid 44 updating version 815 to 816 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 816 (ObjNum 489 snap 160 seq_num 489) dirty exists 2024-03-20T08:06:50.566 INFO:tasks.rados.rados.0.smithi118.stdout:1554: left oid 44 (ObjNum 489 snap 160 seq_num 489) 2024-03-20T08:06:50.567 INFO:tasks.rados.rados.0.smithi118.stdout:1556: finishing rollback tid 0 to smithi11881747-41 2024-03-20T08:06:50.567 INFO:tasks.rados.rados.0.smithi118.stdout:1556: finishing rollback tid 1 to smithi11881747-41 2024-03-20T08:06:50.567 INFO:tasks.rados.rados.0.smithi118.stdout:1556: finishing rollback tid 2 to smithi11881747-41 2024-03-20T08:06:50.567 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1276 (ObjNum 425 snap 141 seq_num 425) dirty exists 2024-03-20T08:06:50.567 INFO:tasks.rados.rados.0.smithi118.stdout:1552: expect (ObjNum 443 snap 148 seq_num 443) 2024-03-20T08:06:50.698 INFO:tasks.rados.rados.0.smithi118.stdout:1549: finishing copy_from racing read to smithi11881747-30 2024-03-20T08:06:50.698 INFO:tasks.rados.rados.0.smithi118.stdout:1549: finishing copy_from to smithi11881747-30 2024-03-20T08:06:50.698 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 787 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:06:50.804 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.542+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.804 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.547+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.804 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.552+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.557+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.562+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.566+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.571+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.576+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.581+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.585+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.590+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.594+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.598+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.604+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.607+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.613+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.805 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.616+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.622+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.625+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.634+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.634+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.643+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.643+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.652+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.661+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.661+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.669+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.672+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.806 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.678+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.681+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.687+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.691+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.697+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.701+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.706+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.710+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.715+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.718+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.723+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.727+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.732+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.807 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.735+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.740+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.745+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.749+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.753+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.757+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.763+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.767+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.772+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.776+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.780+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.783+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.790+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.793+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.808 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.802+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:50.809 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.803+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:50.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:50 smithi196 ceph-mon[87080]: osdmap e382: 8 total, 8 up, 8 in 2024-03-20T08:06:50.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:50.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:50.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:50 smithi196 ceph-mon[87080]: pgmap v250: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 413 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 2.2 MiB/s wr, 15 op/s; 13 MiB/s, 4 objects/s recovering 2024-03-20T08:06:51.060 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.813+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.060 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.816+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.824+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.826+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.834+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.836+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.843+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.845+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.852+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.853+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.861+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.861+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.869+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.871+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.061 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.878+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.880+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.888+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.891+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.899+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.901+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.909+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.912+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.919+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.923+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.929+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.934+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.938+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.944+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.062 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.948+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.954+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.957+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.963+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.965+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.972+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.975+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.981+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.984+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.991+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:50.994+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.000+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.003+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.009+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.012+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.018+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.021+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.028+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.030+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.037+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.040+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.048+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.049+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.058+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.059+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.313 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.070+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.313 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.071+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.313 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.079+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.082+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.089+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.092+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.099+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.103+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.108+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.113+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.119+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.123+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.128+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.134+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.140+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.143+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.314 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.151+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.154+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.162+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.164+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.172+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.175+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.182+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.186+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.193+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.196+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.203+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.205+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.213+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.215+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.224+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.226+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.234+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.236+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.243+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.246+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.254+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.257+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.264+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.267+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.275+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.277+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.285+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.287+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.294+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.299+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.304+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.311+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1549: done (7 left) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1552: done (6 left) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1553: done (5 left) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1554: done (4 left) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1555: done (3 left) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1556: done (2 left) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1558: done (1 left) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1560: done (0 left) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1561: read oid 13 snap -1 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1561: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 161 2024-03-20T08:06:51.537 INFO:tasks.rados.rados.0.smithi118.stdout:1562: seq_num 491 ranges {794624=630784,1425408=401408} 2024-03-20T08:06:51.545 INFO:tasks.rados.rados.0.smithi118.stdout:1562: writing smithi11881747-32 from 794624 to 1425408 tid 1 2024-03-20T08:06:51.548 INFO:tasks.rados.rados.0.smithi118.stdout:1562: writing smithi11881747-32 from 1425408 to 1826816 tid 2 2024-03-20T08:06:51.549 INFO:tasks.rados.rados.0.smithi118.stdout:1563: copy_from oid 3 from oid 14 current snap is 161 2024-03-20T08:06:51.549 INFO:tasks.rados.rados.0.smithi118.stdout:1564: snap_create 2024-03-20T08:06:51.559 INFO:tasks.rados.rados.0.smithi118.stdout:1561: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:06:51.569 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:51.546+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:51.569 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:51.546+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:51.569 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:51.555+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:51.569 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:51.569 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:51.569 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: osdmap e383: 8 total, 8 up, 8 in 2024-03-20T08:06:51.569 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.570 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[93121]: Upgrade: osd.3 is safe to restart 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: osdmap e383: 8 total, 8 up, 8 in 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:06:51.571 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.572 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:51.572 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.572 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.572 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.572 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-20T08:06:51.572 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-20T08:06:51.572 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-mon[95692]: Upgrade: osd.3 is safe to restart 2024-03-20T08:06:51.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.322+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.323+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.332+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.333+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.341+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.344+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.352+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.353+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.362+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.362+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.372+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.373+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.383+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.384+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.394+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.395+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.404+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.405+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.414+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.416+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.424+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.427+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.435+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.438+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.445+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.448+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.456+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.458+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.467+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.574 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.469+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.477+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.480+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.488+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.490+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.499+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.500+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.510+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.510+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.521+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.523+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.538+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.538+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.557+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.557+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.568+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1562: finishing write tid 1 to smithi11881747-32 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1562: finishing write tid 2 to smithi11881747-32 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1562: finishing write tid 3 to smithi11881747-32 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1562: finishing write tid 4 to smithi11881747-32 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1562: oid 32 updating version 0 to 28409 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1562: oid 32 updating version 28409 to 28411 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1562: oid 32 version 28411 is already newer than 28410 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 28411 (ObjNum 491 snap 161 seq_num 491) dirty exists 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1562: left oid 32 (ObjNum 491 snap 161 seq_num 491) 2024-03-20T08:06:51.775 INFO:tasks.rados.rados.0.smithi118.stdout:1563: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:06:51.776 INFO:tasks.rados.rados.0.smithi118.stdout:1563: finishing copy_from to smithi11881747-3 2024-03-20T08:06:51.776 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 787 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:06:51.825 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.571+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.825 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.583+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.825 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.594+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.825 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.597+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.825 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.606+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.608+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.617+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.620+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.627+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.632+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.639+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.645+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.652+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.656+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.663+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.668+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.675+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.679+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.826 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.686+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.690+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.698+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.704+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.710+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.715+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.721+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.727+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.732+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.738+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.745+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.750+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.757+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.763+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.768+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.827 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.776+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.828 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.780+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.828 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.786+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.828 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.791+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.828 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.799+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.828 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.802+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.828 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.812+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.828 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.815+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:51.828 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.823+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:51.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:51.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:51.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: osdmap e383: 8 total, 8 up, 8 in 2024-03-20T08:06:51.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:51.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:06:51.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:06:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:06:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-20T08:06:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-20T08:06:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:51 smithi196 ceph-mon[87080]: Upgrade: osd.3 is safe to restart 2024-03-20T08:06:52.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.826+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.834+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.835+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.844+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.845+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.853+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.856+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.863+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.867+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.872+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.877+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.881+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.886+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.891+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.896+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.901+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.077 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.905+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.077 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.910+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.077 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.915+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.077 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.919+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.077 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.924+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.077 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.930+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.934+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.940+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.944+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.947+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.953+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.957+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.962+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.968+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.973+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.980+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.986+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.078 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.989+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.996+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:51.998+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.005+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.008+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.014+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.017+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.024+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.026+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.035+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.036+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.079 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.047+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.080 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.080 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.056+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.080 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.064+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.080 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.066+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.080 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.074+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.332 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.079+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.084+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.090+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.096+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.102+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.106+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.112+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.117+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.125+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.129+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.139+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.140+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.333 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.149+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.152+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.160+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.163+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.170+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.172+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.179+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.182+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.189+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.334 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.192+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.198+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.203+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.207+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.212+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.217+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.222+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.227+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.232+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.236+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.242+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.247+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.253+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.335 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.257+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.263+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.269+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.274+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.279+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.285+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.289+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.295+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.300+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.307+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.310+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.318+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.322+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.336 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.329+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.543 INFO:tasks.rados.rados.0.smithi118.stdout:1561: done (3 left) 2024-03-20T08:06:52.543 INFO:tasks.rados.rados.0.smithi118.stdout:1562: done (2 left) 2024-03-20T08:06:52.543 INFO:tasks.rados.rados.0.smithi118.stdout:1563: done (1 left) 2024-03-20T08:06:52.543 INFO:tasks.rados.rados.0.smithi118.stdout:1564: done (0 left) 2024-03-20T08:06:52.543 INFO:tasks.rados.rados.0.smithi118.stdout:1565: delete oid 33 current snap is 162 2024-03-20T08:06:52.554 INFO:tasks.rados.rados.0.smithi118.stdout:1565: done (0 left) 2024-03-20T08:06:52.554 INFO:tasks.rados.rados.0.smithi118.stdout:1566: read oid 12 snap -1 2024-03-20T08:06:52.554 INFO:tasks.rados.rados.0.smithi118.stdout:1566: expect (ObjNum 399 snap 135 seq_num 399) 2024-03-20T08:06:52.555 INFO:tasks.rados.rados.0.smithi118.stdout:1567: rmattr oid 26 current snap is 162 2024-03-20T08:06:52.555 INFO:tasks.rados.rados.0.smithi118.stdout:1567: done (1 left) 2024-03-20T08:06:52.555 INFO:tasks.rados.rados.0.smithi118.stdout:1568: copy_from oid 6 from oid 23 current snap is 162 2024-03-20T08:06:52.555 INFO:tasks.rados.rados.0.smithi118.stdout:1569: rollback oid 17 current snap is 162 2024-03-20T08:06:52.555 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 120 2024-03-20T08:06:52.555 INFO:tasks.rados.rados.0.smithi118.stdout:1570: snap_create 2024-03-20T08:06:52.556 INFO:tasks.rados.rados.0.smithi118.stdout:1569: finishing rollback tid 0 to smithi11881747-17 2024-03-20T08:06:52.557 INFO:tasks.rados.rados.0.smithi118.stdout:1569: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:06:52.557 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7036 (ObjNum 65 snap 0 seq_num 1868392561) dirty dne 2024-03-20T08:06:52.575 INFO:tasks.rados.rados.0.smithi118.stdout:1568: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:06:52.579 INFO:tasks.rados.rados.0.smithi118.stdout:1568: finishing copy_from to smithi11881747-6 2024-03-20T08:06:52.579 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 1267 (ObjNum 464 snap 155 seq_num 464) dirty exists 2024-03-20T08:06:52.584 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:52.551+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:52.584 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:52.551+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:52.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-mon[93121]: osdmap e384: 8 total, 8 up, 8 in 2024-03-20T08:06:52.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:52.584 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:52.585 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-mon[93121]: pgmap v253: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 406 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 22 op/s; 3.5 MiB/s, 0 objects/s recovering 2024-03-20T08:06:52.585 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-mon[95692]: osdmap e384: 8 total, 8 up, 8 in 2024-03-20T08:06:52.585 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:52.585 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:52.585 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-mon[95692]: pgmap v253: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 406 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 22 op/s; 3.5 MiB/s, 0 objects/s recovering 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.333+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.340+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.344+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.351+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.354+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.362+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.365+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.374+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.376+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.385+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.387+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.396+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.397+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.409+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.409+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.420+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.420+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.430+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.431+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.439+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.441+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.449+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.450+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.459+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.460+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.469+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.470+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.480+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.481+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.491+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.493+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.501+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.503+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.511+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.513+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.520+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.522+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.529+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.533+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.588 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.543+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.589 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.548+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.589 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.557+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.589 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.564+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.589 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.570+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.589 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.583+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.589 INFO:tasks.rados.rados.0.smithi118.stdout:1566: expect (ObjNum 399 snap 135 seq_num 399) 2024-03-20T08:06:52.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:52.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:52 smithi196 ceph-mon[87080]: osdmap e384: 8 total, 8 up, 8 in 2024-03-20T08:06:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:52.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:52.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:52 smithi196 ceph-mon[87080]: pgmap v253: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 406 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 22 op/s; 3.5 MiB/s, 0 objects/s recovering 2024-03-20T08:06:52.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.593+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.596+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.606+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.606+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.616+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.616+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.626+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.628+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.635+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.641+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.645+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.651+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.657+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.663+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.668+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.675+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.679+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.842 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.686+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.842 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.690+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.842 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.696+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.842 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.700+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.843 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.706+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.843 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.710+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.843 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.717+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.843 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.722+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.843 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.728+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.844 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.733+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.844 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.737+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.844 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.744+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.844 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.747+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.844 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.754+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.844 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.759+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.845 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.765+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.845 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.770+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.845 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.777+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.845 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.781+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.845 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.786+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.845 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.792+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.846 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.797+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.846 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.802+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.846 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.807+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.846 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.815+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.846 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.818+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:52.849 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:52.849 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.834+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.837+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.844+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.849+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.855+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.861+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.865+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.872+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.876+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.883+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.886+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.894+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.897+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.904+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.908+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.916+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.918+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.927+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.930+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.936+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.941+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.947+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.951+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.958+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.963+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.968+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.974+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.978+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.984+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.989+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.994+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:52.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.006+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.010+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.017+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.021+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.028+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.032+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.038+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.049+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.053+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.058+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.064+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.068+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.077+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.084+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.087+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.095+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.097+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.107+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.115+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.118+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.125+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.128+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.135+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.136+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.145+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.146+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.155+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.157+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.165+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.167+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.176+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.177+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.187+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.187+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.195+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.197+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.204+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.214+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.219+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.224+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.229+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.234+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.239+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.350 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.243+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.249+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.253+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.259+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.262+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.268+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.273+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.277+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.283+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.287+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.292+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.297+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.302+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.307+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.312+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.318+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.321+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.327+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.331+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.341+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.346+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.549 INFO:tasks.rados.rados.0.smithi118.stdout:1566: done (3 left) 2024-03-20T08:06:53.549 INFO:tasks.rados.rados.0.smithi118.stdout:1568: done (2 left) 2024-03-20T08:06:53.549 INFO:tasks.rados.rados.0.smithi118.stdout:1569: done (1 left) 2024-03-20T08:06:53.549 INFO:tasks.rados.rados.0.smithi118.stdout:1570: done (0 left) 2024-03-20T08:06:53.549 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 163 2024-03-20T08:06:53.549 INFO:tasks.rados.rados.0.smithi118.stdout:1571: seq_num 493 ranges {3457024=663552,4120576=401408} 2024-03-20T08:06:53.572 INFO:tasks.rados.rados.0.smithi118.stdout:1571: writing smithi11881747-25 from 3457024 to 4120576 tid 1 2024-03-20T08:06:53.576 INFO:tasks.rados.rados.0.smithi118.stdout:1571: writing smithi11881747-25 from 4120576 to 4521984 tid 2 2024-03-20T08:06:53.576 INFO:tasks.rados.rados.0.smithi118.stdout:1572: delete oid 33 current snap is 163 2024-03-20T08:06:53.580 INFO:tasks.rados.rados.0.smithi118.stdout:1571: finishing write tid 1 to smithi11881747-25 2024-03-20T08:06:53.581 INFO:tasks.rados.rados.0.smithi118.stdout:1571: finishing write tid 2 to smithi11881747-25 2024-03-20T08:06:53.581 INFO:tasks.rados.rados.0.smithi118.stdout:1571: finishing write tid 3 to smithi11881747-25 2024-03-20T08:06:53.583 INFO:tasks.rados.rados.0.smithi118.stdout:1571: finishing write tid 4 to smithi11881747-25 2024-03-20T08:06:53.583 INFO:tasks.rados.rados.0.smithi118.stdout:1571: oid 25 updating version 0 to 1009 2024-03-20T08:06:53.583 INFO:tasks.rados.rados.0.smithi118.stdout:1571: oid 25 version 1009 is already newer than 1008 2024-03-20T08:06:53.583 INFO:tasks.rados.rados.0.smithi118.stdout:1571: oid 25 updating version 1009 to 1010 2024-03-20T08:06:53.583 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 1010 (ObjNum 493 snap 163 seq_num 493) dirty exists 2024-03-20T08:06:53.583 INFO:tasks.rados.rados.0.smithi118.stdout:1571: left oid 25 (ObjNum 493 snap 163 seq_num 493) 2024-03-20T08:06:53.586 INFO:tasks.rados.rados.0.smithi118.stdout:1571: done (1 left) 2024-03-20T08:06:53.586 INFO:tasks.rados.rados.0.smithi118.stdout:1572: done (0 left) 2024-03-20T08:06:53.586 INFO:tasks.rados.rados.0.smithi118.stdout:1573: snap_create 2024-03-20T08:06:53.601 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:06:53] "GET /metrics HTTP/1.1" 200 36123 "" "Prometheus/2.43.0" 2024-03-20T08:06:53.602 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-mon[95692]: osdmap e385: 8 total, 8 up, 8 in 2024-03-20T08:06:53.602 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:53.602 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:53.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:53.557+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:53.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:53.557+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:53.602 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:53.559+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:53.603 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-mon[93121]: osdmap e385: 8 total, 8 up, 8 in 2024-03-20T08:06:53.603 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:53.603 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:53.603 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.352+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.357+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.362+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.368+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.373+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.379+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.383+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.390+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.393+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.400+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.402+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.410+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.413+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.420+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.604 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.423+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.430+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.433+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.440+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.443+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.451+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.452+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.462+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.462+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.471+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.472+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.481+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.482+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.491+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.492+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.500+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.502+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.512+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.514+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.521+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.524+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.532+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.536+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.553+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.553+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.566+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.567+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.579+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.583+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.590+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.595+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.601+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.606+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.612+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.616+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.624+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.628+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.637+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.639+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.649+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.650+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.660+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.662+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.674+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.674+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.685+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.686+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.696+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.697+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.862 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.709+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.862 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.709+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.862 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.720+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.862 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.862 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.732+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.732+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.743+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.743+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.754+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.754+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.765+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.766+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.776+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.778+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.786+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.790+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.798+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.800+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.810+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.812+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.822+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.823+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.866 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.834+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.866 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.835+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.866 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.847+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:53.866 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.848+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:53.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:53 smithi196 ceph-mon[87080]: osdmap e385: 8 total, 8 up, 8 in 2024-03-20T08:06:53.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:53.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:53.890 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:06:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:06:53.509Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:06:54.117 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.859+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.118 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.860+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.118 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.871+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.118 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.873+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.118 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.882+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.118 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.884+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.119 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.894+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.119 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.894+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.119 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.906+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.119 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.907+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.119 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.917+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.120 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.918+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.120 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.930+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.120 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.120 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.120 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.943+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.120 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.953+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.121 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.955+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.121 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.965+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.121 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.966+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.121 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.977+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.121 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.978+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.121 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.989+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.121 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:53.990+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.000+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.000+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.011+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.011+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.023+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.023+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.033+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.035+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.046+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.057+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.058+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.067+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.068+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.079+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.079+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.090+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.090+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.101+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.102+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.114+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.115+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.367 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.127+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.128+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.138+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.140+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.149+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.151+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.161+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.163+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.173+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.175+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.184+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.368 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.185+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.198+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.209+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.221+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.222+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.234+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.234+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.245+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.245+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.257+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.257+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.269+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.270+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.282+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.283+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.294+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.297+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.306+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.309+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.319+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.322+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.331+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.335+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.342+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.348+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.354+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.359+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.371 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.365+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.556 INFO:tasks.rados.rados.0.smithi118.stdout:1573: done (0 left) 2024-03-20T08:06:54.556 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 5 current snap is 164 2024-03-20T08:06:54.556 INFO:tasks.rados.rados.0.smithi118.stdout:1574: seq_num 495 ranges {2154496=696320,2850816=401408} 2024-03-20T08:06:54.572 INFO:tasks.rados.rados.0.smithi118.stdout:1574: writing smithi11881747-5 from 2154496 to 2850816 tid 1 2024-03-20T08:06:54.575 INFO:tasks.rados.rados.0.smithi118.stdout:1574: writing smithi11881747-5 from 2850816 to 3252224 tid 2 2024-03-20T08:06:54.575 INFO:tasks.rados.rados.0.smithi118.stdout:1575: rollback oid 11 current snap is 164 2024-03-20T08:06:54.575 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 120 2024-03-20T08:06:54.575 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 164 2024-03-20T08:06:54.575 INFO:tasks.rados.rados.0.smithi118.stdout:1576: seq_num 496 ranges {0=712704,712704=401408} 2024-03-20T08:06:54.579 INFO:tasks.rados.rados.0.smithi118.stdout:1576: writing smithi11881747-4 from 0 to 712704 tid 1 2024-03-20T08:06:54.582 INFO:tasks.rados.rados.0.smithi118.stdout:1576: writing smithi11881747-4 from 712704 to 1114112 tid 2 2024-03-20T08:06:54.582 INFO:tasks.rados.rados.0.smithi118.stdout:1577: delete oid 30 current snap is 164 2024-03-20T08:06:54.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:54.563+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:54.622 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:54.565+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:54.622 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: osdmap e386: 8 total, 8 up, 8 in 2024-03-20T08:06:54.622 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:54.622 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:54.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: Upgrade: Updating osd.3 2024-03-20T08:06:54.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:54.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T08:06:54.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:54.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: Deploying daemon osd.3 on smithi118 2024-03-20T08:06:54.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[93121]: pgmap v256: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 419 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 13 op/s 2024-03-20T08:06:54.623 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: osdmap e386: 8 total, 8 up, 8 in 2024-03-20T08:06:54.623 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:54.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:54.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: Upgrade: Updating osd.3 2024-03-20T08:06:54.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:54.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T08:06:54.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:54.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: Deploying daemon osd.3 on smithi118 2024-03-20T08:06:54.624 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-mon[95692]: pgmap v256: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 419 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 13 op/s 2024-03-20T08:06:54.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.371+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.377+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.382+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.387+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.397+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.402+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.408+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.414+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.418+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.426+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.428+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.438+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.438+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.448+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.449+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.459+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.459+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.469+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.470+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.479+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.482+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.491+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.495+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.502+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.505+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.513+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.515+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.524+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.527+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.534+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.538+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.544+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.550+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.560+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.568+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.629 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.572+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.578+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.584+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.589+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.592+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.600+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.601+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.610+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.611+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.621+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.630 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.622+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.631 INFO:tasks.rados.rados.0.smithi118.stdout:1574: finishing write tid 1 to smithi11881747-5 2024-03-20T08:06:54.631 INFO:tasks.rados.rados.0.smithi118.stdout:1574: finishing write tid 2 to smithi11881747-5 2024-03-20T08:06:54.631 INFO:tasks.rados.rados.0.smithi118.stdout:1574: finishing write tid 3 to smithi11881747-5 2024-03-20T08:06:54.631 INFO:tasks.rados.rados.0.smithi118.stdout:1575: finishing rollback tid 0 to smithi11881747-11 2024-03-20T08:06:54.631 INFO:tasks.rados.rados.0.smithi118.stdout:1575: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:06:54.631 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 1015 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:06:54.631 INFO:tasks.rados.rados.0.smithi118.stdout:1576: finishing write tid 1 to smithi11881747-4 2024-03-20T08:06:54.632 INFO:tasks.rados.rados.0.smithi118.stdout:1575: done (3 left) 2024-03-20T08:06:54.632 INFO:tasks.rados.rados.0.smithi118.stdout:1577: done (2 left) 2024-03-20T08:06:54.632 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 164 2024-03-20T08:06:54.632 INFO:tasks.rados.rados.0.smithi118.stdout:1578: seq_num 498 ranges {0=745472,745472=401408} 2024-03-20T08:06:54.634 INFO:tasks.rados.rados.0.smithi118.stdout:1578: writing smithi11881747-29 from 0 to 745472 tid 1 2024-03-20T08:06:54.638 INFO:tasks.rados.rados.0.smithi118.stdout:1578: writing smithi11881747-29 from 745472 to 1146880 tid 2 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1576: finishing write tid 2 to smithi11881747-4 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1576: finishing write tid 3 to smithi11881747-4 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1574: finishing write tid 4 to smithi11881747-5 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1574: oid 5 updating version 0 to 1011 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1574: oid 5 updating version 1011 to 1012 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1574: oid 5 updating version 1012 to 1013 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 1013 (ObjNum 495 snap 164 seq_num 495) dirty exists 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1574: left oid 5 (ObjNum 495 snap 164 seq_num 495) 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1576: finishing write tid 4 to smithi11881747-4 2024-03-20T08:06:54.639 INFO:tasks.rados.rados.0.smithi118.stdout:1576: oid 4 updating version 0 to 790 2024-03-20T08:06:54.640 INFO:tasks.rados.rados.0.smithi118.stdout:1576: oid 4 version 790 is already newer than 788 2024-03-20T08:06:54.640 INFO:tasks.rados.rados.0.smithi118.stdout:1576: oid 4 version 790 is already newer than 789 2024-03-20T08:06:54.640 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 790 (ObjNum 496 snap 164 seq_num 496) dirty exists 2024-03-20T08:06:54.640 INFO:tasks.rados.rados.0.smithi118.stdout:1576: left oid 4 (ObjNum 496 snap 164 seq_num 496) 2024-03-20T08:06:54.640 INFO:tasks.rados.rados.0.smithi118.stdout:1574: done (2 left) 2024-03-20T08:06:54.640 INFO:tasks.rados.rados.0.smithi118.stdout:1576: done (1 left) 2024-03-20T08:06:54.640 INFO:tasks.rados.rados.0.smithi118.stdout:1579: delete oid 39 current snap is 164 2024-03-20T08:06:54.642 INFO:tasks.rados.rados.0.smithi118.stdout:1578: finishing write tid 1 to smithi11881747-29 2024-03-20T08:06:54.644 INFO:tasks.rados.rados.0.smithi118.stdout:1578: finishing write tid 2 to smithi11881747-29 2024-03-20T08:06:54.644 INFO:tasks.rados.rados.0.smithi118.stdout:1578: finishing write tid 3 to smithi11881747-29 2024-03-20T08:06:54.646 INFO:tasks.rados.rados.0.smithi118.stdout:1578: finishing write tid 4 to smithi11881747-29 2024-03-20T08:06:54.646 INFO:tasks.rados.rados.0.smithi118.stdout:1578: oid 29 updating version 0 to 1280 2024-03-20T08:06:54.646 INFO:tasks.rados.rados.0.smithi118.stdout:1578: oid 29 updating version 1280 to 1281 2024-03-20T08:06:54.646 INFO:tasks.rados.rados.0.smithi118.stdout:1578: oid 29 version 1281 is already newer than 1279 2024-03-20T08:06:54.647 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1281 (ObjNum 498 snap 164 seq_num 498) dirty exists 2024-03-20T08:06:54.647 INFO:tasks.rados.rados.0.smithi118.stdout:1578: left oid 29 (ObjNum 498 snap 164 seq_num 498) 2024-03-20T08:06:54.649 INFO:tasks.rados.rados.0.smithi118.stdout:1578: done (1 left) 2024-03-20T08:06:54.650 INFO:tasks.rados.rados.0.smithi118.stdout:1579: done (0 left) 2024-03-20T08:06:54.650 INFO:tasks.rados.rados.0.smithi118.stdout:1580: rollback oid 37 current snap is 164 2024-03-20T08:06:54.650 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 163 2024-03-20T08:06:54.650 INFO:tasks.rados.rados.0.smithi118.stdout:1581: snap_create 2024-03-20T08:06:54.652 INFO:tasks.rados.rados.0.smithi118.stdout:1580: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:06:54.652 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 328 (ObjNum 177 snap 0 seq_num 322405856) dirty dne 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.637+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.640+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.653+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.655+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.667+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.668+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.679+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.680+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.875 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.690+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.692+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.701+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.702+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.710+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.721+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.724+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.731+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.736+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.742+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.747+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.752+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.758+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.762+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.768+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.772+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.781+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.784+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.794+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.798+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.805+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.811+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.817+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.821+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.829+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.831+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.841+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.842+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.852+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.854+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.863+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.865+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:54.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.874+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:54.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: osdmap e386: 8 total, 8 up, 8 in 2024-03-20T08:06:54.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:54.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:54.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: Upgrade: Updating osd.3 2024-03-20T08:06:54.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:54.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-20T08:06:54.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:06:54.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: Deploying daemon osd.3 on smithi118 2024-03-20T08:06:54.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:54 smithi196 ceph-mon[87080]: pgmap v256: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 419 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 13 op/s 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.878+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.886+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.890+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.897+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.903+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.908+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.914+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.919+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.925+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.930+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.936+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.941+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.948+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.951+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.960+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.961+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.971+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.972+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.981+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.983+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.133 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.993+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:54.994+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.003+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.005+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.014+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.015+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.086+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.089+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.097+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.100+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.108+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.110+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.120+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.122+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.134 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.131+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.133+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.142+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.147+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.153+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.159+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.164+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.172+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.175+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.183+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.186+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.194+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.197+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.207+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.216+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.219+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.227+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.229+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.239+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.240+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.250+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.251+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.261+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.262+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.273+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.273+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.284+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.413 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.285+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.295+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.296+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.306+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.306+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.317+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.320+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.328+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.332+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.339+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.344+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.414 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.349+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.627 INFO:tasks.rados.rados.0.smithi118.stdout:1580: done (1 left) 2024-03-20T08:06:55.627 INFO:tasks.rados.rados.0.smithi118.stdout:1581: done (0 left) 2024-03-20T08:06:55.627 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 165 2024-03-20T08:06:55.627 INFO:tasks.rados.rados.0.smithi118.stdout:1582: seq_num 500 ranges {3686400=778240,4464640=401408} 2024-03-20T08:06:55.651 INFO:tasks.rados.rados.0.smithi118.stdout:1582: writing smithi11881747-38 from 3686400 to 4464640 tid 1 2024-03-20T08:06:55.653 INFO:tasks.rados.rados.0.smithi118.stdout:1582: writing smithi11881747-38 from 4464640 to 4866048 tid 2 2024-03-20T08:06:55.653 INFO:tasks.rados.rados.0.smithi118.stdout:1583: rollback oid 50 current snap is 165 2024-03-20T08:06:55.653 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 50 to 161 2024-03-20T08:06:55.653 INFO:tasks.rados.rados.0.smithi118.stdout:1584: setattr oid 41 current snap is 165 2024-03-20T08:06:55.654 INFO:tasks.rados.rados.0.smithi118.stdout:1585: rollback oid 7 current snap is 165 2024-03-20T08:06:55.654 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 7 to 134 2024-03-20T08:06:55.654 INFO:tasks.rados.rados.0.smithi118.stdout:1586: snap_remove snap 156 2024-03-20T08:06:55.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-mon[93121]: osdmap e387: 8 total, 8 up, 8 in 2024-03-20T08:06:55.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:55.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:55.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-mon[95692]: osdmap e387: 8 total, 8 up, 8 in 2024-03-20T08:06:55.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:55.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.354+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.416+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.421+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.426+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.432+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.438+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.443+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.449+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.455+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.461+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.466+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.471+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.478+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.482+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.490+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.493+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.502+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.503+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.513+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.514+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.524+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.525+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.536+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.536+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.547+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.547+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.559+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.560+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.571+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.582+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.584+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.592+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.596+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:55 smithi196 ceph-mon[87080]: osdmap e387: 8 total, 8 up, 8 in 2024-03-20T08:06:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:55.931 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:55.680+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:55.680+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:55.680+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.603+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.607+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.612+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.681+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.714+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.726+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.932 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.728+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.737+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.740+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.749+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.752+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.760+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.765+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.771+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.783+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.791+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.794+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.803+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.806+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.816+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.819+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.827+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.830+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.839+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.840+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.851+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.851+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.863+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.864+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.875+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.877+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.887+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.889+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.898+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.901+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.909+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.913+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.919+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:55.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.926+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:55.936 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.930+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.183 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.939+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.183 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.943+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.183 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.951+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.183 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.953+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.963+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.964+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.975+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.977+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.987+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.993+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:55.998+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.004+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.010+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.017+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.028+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.035+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.039+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.046+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.050+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.069+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.072+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.079+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.090+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.095+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.101+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.107+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.113+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.121+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.124+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.133+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.136+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.145+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.186 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.147+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.186 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.157+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.186 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.159+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.186 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.169+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.186 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.170+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.186 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.186 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.182+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.192+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.193+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.204+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.204+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.216+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.217+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.227+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.230+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.237+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.285+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.294+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.365+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.369+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.376+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.381+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.388+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.394+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.400+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.405+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.411+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.418+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.422+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.430+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.433+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1583: finishing rollback tid 1 to smithi11881747-50 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 788 (ObjNum 193 snap 0 seq_num 324613632) dirty dne 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1582: finishing write tid 1 to smithi11881747-38 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1582: finishing write tid 2 to smithi11881747-38 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1582: finishing write tid 3 to smithi11881747-38 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1582: finishing write tid 4 to smithi11881747-38 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1582: oid 38 updating version 0 to 7039 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1582: oid 38 version 7039 is already newer than 7038 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1582: oid 38 version 7039 is already newer than 7037 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7039 (ObjNum 500 snap 165 seq_num 500) dirty exists 2024-03-20T08:06:56.637 INFO:tasks.rados.rados.0.smithi118.stdout:1582: left oid 38 (ObjNum 500 snap 165 seq_num 500) 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1283 (ObjNum 425 snap 141 seq_num 425) dirty exists 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1585: finishing rollback tid 1 to smithi11881747-7 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1585: finishing rollback tid 2 to smithi11881747-7 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 7029 (ObjNum 387 snap 132 seq_num 387) dirty exists 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1582: done (4 left) 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1583: done (3 left) 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1584: done (2 left) 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1585: done (1 left) 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1586: done (0 left) 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1587: rollback oid 30 current snap is 165 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 160 2024-03-20T08:06:56.638 INFO:tasks.rados.rados.0.smithi118.stdout:1588: delete oid 31 current snap is 165 2024-03-20T08:06:56.639 INFO:tasks.rados.rados.0.smithi118.stdout:1588: done (1 left) 2024-03-20T08:06:56.639 INFO:tasks.rados.rados.0.smithi118.stdout:1589: snap_remove snap 120 2024-03-20T08:06:56.688 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:56.660+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:56.688 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:56.660+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:56.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-mon[93121]: osdmap e388: 8 total, 8 up, 8 in 2024-03-20T08:06:56.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:56.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:56.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-mon[93121]: pgmap v259: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 421 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 11 op/s 2024-03-20T08:06:56.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-mon[95692]: osdmap e388: 8 total, 8 up, 8 in 2024-03-20T08:06:56.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:56.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:56.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-mon[95692]: pgmap v259: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 421 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 11 op/s 2024-03-20T08:06:56.689 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.442+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.444+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.454+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.455+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.466+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.468+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.478+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.479+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.490+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.491+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.503+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.503+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.515+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.515+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.527+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.528+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.539+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.539+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.551+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.551+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.563+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.563+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.575+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.577+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.587+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.588+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.599+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.692 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.599+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.692 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.610+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.692 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.612+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.692 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.664+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.692 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.667+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.692 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.677+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.692 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.681+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.692 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.688+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:56 smithi196 ceph-mon[87080]: osdmap e388: 8 total, 8 up, 8 in 2024-03-20T08:06:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:56 smithi196 ceph-mon[87080]: pgmap v259: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 421 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 11 op/s 2024-03-20T08:06:56.946 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.693+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.700+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.705+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.712+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.717+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.723+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.730+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.734+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.742+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.745+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.755+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.757+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.767+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.769+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.778+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.781+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.791+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.793+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.803+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.805+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.815+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.817+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.827+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.828+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.838+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.840+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.850+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.853+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.861+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.872+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.925+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.949 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.934+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:56.950 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.935+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:56.950 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.946+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.948+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.959+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.960+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.971+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.971+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.983+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.983+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.995+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:56.996+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.007+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.007+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.018+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.020+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.030+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.032+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.043+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.203 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.044+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.055+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.057+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.068+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.070+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.080+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.091+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.093+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.102+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.106+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.114+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.204 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.118+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.125+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.130+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.136+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.143+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.149+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.156+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.165+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.170+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.178+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.183+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.190+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.196+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.205 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.202+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.454 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.211+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.454 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.214+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.454 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.223+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.454 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.226+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.454 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.235+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.237+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.302+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.304+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.314+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.318+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.326+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.331+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.337+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.344+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.350+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.357+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.362+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.368+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.374+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.381+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.386+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.393+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.397+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.404+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.410+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.416+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.422+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.429+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.433+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.441+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.446+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.454+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.639 INFO:tasks.rados.rados.0.smithi118.stdout:1587: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:06:57.639 INFO:tasks.rados.rados.0.smithi118.stdout:1587: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:06:57.639 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 793 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:06:57.639 INFO:tasks.rados.rados.0.smithi118.stdout:1587: done (1 left) 2024-03-20T08:06:57.639 INFO:tasks.rados.rados.0.smithi118.stdout:1589: done (0 left) 2024-03-20T08:06:57.639 INFO:tasks.rados.rados.0.smithi118.stdout:1590: read oid 12 snap 163 2024-03-20T08:06:57.639 INFO:tasks.rados.rados.0.smithi118.stdout:1590: expect (ObjNum 399 snap 135 seq_num 399) 2024-03-20T08:06:57.639 INFO:tasks.rados.rados.0.smithi118.stdout:1591: snap_remove snap 160 2024-03-20T08:06:57.705 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-mon[93121]: osdmap e389: 8 total, 8 up, 8 in 2024-03-20T08:06:57.705 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:57.705 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:57.705 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:57.642+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:57.705 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:57.642+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:57.705 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:57.642+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:57.706 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-mon[95692]: osdmap e389: 8 total, 8 up, 8 in 2024-03-20T08:06:57.706 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:57.706 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:57.706 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.458+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.706 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.467+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.706 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.470+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.480+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.483+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.493+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.494+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.556+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.558+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.568+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.570+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.580+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.584+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.592+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.599+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.604+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.611+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.617+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.623+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.628+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.639+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.650+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.667+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.679+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.681+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.692+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.694+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.705+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.706+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.719+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.719+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.731+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.731+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.808+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.811+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.820+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.823+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.958 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.833+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.837+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.845+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.849+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.857+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.862+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.869+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.874+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.881+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.959 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.887+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.894+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.900+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.906+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.912+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.919+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.926+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.932+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.938+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.944+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.952+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:57.960 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.957+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:57 smithi196 ceph-mon[87080]: osdmap e389: 8 total, 8 up, 8 in 2024-03-20T08:06:58.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:58.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:58.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.966+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.970+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.979+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.982+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.991+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:57.995+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.007+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.016+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.019+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.058+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.067+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.081+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.093+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.093+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.106+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.215 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.106+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.215 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.118+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.215 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.122+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.215 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.131+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.215 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.134+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.215 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.144+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.215 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.146+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.215 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.157+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.160+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.169+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.173+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.182+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.187+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.194+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.200+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.206+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.216 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.213+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.470 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.221+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.470 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.226+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.470 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.233+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.470 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.239+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.470 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.247+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.251+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.259+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.264+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.272+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.279+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.286+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.299+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.316+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.471 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.324+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.472 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.330+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.472 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.336+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.472 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.344+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.472 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.348+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.358+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.360+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.371+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.371+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.383+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.384+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.396+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.396+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.409+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.410+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.420+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.423+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.433+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.436+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.446+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.449+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.458+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.463+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.470+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.649 INFO:tasks.rados.rados.0.smithi118.stdout:1591: done (1 left) 2024-03-20T08:06:58.649 INFO:tasks.rados.rados.0.smithi118.stdout:1592: snap_create 2024-03-20T08:06:58.649 INFO:tasks.rados.rados.0.smithi118.stdout:1590: expect (ObjNum 399 snap 135 seq_num 399) 2024-03-20T08:06:58.725 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:58.653+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:58.725 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:58.654+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:58.726 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[93121]: osdmap e390: 8 total, 8 up, 8 in 2024-03-20T08:06:58.726 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:58.726 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:58.726 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[93121]: pgmap v262: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 424 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 808 KiB/s wr, 10 op/s 2024-03-20T08:06:58.726 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:58.726 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:58.726 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:58.727 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[95692]: osdmap e390: 8 total, 8 up, 8 in 2024-03-20T08:06:58.727 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:58.727 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:58.727 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[95692]: pgmap v262: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 424 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 808 KiB/s wr, 10 op/s 2024-03-20T08:06:58.727 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:58.727 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:58.727 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.477+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.482+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.489+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.495+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.554+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.561+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.568+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.573+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.581+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.587+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.728 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.594+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.600+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.607+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.612+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.620+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.624+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.633+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.636+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.655+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.655+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.670+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.673+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.684+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.686+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.699+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.729 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.699+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.713+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.726+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.726+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.741+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.741+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.755+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.755+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.886+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.886+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.900+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.901+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.915+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.915+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.928+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.941+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.947+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.955+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.961+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.969+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:58.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.975+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:58.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.983+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:58 smithi196 ceph-mon[87080]: osdmap e390: 8 total, 8 up, 8 in 2024-03-20T08:06:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:58 smithi196 ceph-mon[87080]: pgmap v262: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 424 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 7.2 MiB/s rd, 808 KiB/s wr, 10 op/s 2024-03-20T08:06:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:06:59.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:06:59.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.989+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:58.996+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.002+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.010+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.016+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.024+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.031+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.037+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.045+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.050+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.060+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.063+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.074+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.076+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.086+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.089+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.100+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.102+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.114+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.116+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.128+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.128+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.265+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.266+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.289+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.290+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.302+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.303+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.314+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.317+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.327+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.330+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.340+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.343+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.353+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.358+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.365+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.370+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.379+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.383+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.515 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.393+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.396+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.405+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.410+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.418+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.424+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.430+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.437+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.443+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.450+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.456+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.464+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.469+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.478+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.516 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.481+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.491+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.494+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.651 INFO:tasks.rados.rados.0.smithi118.stdout:1590: done (1 left) 2024-03-20T08:06:59.651 INFO:tasks.rados.rados.0.smithi118.stdout:1592: done (0 left) 2024-03-20T08:06:59.651 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 166 2024-03-20T08:06:59.651 INFO:tasks.rados.rados.0.smithi118.stdout:1593: seq_num 502 ranges {7692288=409600,8101888=417792,8519680=385024} 2024-03-20T08:06:59.696 INFO:tasks.rados.rados.0.smithi118.stdout:1593: writing smithi11881747-41 from 7692288 to 8101888 tid 1 2024-03-20T08:06:59.699 INFO:tasks.rados.rados.0.smithi118.stdout:1593: writing smithi11881747-41 from 8101888 to 8519680 tid 2 2024-03-20T08:06:59.702 INFO:tasks.rados.rados.0.smithi118.stdout:1593: writing smithi11881747-41 from 8519680 to 8904704 tid 3 2024-03-20T08:06:59.702 INFO:tasks.rados.rados.0.smithi118.stdout:1594: snap_remove snap 157 2024-03-20T08:06:59.735 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-mon[93121]: osdmap e391: 8 total, 8 up, 8 in 2024-03-20T08:06:59.735 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:59.735 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:59.735 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:59.666+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:06:59.735 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:59.666+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:59.735 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:06:59.666+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:59.736 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-mon[95692]: osdmap e391: 8 total, 8 up, 8 in 2024-03-20T08:06:59.736 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:06:59.736 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:06:59.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.516+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.516+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.528+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.529+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.542+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.542+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.554+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.556+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.567+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.568+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.580+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.582+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.593+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.596+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.606+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.737 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.608+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.618+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.622+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.631+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.636+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.645+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.664+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.670+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.677+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.684+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.692+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.697+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.708+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.715+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.720+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.728+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.734+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.742+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.747+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:06:59.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.756+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:06:59.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.760+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.769+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.773+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.783+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.027 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.786+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.797+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.799+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.810+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.811+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.862+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.867+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.875+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.888+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.899+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.902+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.912+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.916+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.028 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.925+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.937+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.943+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.949+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.962+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.971+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.976+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.985+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:06:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.989+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:06:59.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.001+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.013+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.014+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.029 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.026+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.030 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.027+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:59 smithi196 ceph-mon[87080]: osdmap e391: 8 total, 8 up, 8 in 2024-03-20T08:07:00.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:00.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:06:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:00.287 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.040+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.287 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.040+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.287 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.052+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.287 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.054+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.287 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.067+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.068+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.080+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.093+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.094+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.106+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.108+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.118+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.121+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.157+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.167+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.177+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.186+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.194+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.204+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.207+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.218+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.222+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.231+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.234+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.245+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.247+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.259+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.260+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.273+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.274+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.287+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.290+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.301+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.304+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.313+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.318+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.326+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.332+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.340+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.623 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.347+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.353+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.361+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.366+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.376+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.380+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.389+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.393+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.402+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.407+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.416+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.420+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.430+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.433+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.445+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.447+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.459+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.460+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.474+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.474+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.489+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.490+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.503+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.504+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.516+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.518+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.530+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.531+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.544+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.544+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: finishing write tid 1 to smithi11881747-41 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: finishing write tid 2 to smithi11881747-41 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: finishing write tid 3 to smithi11881747-41 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: finishing write tid 4 to smithi11881747-41 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: finishing write tid 5 to smithi11881747-41 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: oid 41 updating version 0 to 1286 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: oid 41 updating version 1286 to 1287 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: oid 41 version 1287 is already newer than 1285 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: oid 41 updating version 1287 to 1288 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1288 (ObjNum 502 snap 166 seq_num 502) dirty exists 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: left oid 41 (ObjNum 502 snap 166 seq_num 502) 2024-03-20T08:07:00.666 INFO:tasks.rados.rados.0.smithi118.stdout:1593: done (1 left) 2024-03-20T08:07:00.667 INFO:tasks.rados.rados.0.smithi118.stdout:1594: done (0 left) 2024-03-20T08:07:00.667 INFO:tasks.rados.rados.0.smithi118.stdout:1595: read oid 15 snap -1 2024-03-20T08:07:00.667 INFO:tasks.rados.rados.0.smithi118.stdout:1595: expect (ObjNum 463 snap 155 seq_num 463) 2024-03-20T08:07:00.667 INFO:tasks.rados.rados.0.smithi118.stdout:1596: read oid 12 snap -1 2024-03-20T08:07:00.667 INFO:tasks.rados.rados.0.smithi118.stdout:1596: expect (ObjNum 399 snap 135 seq_num 399) 2024-03-20T08:07:00.667 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 166 2024-03-20T08:07:00.667 INFO:tasks.rados.rados.0.smithi118.stdout:1597: seq_num 503 ranges {999424=425984,1425408=704512,2129920=98304} 2024-03-20T08:07:00.674 INFO:tasks.rados.rados.0.smithi118.stdout:1597: writing smithi11881747-44 from 999424 to 1425408 tid 1 2024-03-20T08:07:00.681 INFO:tasks.rados.rados.0.smithi118.stdout:1597: writing smithi11881747-44 from 1425408 to 2129920 tid 2 2024-03-20T08:07:00.683 INFO:tasks.rados.rados.0.smithi118.stdout:1597: writing smithi11881747-44 from 2129920 to 2228224 tid 3 2024-03-20T08:07:00.683 INFO:tasks.rados.rados.0.smithi118.stdout:1598: rollback oid 14 current snap is 166 2024-03-20T08:07:00.683 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 158 2024-03-20T08:07:00.683 INFO:tasks.rados.rados.0.smithi118.stdout:1599: read oid 20 snap 163 2024-03-20T08:07:00.683 INFO:tasks.rados.rados.0.smithi118.stdout:1599: expect (ObjNum 486 snap 159 seq_num 486) 2024-03-20T08:07:00.683 INFO:tasks.rados.rados.0.smithi118.stdout:1597: finishing write tid 1 to smithi11881747-44 2024-03-20T08:07:00.683 INFO:tasks.rados.rados.0.smithi118.stdout:1600: read oid 7 snap -1 2024-03-20T08:07:00.683 INFO:tasks.rados.rados.0.smithi118.stdout:1600: expect (ObjNum 387 snap 132 seq_num 387) 2024-03-20T08:07:00.684 INFO:tasks.rados.rados.0.smithi118.stdout:1601: read oid 50 snap -1 2024-03-20T08:07:00.684 INFO:tasks.rados.rados.0.smithi118.stdout:1601: expect deleted 2024-03-20T08:07:00.684 INFO:tasks.rados.rados.0.smithi118.stdout:1602: snap_remove snap 141 2024-03-20T08:07:00.875 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:00.670+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:00.875 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:00.673+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:00.876 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-mon[93121]: osdmap e392: 8 total, 8 up, 8 in 2024-03-20T08:07:00.876 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:00.876 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:00.876 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-mon[93121]: pgmap v265: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 416 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 840 KiB/s wr, 10 op/s 2024-03-20T08:07:00.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-mon[95692]: osdmap e392: 8 total, 8 up, 8 in 2024-03-20T08:07:00.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:00.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:00.876 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-mon[95692]: pgmap v265: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 416 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 840 KiB/s wr, 10 op/s 2024-03-20T08:07:00.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.622+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.623+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.636+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.637+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.650+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.671+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.675+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.701+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.716+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.721+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.734+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.737+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.748+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.750+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.762+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.764+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.775+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.777+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.789+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.790+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.803+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.803+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.817+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.817+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.829+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.830+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.843+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.847+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.857+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.861+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.869+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:00.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.875+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:00 smithi196 ceph-mon[87080]: osdmap e392: 8 total, 8 up, 8 in 2024-03-20T08:07:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:01.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:00 smithi196 ceph-mon[87080]: pgmap v265: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 416 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 5.1 MiB/s rd, 840 KiB/s wr, 10 op/s 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.883+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.889+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.897+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.905+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.911+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.920+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.923+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.935+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.937+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.949+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.951+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.964+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.964+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.978+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.978+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.991+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:00.991+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.005+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.005+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.020+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.020+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.033+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.033+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.433 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.047+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.433 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.048+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.433 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.155+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.433 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.155+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.433 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.165+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.178+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.190+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.193+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.203+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.216+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.223+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.229+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.236+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.244+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.251+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.434 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.257+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.266+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.270+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.280+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.284+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.295+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.298+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.309+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.311+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.323+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.325+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.337+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.338+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.350+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.350+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.363+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.377+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.377+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.391+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.406+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.407+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.420+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.420+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.433+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.434+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.668 INFO:tasks.rados.rados.0.smithi118.stdout:1602: done (7 left) 2024-03-20T08:07:01.669 INFO:tasks.rados.rados.0.smithi118.stdout:1603: rollback oid 10 current snap is 166 2024-03-20T08:07:01.669 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 155 2024-03-20T08:07:01.669 INFO:tasks.rados.rados.0.smithi118.stdout:1604: read oid 3 snap -1 2024-03-20T08:07:01.669 INFO:tasks.rados.rados.0.smithi118.stdout:1604: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:07:01.669 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 166 2024-03-20T08:07:01.669 INFO:tasks.rados.rados.0.smithi118.stdout:1605: seq_num 504 ranges {4194304=450560,4644864=589824,5234688=212992} 2024-03-20T08:07:01.683 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-mon[95692]: osdmap e393: 8 total, 8 up, 8 in 2024-03-20T08:07:01.683 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:01.684 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:01.684 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-mon[93121]: osdmap e393: 8 total, 8 up, 8 in 2024-03-20T08:07:01.684 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:01.684 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:01.684 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:01.671+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:01.684 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:01.671+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:01.685 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:01.675+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:01.685 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.447+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.685 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.447+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.685 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.460+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.685 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.461+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.685 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.474+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.685 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.476+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.685 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.488+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.490+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.502+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.504+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.515+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.519+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.529+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.533+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.543+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.547+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.556+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.562+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.570+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.577+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.686 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.584+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.591+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.597+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.607+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.611+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.621+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.624+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.676+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.684+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.695 INFO:tasks.rados.rados.0.smithi118.stdout:1605: writing smithi11881747-2 from 4194304 to 4644864 tid 1 2024-03-20T08:07:01.700 INFO:tasks.rados.rados.0.smithi118.stdout:1605: writing smithi11881747-2 from 4644864 to 5234688 tid 2 2024-03-20T08:07:01.701 INFO:tasks.rados.rados.0.smithi118.stdout:1605: writing smithi11881747-2 from 5234688 to 5447680 tid 3 2024-03-20T08:07:01.704 INFO:tasks.rados.rados.0.smithi118.stdout:1597: finishing write tid 2 to smithi11881747-44 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1597: finishing write tid 3 to smithi11881747-44 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1597: finishing write tid 4 to smithi11881747-44 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1597: finishing write tid 5 to smithi11881747-44 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1597: oid 44 updating version 0 to 818 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1597: oid 44 version 818 is already newer than 817 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1597: oid 44 updating version 818 to 819 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1597: oid 44 updating version 819 to 820 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 820 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1597: left oid 44 (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:07:01.705 INFO:tasks.rados.rados.0.smithi118.stdout:1595: expect (ObjNum 463 snap 155 seq_num 463) 2024-03-20T08:07:01.936 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.705+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.711+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.720+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.725+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.737+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.740+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.752+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.754+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.768+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.768+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.783+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.783+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.802+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.804+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.817+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.821+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.834+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.837+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.852+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.852+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.868+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.868+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.885+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.886+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.902+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.903+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.919+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.920+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:01.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.936+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:01.944 INFO:tasks.rados.rados.0.smithi118.stdout:1595: done (9 left) 2024-03-20T08:07:01.944 INFO:tasks.rados.rados.0.smithi118.stdout:1597: done (8 left) 2024-03-20T08:07:01.944 INFO:tasks.rados.rados.0.smithi118.stdout:1601: done (7 left) 2024-03-20T08:07:01.944 INFO:tasks.rados.rados.0.smithi118.stdout:1606: copy_from oid 40 from oid 38 current snap is 166 2024-03-20T08:07:01.944 INFO:tasks.rados.rados.0.smithi118.stdout:1607: snap_create 2024-03-20T08:07:01.944 INFO:tasks.rados.rados.0.smithi118.stdout:1598: finishing rollback tid 0 to smithi11881747-14 2024-03-20T08:07:01.944 INFO:tasks.rados.rados.0.smithi118.stdout:1596: expect (ObjNum 399 snap 135 seq_num 399) 2024-03-20T08:07:02.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:01 smithi196 ceph-mon[87080]: osdmap e393: 8 total, 8 up, 8 in 2024-03-20T08:07:02.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:02.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.937+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.954+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.955+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.977+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.978+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.994+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:01.994+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.008+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.009+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.024+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.025+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.038+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.039+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.052+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.054+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.066+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.070+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.084+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.085+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.099+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.102+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.114+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.117+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.128+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.132+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.143+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.147+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.159+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.161+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.175+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.175+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.189+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.190+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.391 INFO:tasks.rados.rados.0.smithi118.stdout:1598: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:07:02.391 INFO:tasks.rados.rados.0.smithi118.stdout:1600: expect (ObjNum 387 snap 132 seq_num 387) 2024-03-20T08:07:02.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.205+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.207+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.220+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.221+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.237+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.237+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.253+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.254+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.269+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.270+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.288+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.289+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.306+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.306+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.320+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.323+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.340+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.342+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.357+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.359+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.371+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.373+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.385+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.390+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.401+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.407+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.415+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.423+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.432+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.439+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.447+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.466 INFO:tasks.rados.rados.0.smithi118.stdout:1598: finishing rollback tid 2 to smithi11881747-14 2024-03-20T08:07:02.466 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 31443 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:07:02.466 INFO:tasks.rados.rados.0.smithi118.stdout:1599: expect (ObjNum 486 snap 159 seq_num 486) 2024-03-20T08:07:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:02.705 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:02.692+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:02.705 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:02.692+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:02.706 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.456+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.706 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.464+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.473+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.481+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.491+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.497+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.510+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.514+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.530+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.532+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.550+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.550+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.566+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.566+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.582+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.707 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.583+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.599+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.600+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.615+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.616+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.632+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.632+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.648+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.649+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.663+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.664+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.687+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.691+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.709 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-mon[93121]: osdmap e394: 8 total, 8 up, 8 in 2024-03-20T08:07:02.709 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:02.709 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:02.709 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-mon[93121]: pgmap v268: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 416 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 560 KiB/s wr, 3 op/s 2024-03-20T08:07:02.709 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-mon[95692]: osdmap e394: 8 total, 8 up, 8 in 2024-03-20T08:07:02.709 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:02.709 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:02.709 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-mon[95692]: pgmap v268: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 416 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 560 KiB/s wr, 3 op/s 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.705+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.708+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.725+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.738+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.741+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.753+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.760+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.771+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.961 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.775+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.789+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.792+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.806+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.808+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.822+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.824+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.838+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.841+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.855+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.858+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.872+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.875+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.888+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.962 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.963 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.905+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.963 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.911+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.963 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.920+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.963 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.927+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.963 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.935+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.963 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.944+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:02.963 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.952+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:02.963 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.960+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:02 smithi196 ceph-mon[87080]: osdmap e394: 8 total, 8 up, 8 in 2024-03-20T08:07:03.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:03.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:03.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:02 smithi196 ceph-mon[87080]: pgmap v268: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 416 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 511 B/s rd, 560 KiB/s wr, 3 op/s 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.968+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.975+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.983+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.990+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:02.998+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.006+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.013+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.029+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.038+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.217 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.044+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.055+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.059+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.073+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.074+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.089+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.089+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.104+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.105+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.120+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.121+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.138+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.141+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.218 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.152+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.219 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.157+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.219 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.168+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.219 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.175+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.219 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.185+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.219 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.190+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.219 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.200+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.219 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.219 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.216+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.313 INFO:tasks.rados.rados.0.smithi118.stdout:1603: finishing rollback tid 0 to smithi11881747-10 2024-03-20T08:07:03.313 INFO:tasks.rados.rados.0.smithi118.stdout:1603: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:07:03.313 INFO:tasks.rados.rados.0.smithi118.stdout:1603: finishing rollback tid 2 to smithi11881747-10 2024-03-20T08:07:03.313 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 1004 (ObjNum 441 snap 148 seq_num 441) dirty exists 2024-03-20T08:07:03.313 INFO:tasks.rados.rados.0.smithi118.stdout:1605: finishing write tid 1 to smithi11881747-2 2024-03-20T08:07:03.314 INFO:tasks.rados.rados.0.smithi118.stdout:1604: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:07:03.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.223+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.232+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.239+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.248+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.263+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.271+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.277+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.286+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.292+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.477 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.302+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.478 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.307+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.478 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.318+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.478 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.324+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.478 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.332+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.478 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.340+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.478 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.479 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.355+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.479 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.363+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.479 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.370+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.479 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.380+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.480 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.386+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.480 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.397+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.480 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.400+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.480 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.412+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.480 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.415+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.480 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.429+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.481 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.430+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.481 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.444+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.481 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.446+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.481 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.460+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.481 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.461+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.510 INFO:tasks.rados.rados.0.smithi118.stdout:1605: finishing write tid 2 to smithi11881747-2 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1605: finishing write tid 3 to smithi11881747-2 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1605: finishing write tid 4 to smithi11881747-2 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1605: finishing write tid 5 to smithi11881747-2 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1605: oid 2 updating version 0 to 1269 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1605: oid 2 updating version 1269 to 1271 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1605: oid 2 version 1271 is already newer than 1270 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1605: oid 2 version 1271 is already newer than 1268 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 1271 (ObjNum 504 snap 166 seq_num 504) dirty exists 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1605: left oid 2 (ObjNum 504 snap 166 seq_num 504) 2024-03-20T08:07:03.511 INFO:tasks.rados.rados.0.smithi118.stdout:1606: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1606: finishing copy_from to smithi11881747-40 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 7039 (ObjNum 500 snap 165 seq_num 500) dirty exists 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1596: done (8 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1598: done (7 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1599: done (6 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1600: done (5 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1603: done (4 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1604: done (3 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1605: done (2 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1606: done (1 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1607: done (0 left) 2024-03-20T08:07:03.512 INFO:tasks.rados.rados.0.smithi118.stdout:1608: rmattr oid 27 current snap is 167 2024-03-20T08:07:03.513 INFO:tasks.rados.rados.0.smithi118.stdout:1608: done (0 left) 2024-03-20T08:07:03.513 INFO:tasks.rados.rados.0.smithi118.stdout:1609: rollback oid 10 current snap is 167 2024-03-20T08:07:03.513 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 135 2024-03-20T08:07:03.513 INFO:tasks.rados.rados.0.smithi118.stdout:1610: read oid 14 snap -1 2024-03-20T08:07:03.513 INFO:tasks.rados.rados.0.smithi118.stdout:1610: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:07:03.513 INFO:tasks.rados.rados.0.smithi118.stdout:1611: read oid 42 snap -1 2024-03-20T08:07:03.513 INFO:tasks.rados.rados.0.smithi118.stdout:1611: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:07:03.513 INFO:tasks.rados.rados.0.smithi118.stdout:1612: snap_remove snap 159 2024-03-20T08:07:03.687 INFO:tasks.rados.rados.0.smithi118.stdout:1612: done (3 left) 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:1613: setattr oid 9 current snap is 167 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:1614: rollback oid 33 current snap is 167 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 137 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:1615: copy_from oid 7 from oid 25 current snap is 167 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:1616: delete oid 36 current snap is 167 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:1609: finishing rollback tid 0 to smithi11881747-10 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:1609: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:1609: finishing rollback tid 2 to smithi11881747-10 2024-03-20T08:07:03.688 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 1007 (ObjNum 372 snap 127 seq_num 372) dirty exists 2024-03-20T08:07:03.689 INFO:tasks.rados.rados.0.smithi118.stdout:1611: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:07:03.736 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-mon[93121]: osdmap e395: 8 total, 8 up, 8 in 2024-03-20T08:07:03.736 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:03.736 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:03.737 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-mon[95692]: osdmap e395: 8 total, 8 up, 8 in 2024-03-20T08:07:03.737 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:03.737 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:03.737 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:03.694+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:03.737 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:03.696+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:03.738 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:03.696+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e2=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7],e2=[e2,e1,e0,df,de,dd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:03.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.476+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.478+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.493+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.494+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.738 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.508+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.510+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.529+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.529+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.543+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.544+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.559+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.559+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.575+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.739 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.576+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.589+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.593+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.604+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.740 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:07:03] "GET /metrics HTTP/1.1" 200 36268 "" "Prometheus/2.43.0" 2024-03-20T08:07:03.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.610+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.621+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.626+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.637+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.643+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.653+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.660+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.668+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.679+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.687+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.703+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.719+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.727+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.737+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.885 INFO:tasks.rados.rados.0.smithi118.stdout:1610: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:07:03.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:03 smithi196 ceph-mon[87080]: osdmap e395: 8 total, 8 up, 8 in 2024-03-20T08:07:03.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset da=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:03.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:03.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:03.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:03.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.749+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.753+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.765+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.768+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.783+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.785+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.801+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.801+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.818+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.832+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.847+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.852+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.993 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.873+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.884+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.891+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.902+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.907+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.923+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.927+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.943+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.943+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.958+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.962+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.975+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:03.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.979+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:03.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.992+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.094 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 1016 (ObjNum 471 snap 156 seq_num 471) dirty exists 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:1614: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:1614: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 7726 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:1615: finishing copy_from racing read to smithi11881747-7 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:1615: finishing copy_from to smithi11881747-7 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 1010 (ObjNum 493 snap 163 seq_num 493) dirty exists 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:1609: done (6 left) 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:1610: done (5 left) 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:1611: done (4 left) 2024-03-20T08:07:04.095 INFO:tasks.rados.rados.0.smithi118.stdout:1613: done (3 left) 2024-03-20T08:07:04.096 INFO:tasks.rados.rados.0.smithi118.stdout:1614: done (2 left) 2024-03-20T08:07:04.096 INFO:tasks.rados.rados.0.smithi118.stdout:1615: done (1 left) 2024-03-20T08:07:04.096 INFO:tasks.rados.rados.0.smithi118.stdout:1616: done (0 left) 2024-03-20T08:07:04.096 INFO:tasks.rados.rados.0.smithi118.stdout:1617: read oid 14 snap -1 2024-03-20T08:07:04.096 INFO:tasks.rados.rados.0.smithi118.stdout:1617: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:07:04.096 INFO:tasks.rados.rados.0.smithi118.stdout:1618: copy_from oid 16 from oid 13 current snap is 167 2024-03-20T08:07:04.096 INFO:tasks.rados.rados.0.smithi118.stdout:1619: copy_from oid 49 from oid 48 current snap is 167 2024-03-20T08:07:04.096 INFO:tasks.rados.rados.0.smithi118.stdout:1620: snap_remove snap 166 2024-03-20T08:07:04.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:03.999+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.011+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.015+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.029+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.033+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.048+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.254 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.052+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.063+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.071+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.082+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.090+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.102+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.117+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.124+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.134+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.141+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.152+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.156+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.169+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.172+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.186+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.187+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.203+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.203+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.219+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.223+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.237+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.240+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.253+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.256 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:04 smithi118 systemd[1]: Stopping Ceph osd.3 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:07:04.504 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[61973]: 2024-03-20T08:07:04.420+0000 7f5bcd10f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:07:04.504 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[61973]: 2024-03-20T08:07:04.421+0000 7f5bcd10f700 -1 osd.3 396 *** Got signal Terminated *** 2024-03-20T08:07:04.504 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[61973]: 2024-03-20T08:07:04.421+0000 7f5bcd10f700 -1 osd.3 396 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.258+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.272+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.275+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.289+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.293+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.305+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.309+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.323+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.324+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.341+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.341+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.356+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.358+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.370+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.373+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.384+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.389+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.398+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.406+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.414+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.424+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.429+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.440+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.444+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.456+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.459+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.472+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.474+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.488+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.489+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.504+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.507 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.504+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1620: done (3 left) 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1621: copy_from oid 1 from oid 3 current snap is 167 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1622: rmattr oid 27 current snap is 167 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1622: done (4 left) 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1623: snap_create 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1619: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1618: finishing copy_from racing read to smithi11881747-16 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1618: finishing copy_from to smithi11881747-16 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 997 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1619: finishing copy_from to smithi11881747-49 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 7035 (ObjNum 104 snap 23 seq_num 104) dirty exists 2024-03-20T08:07:04.696 INFO:tasks.rados.rados.0.smithi118.stdout:1617: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:07:04.755 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:04.693+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:04.756 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[93121]: osdmap e396: 8 total, 8 up, 8 in 2024-03-20T08:07:04.756 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:04.756 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset e2=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7],e2=[e2,e1,e0,df,de,dd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:04.757 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[93121]: pgmap v271: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 425 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 24 MiB/s rd, 2.5 MiB/s wr, 12 op/s 2024-03-20T08:07:04.757 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[93121]: osd.3 marked itself down and dead 2024-03-20T08:07:04.757 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[95692]: osdmap e396: 8 total, 8 up, 8 in 2024-03-20T08:07:04.757 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:04.757 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset e2=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7],e2=[e2,e1,e0,df,de,dd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:04.757 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[95692]: pgmap v271: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 425 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 24 MiB/s rd, 2.5 MiB/s wr, 12 op/s 2024-03-20T08:07:04.757 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-mon[95692]: osd.3 marked itself down and dead 2024-03-20T08:07:04.757 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.520+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.521+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.536+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.537+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.552+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.552+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.567+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.567+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.583+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.586+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.598+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.603+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.613+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.758 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.619+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.629+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.636+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.644+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.652+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.660+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.667+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.675+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.684+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.694+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.720+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.721+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.736+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.740+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.754+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:04.759 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.756+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:04.894 INFO:tasks.rados.rados.0.smithi118.stdout:1621: finishing copy_from racing read to smithi11881747-1 2024-03-20T08:07:04.895 INFO:tasks.rados.rados.0.smithi118.stdout:1621: finishing copy_from to smithi11881747-1 2024-03-20T08:07:04.895 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 787 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:07:05.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.773+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.773+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.790+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.790+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.806+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.807+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.822+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.824+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.837+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.841+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.854+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.858+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.870+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.875+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.885+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.900+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.009 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.908+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.917+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.924+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.932+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.940+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.947+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.955+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.961+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.973+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.978+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.989+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:04.992+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.010 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:04 smithi196 ceph-mon[87080]: osdmap e396: 8 total, 8 up, 8 in 2024-03-20T08:07:05.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:05.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset e2=[]:{7b=[6b,65],95=[95,91,8b,8a,89,83],9f=[9f,9d],b1=[b1,b0,af],b7=[b7,b6,b4,b3,b2],bb=[bb,ba,b8],c3=[c3,c2,c1,c0,bf,be,bd,bc],c8=[c8,c7,c6,c5,c4],cc=[cc,cb,ca,c9],cf=[cf,ce,cd],d6=[d6,d4,d3,d1],da=[da,d9,d8,d7],e2=[e2,e1,e0,df,de,dd]} for object 5:c4fe5974:::smithi11881747-36:5a 2024-03-20T08:07:05.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:04 smithi196 ceph-mon[87080]: pgmap v271: 121 pgs: 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 113 active+clean, 2 active+clean+snaptrim_error; 425 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 24 MiB/s rd, 2.5 MiB/s wr, 12 op/s 2024-03-20T08:07:05.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:04 smithi196 ceph-mon[87080]: osd.3 marked itself down and dead 2024-03-20T08:07:05.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.007+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.022+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.037+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.038+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.052+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.053+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.066+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.070+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.082+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.086+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.098+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.102+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.113+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.118+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.127+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.134+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.143+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.151+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.159+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.168+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.178+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.186+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.195+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.203+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.214+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.220+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.229+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.236+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.244+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.254+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.260+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.263 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:05 smithi118 podman[116043]: 2024-03-20 08:07:05.184774624 +0000 UTC m=+0.896793726 container died 79c58e60b2c58d9b0ce81598431545b827497168ef18578607f1945f41fa1ee6 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=reef-78dbc53, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-20T08:07:05.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.270+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.277+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.287+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.292+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.303+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.306+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.319+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.322+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.336+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.337+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.351+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.353+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.366+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.369+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.382+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.385+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.398+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.402+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.415+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.419+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.430+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.435+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.445+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.451+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.461+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.467+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.475+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.484+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.492+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.500+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.508+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.516+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:1617: done (4 left) 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:1618: done (3 left) 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:1619: done (2 left) 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:1621: done (1 left) 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:1623: done (0 left) 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:1624: rollback oid 43 current snap is 168 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 43 to 162 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 168 2024-03-20T08:07:05.710 INFO:tasks.rados.rados.0.smithi118.stdout:1625: seq_num 506 ranges {1114112=483328,1597440=761856,2359296=40960} 2024-03-20T08:07:05.712 INFO:tasks.rados.rados.0.smithi118.stdout:1625: writing smithi11881747-4 from 1114112 to 1597440 tid 1 2024-03-20T08:07:05.718 INFO:tasks.rados.rados.0.smithi118.stdout:1625: writing smithi11881747-4 from 1597440 to 2359296 tid 2 2024-03-20T08:07:05.719 INFO:tasks.rados.rados.0.smithi118.stdout:1625: writing smithi11881747-4 from 2359296 to 2400256 tid 3 2024-03-20T08:07:05.719 INFO:tasks.rados.rados.0.smithi118.stdout:1626: read oid 18 snap -1 2024-03-20T08:07:05.719 INFO:tasks.rados.rados.0.smithi118.stdout:1626: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:07:05.719 INFO:tasks.rados.rados.0.smithi118.stdout:1627: read oid 50 snap -1 2024-03-20T08:07:05.719 INFO:tasks.rados.rados.0.smithi118.stdout:1627: expect deleted 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:1628: rollback oid 2 current snap is 168 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 108 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:1629: copy_from oid 6 from oid 26 current snap is 168 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:1630: delete oid 47 current snap is 168 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:1624: finishing rollback tid 0 to smithi11881747-43 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:1624: finishing rollback tid 1 to smithi11881747-43 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:1624: finishing rollback tid 2 to smithi11881747-43 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 1010 (ObjNum 341 snap 111 seq_num 341) dirty exists 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:1625: finishing write tid 1 to smithi11881747-4 2024-03-20T08:07:05.720 INFO:tasks.rados.rados.0.smithi118.stdout:1626: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:07:05.728 INFO:tasks.rados.rados.0.smithi118.stdout:1628: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:07:05.728 INFO:tasks.rados.rados.0.smithi118.stdout:1625: finishing write tid 2 to smithi11881747-4 2024-03-20T08:07:05.728 INFO:tasks.rados.rados.0.smithi118.stdout:1628: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:07:05.728 INFO:tasks.rados.rados.0.smithi118.stdout:1625: finishing write tid 3 to smithi11881747-4 2024-03-20T08:07:05.728 INFO:tasks.rados.rados.0.smithi118.stdout:1625: finishing write tid 4 to smithi11881747-4 2024-03-20T08:07:05.728 INFO:tasks.rados.rados.0.smithi118.stdout:1625: finishing write tid 5 to smithi11881747-4 2024-03-20T08:07:05.728 INFO:tasks.rados.rados.0.smithi118.stdout:1625: oid 4 updating version 0 to 795 2024-03-20T08:07:05.728 INFO:tasks.rados.rados.0.smithi118.stdout:1625: oid 4 updating version 795 to 796 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1625: oid 4 version 796 is already newer than 794 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1625: oid 4 updating version 796 to 797 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 797 (ObjNum 506 snap 168 seq_num 506) dirty exists 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1625: left oid 4 (ObjNum 506 snap 168 seq_num 506) 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1624: done (6 left) 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1625: done (5 left) 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1626: done (4 left) 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1627: done (3 left) 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1630: done (2 left) 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1631: rollback oid 48 current snap is 168 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 48 to 164 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1628: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 1274 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:1628: done (2 left) 2024-03-20T08:07:05.729 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 168 2024-03-20T08:07:05.730 INFO:tasks.rados.rados.0.smithi118.stdout:1632: seq_num 508 ranges {0=516096,516096=524288,1040384=278528} 2024-03-20T08:07:05.731 INFO:tasks.rados.rados.0.smithi118.stdout:1632: writing smithi11881747-37 from 0 to 516096 tid 1 2024-03-20T08:07:05.734 INFO:tasks.rados.rados.0.smithi118.stdout:1632: writing smithi11881747-37 from 516096 to 1040384 tid 2 2024-03-20T08:07:05.737 INFO:tasks.rados.rados.0.smithi118.stdout:1632: writing smithi11881747-37 from 1040384 to 1318912 tid 3 2024-03-20T08:07:05.737 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 168 2024-03-20T08:07:05.737 INFO:tasks.rados.rados.0.smithi118.stdout:1633: seq_num 509 ranges {4521984=532480,5054464=409600,5464064=393216} 2024-03-20T08:07:05.766 INFO:tasks.rados.rados.0.smithi118.stdout:1633: writing smithi11881747-25 from 4521984 to 5054464 tid 1 2024-03-20T08:07:05.768 INFO:tasks.rados.rados.0.smithi118.stdout:1633: writing smithi11881747-25 from 5054464 to 5464064 tid 2 2024-03-20T08:07:05.773 INFO:tasks.rados.rados.0.smithi118.stdout:1633: writing smithi11881747-25 from 5464064 to 5857280 tid 3 2024-03-20T08:07:05.773 INFO:tasks.rados.rados.0.smithi118.stdout:1629: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1629: finishing copy_from to smithi11881747-6 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 806 (ObjNum 449 snap 149 seq_num 449) dirty exists 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1631: finishing rollback tid 0 to smithi11881747-48 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1631: finishing rollback tid 1 to smithi11881747-48 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1631: finishing rollback tid 2 to smithi11881747-48 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 7042 (ObjNum 104 snap 23 seq_num 104) dirty exists 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1632: finishing write tid 1 to smithi11881747-37 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1632: finishing write tid 2 to smithi11881747-37 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1632: finishing write tid 3 to smithi11881747-37 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1632: finishing write tid 4 to smithi11881747-37 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1632: finishing write tid 5 to smithi11881747-37 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1632: oid 37 updating version 0 to 7043 2024-03-20T08:07:05.774 INFO:tasks.rados.rados.0.smithi118.stdout:1632: oid 37 version 7043 is already newer than 7040 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1632: oid 37 version 7043 is already newer than 7041 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1632: oid 37 version 7043 is already newer than 7042 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 7043 (ObjNum 508 snap 168 seq_num 508) dirty exists 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1632: left oid 37 (ObjNum 508 snap 168 seq_num 508) 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1629: done (3 left) 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1631: done (2 left) 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1632: done (1 left) 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1634: read oid 22 snap -1 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1634: expect deleted 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1635: delete oid 43 current snap is 168 2024-03-20T08:07:05.775 INFO:tasks.rados.rados.0.smithi118.stdout:1633: finishing write tid 1 to smithi11881747-25 2024-03-20T08:07:05.776 INFO:tasks.rados.rados.0.smithi118.stdout:1634: done (2 left) 2024-03-20T08:07:05.776 INFO:tasks.rados.rados.0.smithi118.stdout:1635: done (1 left) 2024-03-20T08:07:05.776 INFO:tasks.rados.rados.0.smithi118.stdout:1636: copy_from oid 23 from oid 31 current snap is 168 2024-03-20T08:07:05.776 INFO:tasks.rados.rados.0.smithi118.stdout:1637: rmattr oid 21 current snap is 168 2024-03-20T08:07:05.776 INFO:tasks.rados.rados.0.smithi118.stdout:1638: snap_remove snap 137 2024-03-20T08:07:05.781 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:05.706+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:05.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-mon[93121]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:07:05.782 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-mon[93121]: osdmap e397: 8 total, 7 up, 8 in 2024-03-20T08:07:05.782 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:05.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-mon[95692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:07:05.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-mon[95692]: osdmap e397: 8 total, 7 up, 8 in 2024-03-20T08:07:05.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:05.782 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:05 smithi118 podman[116043]: 2024-03-20 08:07:05.609683335 +0000 UTC m=+1.321702441 container cleanup 79c58e60b2c58d9b0ce81598431545b827497168ef18578607f1945f41fa1ee6 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:05 smithi118 bash[116043]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.523+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.533+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.538+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.549+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.554+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.565+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.569+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.581+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.585+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.599+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.600+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.615+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.616+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.632+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.635+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.651+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.657+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.668+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.674+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.683+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.691+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.699+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.720+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.728+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.742+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.747+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.759+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:05.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.763+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:05.785 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.782+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.785+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.798+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.799+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.814+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.815+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.831+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.831+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.846+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.847+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.861+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.863+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.876+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.879+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.892+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.897+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.908+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.913+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.924+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.931+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.939+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.948+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.955+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.965+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.970+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.982+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:05.984+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.001+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.001+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.018+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.042 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.021+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.043 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:05 smithi118 podman[116063]: 2024-03-20 08:07:05.903193149 +0000 UTC m=+0.712774701 container remove 79c58e60b2c58d9b0ce81598431545b827497168ef18578607f1945f41fa1ee6 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=reef-78dbc53, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-20T08:07:06.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:05 smithi196 ceph-mon[87080]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:07:06.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:05 smithi196 ceph-mon[87080]: osdmap e397: 8 total, 7 up, 8 in 2024-03-20T08:07:06.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:06.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.038+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.038+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.055+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.057+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.072+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.074+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.089+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.093+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.104+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.112+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.121+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.129+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.139+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.148+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.158+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.166+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.175+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.184+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.192+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.201+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.209+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.219+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.224+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.240+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.241+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.257+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.259+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.273+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.280+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.301 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.290+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.302 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.299+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.558 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:06 smithi118 podman[116142]: 2024-03-20 08:07:06.431250672 +0000 UTC m=+0.028090078 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.309+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.317+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.326+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.336+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.345+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.354+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.364+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.372+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.383+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.399+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.409+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.416+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.427+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.434+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.446+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.451+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.462+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.465+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.479+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.480+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.495+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.496+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.561 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.512+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.561 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.512+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.561 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.528+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.561 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.528+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.561 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.544+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.561 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.545+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.561 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.559+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.714 INFO:tasks.rados.rados.0.smithi118.stdout:1638: done (3 left) 2024-03-20T08:07:06.714 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 168 2024-03-20T08:07:06.714 INFO:tasks.rados.rados.0.smithi118.stdout:1639: seq_num 511 ranges {2031616=565248,2596864=630784,3227648=172032} 2024-03-20T08:07:06.730 INFO:tasks.rados.rados.0.smithi118.stdout:1639: writing smithi11881747-19 from 2031616 to 2596864 tid 1 2024-03-20T08:07:06.734 INFO:tasks.rados.rados.0.smithi118.stdout:1639: writing smithi11881747-19 from 2596864 to 3227648 tid 2 2024-03-20T08:07:06.736 INFO:tasks.rados.rados.0.smithi118.stdout:1639: writing smithi11881747-19 from 3227648 to 3399680 tid 3 2024-03-20T08:07:06.736 INFO:tasks.rados.rados.0.smithi118.stdout:1633: finishing write tid 2 to smithi11881747-25 2024-03-20T08:07:06.736 INFO:tasks.rados.rados.0.smithi118.stdout:1633: finishing write tid 3 to smithi11881747-25 2024-03-20T08:07:06.736 INFO:tasks.rados.rados.0.smithi118.stdout:1633: finishing write tid 4 to smithi11881747-25 2024-03-20T08:07:06.736 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 7043 (ObjNum 443 snap 148 seq_num 443) dirty exists 2024-03-20T08:07:06.736 INFO:tasks.rados.rados.0.smithi118.stdout:1636: finishing copy_from to smithi11881747-23 2024-03-20T08:07:06.736 INFO:tasks.rados.rados.0.smithi118.stdout:1636: got expected ENOENT (src dne) 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1633: finishing write tid 5 to smithi11881747-25 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1633: oid 25 updating version 0 to 1017 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1633: oid 25 updating version 1017 to 1019 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1633: oid 25 updating version 1019 to 1020 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1633: oid 25 version 1020 is already newer than 1018 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 1020 (ObjNum 509 snap 168 seq_num 509) dirty exists 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1633: left oid 25 (ObjNum 509 snap 168 seq_num 509) 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1636: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1633: done (3 left) 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1636: done (2 left) 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1637: done (1 left) 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1640: copy_from oid 45 from oid 38 current snap is 168 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1641: rollback oid 31 current snap is 168 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 161 2024-03-20T08:07:06.737 INFO:tasks.rados.rados.0.smithi118.stdout:1642: rollback oid 16 current snap is 168 2024-03-20T08:07:06.738 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 134 2024-03-20T08:07:06.738 INFO:tasks.rados.rados.0.smithi118.stdout:1643: snap_remove snap 108 2024-03-20T08:07:06.818 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:06.717+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:06.818 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-mon[93121]: osdmap e398: 8 total, 7 up, 8 in 2024-03-20T08:07:06.818 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:06.818 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-mon[93121]: pgmap v274: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 7 active+undersized, 1 active+undersized+degraded, 13 stale+active+clean, 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 92 active+clean; 426 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 26 MiB/s rd, 2.8 MiB/s wr, 19 op/s; 23/951 objects degraded (2.419%) 2024-03-20T08:07:06.819 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.561+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.819 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.576+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.819 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.579+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.819 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.590+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.819 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.597+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.819 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.605+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.820 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.613+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.820 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.620+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.820 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.630+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.820 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.636+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.820 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.648+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.820 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.651+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.820 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.664+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.820 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.667+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.681+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.683+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.701+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.708+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.729+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.734+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.751+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.753+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.769+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.773+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.821 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.786+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.822 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.789+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.822 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.803+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:06.822 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.805+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:06.822 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-mon[95692]: osdmap e398: 8 total, 7 up, 8 in 2024-03-20T08:07:06.822 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:06.822 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-mon[95692]: pgmap v274: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 7 active+undersized, 1 active+undersized+degraded, 13 stale+active+clean, 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 92 active+clean; 426 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 26 MiB/s rd, 2.8 MiB/s wr, 19 op/s; 23/951 objects degraded (2.419%) 2024-03-20T08:07:06.822 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:06 smithi118 podman[116142]: 2024-03-20 08:07:06.563134543 +0000 UTC m=+0.159973950 container create 5bd7c841570222fbf08f6ac3b0d5ec271719c9f7507a7ea381f2eaee7447bb4c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-03-20T08:07:07.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.818+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.823+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.846+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.847+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.863+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.863+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.879+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.880+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.893+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.896+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.909+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.912+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.925+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.933+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.941+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.950+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.956+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.968+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.972+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.984+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:06.986+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.003+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.004+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.020+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.020+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.036+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.038+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.052+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.057+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.068+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.077 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:06 smithi118 podman[116142]: 2024-03-20 08:07:06.99266296 +0000 UTC m=+0.589502365 container init 5bd7c841570222fbf08f6ac3b0d5ec271719c9f7507a7ea381f2eaee7447bb4c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-20T08:07:07.077 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:07 smithi118 podman[116142]: 2024-03-20 08:07:07.000751388 +0000 UTC m=+0.597590804 container start 5bd7c841570222fbf08f6ac3b0d5ec271719c9f7507a7ea381f2eaee7447bb4c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:07:07.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:06 smithi196 ceph-mon[87080]: osdmap e398: 8 total, 7 up, 8 in 2024-03-20T08:07:07.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:07.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:06 smithi196 ceph-mon[87080]: pgmap v274: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 7 active+undersized, 1 active+undersized+degraded, 13 stale+active+clean, 3 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 92 active+clean; 426 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 26 MiB/s rd, 2.8 MiB/s wr, 19 op/s; 23/951 objects degraded (2.419%) 2024-03-20T08:07:07.337 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:07 smithi118 podman[116142]: 2024-03-20 08:07:07.110073668 +0000 UTC m=+0.706913076 container attach 5bd7c841570222fbf08f6ac3b0d5ec271719c9f7507a7ea381f2eaee7447bb4c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.085+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.092+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.102+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.111+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.119+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.130+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.135+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.148+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.152+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.166+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.169+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.185+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.185+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.203+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.205+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.219+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.225+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.237+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.245+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.253+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.264+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.270+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.340 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.281+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.340 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.288+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.340 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.302+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.340 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.305+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.340 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.321+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.340 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.322+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.340 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.338+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.340 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.338+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.598 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:07 smithi118 podman[116142]: 2024-03-20 08:07:07.415651755 +0000 UTC m=+1.012491160 container died 5bd7c841570222fbf08f6ac3b0d5ec271719c9f7507a7ea381f2eaee7447bb4c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:07:07.599 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.356+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.599 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.356+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.599 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.372+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.599 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.374+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.599 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.391+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.599 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.391+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.407+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.410+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.427+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.429+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.446+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.448+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.462+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.466+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.480+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.485+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.498+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.503+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.516+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.522+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.534+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.541+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.549+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.560+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.568+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.578+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.586+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.596+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.712 INFO:tasks.rados.rados.0.smithi118.stdout:1639: finishing write tid 1 to smithi11881747-19 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1642: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1639: finishing write tid 2 to smithi11881747-19 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1639: finishing write tid 3 to smithi11881747-19 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1639: finishing write tid 4 to smithi11881747-19 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1641: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1642: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1642: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 1294 (ObjNum 381 snap 129 seq_num 381) dirty exists 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1639: finishing write tid 5 to smithi11881747-19 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1639: oid 19 updating version 0 to 406 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1639: oid 19 version 406 is already newer than 403 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1639: oid 19 version 406 is already newer than 404 2024-03-20T08:07:07.713 INFO:tasks.rados.rados.0.smithi118.stdout:1639: oid 19 version 406 is already newer than 405 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 406 (ObjNum 511 snap 168 seq_num 511) dirty exists 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1639: left oid 19 (ObjNum 511 snap 168 seq_num 511) 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1641: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 1013 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1640: finishing copy_from racing read to smithi11881747-45 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1640: finishing copy_from to smithi11881747-45 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 7039 (ObjNum 500 snap 165 seq_num 500) dirty exists 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1639: done (4 left) 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1640: done (3 left) 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1641: done (2 left) 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1642: done (1 left) 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1643: done (0 left) 2024-03-20T08:07:07.714 INFO:tasks.rados.rados.0.smithi118.stdout:1644: snap_remove snap 163 2024-03-20T08:07:07.857 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:07.717+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:07.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-mon[93121]: Health check failed: Degraded data redundancy: 23/951 objects degraded (2.419%), 2 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:07.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-mon[93121]: osdmap e399: 8 total, 7 up, 8 in 2024-03-20T08:07:07.858 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:07.858 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:07.734+0000 7fc1fae3c700 -1 osd.1 pg_epoch: 400 pg[5.9s2( v 400'766 (0'0,400'766] local-lis/les=336/337 n=18 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [7,6,1]p7(0) r=2 lpr=336 luod=0'0 crt=400'764 lcod 400'764 mlcod 400'764 active mbc={} ps=60] update_snap_map remove_oid 5:9d5f254b:::smithi11881747-6:aa failed with -2 2024-03-20T08:07:07.858 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:07.734+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e1=[]:{a4=[a2,9f,9d,95],bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7]} for object 5:36b533bd:::smithi11881747-14:ad 2024-03-20T08:07:07.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-mon[95692]: Health check failed: Degraded data redundancy: 23/951 objects degraded (2.419%), 2 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:07.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-mon[95692]: osdmap e399: 8 total, 7 up, 8 in 2024-03-20T08:07:07.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.604+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.614+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.621+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.633+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.639+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.650+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.656+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.669+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.673+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.688+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.693+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.710+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.717+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.727+0000 7f831b6a8700 -1 osd.2 pg_epoch: 400 pg[5.4s1( v 399'201 (0'0,399'201] local-lis/les=376/377 n=6 ec=73/73 lis/c=376/376 les/c/f=377/377/0 sis=376) [7,2,5]p7(0) r=1 lpr=376 luod=0'0 crt=399'201 mlcod 399'201 active mbc={} ps=60] update_snap_map remove_oid 5:27618338:::smithi11881747-17:a8 failed with -2 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.738+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.740+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.752+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.755+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.767+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.770+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.782+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.786+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.799+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.860 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.800+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.815+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.815+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.830+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.830+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.843+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:07.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.846+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:07.861 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.858+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.862+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.873+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.878+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.886+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.895+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.899+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.911+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.915+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.927+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.929+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.112 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.942+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.944+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.957+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.959+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.973+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.976+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.989+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:07.992+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.003+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.008+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.113 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.017+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.024+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.034+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.043+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.050+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.059+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.065+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.081+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.089+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.098+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.104+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.114 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.111+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:07 smithi196 ceph-mon[87080]: Health check failed: Degraded data redundancy: 23/951 objects degraded (2.419%), 2 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:08.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:07 smithi196 ceph-mon[87080]: osdmap e399: 8 total, 7 up, 8 in 2024-03-20T08:07:08.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:08.362 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:08 smithi118 podman[116187]: 2024-03-20 08:07:08.139601117 +0000 UTC m=+0.717466922 container remove 5bd7c841570222fbf08f6ac3b0d5ec271719c9f7507a7ea381f2eaee7447bb4c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True) 2024-03-20T08:07:08.362 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:08 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-20T08:07:08.362 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:08 smithi118 systemd[1]: Stopped Ceph osd.3 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:07:08.362 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:08 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.3.service: Consumed 20.067s CPU time. 2024-03-20T08:07:08.363 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.363 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.127+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.363 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.137+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.363 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.142+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.363 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.155+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.363 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.159+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.171+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.176+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.188+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.193+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.210+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.237+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.247+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.253+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.264+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.268+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.280+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.284+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.364 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.299+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.316+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.330+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.332+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.346+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.348+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.363+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.363+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.619 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:08 smithi118 systemd[1]: Starting Ceph osd.3 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:07:08.619 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.379+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.619 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.379+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.619 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.396+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.396+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.413+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.413+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.430+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.431+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.447+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.447+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.462+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.464+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.480+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.480+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.497+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.498+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.620 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.513+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.517+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.529+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.535+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.546+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.553+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.563+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.572+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.580+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.590+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.622 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.595+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.622 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.604+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.622 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.608+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.622 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.619+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.719 INFO:tasks.rados.rados.0.smithi118.stdout:1644: done (0 left) 2024-03-20T08:07:08.719 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 168 2024-03-20T08:07:08.719 INFO:tasks.rados.rados.0.smithi118.stdout:1645: seq_num 512 ranges {778240=581632,1359872=516096,1875968=286720} 2024-03-20T08:07:08.727 INFO:tasks.rados.rados.0.smithi118.stdout:1645: writing smithi11881747-24 from 778240 to 1359872 tid 1 2024-03-20T08:07:08.731 INFO:tasks.rados.rados.0.smithi118.stdout:1645: writing smithi11881747-24 from 1359872 to 1875968 tid 2 2024-03-20T08:07:08.734 INFO:tasks.rados.rados.0.smithi118.stdout:1645: writing smithi11881747-24 from 1875968 to 2162688 tid 3 2024-03-20T08:07:08.734 INFO:tasks.rados.rados.0.smithi118.stdout:1646: read oid 5 snap -1 2024-03-20T08:07:08.734 INFO:tasks.rados.rados.0.smithi118.stdout:1646: expect (ObjNum 495 snap 164 seq_num 495) 2024-03-20T08:07:08.734 INFO:tasks.rados.rados.0.smithi118.stdout:1647: read oid 33 snap 135 2024-03-20T08:07:08.734 INFO:tasks.rados.rados.0.smithi118.stdout:1647: expect (ObjNum 343 snap 112 seq_num 343) 2024-03-20T08:07:08.734 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 168 2024-03-20T08:07:08.734 INFO:tasks.rados.rados.0.smithi118.stdout:1648: seq_num 513 ranges {1753088=598016,2351104=794624,3145728=8192} 2024-03-20T08:07:08.748 INFO:tasks.rados.rados.0.smithi118.stdout:1648: writing smithi11881747-49 from 1753088 to 2351104 tid 1 2024-03-20T08:07:08.754 INFO:tasks.rados.rados.0.smithi118.stdout:1648: writing smithi11881747-49 from 2351104 to 3145728 tid 2 2024-03-20T08:07:08.754 INFO:tasks.rados.rados.0.smithi118.stdout:1648: writing smithi11881747-49 from 3145728 to 3153920 tid 3 2024-03-20T08:07:08.754 INFO:tasks.rados.rados.0.smithi118.stdout:1649: delete oid 19 current snap is 168 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: finishing write tid 1 to smithi11881747-24 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: finishing write tid 2 to smithi11881747-24 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: finishing write tid 3 to smithi11881747-24 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: finishing write tid 4 to smithi11881747-24 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: finishing write tid 5 to smithi11881747-24 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: oid 24 updating version 0 to 822 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: oid 24 version 822 is already newer than 821 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: oid 24 updating version 822 to 824 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: oid 24 version 824 is already newer than 823 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 824 (ObjNum 512 snap 168 seq_num 512) dirty exists 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1645: left oid 24 (ObjNum 512 snap 168 seq_num 512) 2024-03-20T08:07:08.760 INFO:tasks.rados.rados.0.smithi118.stdout:1646: expect (ObjNum 495 snap 164 seq_num 495) 2024-03-20T08:07:08.874 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:08.728+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:08.874 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:08.741+0000 7f0cedc5b700 -1 osd.0 pg_epoch: 401 pg[5.cs2( v 401'33781 (380'27101,401'33781] local-lis/les=336/337 n=12 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [1,4,0]p1(0) r=2 lpr=336 luod=0'0 crt=400'33779 mlcod 400'33779 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~2,a9~1,ab~1,af~1,b1~1,b5~1,b8~2]] update_snap_map remove_oid 5:332bf1c7:::smithi11881747-32:9f failed with -2 2024-03-20T08:07:08.874 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-mon[93121]: osdmap e400: 8 total, 7 up, 8 in 2024-03-20T08:07:08.874 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:08.874 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset e1=[]:{a4=[a2,9f,9d,95],bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7]} for object 5:36b533bd:::smithi11881747-14:ad 2024-03-20T08:07:08.874 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-mon[93121]: pgmap v277: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 15 active+undersized, 5 active+undersized+degraded, 10 stale+active+clean, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 84 active+clean; 449 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 13 op/s; 90/957 objects degraded (9.404%) 2024-03-20T08:07:08.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-mon[95692]: osdmap e400: 8 total, 7 up, 8 in 2024-03-20T08:07:08.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:08.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset e1=[]:{a4=[a2,9f,9d,95],bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7]} for object 5:36b533bd:::smithi11881747-14:ad 2024-03-20T08:07:08.875 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-mon[95692]: pgmap v277: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 15 active+undersized, 5 active+undersized+degraded, 10 stale+active+clean, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 84 active+clean; 449 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 13 op/s; 90/957 objects degraded (9.404%) 2024-03-20T08:07:08.875 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:08.741+0000 7fc1f9639700 -1 osd.1 pg_epoch: 401 pg[5.cs0( v 401'33781 (380'27101,401'33781] local-lis/les=336/337 n=12 ec=73/73 lis/c=336/336 les/c/f=337/337/0 sis=336) [1,4,0]p1(0) r=0 lpr=336 luod=400'33779 crt=400'33779 lcod 400'33777 mlcod 400'33777 active+clean+snaptrim trimq=44 ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~2,a9~1,ab~1,af~1,b1~1,b5~1,b8~2]] update_snap_map remove_oid 5:332bf1c7:::smithi11881747-32:9f failed with -2 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:08.755+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.624+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.636+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.637+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.652+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.652+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.667+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.668+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.876 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.681+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.681+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.694+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.697+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.710+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.742+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.743+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.764+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.769+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.782+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.785+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.798+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.799+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.877 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.813+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.814+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.827+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.830+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.841+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.844+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.859+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.861+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:08 smithi118 podman[116294]: 2024-03-20 08:07:08.576658966 +0000 UTC m=+0.023410756 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:07:08.878 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:08 smithi118 podman[116294]: 2024-03-20 08:07:08.732898908 +0000 UTC m=+0.179650683 container create 728d736004749ede90b5b8c2f236b2c8bc6b03268d284ecba5b7c619179a6e26 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-20T08:07:08.994 INFO:tasks.rados.rados.0.smithi118.stdout:1648: finishing write tid 1 to smithi11881747-49 2024-03-20T08:07:08.995 INFO:tasks.rados.rados.0.smithi118.stdout:1645: done (4 left) 2024-03-20T08:07:08.995 INFO:tasks.rados.rados.0.smithi118.stdout:1646: done (3 left) 2024-03-20T08:07:08.995 INFO:tasks.rados.rados.0.smithi118.stdout:1649: done (2 left) 2024-03-20T08:07:08.995 INFO:tasks.rados.rados.0.smithi118.stdout:1650: snap_remove snap 162 2024-03-20T08:07:09.129 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 podman[116294]: 2024-03-20 08:07:09.011306238 +0000 UTC m=+0.458058020 container init 728d736004749ede90b5b8c2f236b2c8bc6b03268d284ecba5b7c619179a6e26 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:07:09.129 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 podman[116294]: 2024-03-20 08:07:09.019251816 +0000 UTC m=+0.466003583 container start 728d736004749ede90b5b8c2f236b2c8bc6b03268d284ecba5b7c619179a6e26 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-20T08:07:09.129 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 podman[116294]: 2024-03-20 08:07:09.119600855 +0000 UTC m=+0.566352650 container attach 728d736004749ede90b5b8c2f236b2c8bc6b03268d284ecba5b7c619179a6e26 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True) 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.878+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.879+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.896+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.897+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.912+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.913+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.928+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.929+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.944+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.945+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.959+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.962+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.974+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.978+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.990+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:08.995+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.004+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.011+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.019+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.027+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.036+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.044+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.051+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.058+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.067+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.083+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.092+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.099+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.108+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.115+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.123+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.132 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.129+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:08 smithi196 ceph-mon[87080]: osdmap e400: 8 total, 7 up, 8 in 2024-03-20T08:07:09.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:09.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset e1=[]:{a4=[a2,9f,9d,95],bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7]} for object 5:36b533bd:::smithi11881747-14:ad 2024-03-20T08:07:09.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:08 smithi196 ceph-mon[87080]: pgmap v277: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 15 active+undersized, 5 active+undersized+degraded, 10 stale+active+clean, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 84 active+clean; 449 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 13 op/s; 90/957 objects degraded (9.404%) 2024-03-20T08:07:09.392 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.138+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.392 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.145+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.392 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.154+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.392 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.158+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.169+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.173+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.184+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.186+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.199+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.200+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.214+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.215+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.229+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.244+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.246+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.259+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.261+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.275+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.277+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.289+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.293+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.304+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.309+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.317+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.326+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.333+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.342+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.348+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.359+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.363+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.377+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.377+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.392+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.393+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.408+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.410+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.422+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.426+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.437+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.441+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.451+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.456+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.467+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.470+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.481+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.486+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.496+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.502+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.510+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.519+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.524+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate[116308]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate[116308]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 bash[116294]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-20T08:07:09.651 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 bash[116294]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 bash[116294]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 bash[116294]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 bash[116294]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 bash[116294]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 bash[116294]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate[116308]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate[116308]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate[116308]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-3/block 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate[116308]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate[116308]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:09 smithi118 podman[116294]: 2024-03-20 08:07:09.61892461 +0000 UTC m=+1.065676384 container died 728d736004749ede90b5b8c2f236b2c8bc6b03268d284ecba5b7c619179a6e26 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307) 2024-03-20T08:07:09.652 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.534+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.539+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.549+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.553+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.567+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.568+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.582+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.582+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.597+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.599+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.611+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.616+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.625+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.632+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.640+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.650+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.725 INFO:tasks.rados.rados.0.smithi118.stdout:1650: done (2 left) 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1651: read oid 2 snap -1 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1651: expect (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1652: read oid 27 snap -1 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1652: expect deleted 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1653: rollback oid 1 current snap is 168 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 1 to 135 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1648: finishing write tid 2 to smithi11881747-49 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1648: finishing write tid 3 to smithi11881747-49 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1648: finishing write tid 4 to smithi11881747-49 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1648: finishing write tid 5 to smithi11881747-49 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1648: oid 49 updating version 0 to 7054 2024-03-20T08:07:09.726 INFO:tasks.rados.rados.0.smithi118.stdout:1648: oid 49 updating version 7054 to 7055 2024-03-20T08:07:09.727 INFO:tasks.rados.rados.0.smithi118.stdout:1648: oid 49 updating version 7055 to 7057 2024-03-20T08:07:09.727 INFO:tasks.rados.rados.0.smithi118.stdout:1648: oid 49 version 7057 is already newer than 7056 2024-03-20T08:07:09.727 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 7057 (ObjNum 513 snap 168 seq_num 513) dirty exists 2024-03-20T08:07:09.727 INFO:tasks.rados.rados.0.smithi118.stdout:1648: left oid 49 (ObjNum 513 snap 168 seq_num 513) 2024-03-20T08:07:09.727 INFO:tasks.rados.rados.0.smithi118.stdout:1647: expect (ObjNum 343 snap 112 seq_num 343) 2024-03-20T08:07:09.865 INFO:tasks.rados.rados.0.smithi118.stdout:1653: finishing rollback tid 0 to smithi11881747-1 2024-03-20T08:07:09.865 INFO:tasks.rados.rados.0.smithi118.stdout:1653: finishing rollback tid 1 to smithi11881747-1 2024-03-20T08:07:09.865 INFO:tasks.rados.rados.0.smithi118.stdout:1653: finishing rollback tid 2 to smithi11881747-1 2024-03-20T08:07:09.865 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 1016 (ObjNum 391 snap 134 seq_num 391) dirty exists 2024-03-20T08:07:09.865 INFO:tasks.rados.rados.0.smithi118.stdout:1651: expect (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:07:09.906 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:09.729+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.654+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.664+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.667+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.680+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.680+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.694+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.694+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.707+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.708+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.719+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.724+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.744+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.753+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.761+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.767+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.775+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.783+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.908 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-mon[93121]: osdmap e401: 8 total, 7 up, 8 in 2024-03-20T08:07:09.908 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:09.908 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:09.909 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-mon[95692]: osdmap e401: 8 total, 7 up, 8 in 2024-03-20T08:07:09.909 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:09.909 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:09.909 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:09.727+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:09.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.790+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.798+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.804+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.813+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.818+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.828+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.833+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.843+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.847+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.859+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.860+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.875+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.875+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.892+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:09.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.892+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:09.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.907+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.117 INFO:tasks.rados.rados.0.smithi118.stdout:1647: done (4 left) 2024-03-20T08:07:10.117 INFO:tasks.rados.rados.0.smithi118.stdout:1648: done (3 left) 2024-03-20T08:07:10.117 INFO:tasks.rados.rados.0.smithi118.stdout:1651: done (2 left) 2024-03-20T08:07:10.117 INFO:tasks.rados.rados.0.smithi118.stdout:1652: done (1 left) 2024-03-20T08:07:10.117 INFO:tasks.rados.rados.0.smithi118.stdout:1653: done (0 left) 2024-03-20T08:07:10.117 INFO:tasks.rados.rados.0.smithi118.stdout:1654: rollback oid 10 current snap is 168 2024-03-20T08:07:10.117 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 164 2024-03-20T08:07:10.117 INFO:tasks.rados.rados.0.smithi118.stdout:1655: snap_create 2024-03-20T08:07:10.121 INFO:tasks.rados.rados.0.smithi118.stdout:1654: finishing rollback tid 0 to smithi11881747-10 2024-03-20T08:07:10.121 INFO:tasks.rados.rados.0.smithi118.stdout:1654: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:07:10.121 INFO:tasks.rados.rados.0.smithi118.stdout:1654: finishing rollback tid 2 to smithi11881747-10 2024-03-20T08:07:10.121 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 1019 (ObjNum 472 snap 156 seq_num 472) dirty exists 2024-03-20T08:07:10.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:09 smithi196 ceph-mon[87080]: osdmap e401: 8 total, 7 up, 8 in 2024-03-20T08:07:10.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:10.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:10.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.909+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.921+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.922+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.935+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.936+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.949+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.956+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.962+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.973+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.975+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.987+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:09.988+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.002+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.003+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.016+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.019+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.030+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.036+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.159 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.044+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.052+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.059+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.069+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.073+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.084+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.085+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.098+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.099+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.112+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.114+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.126+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.129+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.140+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.160 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.144+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.161 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.153+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.161 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.158+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.416 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:10 smithi118 podman[116460]: 2024-03-20 08:07:10.34277182 +0000 UTC m=+0.716671446 container remove 728d736004749ede90b5b8c2f236b2c8bc6b03268d284ecba5b7c619179a6e26 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-20T08:07:10.416 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.166+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.416 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.174+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.416 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.179+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.416 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.191+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.416 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.191+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.416 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.204+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.416 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.207+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.216+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.223+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.229+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.238+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.243+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.253+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.259+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.269+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.274+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.417 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.285+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.288+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.300+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.304+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.316+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.318+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.332+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.334+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.347+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.348+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.363+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.364+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.381+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.418 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.381+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.419 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.398+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.419 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.399+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.419 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.416+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.419 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.417+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.668 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.432+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.668 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.435+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.449+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.453+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.466+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.471+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.483+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.487+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.500+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.502+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.516+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.517+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.531+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.532+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.545+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.546+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.669 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.560+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.670 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:10 smithi118 podman[116542]: 2024-03-20 08:07:10.493504902 +0000 UTC m=+0.023877615 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:07:10.670 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:10 smithi118 podman[116542]: 2024-03-20 08:07:10.627357553 +0000 UTC m=+0.157730253 container create c7c15dc3876f3689a9d39e3aac9df13685f0319622c1dfddd5286289b23efc5e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:07:10.670 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.561+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.670 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.575+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.670 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.576+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.670 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.590+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.670 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.591+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.604+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.607+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.618+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.622+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.635+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.637+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.649+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.654+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.663+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.671 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.668+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.717 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:07:10.742 INFO:tasks.rados.rados.0.smithi118.stdout:1654: done (1 left) 2024-03-20T08:07:10.742 INFO:tasks.rados.rados.0.smithi118.stdout:1655: done (0 left) 2024-03-20T08:07:10.742 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 169 2024-03-20T08:07:10.742 INFO:tasks.rados.rados.0.smithi118.stdout:1656: seq_num 515 ranges {1146880=630784,1777664=565248,2342912=237568} 2024-03-20T08:07:10.752 INFO:tasks.rados.rados.0.smithi118.stdout:1656: writing smithi11881747-29 from 1146880 to 1777664 tid 1 2024-03-20T08:07:10.756 INFO:tasks.rados.rados.0.smithi118.stdout:1656: writing smithi11881747-29 from 1777664 to 2342912 tid 2 2024-03-20T08:07:10.758 INFO:tasks.rados.rados.0.smithi118.stdout:1656: writing smithi11881747-29 from 2342912 to 2580480 tid 3 2024-03-20T08:07:10.758 INFO:tasks.rados.rados.0.smithi118.stdout:1657: snap_remove snap 134 2024-03-20T08:07:10.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:10.751+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-mon[93121]: osdmap e402: 8 total, 7 up, 8 in 2024-03-20T08:07:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-mon[93121]: pgmap v280: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 75 active+clean; 431 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.3 MiB/s wr, 14 op/s; 106/912 objects degraded (11.623%) 2024-03-20T08:07:10.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-mon[95692]: osdmap e402: 8 total, 7 up, 8 in 2024-03-20T08:07:10.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:10.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:10.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-mon[95692]: pgmap v280: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 75 active+clean; 431 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.3 MiB/s wr, 14 op/s; 106/912 objects degraded (11.623%) 2024-03-20T08:07:10.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:10.752+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:10.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.678+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.686+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.690+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.701+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.704+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.716+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.719+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.734+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.734+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.756+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.761+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.773+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.776+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.790+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.795+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.806+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.812+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.821+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.829+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.836+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.846+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.852+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.863+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.869+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.878+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.887+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.894+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.902+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:10.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.909+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:10.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.916+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:10 smithi196 ceph-mon[87080]: osdmap e402: 8 total, 7 up, 8 in 2024-03-20T08:07:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:11.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:10 smithi196 ceph-mon[87080]: pgmap v280: 121 pgs: 1 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 75 active+clean; 431 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 5.9 MiB/s rd, 5.3 MiB/s wr, 14 op/s; 106/912 objects degraded (11.623%) 2024-03-20T08:07:11.168 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:07:11.168 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (6m) 26s ago 7m 27.0M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:07:11.168 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (7m) 2m ago 7m 86.9M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:07:11.168 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (8m) 26s ago 8m 15.2M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:07:11.168 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (8m) 2m ago 8m 19.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (3m) 2m ago 12m 449M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (4m) 26s ago 14m 507M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (3m) 26s ago 14m 53.8M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (2m) 2m ago 12m 26.6M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (2m) 26s ago 12m 47.3M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (7m) 26s ago 7m 15.3M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (7m) 2m ago 7m 15.8M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (109s) 26s ago 12m 291M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (73s) 26s ago 11m 213M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (33s) 26s ago 11m 13.7M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 starting - - - 2480M 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (10m) 2m ago 10m 181M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (10m) 2m ago 10m 133M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (9m) 2m ago 9m 117M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:07:11.169 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (9m) 2m ago 9m 115M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:07:11.170 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (3m) 2m ago 8m 56.2M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:07:11.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:10 smithi118 podman[116542]: 2024-03-20 08:07:10.931308205 +0000 UTC m=+0.461680923 container init c7c15dc3876f3689a9d39e3aac9df13685f0319622c1dfddd5286289b23efc5e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-20T08:07:11.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:10 smithi118 podman[116542]: 2024-03-20 08:07:10.938527731 +0000 UTC m=+0.468900433 container start c7c15dc3876f3689a9d39e3aac9df13685f0319622c1dfddd5286289b23efc5e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-20T08:07:11.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-osd[116600]: -- 172.21.15.118:0/673164386 <== mon.2 v2:172.21.15.118:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e67c77b680 con 0x55e67c744c00 2024-03-20T08:07:11.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:11 smithi118 bash[116542]: c7c15dc3876f3689a9d39e3aac9df13685f0319622c1dfddd5286289b23efc5e 2024-03-20T08:07:11.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:11 smithi118 systemd[1]: Started Ceph osd.3 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:07:11.179 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.925+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.931+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.944+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.948+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.960+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.964+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.977+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.979+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.993+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:10.994+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.007+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.008+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.180 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.022+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.022+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.035+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.038+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.052+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.067+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.072+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.085+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.089+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.102+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.119+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.121+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.132+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.137+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.147+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.153+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.161+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.170+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.178+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.189+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.196+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.213+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.221+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.229+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.238+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.246+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.256+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.263+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.273+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.281+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.291+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.296+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.309+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.312+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.328+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.329+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.344+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.359+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.438 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.366+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.438 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.373+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.438 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.386+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.438 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.388+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.438 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.403+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.438 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.403+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.438 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.418+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.438 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.422+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.697 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:11.481+0000 7f5a0f20c7c0 -1 Falling back to public interface 2024-03-20T08:07:11.697 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.436+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.697 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.439+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.453+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.456+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.467+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.474+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.484+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.494+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.506+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.512+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.522+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.529+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.537+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.698 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.548+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.551+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.566+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.568+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.582+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.585+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.600+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.602+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.617+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.699 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.618+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.633+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.634+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.650+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.667+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.667+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.682+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.684+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.697+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.750 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:07:11.750 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:07:11.750 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:07:11.750 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 4, 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 6, 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 8 2024-03-20T08:07:11.751 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:07:11.752 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:07:11.754 INFO:tasks.rados.rados.0.smithi118.stdout:1657: done (1 left) 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1658: rmattr oid 40 current snap is 169 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1659: delete oid 14 current snap is 169 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: finishing write tid 1 to smithi11881747-29 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: finishing write tid 2 to smithi11881747-29 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: finishing write tid 3 to smithi11881747-29 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: finishing write tid 4 to smithi11881747-29 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: finishing write tid 5 to smithi11881747-29 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: oid 29 updating version 0 to 1299 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: oid 29 version 1299 is already newer than 1297 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: oid 29 version 1299 is already newer than 1296 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: oid 29 version 1299 is already newer than 1298 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1299 (ObjNum 515 snap 169 seq_num 515) dirty exists 2024-03-20T08:07:11.755 INFO:tasks.rados.rados.0.smithi118.stdout:1656: left oid 29 (ObjNum 515 snap 169 seq_num 515) 2024-03-20T08:07:11.770 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 7044 (ObjNum 500 snap 165 seq_num 500) dirty exists 2024-03-20T08:07:11.770 INFO:tasks.rados.rados.0.smithi118.stdout:1656: done (2 left) 2024-03-20T08:07:11.771 INFO:tasks.rados.rados.0.smithi118.stdout:1658: done (1 left) 2024-03-20T08:07:11.771 INFO:tasks.rados.rados.0.smithi118.stdout:1659: done (0 left) 2024-03-20T08:07:11.771 INFO:tasks.rados.rados.0.smithi118.stdout:1660: read oid 10 snap -1 2024-03-20T08:07:11.771 INFO:tasks.rados.rados.0.smithi118.stdout:1660: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:07:11.771 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 169 2024-03-20T08:07:11.771 INFO:tasks.rados.rados.0.smithi118.stdout:1661: seq_num 517 ranges {2162688=663552,2826240=786432,3612672=16384} 2024-03-20T08:07:11.787 INFO:tasks.rados.rados.0.smithi118.stdout:1661: writing smithi11881747-24 from 2162688 to 2826240 tid 1 2024-03-20T08:07:11.793 INFO:tasks.rados.rados.0.smithi118.stdout:1661: writing smithi11881747-24 from 2826240 to 3612672 tid 2 2024-03-20T08:07:11.794 INFO:tasks.rados.rados.0.smithi118.stdout:1661: writing smithi11881747-24 from 3612672 to 3629056 tid 3 2024-03-20T08:07:11.794 INFO:tasks.rados.rados.0.smithi118.stdout:1662: copy_from oid 50 from oid 18 current snap is 169 2024-03-20T08:07:11.795 INFO:tasks.rados.rados.0.smithi118.stdout:1663: rollback oid 27 current snap is 169 2024-03-20T08:07:11.795 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 168 2024-03-20T08:07:11.795 INFO:tasks.rados.rados.0.smithi118.stdout:1664: rollback oid 1 current snap is 169 2024-03-20T08:07:11.795 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 1 to 135 2024-03-20T08:07:11.795 INFO:tasks.rados.rados.0.smithi118.stdout:1665: read oid 23 snap -1 2024-03-20T08:07:11.795 INFO:tasks.rados.rados.0.smithi118.stdout:1665: expect (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:07:11.795 INFO:tasks.rados.rados.0.smithi118.stdout:1666: rollback oid 45 current snap is 169 2024-03-20T08:07:11.796 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 45 to 167 2024-03-20T08:07:11.796 INFO:tasks.rados.rados.0.smithi118.stdout:1661: finishing write tid 1 to smithi11881747-24 2024-03-20T08:07:11.796 INFO:tasks.rados.rados.0.smithi118.stdout:1667: read oid 11 snap -1 2024-03-20T08:07:11.796 INFO:tasks.rados.rados.0.smithi118.stdout:1667: expect deleted 2024-03-20T08:07:11.796 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 169 2024-03-20T08:07:11.796 INFO:tasks.rados.rados.0.smithi118.stdout:1668: seq_num 518 ranges {2097152=679936,2777088=671744,3448832=131072} 2024-03-20T08:07:11.812 INFO:tasks.rados.rados.0.smithi118.stdout:1668: writing smithi11881747-21 from 2097152 to 2777088 tid 1 2024-03-20T08:07:11.817 INFO:tasks.rados.rados.0.smithi118.stdout:1668: writing smithi11881747-21 from 2777088 to 3448832 tid 2 2024-03-20T08:07:11.818 INFO:tasks.rados.rados.0.smithi118.stdout:1668: writing smithi11881747-21 from 3448832 to 3579904 tid 3 2024-03-20T08:07:11.818 INFO:tasks.rados.rados.0.smithi118.stdout:1669: read oid 26 snap -1 2024-03-20T08:07:11.818 INFO:tasks.rados.rados.0.smithi118.stdout:1669: expect (ObjNum 449 snap 149 seq_num 449) 2024-03-20T08:07:11.818 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 169 2024-03-20T08:07:11.818 INFO:tasks.rados.rados.0.smithi118.stdout:1670: seq_num 519 ranges {0=696320,696320=548864,1245184=253952} 2024-03-20T08:07:11.823 INFO:tasks.rados.rados.0.smithi118.stdout:1670: writing smithi11881747-43 from 0 to 696320 tid 1 2024-03-20T08:07:11.828 INFO:tasks.rados.rados.0.smithi118.stdout:1670: writing smithi11881747-43 from 696320 to 1245184 tid 2 2024-03-20T08:07:11.829 INFO:tasks.rados.rados.0.smithi118.stdout:1670: writing smithi11881747-43 from 1245184 to 1499136 tid 3 2024-03-20T08:07:11.830 INFO:tasks.rados.rados.0.smithi118.stdout:1671: setattr oid 44 current snap is 169 2024-03-20T08:07:11.832 INFO:tasks.rados.rados.0.smithi118.stdout:1663: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:07:11.832 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7052 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-03-20T08:07:11.832 INFO:tasks.rados.rados.0.smithi118.stdout:1660: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:07:11.948 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:11.756+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:11.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[93121]: from='client.54181 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:11.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[93121]: osdmap e403: 8 total, 7 up, 8 in 2024-03-20T08:07:11.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:11.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:11.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:11.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:11.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:11.949 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[93121]: from='client.34281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:11.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[95692]: from='client.54181 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:11.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[95692]: osdmap e403: 8 total, 7 up, 8 in 2024-03-20T08:07:11.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:11.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:11.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:11.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:11.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:11.950 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-mon[95692]: from='client.34281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:11.950 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.699+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.950 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.715+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.715+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.731+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.735+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.750+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.766+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.778+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.789+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.807+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.823+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.830+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.843+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.847+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.951 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.860+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.952 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.865+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.952 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.879+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.952 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:11.758+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:11.952 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.881+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.952 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.899+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.899+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.916+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.917+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.933+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:11.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.933+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:11.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.949+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:11 smithi196 ceph-mon[87080]: from='client.54181 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:11 smithi196 ceph-mon[87080]: osdmap e403: 8 total, 7 up, 8 in 2024-03-20T08:07:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:11 smithi196 ceph-mon[87080]: from='client.34281 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:12.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.952+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.965+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.971+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.982+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.990+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:11.995+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.006+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.010+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.022+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.024+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.038+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.039+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.053+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.055+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.071+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.071+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.085+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.086+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.098+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.100+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.111+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.116+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.123+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.130+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.135+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.146+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.148+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.161+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.162+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.176+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.177+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.192+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.194+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.208+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:1664: finishing rollback tid 0 to smithi11881747-1 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:1664: finishing rollback tid 1 to smithi11881747-1 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:1664: finishing rollback tid 2 to smithi11881747-1 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 1022 (ObjNum 391 snap 134 seq_num 391) dirty exists 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:1661: finishing write tid 2 to smithi11881747-24 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:1661: finishing write tid 3 to smithi11881747-24 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:1661: finishing write tid 4 to smithi11881747-24 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:1661: finishing write tid 5 to smithi11881747-24 2024-03-20T08:07:12.399 INFO:tasks.rados.rados.0.smithi118.stdout:1661: oid 24 updating version 0 to 825 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1661: oid 24 updating version 825 to 827 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1661: oid 24 version 827 is already newer than 826 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1661: oid 24 updating version 827 to 828 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 828 (ObjNum 517 snap 169 seq_num 517) dirty exists 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1661: left oid 24 (ObjNum 517 snap 169 seq_num 517) 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1666: finishing rollback tid 0 to smithi11881747-45 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1666: finishing rollback tid 1 to smithi11881747-45 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 7046 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1662: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1662: finishing copy_from to smithi11881747-50 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 7048 (ObjNum 33 snap 0 seq_num 324626832) dirty exists 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1668: finishing write tid 1 to smithi11881747-21 2024-03-20T08:07:12.400 INFO:tasks.rados.rados.0.smithi118.stdout:1669: expect (ObjNum 449 snap 149 seq_num 449) 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: finishing write tid 2 to smithi11881747-21 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: finishing write tid 3 to smithi11881747-21 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: finishing write tid 4 to smithi11881747-21 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: finishing write tid 5 to smithi11881747-21 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: oid 21 updating version 0 to 7047 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: oid 21 version 7047 is already newer than 7045 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: oid 21 version 7047 is already newer than 7044 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: oid 21 version 7047 is already newer than 7046 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 7047 (ObjNum 518 snap 169 seq_num 518) dirty exists 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1668: left oid 21 (ObjNum 518 snap 169 seq_num 518) 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1670: finishing write tid 1 to smithi11881747-43 2024-03-20T08:07:12.408 INFO:tasks.rados.rados.0.smithi118.stdout:1665: expect (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:07:12.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.210+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.223+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.225+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.237+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.242+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.254+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.258+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.269+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.274+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.285+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.290+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.299+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.306+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.313+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.322+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.329+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.467 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.344+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.353+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.361+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.369+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.377+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.384+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.391+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.399+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.406+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.416+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.420+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.468 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.435+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.469 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.437+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.469 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.450+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.469 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.451+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.469 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.466+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:12.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:12.656 INFO:tasks.rados.rados.0.smithi118.stdout:1660: done (11 left) 2024-03-20T08:07:12.656 INFO:tasks.rados.rados.0.smithi118.stdout:1661: done (10 left) 2024-03-20T08:07:12.656 INFO:tasks.rados.rados.0.smithi118.stdout:1662: done (9 left) 2024-03-20T08:07:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:1663: done (8 left) 2024-03-20T08:07:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:1664: done (7 left) 2024-03-20T08:07:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:1665: done (6 left) 2024-03-20T08:07:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:1666: done (5 left) 2024-03-20T08:07:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:1667: done (4 left) 2024-03-20T08:07:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:1668: done (3 left) 2024-03-20T08:07:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:1669: done (2 left) 2024-03-20T08:07:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:1672: snap_remove snap 140 2024-03-20T08:07:12.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.467+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.479+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.484+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.492+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.499+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.505+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.513+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.520+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.528+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.533+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.543+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.548+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.558+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.562+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.573+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.577+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.590+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.591+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.603+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.605+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.617+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.622+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.630+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.636+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.644+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.653+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.658+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.670+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.672+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.685+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.685+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.698+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.700+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.710+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.714+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.722+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.753 INFO:tasks.rados.rados.0.smithi118.stdout:1672: done (2 left) 2024-03-20T08:07:12.753 INFO:tasks.rados.rados.0.smithi118.stdout:1673: rmattr oid 6 current snap is 169 2024-03-20T08:07:12.753 INFO:tasks.rados.rados.0.smithi118.stdout:1673: done (2 left) 2024-03-20T08:07:12.753 INFO:tasks.rados.rados.0.smithi118.stdout:1674: rollback oid 13 current snap is 169 2024-03-20T08:07:12.753 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 13 to 145 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: finishing write tid 2 to smithi11881747-43 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: finishing write tid 3 to smithi11881747-43 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: finishing write tid 4 to smithi11881747-43 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: finishing write tid 5 to smithi11881747-43 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: oid 43 updating version 0 to 1023 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: oid 43 updating version 1023 to 1026 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: oid 43 version 1026 is already newer than 1024 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: oid 43 version 1026 is already newer than 1025 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 1026 (ObjNum 519 snap 169 seq_num 519) dirty exists 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: left oid 43 (ObjNum 519 snap 169 seq_num 519) 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 7049 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1670: done (2 left) 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1671: done (1 left) 2024-03-20T08:07:12.754 INFO:tasks.rados.rados.0.smithi118.stdout:1675: rollback oid 40 current snap is 169 2024-03-20T08:07:12.755 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 89 2024-03-20T08:07:12.755 INFO:tasks.rados.rados.0.smithi118.stdout:1676: read oid 2 snap -1 2024-03-20T08:07:12.755 INFO:tasks.rados.rados.0.smithi118.stdout:1676: expect (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:07:12.755 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 169 2024-03-20T08:07:12.755 INFO:tasks.rados.rados.0.smithi118.stdout:1677: seq_num 520 ranges {2580480=712704,3293184=434176,3727360=368640} 2024-03-20T08:07:12.772 INFO:tasks.rados.rados.0.smithi118.stdout:1677: writing smithi11881747-29 from 2580480 to 3293184 tid 1 2024-03-20T08:07:12.775 INFO:tasks.rados.rados.0.smithi118.stdout:1677: writing smithi11881747-29 from 3293184 to 3727360 tid 2 2024-03-20T08:07:12.778 INFO:tasks.rados.rados.0.smithi118.stdout:1677: writing smithi11881747-29 from 3727360 to 4096000 tid 3 2024-03-20T08:07:12.778 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 169 2024-03-20T08:07:12.778 INFO:tasks.rados.rados.0.smithi118.stdout:1678: seq_num 521 ranges {4382720=729088,5111808=720896,5832704=81920} 2024-03-20T08:07:12.807 INFO:tasks.rados.rados.0.smithi118.stdout:1678: writing smithi11881747-1 from 4382720 to 5111808 tid 1 2024-03-20T08:07:12.811 INFO:tasks.rados.rados.0.smithi118.stdout:1678: writing smithi11881747-1 from 5111808 to 5832704 tid 2 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1678: writing smithi11881747-1 from 5832704 to 5914624 tid 3 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1679: rollback oid 41 current snap is 169 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 41 to 135 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1674: finishing rollback tid 0 to smithi11881747-13 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1674: finishing rollback tid 1 to smithi11881747-13 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1674: finishing rollback tid 2 to smithi11881747-13 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 1023 (ObjNum 343 snap 112 seq_num 343) dirty exists 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1675: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1675: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1675: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 7049 (ObjNum 238 snap 84 seq_num 238) dirty exists 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1677: finishing write tid 1 to smithi11881747-29 2024-03-20T08:07:12.813 INFO:tasks.rados.rados.0.smithi118.stdout:1674: done (5 left) 2024-03-20T08:07:12.814 INFO:tasks.rados.rados.0.smithi118.stdout:1675: done (4 left) 2024-03-20T08:07:12.814 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 169 2024-03-20T08:07:12.814 INFO:tasks.rados.rados.0.smithi118.stdout:1677: finishing write tid 2 to smithi11881747-29 2024-03-20T08:07:12.814 INFO:tasks.rados.rados.0.smithi118.stdout:1680: seq_num 522 ranges {4866048=745472,5611520=606208,6217728=196608} 2024-03-20T08:07:12.844 INFO:tasks.rados.rados.0.smithi118.stdout:1680: writing smithi11881747-38 from 4866048 to 5611520 tid 1 2024-03-20T08:07:12.847 INFO:tasks.rados.rados.0.smithi118.stdout:1680: writing smithi11881747-38 from 5611520 to 6217728 tid 2 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1680: writing smithi11881747-38 from 6217728 to 6414336 tid 3 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1681: delete oid 50 current snap is 169 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1677: finishing write tid 3 to smithi11881747-29 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1677: finishing write tid 4 to smithi11881747-29 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1677: finishing write tid 5 to smithi11881747-29 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1677: oid 29 updating version 0 to 1300 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1677: oid 29 updating version 1300 to 1303 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1677: oid 29 version 1303 is already newer than 1301 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:1677: oid 29 version 1303 is already newer than 1302 2024-03-20T08:07:12.849 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1303 (ObjNum 520 snap 169 seq_num 520) dirty exists 2024-03-20T08:07:12.850 INFO:tasks.rados.rados.0.smithi118.stdout:1677: left oid 29 (ObjNum 520 snap 169 seq_num 520) 2024-03-20T08:07:12.850 INFO:tasks.rados.rados.0.smithi118.stdout:1676: expect (ObjNum 299 snap 99 seq_num 299) 2024-03-20T08:07:12.974 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:07:12.743+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 3 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:12.974 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[93121]: osdmap e404: 8 total, 7 up, 8 in 2024-03-20T08:07:12.974 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/2793766090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:12.974 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:12.974 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:12.975 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[93121]: pgmap v283: 121 pgs: 2 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 429 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 11 op/s; 107/909 objects degraded (11.771%) 2024-03-20T08:07:12.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[95692]: osdmap e404: 8 total, 7 up, 8 in 2024-03-20T08:07:12.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/2793766090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:12.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:12.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:12.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-mon[95692]: pgmap v283: 121 pgs: 2 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 429 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 11 op/s; 107/909 objects degraded (11.771%) 2024-03-20T08:07:12.975 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:12.760+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:12.976 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:12.768+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:12.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.729+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.736+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.745+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.750+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.771+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.780+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.792+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.797+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.807+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.810+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.824+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.826+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.839+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.841+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.858+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.860+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.873+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.876+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.889+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.905+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.907+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.923+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.923+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.939+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.941+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.957+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:12.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.958+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:12.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.973+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.110 INFO:tasks.rados.rados.0.smithi118.stdout:1678: finishing write tid 1 to smithi11881747-1 2024-03-20T08:07:13.110 INFO:tasks.rados.rados.0.smithi118.stdout:1676: done (5 left) 2024-03-20T08:07:13.110 INFO:tasks.rados.rados.0.smithi118.stdout:1677: done (4 left) 2024-03-20T08:07:13.110 INFO:tasks.rados.rados.0.smithi118.stdout:1681: done (3 left) 2024-03-20T08:07:13.110 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 169 2024-03-20T08:07:13.110 INFO:tasks.rados.rados.0.smithi118.stdout:1682: seq_num 524 ranges {3579904=786432,4366336=778240,5144576=24576} 2024-03-20T08:07:13.136 INFO:tasks.rados.rados.0.smithi118.stdout:1682: writing smithi11881747-21 from 3579904 to 4366336 tid 1 2024-03-20T08:07:13.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:12 smithi196 ceph-mon[87080]: osdmap e404: 8 total, 7 up, 8 in 2024-03-20T08:07:13.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:12 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/2793766090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:13.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:13.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:13.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:12 smithi196 ceph-mon[87080]: pgmap v283: 121 pgs: 2 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 429 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 11 op/s; 107/909 objects degraded (11.771%) 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1682: writing smithi11881747-21 from 4366336 to 5144576 tid 2 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1682: writing smithi11881747-21 from 5144576 to 5169152 tid 3 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1683: read oid 11 snap -1 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1683: expect deleted 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1684: snap_create 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1678: finishing write tid 2 to smithi11881747-1 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1678: finishing write tid 3 to smithi11881747-1 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1678: finishing write tid 4 to smithi11881747-1 2024-03-20T08:07:13.142 INFO:tasks.rados.rados.0.smithi118.stdout:1679: finishing rollback tid 0 to smithi11881747-41 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1679: finishing rollback tid 1 to smithi11881747-41 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1679: finishing rollback tid 2 to smithi11881747-41 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1307 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1678: finishing write tid 5 to smithi11881747-1 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1678: oid 1 updating version 0 to 1030 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1678: oid 1 version 1030 is already newer than 1028 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1678: oid 1 version 1030 is already newer than 1029 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1678: oid 1 version 1030 is already newer than 1027 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 1030 (ObjNum 521 snap 169 seq_num 521) dirty exists 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1678: left oid 1 (ObjNum 521 snap 169 seq_num 521) 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1680: finishing write tid 1 to smithi11881747-38 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1680: finishing write tid 2 to smithi11881747-38 2024-03-20T08:07:13.143 INFO:tasks.rados.rados.0.smithi118.stdout:1680: finishing write tid 3 to smithi11881747-38 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:1680: finishing write tid 4 to smithi11881747-38 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:1680: finishing write tid 5 to smithi11881747-38 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:1680: oid 38 updating version 0 to 7050 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:1680: oid 38 version 7050 is already newer than 7049 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:1680: oid 38 updating version 7050 to 7051 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:1680: oid 38 version 7051 is already newer than 7048 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7051 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:1680: left oid 38 (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:07:13.144 INFO:tasks.rados.rados.0.smithi118.stdout:1682: finishing write tid 1 to smithi11881747-21 2024-03-20T08:07:13.145 INFO:tasks.rados.rados.0.smithi118.stdout:1682: finishing write tid 2 to smithi11881747-21 2024-03-20T08:07:13.146 INFO:tasks.rados.rados.0.smithi118.stdout:1682: finishing write tid 3 to smithi11881747-21 2024-03-20T08:07:13.146 INFO:tasks.rados.rados.0.smithi118.stdout:1682: finishing write tid 4 to smithi11881747-21 2024-03-20T08:07:13.147 INFO:tasks.rados.rados.0.smithi118.stdout:1682: finishing write tid 5 to smithi11881747-21 2024-03-20T08:07:13.148 INFO:tasks.rados.rados.0.smithi118.stdout:1682: oid 21 updating version 0 to 7052 2024-03-20T08:07:13.148 INFO:tasks.rados.rados.0.smithi118.stdout:1682: oid 21 updating version 7052 to 7053 2024-03-20T08:07:13.148 INFO:tasks.rados.rados.0.smithi118.stdout:1682: oid 21 updating version 7053 to 7054 2024-03-20T08:07:13.148 INFO:tasks.rados.rados.0.smithi118.stdout:1682: oid 21 updating version 7054 to 7055 2024-03-20T08:07:13.148 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 7055 (ObjNum 524 snap 169 seq_num 524) dirty exists 2024-03-20T08:07:13.148 INFO:tasks.rados.rados.0.smithi118.stdout:1682: left oid 21 (ObjNum 524 snap 169 seq_num 524) 2024-03-20T08:07:13.225 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.976+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.225 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.993+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:12.994+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.009+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.010+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.026+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.026+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.042+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.043+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.058+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.059+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.074+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.077+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.092+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.226 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.093+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.109+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.112+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.127+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.130+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.144+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.149+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.160+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.169+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.177+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.186+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.193+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.202+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.209+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.217+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.227 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.225+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.490 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.233+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.240+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.248+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.256+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.263+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.273+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.280+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.289+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.297+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.306+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.321+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.491 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.331+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.339+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.347+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.355+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.361+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.369+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.375+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.383+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.390+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.492 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.399+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.408+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.415+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.426+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.431+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.442+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.446+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.458+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.461+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.475+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.475+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.490+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.490+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.740 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:07:13] "GET /metrics HTTP/1.1" 200 36270 "" "Prometheus/2.43.0" 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.506+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.506+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.521+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.521+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.537+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.538+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.550+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.555+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.741 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.566+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.570+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.582+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.587+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.599+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.603+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.615+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.620+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.630+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.637+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.648+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.652+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.665+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.670+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.681+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.686+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.696+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.703+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.710+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.727+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.738+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.743 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.740+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.760 INFO:tasks.rados.rados.0.smithi118.stdout:1678: done (5 left) 2024-03-20T08:07:13.760 INFO:tasks.rados.rados.0.smithi118.stdout:1679: done (4 left) 2024-03-20T08:07:13.761 INFO:tasks.rados.rados.0.smithi118.stdout:1680: done (3 left) 2024-03-20T08:07:13.761 INFO:tasks.rados.rados.0.smithi118.stdout:1682: done (2 left) 2024-03-20T08:07:13.761 INFO:tasks.rados.rados.0.smithi118.stdout:1683: done (1 left) 2024-03-20T08:07:13.761 INFO:tasks.rados.rados.0.smithi118.stdout:1684: done (0 left) 2024-03-20T08:07:13.761 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 170 2024-03-20T08:07:13.761 INFO:tasks.rados.rados.0.smithi118.stdout:1685: seq_num 525 ranges {0=802816,802816=712704,1515520=90112} 2024-03-20T08:07:13.766 INFO:tasks.rados.rados.0.smithi118.stdout:1685: writing smithi11881747-27 from 0 to 802816 tid 1 2024-03-20T08:07:13.771 INFO:tasks.rados.rados.0.smithi118.stdout:1685: writing smithi11881747-27 from 802816 to 1515520 tid 2 2024-03-20T08:07:13.772 INFO:tasks.rados.rados.0.smithi118.stdout:1685: writing smithi11881747-27 from 1515520 to 1605632 tid 3 2024-03-20T08:07:13.772 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 170 2024-03-20T08:07:13.772 INFO:tasks.rados.rados.0.smithi118.stdout:1686: seq_num 526 ranges {3948544=417792,4366336=589824,4956160=614400} 2024-03-20T08:07:13.796 INFO:tasks.rados.rados.0.smithi118.stdout:1686: writing smithi11881747-15 from 3948544 to 4366336 tid 1 2024-03-20T08:07:13.800 INFO:tasks.rados.rados.0.smithi118.stdout:1686: writing smithi11881747-15 from 4366336 to 4956160 tid 2 2024-03-20T08:07:13.805 INFO:tasks.rados.rados.0.smithi118.stdout:1686: writing smithi11881747-15 from 4956160 to 5570560 tid 3 2024-03-20T08:07:13.805 INFO:tasks.rados.rados.0.smithi118.stdout:1685: finishing write tid 1 to smithi11881747-27 2024-03-20T08:07:13.805 INFO:tasks.rados.rados.0.smithi118.stdout:1685: finishing write tid 2 to smithi11881747-27 2024-03-20T08:07:13.805 INFO:tasks.rados.rados.0.smithi118.stdout:1685: finishing write tid 3 to smithi11881747-27 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1685: finishing write tid 4 to smithi11881747-27 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1685: finishing write tid 5 to smithi11881747-27 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1685: oid 27 updating version 0 to 7060 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1685: oid 27 version 7060 is already newer than 7059 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1685: oid 27 updating version 7060 to 7061 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1685: oid 27 version 7061 is already newer than 7058 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7061 (ObjNum 525 snap 170 seq_num 525) dirty exists 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1685: left oid 27 (ObjNum 525 snap 170 seq_num 525) 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1686: finishing write tid 1 to smithi11881747-15 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1685: done (1 left) 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1687: read oid 26 snap -1 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1687: expect (ObjNum 449 snap 149 seq_num 449) 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 170 2024-03-20T08:07:13.806 INFO:tasks.rados.rados.0.smithi118.stdout:1688: seq_num 527 ranges {327680=434176,761856=475136,1236992=565248,1802240=163840} 2024-03-20T08:07:13.809 INFO:tasks.rados.rados.0.smithi118.stdout:1688: writing smithi11881747-6 from 327680 to 761856 tid 1 2024-03-20T08:07:13.812 INFO:tasks.rados.rados.0.smithi118.stdout:1688: writing smithi11881747-6 from 761856 to 1236992 tid 2 2024-03-20T08:07:13.816 INFO:tasks.rados.rados.0.smithi118.stdout:1688: writing smithi11881747-6 from 1236992 to 1802240 tid 3 2024-03-20T08:07:13.817 INFO:tasks.rados.rados.0.smithi118.stdout:1688: writing smithi11881747-6 from 1802240 to 1966080 tid 4 2024-03-20T08:07:13.818 INFO:tasks.rados.rados.0.smithi118.stdout:1689: read oid 37 snap -1 2024-03-20T08:07:13.818 INFO:tasks.rados.rados.0.smithi118.stdout:1689: expect (ObjNum 508 snap 168 seq_num 508) 2024-03-20T08:07:13.818 INFO:tasks.rados.rados.0.smithi118.stdout:1686: finishing write tid 2 to smithi11881747-15 2024-03-20T08:07:13.818 INFO:tasks.rados.rados.0.smithi118.stdout:1687: expect (ObjNum 449 snap 149 seq_num 449) 2024-03-20T08:07:13.830 INFO:tasks.rados.rados.0.smithi118.stdout:1686: finishing write tid 3 to smithi11881747-15 2024-03-20T08:07:13.830 INFO:tasks.rados.rados.0.smithi118.stdout:1686: finishing write tid 4 to smithi11881747-15 2024-03-20T08:07:13.830 INFO:tasks.rados.rados.0.smithi118.stdout:1687: done (3 left) 2024-03-20T08:07:13.830 INFO:tasks.rados.rados.0.smithi118.stdout:1690: delete oid 13 current snap is 170 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1686: finishing write tid 5 to smithi11881747-15 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1686: oid 15 updating version 0 to 425 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1686: oid 15 updating version 425 to 428 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1686: oid 15 version 428 is already newer than 426 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1686: oid 15 version 428 is already newer than 427 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 428 (ObjNum 526 snap 170 seq_num 526) dirty exists 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1686: left oid 15 (ObjNum 526 snap 170 seq_num 526) 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1688: finishing write tid 1 to smithi11881747-6 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1688: finishing write tid 2 to smithi11881747-6 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1688: finishing write tid 3 to smithi11881747-6 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1688: finishing write tid 4 to smithi11881747-6 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1688: finishing write tid 5 to smithi11881747-6 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1688: finishing write tid 6 to smithi11881747-6 2024-03-20T08:07:13.831 INFO:tasks.rados.rados.0.smithi118.stdout:1688: oid 6 updating version 0 to 1279 2024-03-20T08:07:13.832 INFO:tasks.rados.rados.0.smithi118.stdout:1688: oid 6 version 1279 is already newer than 1276 2024-03-20T08:07:13.832 INFO:tasks.rados.rados.0.smithi118.stdout:1688: oid 6 version 1279 is already newer than 1277 2024-03-20T08:07:13.832 INFO:tasks.rados.rados.0.smithi118.stdout:1688: oid 6 version 1279 is already newer than 1278 2024-03-20T08:07:13.832 INFO:tasks.rados.rados.0.smithi118.stdout:1688: oid 6 version 1279 is already newer than 1275 2024-03-20T08:07:13.832 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 1279 (ObjNum 527 snap 170 seq_num 527) dirty exists 2024-03-20T08:07:13.832 INFO:tasks.rados.rados.0.smithi118.stdout:1688: left oid 6 (ObjNum 527 snap 170 seq_num 527) 2024-03-20T08:07:13.833 INFO:tasks.rados.rados.0.smithi118.stdout:1686: done (3 left) 2024-03-20T08:07:13.834 INFO:tasks.rados.rados.0.smithi118.stdout:1688: done (2 left) 2024-03-20T08:07:13.834 INFO:tasks.rados.rados.0.smithi118.stdout:1690: done (1 left) 2024-03-20T08:07:13.834 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 170 2024-03-20T08:07:13.834 INFO:tasks.rados.rados.0.smithi118.stdout:1691: seq_num 529 ranges {7110656=466944,7577600=647168,8224768=532480,8757248=24576} 2024-03-20T08:07:13.875 INFO:tasks.rados.rados.0.smithi118.stdout:1691: writing smithi11881747-12 from 7110656 to 7577600 tid 1 2024-03-20T08:07:13.879 INFO:tasks.rados.rados.0.smithi118.stdout:1691: writing smithi11881747-12 from 7577600 to 8224768 tid 2 2024-03-20T08:07:13.882 INFO:tasks.rados.rados.0.smithi118.stdout:1691: writing smithi11881747-12 from 8224768 to 8757248 tid 3 2024-03-20T08:07:13.882 INFO:tasks.rados.rados.0.smithi118.stdout:1691: writing smithi11881747-12 from 8757248 to 8781824 tid 4 2024-03-20T08:07:13.882 INFO:tasks.rados.rados.0.smithi118.stdout:1692: snap_remove snap 155 2024-03-20T08:07:13.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:13 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 3 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:13.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:13 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 107/909 objects degraded (11.771%), 8 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:13.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:13 smithi196 ceph-mon[87080]: osdmap e405: 8 total, 7 up, 8 in 2024-03-20T08:07:13.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:13.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:13.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:13.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:13.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:13.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:13.992 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:13.782+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:13.992 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 3 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:13.992 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 107/909 objects degraded (11.771%), 8 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[93121]: osdmap e405: 8 total, 7 up, 8 in 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 3 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 107/909 objects degraded (11.771%), 8 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[95692]: osdmap e405: 8 total, 7 up, 8 in 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:13.993 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:13.994 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:13.994 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:13.994 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:13.774+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:13.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.756+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.759+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.994 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.778+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.786+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.800+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.805+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.821+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.825+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.841+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.842+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.857+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.859+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.872+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.876+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.887+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.900+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.995 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.908+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.915+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.926+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.931+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.947+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.960+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.962+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.977+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.978+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.993+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:13.996 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:13.993+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.008+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.009+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.024+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.024+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.039+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.039+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.055+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.246 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.056+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.071+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.071+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.085+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.086+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.100+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.102+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.116+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.120+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.132+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.137+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.151+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.155+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.168+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.247 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.172+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.185+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.190+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.202+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.219+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.227+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.236+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.245+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.496 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:14.389+0000 7f5a0f20c7c0 -1 osd.3 396 log_to_monitors true 2024-03-20T08:07:14.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.252+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.264+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.268+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.281+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.283+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.297+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.497 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.297+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.312+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.314+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.325+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.331+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.340+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.354+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.365+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.371+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.382+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.387+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.399+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.402+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.414+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.416+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.429+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.429+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.443+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.444+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.456+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.458+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.468+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.472+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.482+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.487+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.497+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.505+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.512+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.520+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.524+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.534+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.539+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.550+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.554+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.566+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.569+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.582+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.586+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.597+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.600+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.612+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.614+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.626+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.626+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.640+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.641+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.652+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.654+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.665+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.670+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.680+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.685+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.694+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.700+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.708+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.716+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.722+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.731+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.735+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.750 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.745+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:14.750 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.747+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:14.766 INFO:tasks.rados.rados.0.smithi118.stdout:1692: done (2 left) 2024-03-20T08:07:14.766 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 170 2024-03-20T08:07:14.766 INFO:tasks.rados.rados.0.smithi118.stdout:1693: seq_num 530 ranges {327680=483328,811008=532480,1343488=671744} 2024-03-20T08:07:14.771 INFO:tasks.rados.rados.0.smithi118.stdout:1693: writing smithi11881747-26 from 327680 to 811008 tid 1 2024-03-20T08:07:14.775 INFO:tasks.rados.rados.0.smithi118.stdout:1693: writing smithi11881747-26 from 811008 to 1343488 tid 2 2024-03-20T08:07:14.782 INFO:tasks.rados.rados.0.smithi118.stdout:1693: writing smithi11881747-26 from 1343488 to 2015232 tid 3 2024-03-20T08:07:14.782 INFO:tasks.rados.rados.0.smithi118.stdout:1694: snap_create 2024-03-20T08:07:14.782 INFO:tasks.rados.rados.0.smithi118.stdout:1689: expect (ObjNum 508 snap 168 seq_num 508) 2024-03-20T08:07:14.827 INFO:tasks.rados.rados.0.smithi118.stdout:1691: finishing write tid 1 to smithi11881747-12 2024-03-20T08:07:14.827 INFO:tasks.rados.rados.0.smithi118.stdout:1691: finishing write tid 2 to smithi11881747-12 2024-03-20T08:07:14.827 INFO:tasks.rados.rados.0.smithi118.stdout:1691: finishing write tid 3 to smithi11881747-12 2024-03-20T08:07:14.827 INFO:tasks.rados.rados.0.smithi118.stdout:1691: finishing write tid 4 to smithi11881747-12 2024-03-20T08:07:14.827 INFO:tasks.rados.rados.0.smithi118.stdout:1691: finishing write tid 5 to smithi11881747-12 2024-03-20T08:07:14.827 INFO:tasks.rados.rados.0.smithi118.stdout:1691: finishing write tid 6 to smithi11881747-12 2024-03-20T08:07:14.827 INFO:tasks.rados.rados.0.smithi118.stdout:1691: oid 12 updating version 0 to 9893 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1691: oid 12 version 9893 is already newer than 9891 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1691: oid 12 updating version 9893 to 9894 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1691: oid 12 updating version 9894 to 9895 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1691: oid 12 version 9895 is already newer than 9892 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 9895 (ObjNum 529 snap 170 seq_num 529) dirty exists 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1691: left oid 12 (ObjNum 529 snap 170 seq_num 529) 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: finishing write tid 1 to smithi11881747-26 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: finishing write tid 2 to smithi11881747-26 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: finishing write tid 3 to smithi11881747-26 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: finishing write tid 4 to smithi11881747-26 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: finishing write tid 5 to smithi11881747-26 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: oid 26 updating version 0 to 7052 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: oid 26 version 7052 is already newer than 7051 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: oid 26 updating version 7052 to 7054 2024-03-20T08:07:14.828 INFO:tasks.rados.rados.0.smithi118.stdout:1693: oid 26 version 7054 is already newer than 7053 2024-03-20T08:07:14.829 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 7054 (ObjNum 530 snap 170 seq_num 530) dirty exists 2024-03-20T08:07:14.829 INFO:tasks.rados.rados.0.smithi118.stdout:1693: left oid 26 (ObjNum 530 snap 170 seq_num 530) 2024-03-20T08:07:15.005 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[93121]: osdmap e406: 8 total, 7 up, 8 in 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[93121]: pgmap v286: 121 pgs: 2 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 425 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 25 op/s; 108/921 objects degraded (11.726%) 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[93121]: from='osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[93121]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[95692]: osdmap e406: 8 total, 7 up, 8 in 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:15.006 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[95692]: pgmap v286: 121 pgs: 2 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 425 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 25 op/s; 108/921 objects degraded (11.726%) 2024-03-20T08:07:15.007 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[95692]: from='osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T08:07:15.007 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-mon[95692]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T08:07:15.007 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:14.776+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:15.007 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:14.779+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:15.007 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.762+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.762+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.790+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.794+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.813+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.817+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.843+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.868+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.969+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.974+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.985+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:14.990+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.008 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.003+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:14 smithi196 ceph-mon[87080]: osdmap e406: 8 total, 7 up, 8 in 2024-03-20T08:07:15.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:15.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:15.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:14 smithi196 ceph-mon[87080]: pgmap v286: 121 pgs: 2 active+clean+snaptrim_error, 1 active+undersized+degraded+snaptrim_error, 32 active+undersized, 7 active+undersized+degraded, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 425 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 25 op/s; 108/921 objects degraded (11.726%) 2024-03-20T08:07:15.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:14 smithi196 ceph-mon[87080]: from='osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T08:07:15.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:14 smithi196 ceph-mon[87080]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-20T08:07:15.257 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:15.075+0000 7f59fc65c700 -1 osd.3 396 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T08:07:15.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.006+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.023+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.024+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.041+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.057+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.257 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.062+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.074+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.079+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.091+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.095+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.107+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.113+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.125+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.128+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.141+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.143+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.154+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.160+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.167+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.176+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.180+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.191+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.194+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.206+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.207+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.219+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.220+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.232+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.234+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.246+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.249+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.257+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.265+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.271+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.281+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.285+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.297+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.299+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.314+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.327+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.328+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.342+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.346+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.356+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.363+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.369+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.379+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.384+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.394+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.398+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.411+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.413+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.426+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.426+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.444+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.444+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.456+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.459+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.469+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.474+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.483+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.490+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.497+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.505+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.511+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.763 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.518+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.763 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.525+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.534+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.538+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.550+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.551+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.565+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.764 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.566+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.765 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.579+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.765 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.581+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.765 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.592+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.765 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.596+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.765 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.604+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.765 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.611+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.617+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.626+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.629+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.641+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.644+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.657+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.657+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.670+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.672+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.683+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.688+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.696+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.703+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.709+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.720+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.722+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.768 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.734+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.735+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.750+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:15.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.750+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.762+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:15.773 INFO:tasks.rados.rados.0.smithi118.stdout:1689: done (3 left) 2024-03-20T08:07:15.773 INFO:tasks.rados.rados.0.smithi118.stdout:1691: done (2 left) 2024-03-20T08:07:15.773 INFO:tasks.rados.rados.0.smithi118.stdout:1693: done (1 left) 2024-03-20T08:07:15.774 INFO:tasks.rados.rados.0.smithi118.stdout:1694: done (0 left) 2024-03-20T08:07:15.774 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 171 2024-03-20T08:07:15.774 INFO:tasks.rados.rados.0.smithi118.stdout:1695: seq_num 531 ranges {4071424=499712,4571136=417792,4988928=548864,5537792=237568} 2024-03-20T08:07:15.800 INFO:tasks.rados.rados.0.smithi118.stdout:1695: writing smithi11881747-3 from 4071424 to 4571136 tid 1 2024-03-20T08:07:15.802 INFO:tasks.rados.rados.0.smithi118.stdout:1695: writing smithi11881747-3 from 4571136 to 4988928 tid 2 2024-03-20T08:07:15.806 INFO:tasks.rados.rados.0.smithi118.stdout:1695: writing smithi11881747-3 from 4988928 to 5537792 tid 3 2024-03-20T08:07:15.810 INFO:tasks.rados.rados.0.smithi118.stdout:1695: writing smithi11881747-3 from 5537792 to 5775360 tid 4 2024-03-20T08:07:15.810 INFO:tasks.rados.rados.0.smithi118.stdout:1696: read oid 23 snap -1 2024-03-20T08:07:15.811 INFO:tasks.rados.rados.0.smithi118.stdout:1696: expect (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:07:15.811 INFO:tasks.rados.rados.0.smithi118.stdout:1697: snap_create 2024-03-20T08:07:15.817 INFO:tasks.rados.rados.0.smithi118.stdout:1695: finishing write tid 1 to smithi11881747-3 2024-03-20T08:07:15.817 INFO:tasks.rados.rados.0.smithi118.stdout:1695: finishing write tid 2 to smithi11881747-3 2024-03-20T08:07:15.817 INFO:tasks.rados.rados.0.smithi118.stdout:1695: finishing write tid 3 to smithi11881747-3 2024-03-20T08:07:15.818 INFO:tasks.rados.rados.0.smithi118.stdout:1695: finishing write tid 4 to smithi11881747-3 2024-03-20T08:07:15.824 INFO:tasks.rados.rados.0.smithi118.stdout:1695: finishing write tid 5 to smithi11881747-3 2024-03-20T08:07:15.834 INFO:tasks.rados.rados.0.smithi118.stdout:1695: finishing write tid 6 to smithi11881747-3 2024-03-20T08:07:15.834 INFO:tasks.rados.rados.0.smithi118.stdout:1695: oid 3 updating version 0 to 10751 2024-03-20T08:07:15.834 INFO:tasks.rados.rados.0.smithi118.stdout:1695: oid 3 version 10751 is already newer than 10750 2024-03-20T08:07:15.834 INFO:tasks.rados.rados.0.smithi118.stdout:1695: oid 3 updating version 10751 to 10753 2024-03-20T08:07:15.834 INFO:tasks.rados.rados.0.smithi118.stdout:1695: oid 3 updating version 10753 to 10754 2024-03-20T08:07:15.834 INFO:tasks.rados.rados.0.smithi118.stdout:1695: oid 3 version 10754 is already newer than 10752 2024-03-20T08:07:15.834 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 10754 (ObjNum 531 snap 171 seq_num 531) dirty exists 2024-03-20T08:07:15.834 INFO:tasks.rados.rados.0.smithi118.stdout:1695: left oid 3 (ObjNum 531 snap 171 seq_num 531) 2024-03-20T08:07:16.014 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:15.780+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:16.015 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[93121]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-20T08:07:16.015 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[93121]: osdmap e407: 8 total, 7 up, 8 in 2024-03-20T08:07:16.015 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[93121]: from='osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:07:16.015 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[93121]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:07:16.015 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:16.015 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:16.015 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[93121]: from='osd.3 ' entity='osd.3' 2024-03-20T08:07:16.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.765+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.794+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.795+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.813+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.829+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.835+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.846+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.850+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.869+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.883+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.884+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.905+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.906+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.927+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.948+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[95692]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-20T08:07:16.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[95692]: osdmap e407: 8 total, 7 up, 8 in 2024-03-20T08:07:16.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[95692]: from='osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:07:16.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[95692]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:07:16.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:16.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:16.017 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-mon[95692]: from='osd.3 ' entity='osd.3' 2024-03-20T08:07:16.017 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:15.787+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:16.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.950+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.963+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.966+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.978+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.983+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.992+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:15.998+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.018 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.009+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:15 smithi196 ceph-mon[87080]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-20T08:07:16.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:15 smithi196 ceph-mon[87080]: osdmap e407: 8 total, 7 up, 8 in 2024-03-20T08:07:16.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:15 smithi196 ceph-mon[87080]: from='osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:07:16.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:15 smithi196 ceph-mon[87080]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi118", "root=default"]}]: dispatch 2024-03-20T08:07:16.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:16.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:16.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:15 smithi196 ceph-mon[87080]: from='osd.3 ' entity='osd.3' 2024-03-20T08:07:16.273 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.014+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.023+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.031+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.038+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.052+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.062+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.068+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.077+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.085+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.092+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.099+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.274 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.107+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.114+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.122+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.128+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.140+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.142+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.156+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.157+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.171+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.173+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.186+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.187+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.201+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.201+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.275 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.216+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.217+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.231+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.233+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.244+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.249+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.259+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.263+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.276 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.273+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.280+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.287+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.525 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.302+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.311+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.318+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.327+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.333+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.342+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.348+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.357+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.362+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.373+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.377+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.388+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.392+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.404+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.407+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.419+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.422+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.434+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.437+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.451+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.453+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.467+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.468+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.482+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.483+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.497+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.500+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.510+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.516+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.528 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.525+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731] boot 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: osdmap e408: 8 total, 8 up, 8 in 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:16.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[93121]: pgmap v289: 121 pgs: 5 peering, 2 active+clean+snaptrim_error, 28 active+undersized, 6 active+undersized+degraded, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 439 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 3.3 MiB/s wr, 30 op/s; 85/939 objects degraded (9.052%) 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731] boot 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: osdmap e408: 8 total, 8 up, 8 in 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:16.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-mon[95692]: pgmap v289: 121 pgs: 5 peering, 2 active+clean+snaptrim_error, 28 active+undersized, 6 active+undersized+degraded, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 439 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 3.3 MiB/s wr, 30 op/s; 85/939 objects degraded (9.052%) 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.532+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.539+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.549+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.554+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.564+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.567+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.581+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.582+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.782 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.595+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.598+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.613+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.615+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.630+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.631+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.646+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.646+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.660+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.661+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.674+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.676+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.688+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.692+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.704+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.783 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.707+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.718+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.722+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.734+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.739+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.748+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.754+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.762+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:16.784 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.773+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:16.784 INFO:tasks.rados.rados.0.smithi118.stdout:1695: done (2 left) 2024-03-20T08:07:16.784 INFO:tasks.rados.rados.0.smithi118.stdout:1697: done (1 left) 2024-03-20T08:07:16.785 INFO:tasks.rados.rados.0.smithi118.stdout:1698: snap_create 2024-03-20T08:07:16.830 INFO:tasks.rados.rados.0.smithi118.stdout:1696: expect (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:07:17.037 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:16.779+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:17.038 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:16.801+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:17.038 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:16.791+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:17.038 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:16.829+0000 7f59e9e37700 -1 osd.3 pg_epoch: 409 pg[5.1s1( v 398'381 (0'0,398'381] local-lis/les=408/409 n=11 ec=73/73 lis/c=408/408 les/c/f=409/409/0 sis=408) [4,3,7]p4(0) r=1 lpr=408 luod=0'0 crt=398'381 mlcod 0'0 active mbc={} ps=58] update_snap_map remove_oid 5:8a455596:::smithi11881747-30:bb failed with -2 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:16.832+0000 7f59e5e2f700 -1 osd.3 pg_epoch: 409 pg[5.1s1( v 409'385 (0'0,409'385] local-lis/les=408/409 n=10 ec=73/73 lis/c=408/408 les/c/f=409/409/0 sis=408) [4,3,7]p4(0) r=1 lpr=408 luod=0'0 crt=409'385 mlcod 409'385 active mbc={} ps=60] update_snap_map remove_oid 5:8a455596:::smithi11881747-30:c6 failed with -2 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:16.833+0000 7f59e5e2f700 -1 osd.3 pg_epoch: 409 pg[5.1s1( v 409'387 (0'0,409'387] local-lis/les=408/409 n=9 ec=73/73 lis/c=408/408 les/c/f=409/409/0 sis=408) [4,3,7]p4(0) r=1 lpr=408 luod=0'0 crt=409'385 lcod 409'385 mlcod 409'385 active mbc={} ps=60] update_snap_map remove_oid 5:89304740:::smithi11881747-4:c8 failed with -2 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:16.836+0000 7f59e5e2f700 -1 osd.3 pg_epoch: 409 pg[5.1s1( v 409'389 (0'0,409'389] local-lis/les=408/409 n=8 ec=73/73 lis/c=408/408 les/c/f=409/409/0 sis=408) [4,3,7]p4(0) r=1 lpr=408 luod=0'0 crt=409'389 mlcod 409'389 active mbc={} ps=61] update_snap_map remove_oid 5:8a455596:::smithi11881747-30:cc failed with -2 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:16.844+0000 7f59ea638700 -1 osd.3 pg_epoch: 409 pg[5.0s0( v 406'1110 (0'0,406'1110] local-lis/les=408/409 n=26 ec=73/73 lis/c=408/408 les/c/f=409/409/0 sis=408) [3,1,4]p3(0) r=0 lpr=408 crt=406'1110 mlcod 0'0 active+clean+snaptrim trimq=[a8~1,c2~1,c5~1,c8~1,d7~1,de~2,e2~1] ps=58] update_snap_map remove_oid 5:02e2296c:::smithi11881747-49:b7 failed with -2 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:16.846+0000 7f59ea638700 -1 osd.3 pg_epoch: 409 pg[5.0s0( v 409'1112 (0'0,409'1112] local-lis/les=408/409 n=25 ec=73/73 lis/c=408/408 les/c/f=409/409/0 sis=408) [3,1,4]p3(0) r=0 lpr=408 luod=406'1110 crt=406'1110 lcod 406'1110 mlcod 0'0 active+clean+snaptrim trimq=[a8~1,c2~1,c5~1,c8~1,d7~1,de~2,e2~1] ps=58] update_snap_map remove_oid 5:0e32d9f9:::smithi11881747-27:ad failed with -2 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:16.850+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.799+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.804+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.824+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.039 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.828+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.847+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.848+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.865+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.865+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.882+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.883+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.901+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.904+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.919+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.922+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.937+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.957+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.961+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.973+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.977+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.989+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:16.995+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.006+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.010+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.022+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.027+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.038+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: osd.3 [v2:172.21.15.118:6826/2186452731,v1:172.21.15.118:6827/2186452731] boot 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: osdmap e408: 8 total, 8 up, 8 in 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:17.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:16 smithi196 ceph-mon[87080]: pgmap v289: 121 pgs: 5 peering, 2 active+clean+snaptrim_error, 28 active+undersized, 6 active+undersized+degraded, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 75 active+clean; 439 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 16 MiB/s rd, 3.3 MiB/s wr, 30 op/s; 85/939 objects degraded (9.052%) 2024-03-20T08:07:17.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.055+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.062+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.070+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.078+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.086+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.095+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.101+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.109+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.116+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.125+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.133+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.140+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.148+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.155+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.162+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.171+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.178+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.188+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.192+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.204+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.206+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.219+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.221+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.236+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.236+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.253+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.254+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.268+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.270+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.283+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.286+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.300 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.298+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.557 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.302+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.557 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.312+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.557 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.318+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.326+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.335+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.341+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.351+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.356+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.366+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.370+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.383+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.387+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.401+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.401+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.415+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.558 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.417+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.430+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.435+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.445+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.450+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.460+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.465+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.475+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.480+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.489+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.496+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.505+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.511+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.520+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.526+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.533+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.543+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.547+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.557+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.775 INFO:tasks.rados.rados.0.smithi118.stdout:1696: done (1 left) 2024-03-20T08:07:17.775 INFO:tasks.rados.rados.0.smithi118.stdout:1698: done (0 left) 2024-03-20T08:07:17.775 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 40 current snap is 173 2024-03-20T08:07:17.775 INFO:tasks.rados.rados.0.smithi118.stdout:1699: seq_num 532 ranges {1597440=516096,2113536=696320,2809856=507904} 2024-03-20T08:07:17.787 INFO:tasks.rados.rados.0.smithi118.stdout:1699: writing smithi11881747-40 from 1597440 to 2113536 tid 1 2024-03-20T08:07:17.792 INFO:tasks.rados.rados.0.smithi118.stdout:1699: writing smithi11881747-40 from 2113536 to 2809856 tid 2 2024-03-20T08:07:17.796 INFO:tasks.rados.rados.0.smithi118.stdout:1699: writing smithi11881747-40 from 2809856 to 3317760 tid 3 2024-03-20T08:07:17.797 INFO:tasks.rados.rados.0.smithi118.stdout:1700: snap_create 2024-03-20T08:07:17.801 INFO:tasks.rados.rados.0.smithi118.stdout:1699: finishing write tid 1 to smithi11881747-40 2024-03-20T08:07:17.802 INFO:tasks.rados.rados.0.smithi118.stdout:1699: finishing write tid 2 to smithi11881747-40 2024-03-20T08:07:17.802 INFO:tasks.rados.rados.0.smithi118.stdout:1699: finishing write tid 3 to smithi11881747-40 2024-03-20T08:07:17.802 INFO:tasks.rados.rados.0.smithi118.stdout:1699: finishing write tid 4 to smithi11881747-40 2024-03-20T08:07:17.804 INFO:tasks.rados.rados.0.smithi118.stdout:1699: finishing write tid 5 to smithi11881747-40 2024-03-20T08:07:17.804 INFO:tasks.rados.rados.0.smithi118.stdout:1699: oid 40 updating version 0 to 7051 2024-03-20T08:07:17.804 INFO:tasks.rados.rados.0.smithi118.stdout:1699: oid 40 updating version 7051 to 7053 2024-03-20T08:07:17.804 INFO:tasks.rados.rados.0.smithi118.stdout:1699: oid 40 version 7053 is already newer than 7052 2024-03-20T08:07:17.804 INFO:tasks.rados.rados.0.smithi118.stdout:1699: oid 40 version 7053 is already newer than 7050 2024-03-20T08:07:17.804 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 7053 (ObjNum 532 snap 173 seq_num 532) dirty exists 2024-03-20T08:07:17.804 INFO:tasks.rados.rados.0.smithi118.stdout:1699: left oid 40 (ObjNum 532 snap 173 seq_num 532) 2024-03-20T08:07:17.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[93121]: osdmap e409: 8 total, 8 up, 8 in 2024-03-20T08:07:17.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:17.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:17.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:17.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:17.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:17.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:17.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[95692]: osdmap e409: 8 total, 8 up, 8 in 2024-03-20T08:07:17.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:17.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:17.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:17.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:17.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:17.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:17.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:17.794+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:17.811 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:17.794+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:17.812 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:17.796+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:17.812 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:17.790+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:17.812 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:17.776+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:17.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.563+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.573+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.579+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.590+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.593+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.606+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.608+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.622+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.622+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.637+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.640+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.653+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.656+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.666+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.672+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.680+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.688+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.695+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.704+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.712+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.726+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.740+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.753+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.754+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.770+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.770+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.791+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:17.815 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.792+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.815 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.810+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:17.815 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.811+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.825+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.825+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.839+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.841+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.854+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.855+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.870+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.871+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.063 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.886+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.887+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.901+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.903+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.917+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.920+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.933+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.935+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.948+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.951+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.965+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.968+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.981+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.985+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.064 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:17.996+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.002+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.013+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.018+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.026+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.033+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.040+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.048+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.053+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.065 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.063+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:17 smithi196 ceph-mon[87080]: osdmap e409: 8 total, 8 up, 8 in 2024-03-20T08:07:18.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:18.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:18.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:18.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:18.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:17 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:18.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:17 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:18.324 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.068+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.324 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.080+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.324 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.082+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.324 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.096+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.324 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.096+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.110+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.112+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.123+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.128+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.135+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.143+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.150+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.159+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.164+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.174+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.175+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.188+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.189+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.200+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.204+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.213+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.218+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.225+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.232+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.238+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.326 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.248+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.251+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.263+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.265+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.279+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.279+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.294+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.308+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.312+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.325+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.330+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.341+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.345+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.358+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.362+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.372+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.378+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.385+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.398+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.407+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.411+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.422+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.423+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.436+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.437+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.450+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.454+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.464+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.470+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.480+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.486+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.493+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.500+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.506+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.515+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.519+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.529+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.531+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.543+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.544+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.557+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.558+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.570+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.573+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.584+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.787 INFO:tasks.rados.rados.0.smithi118.stdout:1699: done (1 left) 2024-03-20T08:07:18.787 INFO:tasks.rados.rados.0.smithi118.stdout:1700: done (0 left) 2024-03-20T08:07:18.787 INFO:tasks.rados.rados.0.smithi118.stdout:1701: copy_from oid 50 from oid 10 current snap is 174 2024-03-20T08:07:18.787 INFO:tasks.rados.rados.0.smithi118.stdout:1702: snap_remove snap 170 2024-03-20T08:07:18.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[93121]: osdmap e410: 8 total, 8 up, 8 in 2024-03-20T08:07:18.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:18.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:18.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:18.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:18.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[93121]: pgmap v292: 121 pgs: 5 peering, 2 active+clean+snaptrim_error, 22 active+undersized, 3 active+undersized+degraded, 1 peering+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 83 active+clean; 445 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.3 MiB/s wr, 14 op/s; 43/948 objects degraded (4.536%); 6.1 MiB/s, 1 objects/s recovering 2024-03-20T08:07:18.837 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:18.797+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:18.837 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:18.798+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:18.837 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:18.797+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:18.837 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[95692]: osdmap e410: 8 total, 8 up, 8 in 2024-03-20T08:07:18.837 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:18.838 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:18.838 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:18.838 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:18.838 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-mon[95692]: pgmap v292: 121 pgs: 5 peering, 2 active+clean+snaptrim_error, 22 active+undersized, 3 active+undersized+degraded, 1 peering+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 83 active+clean; 445 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.3 MiB/s wr, 14 op/s; 43/948 objects degraded (4.536%); 6.1 MiB/s, 1 objects/s recovering 2024-03-20T08:07:18.838 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:18.791+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:18.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.589+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.597+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.605+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.610+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.620+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.622+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.634+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.635+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.648+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.648+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.661+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.663+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.674+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.677+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.688+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.691+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.700+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.706+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.714+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.727+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.740+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.751+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.753+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.765+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.766+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.778+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.781+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.795+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.803+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.813+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.821+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:18.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.828+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:18.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.836+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.087 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.841+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.851+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.854+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.869+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.881+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.881+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.893+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.896+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.906+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.910+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.919+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.924+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.088 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.931+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.939+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.944+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.953+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.958+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.969+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.971+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.983+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.983+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.999+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.089 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:18.999+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.012+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.014+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.026+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.029+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.039+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.054+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.058+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.066+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.073+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.079+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.087+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:18 smithi196 ceph-mon[87080]: osdmap e410: 8 total, 8 up, 8 in 2024-03-20T08:07:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:18 smithi196 ceph-mon[87080]: pgmap v292: 121 pgs: 5 peering, 2 active+clean+snaptrim_error, 22 active+undersized, 3 active+undersized+degraded, 1 peering+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 83 active+clean; 445 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.3 MiB/s wr, 14 op/s; 43/948 objects degraded (4.536%); 6.1 MiB/s, 1 objects/s recovering 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.092+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.103+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.106+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.121+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.136+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.138+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.151+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.153+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.165+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.170+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.179+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.185+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.191+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.201+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.203+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.215+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.217+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.230+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.244+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.245+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.257+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.259+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.271+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.275+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.287+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.290+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.305+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.305+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.319+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.320+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.335+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.352+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.354+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.609 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.369+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.609 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.371+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.383+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.385+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.396+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.399+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.409+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.414+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.422+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.429+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.436+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.444+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.449+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.460+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.462+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.475+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.476+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.490+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.491+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.503+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.506+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.517+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.522+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.530+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.537+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.543+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.551+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.557+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.612 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.566+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.612 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.569+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.612 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.581+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.612 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.582+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.612 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.596+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.612 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.596+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.796 INFO:tasks.rados.rados.0.smithi118.stdout:1702: done (1 left) 2024-03-20T08:07:19.796 INFO:tasks.rados.rados.0.smithi118.stdout:1703: copy_from oid 5 from oid 2 current snap is 174 2024-03-20T08:07:19.796 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 174 2024-03-20T08:07:19.796 INFO:tasks.rados.rados.0.smithi118.stdout:1704: seq_num 533 ranges {0=532480,532480=630784,1163264=516096,1679360=57344} 2024-03-20T08:07:19.798 INFO:tasks.rados.rados.0.smithi118.stdout:1704: writing smithi11881747-11 from 0 to 532480 tid 1 2024-03-20T08:07:19.802 INFO:tasks.rados.rados.0.smithi118.stdout:1704: writing smithi11881747-11 from 532480 to 1163264 tid 2 2024-03-20T08:07:19.806 INFO:tasks.rados.rados.0.smithi118.stdout:1704: writing smithi11881747-11 from 1163264 to 1679360 tid 3 2024-03-20T08:07:19.806 INFO:tasks.rados.rados.0.smithi118.stdout:1704: writing smithi11881747-11 from 1679360 to 1736704 tid 4 2024-03-20T08:07:19.807 INFO:tasks.rados.rados.0.smithi118.stdout:1705: snap_create 2024-03-20T08:07:19.807 INFO:tasks.rados.rados.0.smithi118.stdout:1701: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:07:19.807 INFO:tasks.rados.rados.0.smithi118.stdout:1701: finishing copy_from to smithi11881747-50 2024-03-20T08:07:19.807 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 1019 (ObjNum 472 snap 156 seq_num 472) dirty exists 2024-03-20T08:07:19.811 INFO:tasks.rados.rados.0.smithi118.stdout:1704: finishing write tid 1 to smithi11881747-11 2024-03-20T08:07:19.811 INFO:tasks.rados.rados.0.smithi118.stdout:1704: finishing write tid 2 to smithi11881747-11 2024-03-20T08:07:19.815 INFO:tasks.rados.rados.0.smithi118.stdout:1704: finishing write tid 3 to smithi11881747-11 2024-03-20T08:07:19.815 INFO:tasks.rados.rados.0.smithi118.stdout:1704: finishing write tid 4 to smithi11881747-11 2024-03-20T08:07:19.815 INFO:tasks.rados.rados.0.smithi118.stdout:1704: finishing write tid 5 to smithi11881747-11 2024-03-20T08:07:19.816 INFO:tasks.rados.rados.0.smithi118.stdout:1704: finishing write tid 6 to smithi11881747-11 2024-03-20T08:07:19.816 INFO:tasks.rados.rados.0.smithi118.stdout:1704: oid 11 updating version 0 to 1029 2024-03-20T08:07:19.816 INFO:tasks.rados.rados.0.smithi118.stdout:1704: oid 11 version 1029 is already newer than 1028 2024-03-20T08:07:19.816 INFO:tasks.rados.rados.0.smithi118.stdout:1704: oid 11 version 1029 is already newer than 1025 2024-03-20T08:07:19.816 INFO:tasks.rados.rados.0.smithi118.stdout:1704: oid 11 version 1029 is already newer than 1026 2024-03-20T08:07:19.816 INFO:tasks.rados.rados.0.smithi118.stdout:1704: oid 11 version 1029 is already newer than 1027 2024-03-20T08:07:19.816 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 1029 (ObjNum 533 snap 174 seq_num 533) dirty exists 2024-03-20T08:07:19.816 INFO:tasks.rados.rados.0.smithi118.stdout:1704: left oid 11 (ObjNum 533 snap 174 seq_num 533) 2024-03-20T08:07:19.828 INFO:tasks.rados.rados.0.smithi118.stdout:1703: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:07:19.829 INFO:tasks.rados.rados.0.smithi118.stdout:1703: finishing copy_from to smithi11881747-5 2024-03-20T08:07:19.829 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 1274 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:07:19.866 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:19.801+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:19.866 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:19.802+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:19.866 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:19.802+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:19.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.609+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.612+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.624+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.628+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.641+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.645+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.656+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.659+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.669+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.675+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.682+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.690+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.695+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.703+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.708+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.717+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.720+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.732+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.733+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.745+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.746+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.757+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.761+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.770+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.775+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.783+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.794+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.802+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.822+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.822+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.838+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.839+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.853+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:19.869 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.854+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:19.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 43/948 objects degraded (4.536%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:19.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[93121]: osdmap e411: 8 total, 8 up, 8 in 2024-03-20T08:07:19.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:19.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:19.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:19.870 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:19.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 43/948 objects degraded (4.536%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:19.870 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[95692]: osdmap e411: 8 total, 8 up, 8 in 2024-03-20T08:07:19.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:19.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:19.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:19.871 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:19.871 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:19.795+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:19.871 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:19.812+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:20.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.867+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.871+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.882+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.886+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.897+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.903+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.912+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.122 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.918+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.929+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.932+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.945+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.948+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.958+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.963+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.972+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.981+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.987+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.123 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:19.996+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.002+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.011+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.017+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.027+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.031+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.042+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.046+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.059+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.060+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.074+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.091+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.091+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.107+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.124 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.109+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.125 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.121+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:19 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 43/948 objects degraded (4.536%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:20.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:19 smithi196 ceph-mon[87080]: osdmap e411: 8 total, 8 up, 8 in 2024-03-20T08:07:20.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:20.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:20.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:20.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:20.374 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.123+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.136+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.138+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.151+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.154+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.166+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.170+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.180+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.185+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.195+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.201+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.211+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.216+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.228+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.232+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.245+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.247+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.260+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.263+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.276+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.278+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.291+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.294+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.305+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.311+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.318+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.326+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.332+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.342+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.347+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.358+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.362+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.374+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.378+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.391+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.392+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.406+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.407+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.421+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.422+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.438+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.438+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.453+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.453+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.468+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.468+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.483+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.485+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.496+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.500+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.510+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.517+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.627 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.524+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.533+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.539+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.548+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.553+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.563+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.568+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.578+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.582+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.593+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.595+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.610+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.610+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.623+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.628 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.626+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.800 INFO:tasks.rados.rados.0.smithi118.stdout:1701: done (3 left) 2024-03-20T08:07:20.800 INFO:tasks.rados.rados.0.smithi118.stdout:1703: done (2 left) 2024-03-20T08:07:20.800 INFO:tasks.rados.rados.0.smithi118.stdout:1704: done (1 left) 2024-03-20T08:07:20.801 INFO:tasks.rados.rados.0.smithi118.stdout:1705: done (0 left) 2024-03-20T08:07:20.801 INFO:tasks.rados.rados.0.smithi118.stdout:1706: read oid 9 snap -1 2024-03-20T08:07:20.801 INFO:tasks.rados.rados.0.smithi118.stdout:1706: expect (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:07:20.801 INFO:tasks.rados.rados.0.smithi118.stdout:1707: setattr oid 15 current snap is 175 2024-03-20T08:07:20.803 INFO:tasks.rados.rados.0.smithi118.stdout:1708: snap_remove snap 171 2024-03-20T08:07:20.881 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:20.805+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:20.881 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:20.816+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:20.881 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[95692]: osdmap e412: 8 total, 8 up, 8 in 2024-03-20T08:07:20.881 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:20.881 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[95692]: pgmap v295: 121 pgs: 5 peering, 3 active+clean+snaptrim_error, 1 peering+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 107 active+clean; 450 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 10 op/s; 8.5 MiB/s, 2 objects/s recovering 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:07:20.790+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 4 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[93121]: osdmap e412: 8 total, 8 up, 8 in 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:20.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-mon[93121]: pgmap v295: 121 pgs: 5 peering, 3 active+clean+snaptrim_error, 1 peering+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 107 active+clean; 450 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 10 op/s; 8.5 MiB/s, 2 objects/s recovering 2024-03-20T08:07:20.883 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:20.800+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:20.883 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:20.813+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:20.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.637+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.642+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.652+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.659+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.667+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.677+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.683+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.694+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.697+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.711+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.712+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.726+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.727+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.739+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.742+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.752+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.757+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.766+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.772+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.778+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.787+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.792+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.807+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.812+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.825+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.828+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.841+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.842+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.855+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.885 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.856+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.886 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.869+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:20.886 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.870+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:20.886 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.882+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.884+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.896+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.900+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.911+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.915+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.923+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.930+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.936+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.944+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.948+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.958+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.961+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.974+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.975+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.988+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:20.989+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.138 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.002+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:20 smithi196 ceph-mon[87080]: osdmap e412: 8 total, 8 up, 8 in 2024-03-20T08:07:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:21.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:20 smithi196 ceph-mon[87080]: pgmap v295: 121 pgs: 5 peering, 3 active+clean+snaptrim_error, 1 peering+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 107 active+clean; 450 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 10 op/s; 8.5 MiB/s, 2 objects/s recovering 2024-03-20T08:07:21.139 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.003+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.139 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.017+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.139 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.020+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.031+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.035+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.045+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.050+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.058+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.064+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.070+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.079+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.086+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.093+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.099+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.108+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.111+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.124+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.124+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.137+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.393 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.139+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.150+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.155+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.164+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.170+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.176+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.184+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.188+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.394 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.199+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.201+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.213+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.214+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.228+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.228+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.240+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.244+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.255+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.259+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.268+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.395 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.276+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.282+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.293+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.295+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.308+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.308+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.322+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.322+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.336+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.351+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.351+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.363+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.365+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.377+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.396 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.381+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.393+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.646 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:07:21.623+0000 7ff9af1ba700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-03-20T08:07:21.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.398+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.410+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.414+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.424+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.427+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.436+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.441+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.449+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.455+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.462+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.473+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.477+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.489+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.494+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.505+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.509+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.520+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.524+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.535+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.539+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.549+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.553+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.565+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.567+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.582+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.583+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.595+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.598+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.608+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.613+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.649 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.622+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.630+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.638+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.650 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.646+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.805 INFO:tasks.rados.rados.0.smithi118.stdout:1708: done (2 left) 2024-03-20T08:07:21.805 INFO:tasks.rados.rados.0.smithi118.stdout:1709: snap_remove snap 174 2024-03-20T08:07:21.897 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:21.805+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:21.898 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:21.811+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:21.898 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:21.811+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:21.898 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:21.812+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 4 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/948 objects degraded (4.536%), 3 pgs degraded) 2024-03-20T08:07:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: osdmap e413: 8 total, 8 up, 8 in 2024-03-20T08:07:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:21.899 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:21.900 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:21.900 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:21.900 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:21.900 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:07:21.900 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:07:21.900 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:21.901 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:21.901 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:21.901 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:21.901 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:21.901 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:21.902 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 4 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:21.902 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/948 objects degraded (4.536%), 3 pgs degraded) 2024-03-20T08:07:21.902 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:21.902 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: osdmap e413: 8 total, 8 up, 8 in 2024-03-20T08:07:21.902 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:21.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:21.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:21.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:21.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:21.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:07:21.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:07:21.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:21.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:21.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:21.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:21.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:21.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:21.905 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:21.815+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:21.905 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.652+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.661+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.667+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.676+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.680+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.691+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.692+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.706+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.706+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.720+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.723+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.907 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.733+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.739+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.746+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.754+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.759+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.770+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.908 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.772+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.784+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.785+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.800+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.801+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.818+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.820+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.831+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.835+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.843+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.851+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.857+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.871+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.880+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.884+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:21.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.896+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:21.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.898+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 4 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/948 objects degraded (4.536%), 3 pgs degraded) 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: osdmap e413: 8 total, 8 up, 8 in 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:22.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:07:22.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:07:22.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:22.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:22.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:22.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:22.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:22.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:22.150 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.911+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.150 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.912+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.150 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.925+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.150 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.927+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.150 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.938+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.150 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.941+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.150 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.952+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.150 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.956+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.965+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.970+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.978+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.984+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.991+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:21.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.003+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.013+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.015+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.028+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.028+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.042+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.055+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.060+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.069+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.075+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.081+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.091+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.095+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.108+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.120+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.133+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.153 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.135+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.153 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.146+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.153 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.150+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.159+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.165+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.173+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.181+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.186+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.199+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.210+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.211+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.224+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.225+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.237+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.240+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.249+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.262+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.407 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.269+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.275+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.285+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.289+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.300+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.301+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.313+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.315+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.326+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.329+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.339+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.345+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.352+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.360+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.408 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.365+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.409 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.376+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.409 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.379+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.409 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.390+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.409 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.393+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.409 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.406+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.409 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.406+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.638 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:22.661 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.419+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.420+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.434+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.436+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.448+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.451+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.462+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.467+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.475+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.481+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.488+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.495+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.500+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.662 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.510+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.514+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.526+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.530+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.541+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.543+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.557+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.557+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.570+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.581+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.587+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.595+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.603+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.610+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.618+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.622+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.634+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.635+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.648+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.649+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.661+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.813 INFO:tasks.rados.rados.0.smithi118.stdout:1709: done (2 left) 2024-03-20T08:07:22.813 INFO:tasks.rados.rados.0.smithi118.stdout:1710: copy_from oid 7 from oid 33 current snap is 175 2024-03-20T08:07:22.813 INFO:tasks.rados.rados.0.smithi118.stdout:1711: rollback oid 11 current snap is 175 2024-03-20T08:07:22.813 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 172 2024-03-20T08:07:22.813 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 434 (ObjNum 526 snap 170 seq_num 526) dirty exists 2024-03-20T08:07:22.814 INFO:tasks.rados.rados.0.smithi118.stdout:1706: expect (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:07:22.911 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:22.810+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:22.912 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:22.809+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:22.912 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:22.809+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:22.912 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:22.912 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[93121]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-03-20T08:07:22.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[93121]: osdmap e414: 8 total, 8 up, 8 in 2024-03-20T08:07:22.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:22.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:22.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:22.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:22.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[93121]: pgmap v298: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 455 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 5 op/s; 2.4 MiB/s, 1 objects/s recovering 2024-03-20T08:07:22.914 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:22.819+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:22.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:22.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[95692]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-03-20T08:07:22.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[95692]: osdmap e414: 8 total, 8 up, 8 in 2024-03-20T08:07:22.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:22.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:22.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:22.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:22.916 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-mon[95692]: pgmap v298: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 455 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 5 op/s; 2.4 MiB/s, 1 objects/s recovering 2024-03-20T08:07:22.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.665+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.674+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.679+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.687+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.694+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.701+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.918 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.709+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.918 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.716+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.918 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.724+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.918 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.730+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.918 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.739+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.918 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.743+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.919 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.753+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.919 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.756+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.919 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.768+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.919 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.769+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.919 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.782+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.919 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.783+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.920 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.796+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.920 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.798+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.920 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.814+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.920 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.822+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.920 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.832+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.921 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.838+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.921 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.851+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.921 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.867+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.921 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.868+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.921 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.883+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.921 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.883+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.898+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.898+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.911+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:22.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.912+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:22.993 INFO:tasks.rados.rados.0.smithi118.stdout:1706: done (3 left) 2024-03-20T08:07:22.993 INFO:tasks.rados.rados.0.smithi118.stdout:1707: done (2 left) 2024-03-20T08:07:22.993 INFO:tasks.rados.rados.0.smithi118.stdout:1712: read oid 16 snap -1 2024-03-20T08:07:22.993 INFO:tasks.rados.rados.0.smithi118.stdout:1712: expect (ObjNum 381 snap 129 seq_num 381) 2024-03-20T08:07:22.994 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 175 2024-03-20T08:07:22.994 INFO:tasks.rados.rados.0.smithi118.stdout:1711: finishing rollback tid 0 to smithi11881747-11 2024-03-20T08:07:22.994 INFO:tasks.rados.rados.0.smithi118.stdout:1713: seq_num 534 ranges {4096000=548864,4644864=516096,5160960=688128} 2024-03-20T08:07:23.019 INFO:tasks.rados.rados.0.smithi118.stdout:1713: writing smithi11881747-29 from 4096000 to 4644864 tid 1 2024-03-20T08:07:23.022 INFO:tasks.rados.rados.0.smithi118.stdout:1713: writing smithi11881747-29 from 4644864 to 5160960 tid 2 2024-03-20T08:07:23.028 INFO:tasks.rados.rados.0.smithi118.stdout:1713: writing smithi11881747-29 from 5160960 to 5849088 tid 3 2024-03-20T08:07:23.028 INFO:tasks.rados.rados.0.smithi118.stdout:1714: copy_from oid 40 from oid 23 current snap is 175 2024-03-20T08:07:23.028 INFO:tasks.rados.rados.0.smithi118.stdout:1711: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:07:23.028 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 1276 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-03-20T08:07:23.028 INFO:tasks.rados.rados.0.smithi118.stdout:1710: finishing copy_from racing read to smithi11881747-7 2024-03-20T08:07:23.028 INFO:tasks.rados.rados.0.smithi118.stdout:1710: finishing copy_from to smithi11881747-7 2024-03-20T08:07:23.028 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 7726 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:07:23.028 INFO:tasks.rados.rados.0.smithi118.stdout:1712: expect (ObjNum 381 snap 129 seq_num 381) 2024-03-20T08:07:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-mon[87080]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-03-20T08:07:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-mon[87080]: osdmap e414: 8 total, 8 up, 8 in 2024-03-20T08:07:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:22 smithi196 ceph-mon[87080]: pgmap v298: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 455 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 5 op/s; 2.4 MiB/s, 1 objects/s recovering 2024-03-20T08:07:23.164 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.924+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.164 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.927+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.164 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.940+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.164 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.164 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.953+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.164 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.959+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.967+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.973+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.981+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.989+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:22.996+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.014+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.025+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.030+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.048+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.062+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.065+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.077+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.079+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.091+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.092+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.106+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.107+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.121+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.122+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.135+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.166 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.149+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.167 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.150+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.167 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.164+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.286 INFO:tasks.rados.rados.0.smithi118.stdout:1713: finishing write tid 1 to smithi11881747-29 2024-03-20T08:07:23.286 INFO:tasks.rados.rados.0.smithi118.stdout:1713: finishing write tid 2 to smithi11881747-29 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1713: finishing write tid 3 to smithi11881747-29 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1713: finishing write tid 4 to smithi11881747-29 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1713: finishing write tid 5 to smithi11881747-29 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1713: oid 29 updating version 0 to 1309 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1713: oid 29 updating version 1309 to 1312 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1713: oid 29 version 1312 is already newer than 1311 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1713: oid 29 version 1312 is already newer than 1310 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1312 (ObjNum 534 snap 175 seq_num 534) dirty exists 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1713: left oid 29 (ObjNum 534 snap 175 seq_num 534) 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1714: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1714: finishing copy_from to smithi11881747-40 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 1267 (ObjNum 464 snap 155 seq_num 464) dirty exists 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1710: done (4 left) 2024-03-20T08:07:23.287 INFO:tasks.rados.rados.0.smithi118.stdout:1711: done (3 left) 2024-03-20T08:07:23.288 INFO:tasks.rados.rados.0.smithi118.stdout:1712: done (2 left) 2024-03-20T08:07:23.288 INFO:tasks.rados.rados.0.smithi118.stdout:1713: done (1 left) 2024-03-20T08:07:23.288 INFO:tasks.rados.rados.0.smithi118.stdout:1714: done (0 left) 2024-03-20T08:07:23.288 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 175 2024-03-20T08:07:23.288 INFO:tasks.rados.rados.0.smithi118.stdout:1715: seq_num 535 ranges {3661824=565248,4227072=802816,5029888=401408} 2024-03-20T08:07:23.309 INFO:tasks.rados.rados.0.smithi118.stdout:1715: writing smithi11881747-46 from 3661824 to 4227072 tid 1 2024-03-20T08:07:23.314 INFO:tasks.rados.rados.0.smithi118.stdout:1715: writing smithi11881747-46 from 4227072 to 5029888 tid 2 2024-03-20T08:07:23.317 INFO:tasks.rados.rados.0.smithi118.stdout:1715: writing smithi11881747-46 from 5029888 to 5431296 tid 3 2024-03-20T08:07:23.317 INFO:tasks.rados.rados.0.smithi118.stdout:1716: copy_from oid 18 from oid 20 current snap is 175 2024-03-20T08:07:23.318 INFO:tasks.rados.rados.0.smithi118.stdout:1715: finishing write tid 1 to smithi11881747-46 2024-03-20T08:07:23.318 INFO:tasks.rados.rados.0.smithi118.stdout:1717: delete oid 45 current snap is 175 2024-03-20T08:07:23.318 INFO:tasks.rados.rados.0.smithi118.stdout:1715: finishing write tid 2 to smithi11881747-46 2024-03-20T08:07:23.320 INFO:tasks.rados.rados.0.smithi118.stdout:1717: done (2 left) 2024-03-20T08:07:23.320 INFO:tasks.rados.rados.0.smithi118.stdout:1718: snap_remove snap 165 2024-03-20T08:07:23.421 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.166+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.421 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.177+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.421 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.421 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.192+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.421 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.196+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.421 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.206+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.421 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.210+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.421 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.221+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.225+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.236+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.240+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.250+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.264+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.270+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.279+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.283+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.294+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.298+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.309+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.325+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.331+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.340+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.349+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.358+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.365+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.371+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.378+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.384+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.397+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.407+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.409+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.421+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.421+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:07:23] "GET /metrics HTTP/1.1" 200 36270 "" "Prometheus/2.43.0" 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.434+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.435+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.447+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.449+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.463+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.463+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.475+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.477+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.489+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.492+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.502+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.505+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.515+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.518+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.530+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.533+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.544+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.547+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.557+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.562+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.569+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.578+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.584+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.591+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.597+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.604+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.611+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.619+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.625+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.633+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.638+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.647+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.651+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.662+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.665+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.806 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:23.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:23.816 INFO:tasks.rados.rados.0.smithi118.stdout:1718: done (2 left) 2024-03-20T08:07:23.816 INFO:tasks.rados.rados.0.smithi118.stdout:1719: read oid 1 snap 173 2024-03-20T08:07:23.816 INFO:tasks.rados.rados.0.smithi118.stdout:1719: expect (ObjNum 521 snap 169 seq_num 521) 2024-03-20T08:07:23.816 INFO:tasks.rados.rados.0.smithi118.stdout:1720: rmattr oid 38 current snap is 175 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: finishing write tid 3 to smithi11881747-46 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: finishing write tid 4 to smithi11881747-46 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: finishing write tid 5 to smithi11881747-46 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: oid 46 updating version 0 to 1280 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: oid 46 updating version 1280 to 1282 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: oid 46 version 1282 is already newer than 1281 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: oid 46 updating version 1282 to 1283 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 1283 (ObjNum 535 snap 175 seq_num 535) dirty exists 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: left oid 46 (ObjNum 535 snap 175 seq_num 535) 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1716: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1716: finishing copy_from to smithi11881747-18 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 7051 (ObjNum 486 snap 159 seq_num 486) dirty exists 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1715: done (3 left) 2024-03-20T08:07:23.817 INFO:tasks.rados.rados.0.smithi118.stdout:1716: done (2 left) 2024-03-20T08:07:23.818 INFO:tasks.rados.rados.0.smithi118.stdout:1721: read oid 13 snap -1 2024-03-20T08:07:23.818 INFO:tasks.rados.rados.0.smithi118.stdout:1721: expect deleted 2024-03-20T08:07:23.818 INFO:tasks.rados.rados.0.smithi118.stdout:1722: copy_from oid 7 from oid 34 current snap is 175 2024-03-20T08:07:23.818 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 175 2024-03-20T08:07:23.818 INFO:tasks.rados.rados.0.smithi118.stdout:1723: seq_num 537 ranges {0=598016,598016=573440,1171456=491520,1662976=139264} 2024-03-20T08:07:23.820 INFO:tasks.rados.rados.0.smithi118.stdout:1723: writing smithi11881747-45 from 0 to 598016 tid 1 2024-03-20T08:07:23.824 INFO:tasks.rados.rados.0.smithi118.stdout:1723: writing smithi11881747-45 from 598016 to 1171456 tid 2 2024-03-20T08:07:23.828 INFO:tasks.rados.rados.0.smithi118.stdout:1723: writing smithi11881747-45 from 1171456 to 1662976 tid 3 2024-03-20T08:07:23.829 INFO:tasks.rados.rados.0.smithi118.stdout:1723: writing smithi11881747-45 from 1662976 to 1802240 tid 4 2024-03-20T08:07:23.829 INFO:tasks.rados.rados.0.smithi118.stdout:1724: setattr oid 28 current snap is 175 2024-03-20T08:07:23.829 INFO:tasks.rados.rados.0.smithi118.stdout:1722: finishing copy_from to smithi11881747-7 2024-03-20T08:07:23.829 INFO:tasks.rados.rados.0.smithi118.stdout:1722: got expected ENOENT (src dne) 2024-03-20T08:07:23.829 INFO:tasks.rados.rados.0.smithi118.stdout:1722: finishing copy_from racing read to smithi11881747-7 2024-03-20T08:07:23.829 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7056 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:07:23.830 INFO:tasks.rados.rados.0.smithi118.stdout:1720: done (5 left) 2024-03-20T08:07:23.831 INFO:tasks.rados.rados.0.smithi118.stdout:1721: done (4 left) 2024-03-20T08:07:23.831 INFO:tasks.rados.rados.0.smithi118.stdout:1722: done (3 left) 2024-03-20T08:07:23.831 INFO:tasks.rados.rados.0.smithi118.stdout:1725: snap_remove snap 168 2024-03-20T08:07:23.936 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:23.936 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:23.936 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[95692]: osdmap e415: 8 total, 8 up, 8 in 2024-03-20T08:07:23.936 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:23.936 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:23.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:23.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:23.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[93121]: osdmap e415: 8 total, 8 up, 8 in 2024-03-20T08:07:23.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:23.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:23.937 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:23.814+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:23.937 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:23.814+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:23.937 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:23.814+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:23.938 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:23.815+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:23.938 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:23.826+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:23.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.677+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.678+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.691+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.692+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.704+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.706+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.718+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.722+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.731+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.746+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.750+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.758+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.764+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.770+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.778+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.783+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.792+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.797+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.806+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.811+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.830+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.837+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.852+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.856+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.868+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.871+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.880+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.940 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.884+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.941 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.895+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.941 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.897+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.941 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.907+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.941 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.911+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.941 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.922+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.941 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.924+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:23.941 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.936+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:23.941 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.937+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:24.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:24.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:23 smithi196 ceph-mon[87080]: osdmap e415: 8 total, 8 up, 8 in 2024-03-20T08:07:24.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:24.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:24.192 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.950+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.192 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.951+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.192 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.963+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.192 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.965+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.192 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.975+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.979+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.988+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:23.992+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.000+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.006+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.013+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.019+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.025+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.033+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.040+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.048+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.193 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.054+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.063+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.066+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.076+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.079+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.091+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.092+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.105+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.105+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.119+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.194 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.119+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.132+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.133+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.145+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.148+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.159+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.163+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.173+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.178+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.185+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.192+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.198+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.206+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.212+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.220+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.227+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.234+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.240+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.248+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.253+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.261+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.445 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.269+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.446 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.276+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.446 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.284+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.446 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.291+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.446 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.297+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.446 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.305+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.446 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.311+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.446 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.320+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.325+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.334+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.339+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.351+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.362+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.364+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.376+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.378+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.389+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.391+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.403+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.404+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.415+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.447 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.418+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.428+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.431+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.442+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.445+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.708 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.457+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.458+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.470+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.470+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.484+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.484+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.499+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.499+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.512+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.513+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.524+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.526+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.537+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.540+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.709 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.549+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.710 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.554+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.710 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.563+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.710 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.570+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.710 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.576+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.710 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.583+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.710 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.588+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.597+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.602+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.612+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.616+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.627+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.629+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.642+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.642+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.654+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.655+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.669+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.669+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.682+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.711 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.682+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.712 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.695+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.712 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.695+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.712 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.708+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.712 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.709+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1725: done (3 left) 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1726: rollback oid 13 current snap is 175 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 13 to 167 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1723: finishing write tid 1 to smithi11881747-45 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1723: finishing write tid 2 to smithi11881747-45 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1723: finishing write tid 3 to smithi11881747-45 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1723: finishing write tid 4 to smithi11881747-45 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1723: finishing write tid 5 to smithi11881747-45 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 1314 (ObjNum 446 snap 148 seq_num 446) dirty exists 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1723: finishing write tid 6 to smithi11881747-45 2024-03-20T08:07:24.819 INFO:tasks.rados.rados.0.smithi118.stdout:1723: oid 45 updating version 0 to 7058 2024-03-20T08:07:24.820 INFO:tasks.rados.rados.0.smithi118.stdout:1723: oid 45 version 7058 is already newer than 7057 2024-03-20T08:07:24.820 INFO:tasks.rados.rados.0.smithi118.stdout:1723: oid 45 version 7058 is already newer than 7055 2024-03-20T08:07:24.820 INFO:tasks.rados.rados.0.smithi118.stdout:1723: oid 45 version 7058 is already newer than 7056 2024-03-20T08:07:24.820 INFO:tasks.rados.rados.0.smithi118.stdout:1723: oid 45 version 7058 is already newer than 7054 2024-03-20T08:07:24.820 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 7058 (ObjNum 537 snap 175 seq_num 537) dirty exists 2024-03-20T08:07:24.820 INFO:tasks.rados.rados.0.smithi118.stdout:1723: left oid 45 (ObjNum 537 snap 175 seq_num 537) 2024-03-20T08:07:24.820 INFO:tasks.rados.rados.0.smithi118.stdout:1719: expect (ObjNum 521 snap 169 seq_num 521) 2024-03-20T08:07:24.962 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:24.825+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:24.962 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:24.828+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:24.962 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[93121]: osdmap e416: 8 total, 8 up, 8 in 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[93121]: pgmap v301: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 459 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[95692]: osdmap e416: 8 total, 8 up, 8 in 2024-03-20T08:07:24.963 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:24.964 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-mon[95692]: pgmap v301: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 459 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-03-20T08:07:24.964 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:24.818+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:24.964 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:24.831+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.722+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.722+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.735+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.748+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.750+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.760+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.765+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.773+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.778+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.787+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.965 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.793+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.802+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.808+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.817+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.831+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.840+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.847+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.853+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.859+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.870+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.877+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.885+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.891+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.898+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.904+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.966 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.913+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.967 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.920+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.967 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.927+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.967 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.933+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.967 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.941+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.967 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.948+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:24.967 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.956+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:24.967 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.963+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:25.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:25.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:25.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:24 smithi196 ceph-mon[87080]: osdmap e416: 8 total, 8 up, 8 in 2024-03-20T08:07:25.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:25.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:24 smithi196 ceph-mon[87080]: pgmap v301: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 459 MiB data, 3.4 GiB used, 712 GiB / 715 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 8 op/s 2024-03-20T08:07:25.187 INFO:tasks.rados.rados.0.smithi118.stdout:1726: finishing rollback tid 1 to smithi11881747-13 2024-03-20T08:07:25.187 INFO:tasks.rados.rados.0.smithi118.stdout:1726: finishing rollback tid 2 to smithi11881747-13 2024-03-20T08:07:25.187 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 1278 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:07:25.187 INFO:tasks.rados.rados.0.smithi118.stdout:1719: done (3 left) 2024-03-20T08:07:25.187 INFO:tasks.rados.rados.0.smithi118.stdout:1723: done (2 left) 2024-03-20T08:07:25.187 INFO:tasks.rados.rados.0.smithi118.stdout:1724: done (1 left) 2024-03-20T08:07:25.188 INFO:tasks.rados.rados.0.smithi118.stdout:1726: done (0 left) 2024-03-20T08:07:25.188 INFO:tasks.rados.rados.0.smithi118.stdout:1727: read oid 31 snap 89 2024-03-20T08:07:25.188 INFO:tasks.rados.rados.0.smithi118.stdout:1727: expect (ObjNum 207 snap 67 seq_num 207) 2024-03-20T08:07:25.188 INFO:tasks.rados.rados.0.smithi118.stdout:1728: rollback oid 26 current snap is 175 2024-03-20T08:07:25.188 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 161 2024-03-20T08:07:25.188 INFO:tasks.rados.rados.0.smithi118.stdout:1729: snap_remove snap 135 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.971+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.978+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.984+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.991+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:24.998+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.005+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.012+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.020+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.026+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.222 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.035+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.039+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.050+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.052+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.064+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.065+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.076+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.079+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.091+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.092+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.105+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.106+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.119+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.121+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.132+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.223 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.136+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.146+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.150+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.159+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.163+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.173+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.178+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.187+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.205+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.212+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.224 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.221+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.230+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.238+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.473 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.244+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.251+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.257+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.264+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.270+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.277+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.284+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.291+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.297+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.303+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.311+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.317+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.324+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.330+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.474 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.338+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.475 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.344+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.475 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.352+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.475 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.357+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.475 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.367+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.475 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.371+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.475 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.382+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.475 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.383+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.475 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.395+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.395+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.408+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.409+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.421+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.422+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.435+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.436+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.448+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.449+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.461+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.461+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.473+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.476 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.475+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.733 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.487+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.488+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.499+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.501+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.512+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.515+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.525+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.529+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.538+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.542+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.552+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.557+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.565+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.734 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.578+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.585+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.590+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.598+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.603+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.612+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.616+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.735 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.626+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.631+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.640+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.644+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.655+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.657+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.669+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.669+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.681+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.682+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.694+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.695+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.707+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.708+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.721+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:25.736 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.733+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:25.824 INFO:tasks.rados.rados.0.smithi118.stdout:1729: done (2 left) 2024-03-20T08:07:25.824 INFO:tasks.rados.rados.0.smithi118.stdout:1730: rollback oid 29 current snap is 175 2024-03-20T08:07:25.824 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 158 2024-03-20T08:07:25.824 INFO:tasks.rados.rados.0.smithi118.stdout:1728: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:07:25.825 INFO:tasks.rados.rados.0.smithi118.stdout:1728: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:07:25.825 INFO:tasks.rados.rados.0.smithi118.stdout:1728: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:07:25.825 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 7057 (ObjNum 449 snap 149 seq_num 449) dirty exists 2024-03-20T08:07:25.825 INFO:tasks.rados.rados.0.smithi118.stdout:1727: expect (ObjNum 207 snap 67 seq_num 207) 2024-03-20T08:07:26.000 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:25.831+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:26.000 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:25.831+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:26.000 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:25.835+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:26.000 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:26.000 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[93121]: osdmap e417: 8 total, 8 up, 8 in 2024-03-20T08:07:26.000 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:26.000 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:26.001 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:26.001 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:26.001 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[95692]: osdmap e417: 8 total, 8 up, 8 in 2024-03-20T08:07:26.001 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:26.001 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:26.001 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:26.001 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.736+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.001 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.747+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.751+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.760+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.765+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.772+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.780+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.786+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.794+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.799+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.809+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.812+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.833+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap bf after trimming was started 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.845+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e2=[]:{97=[95],cf=[cf,ce,cd,cb,ca,c9,c8,c6,c5],d4=[d4,d3,d1],dd=[dd,dc,db,da,d9,d8,d7]} for object 5:66a7f7bc:::smithi11881747-33:c3 2024-03-20T08:07:26.002 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.852+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.003 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.869+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.003 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:25.834+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:26.003 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:25.823+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:26.003 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.887+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.003 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.903+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.919+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.935+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.952+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.968+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:25.983+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.000+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:25 smithi196 ceph-mon[87080]: osdmap e417: 8 total, 8 up, 8 in 2024-03-20T08:07:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:26.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.016+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.032+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.048+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.065+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.083+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.100+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.115+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.132+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.148+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.164+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.181+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.197+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.213+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.245+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.263+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.280+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.295+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.311+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.329+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.347+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.363+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.381+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.397+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.414+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.431+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.448+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.463+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.479+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.496+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.513+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.531 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.530+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.583 INFO:tasks.rados.rados.0.smithi118.stdout:1730: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1730: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1730: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1317 (ObjNum 423 snap 141 seq_num 423) dirty exists 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1727: done (2 left) 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1728: done (1 left) 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1730: done (0 left) 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1731: copy_from oid 19 from oid 31 current snap is 175 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1732: copy_from oid 3 from oid 21 current snap is 175 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1733: read oid 12 snap -1 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1733: expect (ObjNum 529 snap 170 seq_num 529) 2024-03-20T08:07:26.584 INFO:tasks.rados.rados.0.smithi118.stdout:1734: snap_create 2024-03-20T08:07:26.590 INFO:tasks.rados.rados.0.smithi118.stdout:1731: finishing copy_from racing read to smithi11881747-19 2024-03-20T08:07:26.594 INFO:tasks.rados.rados.0.smithi118.stdout:1731: finishing copy_from to smithi11881747-19 2024-03-20T08:07:26.594 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 1013 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:07:26.624 INFO:tasks.rados.rados.0.smithi118.stdout:1732: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:07:26.631 INFO:tasks.rados.rados.0.smithi118.stdout:1733: expect (ObjNum 529 snap 170 seq_num 529) 2024-03-20T08:07:26.794 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.546+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.794 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.563+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.581+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.630+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.646+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.663+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.679+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.694+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.710+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.726+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.743+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.762+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.779+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:26.795 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.794+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.048 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:26.840+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:27.048 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:26.837+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:27.048 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:26.837+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:27.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:27.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[95692]: osdmap e418: 8 total, 8 up, 8 in 2024-03-20T08:07:27.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:27.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:27.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:27.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset e2=[]:{97=[95],cf=[cf,ce,cd,cb,ca,c9,c8,c6,c5],d4=[d4,d3,d1],dd=[dd,dc,db,da,d9,d8,d7]} for object 5:66a7f7bc:::smithi11881747-33:c3 2024-03-20T08:07:27.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[95692]: pgmap v304: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 468 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-03-20T08:07:27.049 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:26.829+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:27.049 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:27.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[93121]: osdmap e418: 8 total, 8 up, 8 in 2024-03-20T08:07:27.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:27.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:27.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:27.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset e2=[]:{97=[95],cf=[cf,ce,cd,cb,ca,c9,c8,c6,c5],d4=[d4,d3,d1],dd=[dd,dc,db,da,d9,d8,d7]} for object 5:66a7f7bc:::smithi11881747-33:c3 2024-03-20T08:07:27.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-mon[93121]: pgmap v304: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 468 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-03-20T08:07:27.050 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.811+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.050 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.050 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.849+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.050 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.863+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.050 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.880+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.050 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.897+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.915+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.931+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.948+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.964+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.981+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:26.998+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.016+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.033+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.048+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:26 smithi196 ceph-mon[87080]: osdmap e418: 8 total, 8 up, 8 in 2024-03-20T08:07:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset e2=[]:{97=[95],cf=[cf,ce,cd,cb,ca,c9,c8,c6,c5],d4=[d4,d3,d1],dd=[dd,dc,db,da,d9,d8,d7]} for object 5:66a7f7bc:::smithi11881747-33:c3 2024-03-20T08:07:27.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:26 smithi196 ceph-mon[87080]: pgmap v304: 121 pgs: 4 active+clean+snaptrim_error, 3 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 468 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 12 op/s 2024-03-20T08:07:27.266 INFO:tasks.rados.rados.0.smithi118.stdout:1731: done (3 left) 2024-03-20T08:07:27.266 INFO:tasks.rados.rados.0.smithi118.stdout:1733: done (2 left) 2024-03-20T08:07:27.266 INFO:tasks.rados.rados.0.smithi118.stdout:1734: done (1 left) 2024-03-20T08:07:27.266 INFO:tasks.rados.rados.0.smithi118.stdout:1735: setattr oid 41 current snap is 176 2024-03-20T08:07:27.266 INFO:tasks.rados.rados.0.smithi118.stdout:1732: finishing copy_from to smithi11881747-3 2024-03-20T08:07:27.266 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 7055 (ObjNum 524 snap 169 seq_num 524) dirty exists 2024-03-20T08:07:27.267 INFO:tasks.rados.rados.0.smithi118.stdout:1732: done (1 left) 2024-03-20T08:07:27.267 INFO:tasks.rados.rados.0.smithi118.stdout:1736: read oid 46 snap -1 2024-03-20T08:07:27.267 INFO:tasks.rados.rados.0.smithi118.stdout:1736: expect (ObjNum 535 snap 175 seq_num 535) 2024-03-20T08:07:27.267 INFO:tasks.rados.rados.0.smithi118.stdout:1737: snap_create 2024-03-20T08:07:27.270 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1319 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:07:27.293 INFO:tasks.rados.rados.0.smithi118.stdout:1736: expect (ObjNum 535 snap 175 seq_num 535) 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.064+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.080+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.097+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.114+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.130+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.147+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.164+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.211+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.227+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.244+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.259+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.277+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.295+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.310+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.569 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.328+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.345+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.362+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.378+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.394+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.410+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.426+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.443+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.458+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.474+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.490+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.506+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.522+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.570 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.537+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.571 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.553+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.571 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.569+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.586+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.602+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.617+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.633+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.649+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.664+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.679+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.694+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.709+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.832 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.724+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.832 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.740+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.832 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.755+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.832 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.770+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.832 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.785+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.832 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.801+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.832 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.818+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:27.839 INFO:tasks.rados.rados.0.smithi118.stdout:1735: done (2 left) 2024-03-20T08:07:27.839 INFO:tasks.rados.rados.0.smithi118.stdout:1736: done (1 left) 2024-03-20T08:07:27.839 INFO:tasks.rados.rados.0.smithi118.stdout:1737: done (0 left) 2024-03-20T08:07:27.839 INFO:tasks.rados.rados.0.smithi118.stdout:1738: copy_from oid 12 from oid 38 current snap is 177 2024-03-20T08:07:27.840 INFO:tasks.rados.rados.0.smithi118.stdout:1739: read oid 23 snap 175 2024-03-20T08:07:27.840 INFO:tasks.rados.rados.0.smithi118.stdout:1739: expect (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:07:27.840 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 177 2024-03-20T08:07:27.840 INFO:tasks.rados.rados.0.smithi118.stdout:1740: seq_num 538 ranges {1605632=614400,2220032=450560,2670592=704512,3375104=49152} 2024-03-20T08:07:27.853 INFO:tasks.rados.rados.0.smithi118.stdout:1740: writing smithi11881747-27 from 1605632 to 2220032 tid 1 2024-03-20T08:07:27.856 INFO:tasks.rados.rados.0.smithi118.stdout:1740: writing smithi11881747-27 from 2220032 to 2670592 tid 2 2024-03-20T08:07:27.861 INFO:tasks.rados.rados.0.smithi118.stdout:1740: writing smithi11881747-27 from 2670592 to 3375104 tid 3 2024-03-20T08:07:27.861 INFO:tasks.rados.rados.0.smithi118.stdout:1740: writing smithi11881747-27 from 3375104 to 3424256 tid 4 2024-03-20T08:07:27.861 INFO:tasks.rados.rados.0.smithi118.stdout:1741: read oid 9 snap -1 2024-03-20T08:07:27.862 INFO:tasks.rados.rados.0.smithi118.stdout:1741: expect (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:07:27.862 INFO:tasks.rados.rados.0.smithi118.stdout:1742: rmattr oid 50 current snap is 177 2024-03-20T08:07:27.862 INFO:tasks.rados.rados.0.smithi118.stdout:1742: done (4 left) 2024-03-20T08:07:27.862 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 177 2024-03-20T08:07:27.862 INFO:tasks.rados.rados.0.smithi118.stdout:1743: seq_num 539 ranges {4751360=630784,5382144=737280,6119424=466944} 2024-03-20T08:07:27.892 INFO:tasks.rados.rados.0.smithi118.stdout:1743: writing smithi11881747-33 from 4751360 to 5382144 tid 1 2024-03-20T08:07:27.897 INFO:tasks.rados.rados.0.smithi118.stdout:1743: writing smithi11881747-33 from 5382144 to 6119424 tid 2 2024-03-20T08:07:27.901 INFO:tasks.rados.rados.0.smithi118.stdout:1743: writing smithi11881747-33 from 6119424 to 6586368 tid 3 2024-03-20T08:07:27.901 INFO:tasks.rados.rados.0.smithi118.stdout:1744: rollback oid 17 current snap is 177 2024-03-20T08:07:27.901 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 169 2024-03-20T08:07:27.901 INFO:tasks.rados.rados.0.smithi118.stdout:1740: finishing write tid 1 to smithi11881747-27 2024-03-20T08:07:27.901 INFO:tasks.rados.rados.0.smithi118.stdout:1740: finishing write tid 2 to smithi11881747-27 2024-03-20T08:07:27.901 INFO:tasks.rados.rados.0.smithi118.stdout:1739: expect (ObjNum 464 snap 155 seq_num 464) 2024-03-20T08:07:28.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:27.846+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:28.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:27.846+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:28.095 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:27.847+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:28.096 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:27.838+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:28.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:28.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[93121]: osdmap e419: 8 total, 8 up, 8 in 2024-03-20T08:07:28.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:28.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:28.096 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:28.097 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:28.097 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[95692]: osdmap e419: 8 total, 8 up, 8 in 2024-03-20T08:07:28.097 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:28.097 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:28.097 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:28.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.835+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.860+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.881+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.917+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.934+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.950+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.965+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.981+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:27.999+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.014+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.030+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.047+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.064+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.080+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.099 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:27.853+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:28.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:28.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:27 smithi196 ceph-mon[87080]: osdmap e419: 8 total, 8 up, 8 in 2024-03-20T08:07:28.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:28.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:28.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:28.157 INFO:tasks.rados.rados.0.smithi118.stdout:1740: finishing write tid 3 to smithi11881747-27 2024-03-20T08:07:28.157 INFO:tasks.rados.rados.0.smithi118.stdout:1740: finishing write tid 4 to smithi11881747-27 2024-03-20T08:07:28.157 INFO:tasks.rados.rados.0.smithi118.stdout:1740: finishing write tid 5 to smithi11881747-27 2024-03-20T08:07:28.157 INFO:tasks.rados.rados.0.smithi118.stdout:1740: finishing write tid 6 to smithi11881747-27 2024-03-20T08:07:28.158 INFO:tasks.rados.rados.0.smithi118.stdout:1740: oid 27 updating version 0 to 7064 2024-03-20T08:07:28.158 INFO:tasks.rados.rados.0.smithi118.stdout:1740: oid 27 version 7064 is already newer than 7062 2024-03-20T08:07:28.158 INFO:tasks.rados.rados.0.smithi118.stdout:1740: oid 27 updating version 7064 to 7065 2024-03-20T08:07:28.158 INFO:tasks.rados.rados.0.smithi118.stdout:1740: oid 27 updating version 7065 to 7066 2024-03-20T08:07:28.158 INFO:tasks.rados.rados.0.smithi118.stdout:1740: oid 27 version 7066 is already newer than 7063 2024-03-20T08:07:28.158 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7066 (ObjNum 538 snap 177 seq_num 538) dirty exists 2024-03-20T08:07:28.158 INFO:tasks.rados.rados.0.smithi118.stdout:1740: left oid 27 (ObjNum 538 snap 177 seq_num 538) 2024-03-20T08:07:28.158 INFO:tasks.rados.rados.0.smithi118.stdout:1741: expect (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:07:28.355 INFO:tasks.rados.rados.0.smithi118.stdout:1738: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1744: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7036 (ObjNum 65 snap 0 seq_num 1582653295) dirty dne 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1738: finishing copy_from to smithi11881747-12 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 7056 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1743: finishing write tid 1 to smithi11881747-33 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1743: finishing write tid 2 to smithi11881747-33 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1743: finishing write tid 3 to smithi11881747-33 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1743: finishing write tid 4 to smithi11881747-33 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1743: finishing write tid 5 to smithi11881747-33 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1743: oid 33 updating version 0 to 13578 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1743: oid 33 version 13578 is already newer than 13577 2024-03-20T08:07:28.356 INFO:tasks.rados.rados.0.smithi118.stdout:1743: oid 33 updating version 13578 to 13579 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1743: oid 33 version 13579 is already newer than 13576 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 13579 (ObjNum 539 snap 177 seq_num 539) dirty exists 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1743: left oid 33 (ObjNum 539 snap 177 seq_num 539) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1738: done (5 left) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1739: done (4 left) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1740: done (3 left) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1741: done (2 left) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1743: done (1 left) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1744: done (0 left) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1745: read oid 38 snap -1 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1745: expect (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1746: read oid 32 snap -1 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1746: expect (ObjNum 491 snap 161 seq_num 491) 2024-03-20T08:07:28.357 INFO:tasks.rados.rados.0.smithi118.stdout:1747: setattr oid 22 current snap is 177 2024-03-20T08:07:28.358 INFO:tasks.rados.rados.0.smithi118.stdout:1748: rmattr oid 44 current snap is 177 2024-03-20T08:07:28.359 INFO:tasks.rados.rados.0.smithi118.stdout:1749: snap_create 2024-03-20T08:07:28.363 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 7058 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.095+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.111+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.127+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.143+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.160+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.177+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.192+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.210+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.365 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.227+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.366 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.245+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.366 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.263+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.366 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.279+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.366 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.366 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.366 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.330+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.366 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.346+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.366 INFO:tasks.rados.rados.0.smithi118.stdout:1746: expect (ObjNum 491 snap 161 seq_num 491) 2024-03-20T08:07:28.444 INFO:tasks.rados.rados.0.smithi118.stdout:1745: expect (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.365+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.385+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.402+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.418+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.434+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.450+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.467+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.482+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.499+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.514+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.530+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.546+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.563+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.578+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.593+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.608+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.626 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.625+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.881 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:07:28.836+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 5 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:28.881 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:28.881 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: osdmap e420: 8 total, 8 up, 8 in 2024-03-20T08:07:28.881 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:28.881 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:28.881 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: pgmap v307: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 486 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 24 op/s 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: osdmap e420: 8 total, 8 up, 8 in 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: pgmap v307: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 486 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 24 op/s 2024-03-20T08:07:28.882 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:28.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:28.883 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.641+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.656+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.674+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.690+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.707+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.723+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.739+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.754+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.883 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.769+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.785+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.801+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.818+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.833+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.851+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.867+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.884 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.881+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:28.932 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 7057 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-03-20T08:07:29.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: osdmap e420: 8 total, 8 up, 8 in 2024-03-20T08:07:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: pgmap v307: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 486 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 14 MiB/s rd, 6.3 MiB/s wr, 24 op/s 2024-03-20T08:07:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:29.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.897+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.913+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.946+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.961+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.976+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:28.992+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.006+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.038+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.052+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.068+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.082+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.097+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.112+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.142 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.126+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.142 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.141+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.334 INFO:tasks.rados.rados.0.smithi118.stdout:1745: done (4 left) 2024-03-20T08:07:29.453 INFO:tasks.rados.rados.0.smithi118.stdout:1746: done (3 left) 2024-03-20T08:07:29.454 INFO:tasks.rados.rados.0.smithi118.stdout:1747: done (2 left) 2024-03-20T08:07:29.454 INFO:tasks.rados.rados.0.smithi118.stdout:1748: done (1 left) 2024-03-20T08:07:29.454 INFO:tasks.rados.rados.0.smithi118.stdout:1749: done (0 left) 2024-03-20T08:07:29.454 INFO:tasks.rados.rados.0.smithi118.stdout:1750: snap_remove snap 176 2024-03-20T08:07:29.454 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:29.339+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:29.455 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:29.340+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:29.455 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:29.333+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:29.455 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:29.344+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:29.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.157+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.171+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.187+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.201+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.218+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.233+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.248+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.265+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.279+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.295+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.310+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.326+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.345+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.361+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.376+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.391+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.405+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.420+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.434+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.451+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.466+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.481+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.496+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.510+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.525+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.540+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.659 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.554+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.569+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.583+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.598+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.613+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.628+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.643+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.660 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.659+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 5 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[93121]: osdmap e421: 8 total, 8 up, 8 in 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 5 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:29.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[95692]: osdmap e421: 8 total, 8 up, 8 in 2024-03-20T08:07:29.916 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:29.916 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:29.916 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.675+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.691+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.705+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.720+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.735+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.750+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.764+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.778+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.916 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.793+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.810+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.825+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.842+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.857+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.873+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.887+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.902+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:29.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.915+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:29 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 5 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:07:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:29 smithi196 ceph-mon[87080]: osdmap e421: 8 total, 8 up, 8 in 2024-03-20T08:07:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:30.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:30.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.931+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.945+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.961+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.977+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:29.992+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.037+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.052+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.067+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.096+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.113+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.127+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.142+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.158+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.173+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.335 INFO:tasks.rados.rados.0.smithi118.stdout:1750: done (0 left) 2024-03-20T08:07:30.335 INFO:tasks.rados.rados.0.smithi118.stdout:1751: delete oid 49 current snap is 178 2024-03-20T08:07:30.342 INFO:tasks.rados.rados.0.smithi118.stdout:1751: done (0 left) 2024-03-20T08:07:30.342 INFO:tasks.rados.rados.0.smithi118.stdout:1752: delete oid 40 current snap is 178 2024-03-20T08:07:30.357 INFO:tasks.rados.rados.0.smithi118.stdout:1752: done (0 left) 2024-03-20T08:07:30.357 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 178 2024-03-20T08:07:30.357 INFO:tasks.rados.rados.0.smithi118.stdout:1753: seq_num 542 ranges {5570560=679936,6250496=442368,6692864=679936,7372800=81920} 2024-03-20T08:07:30.392 INFO:tasks.rados.rados.0.smithi118.stdout:1753: writing smithi11881747-15 from 5570560 to 6250496 tid 1 2024-03-20T08:07:30.395 INFO:tasks.rados.rados.0.smithi118.stdout:1753: writing smithi11881747-15 from 6250496 to 6692864 tid 2 2024-03-20T08:07:30.399 INFO:tasks.rados.rados.0.smithi118.stdout:1753: writing smithi11881747-15 from 6692864 to 7372800 tid 3 2024-03-20T08:07:30.400 INFO:tasks.rados.rados.0.smithi118.stdout:1753: writing smithi11881747-15 from 7372800 to 7454720 tid 4 2024-03-20T08:07:30.400 INFO:tasks.rados.rados.0.smithi118.stdout:1754: delete oid 41 current snap is 178 2024-03-20T08:07:30.400 INFO:tasks.rados.rados.0.smithi118.stdout:1753: finishing write tid 1 to smithi11881747-15 2024-03-20T08:07:30.400 INFO:tasks.rados.rados.0.smithi118.stdout:1753: finishing write tid 2 to smithi11881747-15 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1753: finishing write tid 3 to smithi11881747-15 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1754: done (1 left) 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1755: read oid 43 snap -1 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1755: expect (ObjNum 519 snap 169 seq_num 519) 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1756: read oid 50 snap -1 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1756: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1757: rollback oid 41 current snap is 178 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 41 to 169 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1753: finishing write tid 4 to smithi11881747-15 2024-03-20T08:07:30.405 INFO:tasks.rados.rados.0.smithi118.stdout:1758: rollback oid 29 current snap is 178 2024-03-20T08:07:30.406 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 177 2024-03-20T08:07:30.406 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 178 2024-03-20T08:07:30.406 INFO:tasks.rados.rados.0.smithi118.stdout:1759: seq_num 544 ranges {5857280=720896,6578176=614400,7192576=589824} 2024-03-20T08:07:30.434 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:30.339+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:30.434 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:30.339+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:30.434 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:30.342+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:30.435 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:30.345+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:30.435 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:30.334+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:30.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.188+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.203+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.435 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.218+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.234+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.249+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.263+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.278+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.294+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.309+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.324+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.347+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.363+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.377+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.436 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.393+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.415+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.437 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.434+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.442 INFO:tasks.rados.rados.0.smithi118.stdout:1759: writing smithi11881747-25 from 5857280 to 6578176 tid 1 2024-03-20T08:07:30.445 INFO:tasks.rados.rados.0.smithi118.stdout:1759: writing smithi11881747-25 from 6578176 to 7192576 tid 2 2024-03-20T08:07:30.449 INFO:tasks.rados.rados.0.smithi118.stdout:1759: writing smithi11881747-25 from 7192576 to 7782400 tid 3 2024-03-20T08:07:30.449 INFO:tasks.rados.rados.0.smithi118.stdout:1760: snap_create 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:1753: finishing write tid 5 to smithi11881747-15 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:1753: finishing write tid 6 to smithi11881747-15 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:1753: oid 15 updating version 0 to 1014 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:1753: oid 15 updating version 1014 to 1017 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:1753: oid 15 version 1017 is already newer than 1016 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:1753: oid 15 version 1017 is already newer than 1015 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:1753: oid 15 updating version 1017 to 1018 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 1018 (ObjNum 542 snap 178 seq_num 542) dirty exists 2024-03-20T08:07:30.450 INFO:tasks.rados.rados.0.smithi118.stdout:1753: left oid 15 (ObjNum 542 snap 178 seq_num 542) 2024-03-20T08:07:30.451 INFO:tasks.rados.rados.0.smithi118.stdout:1757: finishing rollback tid 1 to smithi11881747-41 2024-03-20T08:07:30.451 INFO:tasks.rados.rados.0.smithi118.stdout:1757: finishing rollback tid 2 to smithi11881747-41 2024-03-20T08:07:30.451 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 1323 (ObjNum 117 snap 30 seq_num 117) dirty exists 2024-03-20T08:07:30.451 INFO:tasks.rados.rados.0.smithi118.stdout:1758: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:07:30.451 INFO:tasks.rados.rados.0.smithi118.stdout:1755: expect (ObjNum 519 snap 169 seq_num 519) 2024-03-20T08:07:30.507 INFO:tasks.rados.rados.0.smithi118.stdout:1758: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:07:30.507 INFO:tasks.rados.rados.0.smithi118.stdout:1758: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:07:30.507 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1327 (ObjNum 423 snap 141 seq_num 423) dirty exists 2024-03-20T08:07:30.508 INFO:tasks.rados.rados.0.smithi118.stdout:1756: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:07:30.689 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.453+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.689 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.468+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.689 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.484+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.689 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.500+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.689 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.515+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.689 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.531+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.547+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.563+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.580+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.594+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.610+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.627+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.642+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.657+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.674+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.689+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.705+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.752+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.769+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.785+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.801+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.817+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.832+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.849+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.947 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.867+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.883+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.899+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.914+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.948 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.929+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:30.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[93121]: pgmap v309: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 464 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 4.5 MiB/s wr, 27 op/s 2024-03-20T08:07:30.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:30.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[93121]: osdmap e422: 8 total, 8 up, 8 in 2024-03-20T08:07:30.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:30.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:30.948 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:30.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[95692]: pgmap v309: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 464 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 4.5 MiB/s wr, 27 op/s 2024-03-20T08:07:30.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:30.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[95692]: osdmap e422: 8 total, 8 up, 8 in 2024-03-20T08:07:30.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:30.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:30.949 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:31.054 INFO:tasks.rados.rados.0.smithi118.stdout:1759: finishing write tid 1 to smithi11881747-25 2024-03-20T08:07:31.054 INFO:tasks.rados.rados.0.smithi118.stdout:1759: finishing write tid 2 to smithi11881747-25 2024-03-20T08:07:31.054 INFO:tasks.rados.rados.0.smithi118.stdout:1759: finishing write tid 3 to smithi11881747-25 2024-03-20T08:07:31.054 INFO:tasks.rados.rados.0.smithi118.stdout:1759: finishing write tid 4 to smithi11881747-25 2024-03-20T08:07:31.054 INFO:tasks.rados.rados.0.smithi118.stdout:1759: finishing write tid 5 to smithi11881747-25 2024-03-20T08:07:31.054 INFO:tasks.rados.rados.0.smithi118.stdout:1759: oid 25 updating version 0 to 1280 2024-03-20T08:07:31.055 INFO:tasks.rados.rados.0.smithi118.stdout:1759: oid 25 updating version 1280 to 1282 2024-03-20T08:07:31.055 INFO:tasks.rados.rados.0.smithi118.stdout:1759: oid 25 version 1282 is already newer than 1281 2024-03-20T08:07:31.055 INFO:tasks.rados.rados.0.smithi118.stdout:1759: oid 25 version 1282 is already newer than 1279 2024-03-20T08:07:31.055 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 1282 (ObjNum 544 snap 178 seq_num 544) dirty exists 2024-03-20T08:07:31.055 INFO:tasks.rados.rados.0.smithi118.stdout:1759: left oid 25 (ObjNum 544 snap 178 seq_num 544) 2024-03-20T08:07:31.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:30 smithi196 ceph-mon[87080]: pgmap v309: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 464 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 22 MiB/s rd, 4.5 MiB/s wr, 27 op/s 2024-03-20T08:07:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset e3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:30 smithi196 ceph-mon[87080]: osdmap e422: 8 total, 8 up, 8 in 2024-03-20T08:07:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:31.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.947+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.963+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.978+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:30.992+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.007+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.023+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.039+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.053+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.068+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.082+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.097+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.112+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.126+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.141+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.155+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.171+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.186+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.200+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.337 INFO:tasks.rados.rados.0.smithi118.stdout:1753: done (6 left) 2024-03-20T08:07:31.337 INFO:tasks.rados.rados.0.smithi118.stdout:1755: done (5 left) 2024-03-20T08:07:31.337 INFO:tasks.rados.rados.0.smithi118.stdout:1756: done (4 left) 2024-03-20T08:07:31.337 INFO:tasks.rados.rados.0.smithi118.stdout:1757: done (3 left) 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:1758: done (2 left) 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:1759: done (1 left) 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:1760: done (0 left) 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:1761: rollback oid 13 current snap is 179 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 13 to 161 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:1762: read oid 46 snap -1 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:1762: expect (ObjNum 535 snap 175 seq_num 535) 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 179 2024-03-20T08:07:31.338 INFO:tasks.rados.rados.0.smithi118.stdout:1763: seq_num 545 ranges {5169152=737280,5906432=491520,6397952=450560,6848512=262144} 2024-03-20T08:07:31.371 INFO:tasks.rados.rados.0.smithi118.stdout:1763: writing smithi11881747-21 from 5169152 to 5906432 tid 1 2024-03-20T08:07:31.373 INFO:tasks.rados.rados.0.smithi118.stdout:1763: writing smithi11881747-21 from 5906432 to 6397952 tid 2 2024-03-20T08:07:31.376 INFO:tasks.rados.rados.0.smithi118.stdout:1763: writing smithi11881747-21 from 6397952 to 6848512 tid 3 2024-03-20T08:07:31.378 INFO:tasks.rados.rados.0.smithi118.stdout:1763: writing smithi11881747-21 from 6848512 to 7110656 tid 4 2024-03-20T08:07:31.378 INFO:tasks.rados.rados.0.smithi118.stdout:1764: copy_from oid 42 from oid 44 current snap is 179 2024-03-20T08:07:31.379 INFO:tasks.rados.rados.0.smithi118.stdout:1761: finishing rollback tid 0 to smithi11881747-13 2024-03-20T08:07:31.379 INFO:tasks.rados.rados.0.smithi118.stdout:1761: finishing rollback tid 1 to smithi11881747-13 2024-03-20T08:07:31.379 INFO:tasks.rados.rados.0.smithi118.stdout:1761: finishing rollback tid 2 to smithi11881747-13 2024-03-20T08:07:31.379 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 1285 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:07:31.379 INFO:tasks.rados.rados.0.smithi118.stdout:1762: expect (ObjNum 535 snap 175 seq_num 535) 2024-03-20T08:07:31.461 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:31.341+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:31.461 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:31.346+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:31.462 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:31.343+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:31.462 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:31.337+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:31.462 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.216+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.462 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.232+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.247+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.262+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.277+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.291+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.306+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.321+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.338+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.362+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.379+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.399+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.415+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.430+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.464 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.446+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.464 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.462+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.721 INFO:tasks.rados.rados.0.smithi118.stdout:1763: finishing write tid 1 to smithi11881747-21 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: finishing write tid 2 to smithi11881747-21 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: finishing write tid 3 to smithi11881747-21 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: finishing write tid 4 to smithi11881747-21 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: finishing write tid 5 to smithi11881747-21 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1761: done (3 left) 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1762: done (2 left) 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1765: copy_from oid 8 from oid 19 current snap is 179 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: finishing write tid 6 to smithi11881747-21 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: oid 21 updating version 0 to 7059 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: oid 21 updating version 7059 to 7061 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: oid 21 version 7061 is already newer than 7058 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: oid 21 updating version 7061 to 7062 2024-03-20T08:07:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:1763: oid 21 version 7062 is already newer than 7060 2024-03-20T08:07:31.723 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 7062 (ObjNum 545 snap 179 seq_num 545) dirty exists 2024-03-20T08:07:31.723 INFO:tasks.rados.rados.0.smithi118.stdout:1763: left oid 21 (ObjNum 545 snap 179 seq_num 545) 2024-03-20T08:07:31.723 INFO:tasks.rados.rados.0.smithi118.stdout:1764: finishing copy_from racing read to smithi11881747-42 2024-03-20T08:07:31.723 INFO:tasks.rados.rados.0.smithi118.stdout:1764: finishing copy_from to smithi11881747-42 2024-03-20T08:07:31.723 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 7058 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:07:31.723 INFO:tasks.rados.rados.0.smithi118.stdout:1763: done (2 left) 2024-03-20T08:07:31.723 INFO:tasks.rados.rados.0.smithi118.stdout:1764: done (1 left) 2024-03-20T08:07:31.723 INFO:tasks.rados.rados.0.smithi118.stdout:1766: snap_create 2024-03-20T08:07:31.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.477+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.493+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.509+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.525+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.539+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.555+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.586+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.602+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.618+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.635+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.650+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.666+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.680+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.696+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.711+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.730+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.734 INFO:tasks.rados.rados.0.smithi118.stdout:1765: finishing copy_from racing read to smithi11881747-8 2024-03-20T08:07:31.734 INFO:tasks.rados.rados.0.smithi118.stdout:1765: finishing copy_from to smithi11881747-8 2024-03-20T08:07:31.735 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 1013 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:07:31.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.751+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.766+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.780+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.795+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.810+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.985 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.825+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.840+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.855+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.870+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.884+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.898+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.912+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.926+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.941+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.956+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.970+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:31.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:31.985+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.241 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.000+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.241 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.015+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.030+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.059+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.075+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.092+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.106+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.136+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.152+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.168+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.183+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.197+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.212+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.242 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.227+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.243 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.241+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.345 INFO:tasks.rados.rados.0.smithi118.stdout:1765: done (1 left) 2024-03-20T08:07:32.346 INFO:tasks.rados.rados.0.smithi118.stdout:1766: done (0 left) 2024-03-20T08:07:32.346 INFO:tasks.rados.rados.0.smithi118.stdout:1767: delete oid 20 current snap is 180 2024-03-20T08:07:32.350 INFO:tasks.rados.rados.0.smithi118.stdout:1767: done (0 left) 2024-03-20T08:07:32.350 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 180 2024-03-20T08:07:32.350 INFO:tasks.rados.rados.0.smithi118.stdout:1768: seq_num 547 ranges {2293760=770048,3063808=663552,3727360=466944,4194304=73728} 2024-03-20T08:07:32.367 INFO:tasks.rados.rados.0.smithi118.stdout:1768: writing smithi11881747-29 from 2293760 to 3063808 tid 1 2024-03-20T08:07:32.372 INFO:tasks.rados.rados.0.smithi118.stdout:1768: writing smithi11881747-29 from 3063808 to 3727360 tid 2 2024-03-20T08:07:32.375 INFO:tasks.rados.rados.0.smithi118.stdout:1768: writing smithi11881747-29 from 3727360 to 4194304 tid 3 2024-03-20T08:07:32.375 INFO:tasks.rados.rados.0.smithi118.stdout:1768: writing smithi11881747-29 from 4194304 to 4268032 tid 4 2024-03-20T08:07:32.376 INFO:tasks.rados.rados.0.smithi118.stdout:1769: rmattr oid 24 current snap is 180 2024-03-20T08:07:32.376 INFO:tasks.rados.rados.0.smithi118.stdout:1769: done (1 left) 2024-03-20T08:07:32.376 INFO:tasks.rados.rados.0.smithi118.stdout:1770: snap_remove snap 173 2024-03-20T08:07:32.491 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:32.347+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:32.491 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:32.347+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:32.491 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:32.347+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[93121]: osdmap e423: 8 total, 8 up, 8 in 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[93121]: pgmap v312: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 467 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 17 MiB/s rd, 1.5 MiB/s wr, 15 op/s 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[95692]: osdmap e423: 8 total, 8 up, 8 in 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:32.492 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:32.493 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-mon[95692]: pgmap v312: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 467 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 17 MiB/s rd, 1.5 MiB/s wr, 15 op/s 2024-03-20T08:07:32.493 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:32.340+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:32.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.256+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.270+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.286+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.301+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.316+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.493 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.331+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.352+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.369+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.387+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.402+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.416+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.431+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.446+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.461+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.476+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.494 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:32.353+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:32 smithi196 ceph-mon[87080]: osdmap e423: 8 total, 8 up, 8 in 2024-03-20T08:07:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:32 smithi196 ceph-mon[87080]: pgmap v312: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 467 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 17 MiB/s rd, 1.5 MiB/s wr, 15 op/s 2024-03-20T08:07:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.491+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.508+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.523+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.538+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.553+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.567+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.583+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.597+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.612+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.627+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.746 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.644+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.746 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.659+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.746 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.674+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.746 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.690+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.746 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.704+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.746 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.717+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.746 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.732+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.746 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.745+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.759+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.774+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.790+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.808+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.823+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.838+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.854+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.869+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.883+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.898+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.912+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:32.999 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.926+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.000 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.940+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.000 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.955+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.000 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.970+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.000 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.984+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.000 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:32.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.014+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.030+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.044+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.058+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.073+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.088+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.119+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.134+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.150+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.165+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.210+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.224+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.240+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.256 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1770: done (1 left) 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1771: delete oid 8 current snap is 180 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: finishing write tid 1 to smithi11881747-29 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: finishing write tid 2 to smithi11881747-29 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: finishing write tid 3 to smithi11881747-29 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: finishing write tid 4 to smithi11881747-29 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: finishing write tid 5 to smithi11881747-29 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: finishing write tid 6 to smithi11881747-29 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: oid 29 updating version 0 to 1330 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: oid 29 updating version 1330 to 1333 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: oid 29 version 1333 is already newer than 1331 2024-03-20T08:07:33.347 INFO:tasks.rados.rados.0.smithi118.stdout:1768: oid 29 version 1333 is already newer than 1329 2024-03-20T08:07:33.348 INFO:tasks.rados.rados.0.smithi118.stdout:1768: oid 29 version 1333 is already newer than 1332 2024-03-20T08:07:33.348 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1333 (ObjNum 547 snap 180 seq_num 547) dirty exists 2024-03-20T08:07:33.348 INFO:tasks.rados.rados.0.smithi118.stdout:1768: left oid 29 (ObjNum 547 snap 180 seq_num 547) 2024-03-20T08:07:33.362 INFO:tasks.rados.rados.0.smithi118.stdout:1768: done (1 left) 2024-03-20T08:07:33.363 INFO:tasks.rados.rados.0.smithi118.stdout:1771: done (0 left) 2024-03-20T08:07:33.363 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 180 2024-03-20T08:07:33.363 INFO:tasks.rados.rados.0.smithi118.stdout:1772: seq_num 549 ranges {6045696=802816,6848512=483328,7331840=434176,7766016=286720} 2024-03-20T08:07:33.402 INFO:tasks.rados.rados.0.smithi118.stdout:1772: writing smithi11881747-50 from 6045696 to 6848512 tid 1 2024-03-20T08:07:33.405 INFO:tasks.rados.rados.0.smithi118.stdout:1772: writing smithi11881747-50 from 6848512 to 7331840 tid 2 2024-03-20T08:07:33.408 INFO:tasks.rados.rados.0.smithi118.stdout:1772: writing smithi11881747-50 from 7331840 to 7766016 tid 3 2024-03-20T08:07:33.410 INFO:tasks.rados.rados.0.smithi118.stdout:1772: writing smithi11881747-50 from 7766016 to 8052736 tid 4 2024-03-20T08:07:33.410 INFO:tasks.rados.rados.0.smithi118.stdout:1773: snap_remove snap 145 2024-03-20T08:07:33.506 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:33.350+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:33.506 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:33.353+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:33.506 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:33.506 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[93121]: osdmap e424: 8 total, 8 up, 8 in 2024-03-20T08:07:33.506 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:33.506 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:33.507 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:33.507 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:33.507 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[95692]: osdmap e424: 8 total, 8 up, 8 in 2024-03-20T08:07:33.507 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:33.507 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:33.507 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:33.507 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:33.345+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:33.508 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:33.361+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:33.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.269+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.285+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.300+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.314+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.329+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.508 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.345+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.367+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.381+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.397+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.414+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.430+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.445+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.460+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.475+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.492+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.509 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.506+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:33.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:33 smithi196 ceph-mon[87080]: osdmap e424: 8 total, 8 up, 8 in 2024-03-20T08:07:33.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:33.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:33.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:33.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:33.504Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:33.759 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:07:33] "GET /metrics HTTP/1.1" 200 36262 "" "Prometheus/2.43.0" 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.521+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.538+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.551+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.566+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.580+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.596+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.611+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.626+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.641+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.655+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.671+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.760 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.685+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.761 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.700+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.761 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.714+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.761 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.728+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.761 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.744+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:33.761 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.759+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.775+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.791+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.807+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.821+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.835+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.850+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.881+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.895+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.911+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.925+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.015 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.940+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.954+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.969+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.985+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:33.999+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.016 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.015+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.029+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.042+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.059+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.073+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.088+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.103+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.117+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.132+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.146+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.163+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.280 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.178+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.192+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.206+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.221+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.236+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.251+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.281 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.264+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: finishing write tid 1 to smithi11881747-50 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: finishing write tid 2 to smithi11881747-50 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: finishing write tid 3 to smithi11881747-50 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: finishing write tid 4 to smithi11881747-50 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: finishing write tid 5 to smithi11881747-50 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: finishing write tid 6 to smithi11881747-50 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: oid 50 updating version 0 to 7060 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: oid 50 updating version 7060 to 7061 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: oid 50 updating version 7061 to 7063 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: oid 50 version 7063 is already newer than 7059 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:1772: oid 50 version 7063 is already newer than 7062 2024-03-20T08:07:34.353 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 7063 (ObjNum 549 snap 180 seq_num 549) dirty exists 2024-03-20T08:07:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:1772: left oid 50 (ObjNum 549 snap 180 seq_num 549) 2024-03-20T08:07:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:1772: done (1 left) 2024-03-20T08:07:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:1773: done (0 left) 2024-03-20T08:07:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:1774: read oid 32 snap -1 2024-03-20T08:07:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:1774: expect (ObjNum 491 snap 161 seq_num 491) 2024-03-20T08:07:34.354 INFO:tasks.rados.rados.0.smithi118.stdout:1775: delete oid 42 current snap is 180 2024-03-20T08:07:34.364 INFO:tasks.rados.rados.0.smithi118.stdout:1775: done (1 left) 2024-03-20T08:07:34.364 INFO:tasks.rados.rados.0.smithi118.stdout:1776: read oid 37 snap 175 2024-03-20T08:07:34.364 INFO:tasks.rados.rados.0.smithi118.stdout:1776: expect (ObjNum 508 snap 168 seq_num 508) 2024-03-20T08:07:34.364 INFO:tasks.rados.rados.0.smithi118.stdout:1777: read oid 33 snap 164 2024-03-20T08:07:34.364 INFO:tasks.rados.rados.0.smithi118.stdout:1777: expect deleted 2024-03-20T08:07:34.364 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 40 current snap is 180 2024-03-20T08:07:34.365 INFO:tasks.rados.rados.0.smithi118.stdout:1778: seq_num 551 ranges {0=434176,434176=647168,1081344=450560,1531904=507904} 2024-03-20T08:07:34.367 INFO:tasks.rados.rados.0.smithi118.stdout:1778: writing smithi11881747-40 from 0 to 434176 tid 1 2024-03-20T08:07:34.371 INFO:tasks.rados.rados.0.smithi118.stdout:1778: writing smithi11881747-40 from 434176 to 1081344 tid 2 2024-03-20T08:07:34.375 INFO:tasks.rados.rados.0.smithi118.stdout:1778: writing smithi11881747-40 from 1081344 to 1531904 tid 3 2024-03-20T08:07:34.378 INFO:tasks.rados.rados.0.smithi118.stdout:1778: writing smithi11881747-40 from 1531904 to 2039808 tid 4 2024-03-20T08:07:34.378 INFO:tasks.rados.rados.0.smithi118.stdout:1779: rollback oid 42 current snap is 180 2024-03-20T08:07:34.378 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 177 2024-03-20T08:07:34.379 INFO:tasks.rados.rados.0.smithi118.stdout:1774: expect (ObjNum 491 snap 161 seq_num 491) 2024-03-20T08:07:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:1774: done (4 left) 2024-03-20T08:07:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:1777: done (3 left) 2024-03-20T08:07:34.461 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 180 2024-03-20T08:07:34.461 INFO:tasks.rados.rados.0.smithi118.stdout:1779: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:07:34.461 INFO:tasks.rados.rados.0.smithi118.stdout:1779: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:07:34.461 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 7061 (ObjNum 326 snap 106 seq_num 326) dirty exists 2024-03-20T08:07:34.461 INFO:tasks.rados.rados.0.smithi118.stdout:1778: finishing write tid 1 to smithi11881747-40 2024-03-20T08:07:34.461 INFO:tasks.rados.rados.0.smithi118.stdout:1776: expect (ObjNum 508 snap 168 seq_num 508) 2024-03-20T08:07:34.509 INFO:tasks.rados.rados.0.smithi118.stdout:1778: finishing write tid 2 to smithi11881747-40 2024-03-20T08:07:34.509 INFO:tasks.rados.rados.0.smithi118.stdout:1778: finishing write tid 3 to smithi11881747-40 2024-03-20T08:07:34.509 INFO:tasks.rados.rados.0.smithi118.stdout:1778: finishing write tid 4 to smithi11881747-40 2024-03-20T08:07:34.509 INFO:tasks.rados.rados.0.smithi118.stdout:1780: seq_num 552 ranges {0=450560,450560=532480,983040=655360,1638400=417792} 2024-03-20T08:07:34.511 INFO:tasks.rados.rados.0.smithi118.stdout:1780: writing smithi11881747-17 from 0 to 450560 tid 1 2024-03-20T08:07:34.515 INFO:tasks.rados.rados.0.smithi118.stdout:1780: writing smithi11881747-17 from 450560 to 983040 tid 2 2024-03-20T08:07:34.519 INFO:tasks.rados.rados.0.smithi118.stdout:1780: writing smithi11881747-17 from 983040 to 1638400 tid 3 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1780: writing smithi11881747-17 from 1638400 to 2056192 tid 4 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1776: done (3 left) 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1779: done (2 left) 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1781: setattr oid 15 current snap is 180 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1778: finishing write tid 5 to smithi11881747-40 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1778: finishing write tid 6 to smithi11881747-40 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1778: oid 40 updating version 0 to 7063 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1778: oid 40 updating version 7063 to 7064 2024-03-20T08:07:34.522 INFO:tasks.rados.rados.0.smithi118.stdout:1778: oid 40 version 7064 is already newer than 7060 2024-03-20T08:07:34.523 INFO:tasks.rados.rados.0.smithi118.stdout:1778: oid 40 version 7064 is already newer than 7061 2024-03-20T08:07:34.523 INFO:tasks.rados.rados.0.smithi118.stdout:1778: oid 40 version 7064 is already newer than 7062 2024-03-20T08:07:34.523 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 7064 (ObjNum 551 snap 180 seq_num 551) dirty exists 2024-03-20T08:07:34.523 INFO:tasks.rados.rados.0.smithi118.stdout:1778: left oid 40 (ObjNum 551 snap 180 seq_num 551) 2024-03-20T08:07:34.523 INFO:tasks.rados.rados.0.smithi118.stdout:1780: finishing write tid 1 to smithi11881747-17 2024-03-20T08:07:34.523 INFO:tasks.rados.rados.0.smithi118.stdout:1780: finishing write tid 2 to smithi11881747-17 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1780: finishing write tid 3 to smithi11881747-17 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1778: done (2 left) 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1782: copy_from oid 18 from oid 19 current snap is 180 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1783: read oid 9 snap -1 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1783: expect (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1784: rmattr oid 7 current snap is 180 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1784: done (4 left) 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1785: read oid 41 snap -1 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1785: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1786: snap_create 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1780: finishing write tid 4 to smithi11881747-17 2024-03-20T08:07:34.525 INFO:tasks.rados.rados.0.smithi118.stdout:1780: finishing write tid 5 to smithi11881747-17 2024-03-20T08:07:34.527 INFO:tasks.rados.rados.0.smithi118.stdout:1780: finishing write tid 6 to smithi11881747-17 2024-03-20T08:07:34.527 INFO:tasks.rados.rados.0.smithi118.stdout:1780: oid 17 updating version 0 to 7040 2024-03-20T08:07:34.527 INFO:tasks.rados.rados.0.smithi118.stdout:1780: oid 17 updating version 7040 to 7041 2024-03-20T08:07:34.527 INFO:tasks.rados.rados.0.smithi118.stdout:1780: oid 17 version 7041 is already newer than 7038 2024-03-20T08:07:34.527 INFO:tasks.rados.rados.0.smithi118.stdout:1780: oid 17 version 7041 is already newer than 7039 2024-03-20T08:07:34.527 INFO:tasks.rados.rados.0.smithi118.stdout:1780: oid 17 version 7041 is already newer than 7037 2024-03-20T08:07:34.528 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7041 (ObjNum 552 snap 180 seq_num 552) dirty exists 2024-03-20T08:07:34.528 INFO:tasks.rados.rados.0.smithi118.stdout:1780: left oid 17 (ObjNum 552 snap 180 seq_num 552) 2024-03-20T08:07:34.529 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 1019 (ObjNum 542 snap 178 seq_num 542) dirty exists 2024-03-20T08:07:34.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:34.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[93121]: osdmap e425: 8 total, 8 up, 8 in 2024-03-20T08:07:34.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:34.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:34.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:34.530 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[93121]: pgmap v315: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 486 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 17 op/s 2024-03-20T08:07:34.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:34.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[95692]: osdmap e425: 8 total, 8 up, 8 in 2024-03-20T08:07:34.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:34.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:34.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:34.531 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-mon[95692]: pgmap v315: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 486 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 17 op/s 2024-03-20T08:07:34.531 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:34.359+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:34.531 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:34.359+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:34.532 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:34.361+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:34.532 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:34.364+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:34.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.279+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.295+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.310+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.324+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.532 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.339+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.361+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.385+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.401+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.417+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.434+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.450+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.466+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.481+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.496+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.512+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.530+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:34.356+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:34.533 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:34.362+0000 7f59e9e37700 -1 osd.3 pg_epoch: 426 pg[5.1s1( v 418'399 (0'0,418'399] local-lis/les=408/409 n=6 ec=73/73 lis/c=408/408 les/c/f=409/409/0 sis=408) [4,3,7]p4(0) r=1 lpr=408 luod=0'0 crt=418'399 mlcod 418'399 active mbc={} ps=74] update_snap_map remove_oid 5:8a455596:::smithi11881747-30:ce failed with -2 2024-03-20T08:07:34.538 INFO:tasks.rados.rados.0.smithi118.stdout:1782: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:07:34.540 INFO:tasks.rados.rados.0.smithi118.stdout:1783: expect (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:07:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:34 smithi196 ceph-mon[87080]: osdmap e425: 8 total, 8 up, 8 in 2024-03-20T08:07:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:34 smithi196 ceph-mon[87080]: pgmap v315: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 486 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 17 op/s 2024-03-20T08:07:34.728 INFO:tasks.rados.rados.0.smithi118.stdout:1782: finishing copy_from to smithi11881747-18 2024-03-20T08:07:34.728 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 1013 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:07:34.729 INFO:tasks.rados.rados.0.smithi118.stdout:1785: expect (ObjNum 117 snap 30 seq_num 117) 2024-03-20T08:07:34.789 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.551+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.789 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.567+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.789 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.584+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.789 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.601+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.789 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.618+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.634+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.650+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.665+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.680+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.695+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.711+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.727+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.743+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.758+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.774+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:34.790 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.789+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.805+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.821+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.837+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.854+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.870+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.885+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.900+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.914+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.045 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.927+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.046 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.046 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.046 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.972+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.046 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:34.987+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.046 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.001+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.046 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.015+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.046 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.030+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.046 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.061+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.075+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.091+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.107+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.121+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.136+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.151+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.166+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.194+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.210+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.225+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.241+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.269+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.283+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.358 INFO:tasks.rados.rados.0.smithi118.stdout:1780: done (5 left) 2024-03-20T08:07:35.358 INFO:tasks.rados.rados.0.smithi118.stdout:1781: done (4 left) 2024-03-20T08:07:35.358 INFO:tasks.rados.rados.0.smithi118.stdout:1782: done (3 left) 2024-03-20T08:07:35.358 INFO:tasks.rados.rados.0.smithi118.stdout:1783: done (2 left) 2024-03-20T08:07:35.358 INFO:tasks.rados.rados.0.smithi118.stdout:1785: done (1 left) 2024-03-20T08:07:35.358 INFO:tasks.rados.rados.0.smithi118.stdout:1786: done (0 left) 2024-03-20T08:07:35.358 INFO:tasks.rados.rados.0.smithi118.stdout:1787: setattr oid 36 current snap is 181 2024-03-20T08:07:35.359 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 181 2024-03-20T08:07:35.360 INFO:tasks.rados.rados.0.smithi118.stdout:1788: seq_num 553 ranges {4194304=466944,4661248=417792,5079040=417792,5496832=516096,6012928=253952} 2024-03-20T08:07:35.386 INFO:tasks.rados.rados.0.smithi118.stdout:1788: writing smithi11881747-2 from 4194304 to 4661248 tid 1 2024-03-20T08:07:35.389 INFO:tasks.rados.rados.0.smithi118.stdout:1788: writing smithi11881747-2 from 4661248 to 5079040 tid 2 2024-03-20T08:07:35.392 INFO:tasks.rados.rados.0.smithi118.stdout:1788: writing smithi11881747-2 from 5079040 to 5496832 tid 3 2024-03-20T08:07:35.395 INFO:tasks.rados.rados.0.smithi118.stdout:1788: writing smithi11881747-2 from 5496832 to 6012928 tid 4 2024-03-20T08:07:35.397 INFO:tasks.rados.rados.0.smithi118.stdout:1788: writing smithi11881747-2 from 6012928 to 6266880 tid 5 2024-03-20T08:07:35.397 INFO:tasks.rados.rados.0.smithi118.stdout:1789: read oid 25 snap -1 2024-03-20T08:07:35.397 INFO:tasks.rados.rados.0.smithi118.stdout:1789: expect (ObjNum 544 snap 178 seq_num 544) 2024-03-20T08:07:35.397 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 181 2024-03-20T08:07:35.397 INFO:tasks.rados.rados.0.smithi118.stdout:1790: seq_num 554 ranges {0=483328,483328=704512,1187840=622592,1810432=278528} 2024-03-20T08:07:35.400 INFO:tasks.rados.rados.0.smithi118.stdout:1790: writing smithi11881747-22 from 0 to 483328 tid 1 2024-03-20T08:07:35.405 INFO:tasks.rados.rados.0.smithi118.stdout:1790: writing smithi11881747-22 from 483328 to 1187840 tid 2 2024-03-20T08:07:35.410 INFO:tasks.rados.rados.0.smithi118.stdout:1790: writing smithi11881747-22 from 1187840 to 1810432 tid 3 2024-03-20T08:07:35.412 INFO:tasks.rados.rados.0.smithi118.stdout:1790: writing smithi11881747-22 from 1810432 to 2088960 tid 4 2024-03-20T08:07:35.412 INFO:tasks.rados.rados.0.smithi118.stdout:1791: delete oid 6 current snap is 181 2024-03-20T08:07:35.412 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 7047 (ObjNum 49 snap 0 seq_num 1953066355) dirty exists 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: finishing write tid 1 to smithi11881747-2 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: finishing write tid 2 to smithi11881747-2 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: finishing write tid 3 to smithi11881747-2 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: finishing write tid 4 to smithi11881747-2 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: finishing write tid 5 to smithi11881747-2 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: finishing write tid 6 to smithi11881747-2 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: finishing write tid 7 to smithi11881747-2 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: oid 2 updating version 0 to 1289 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: oid 2 version 1289 is already newer than 1287 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: oid 2 version 1289 is already newer than 1286 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: oid 2 version 1289 is already newer than 1284 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: oid 2 version 1289 is already newer than 1288 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:1788: oid 2 version 1289 is already newer than 1285 2024-03-20T08:07:35.413 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 1289 (ObjNum 553 snap 181 seq_num 553) dirty exists 2024-03-20T08:07:35.414 INFO:tasks.rados.rados.0.smithi118.stdout:1788: left oid 2 (ObjNum 553 snap 181 seq_num 553) 2024-03-20T08:07:35.414 INFO:tasks.rados.rados.0.smithi118.stdout:1790: finishing write tid 1 to smithi11881747-22 2024-03-20T08:07:35.414 INFO:tasks.rados.rados.0.smithi118.stdout:1790: finishing write tid 2 to smithi11881747-22 2024-03-20T08:07:35.416 INFO:tasks.rados.rados.0.smithi118.stdout:1787: done (4 left) 2024-03-20T08:07:35.416 INFO:tasks.rados.rados.0.smithi118.stdout:1788: done (3 left) 2024-03-20T08:07:35.416 INFO:tasks.rados.rados.0.smithi118.stdout:1791: done (2 left) 2024-03-20T08:07:35.416 INFO:tasks.rados.rados.0.smithi118.stdout:1792: snap_remove snap 177 2024-03-20T08:07:35.556 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[93121]: osdmap e426: 8 total, 8 up, 8 in 2024-03-20T08:07:35.556 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:35.557 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:35.557 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:35.557 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:35.557 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[95692]: osdmap e426: 8 total, 8 up, 8 in 2024-03-20T08:07:35.557 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:35.557 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:35.557 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:35.557 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:35.558 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:35.367+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:35.558 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:35.367+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:35.558 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:35.371+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:35.558 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:35.358+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.298+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.328+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.342+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.359+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.381+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.398+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.422+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.438+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.452+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.467+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.481+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.559 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.495+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.511+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.525+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.541+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.557+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:35 smithi196 ceph-mon[87080]: osdmap e426: 8 total, 8 up, 8 in 2024-03-20T08:07:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.587+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.601+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.619+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.634+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.649+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.663+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.678+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.693+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.709+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.723+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.737+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.752+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.766+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.781+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.796+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:35.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.813+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.841+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.857+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.872+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.887+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.902+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.917+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.933+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.947+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.963+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.978+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:35.995+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.010+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.025+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.039+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.069+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.330 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.083+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.330 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.099+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.114+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.133+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.149+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.163+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.178+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.193+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.223+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.239+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.254+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.269+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.284+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.300+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.331 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.315+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.332 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.329+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.367 INFO:tasks.rados.rados.0.smithi118.stdout:1790: finishing write tid 3 to smithi11881747-22 2024-03-20T08:07:36.367 INFO:tasks.rados.rados.0.smithi118.stdout:1790: finishing write tid 4 to smithi11881747-22 2024-03-20T08:07:36.367 INFO:tasks.rados.rados.0.smithi118.stdout:1790: finishing write tid 5 to smithi11881747-22 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:1790: finishing write tid 6 to smithi11881747-22 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:1790: oid 22 updating version 0 to 7063 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:1790: oid 22 updating version 7063 to 7065 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:1790: oid 22 updating version 7065 to 7066 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:1790: oid 22 version 7066 is already newer than 7064 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:1790: oid 22 updating version 7066 to 7067 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 7067 (ObjNum 554 snap 181 seq_num 554) dirty exists 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:1790: left oid 22 (ObjNum 554 snap 181 seq_num 554) 2024-03-20T08:07:36.368 INFO:tasks.rados.rados.0.smithi118.stdout:1789: expect (ObjNum 544 snap 178 seq_num 544) 2024-03-20T08:07:36.591 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:36.366+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:36.591 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:36.366+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:36.591 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:36.367+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:36.591 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[93121]: osdmap e427: 8 total, 8 up, 8 in 2024-03-20T08:07:36.591 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:36.592 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:36.592 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:36.592 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:36.592 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[93121]: pgmap v318: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 495 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 23 op/s 2024-03-20T08:07:36.592 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.345+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.592 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.361+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.592 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.387+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.592 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.402+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.592 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.417+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.592 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.432+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.593 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.448+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.593 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.464+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.593 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.479+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.593 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.494+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.593 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.510+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.593 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.574+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.593 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:36.371+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:36.593 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[95692]: osdmap e427: 8 total, 8 up, 8 in 2024-03-20T08:07:36.594 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:36.594 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:36.594 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:36.594 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:36.594 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-mon[95692]: pgmap v318: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 495 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 23 op/s 2024-03-20T08:07:36.594 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:36.363+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:36.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:36 smithi196 ceph-mon[87080]: osdmap e427: 8 total, 8 up, 8 in 2024-03-20T08:07:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:36 smithi196 ceph-mon[87080]: pgmap v318: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 495 MiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 23 op/s 2024-03-20T08:07:36.856 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.590+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.856 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.606+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.856 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.621+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.856 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.637+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.652+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.667+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.684+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.701+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.715+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.731+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.747+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.761+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.775+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.793+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.807+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.823+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.838+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.857 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.855+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:36.998 INFO:tasks.rados.rados.0.smithi118.stdout:1789: done (2 left) 2024-03-20T08:07:36.998 INFO:tasks.rados.rados.0.smithi118.stdout:1790: done (1 left) 2024-03-20T08:07:36.998 INFO:tasks.rados.rados.0.smithi118.stdout:1792: done (0 left) 2024-03-20T08:07:36.998 INFO:tasks.rados.rados.0.smithi118.stdout:1793: delete oid 24 current snap is 181 2024-03-20T08:07:37.000 INFO:tasks.rados.rados.0.smithi118.stdout:1793: done (0 left) 2024-03-20T08:07:37.000 INFO:tasks.rados.rados.0.smithi118.stdout:1794: delete oid 23 current snap is 181 2024-03-20T08:07:37.003 INFO:tasks.rados.rados.0.smithi118.stdout:1794: done (0 left) 2024-03-20T08:07:37.003 INFO:tasks.rados.rados.0.smithi118.stdout:1795: copy_from oid 6 from oid 30 current snap is 181 2024-03-20T08:07:37.003 INFO:tasks.rados.rados.0.smithi118.stdout:1796: snap_remove snap 178 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.872+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.889+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.907+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.923+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.938+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.955+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.971+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:36.986+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.001+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.019+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.035+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.050+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.117 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.066+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.117 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.117 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.097+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.117 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.114+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.371 INFO:tasks.rados.rados.0.smithi118.stdout:1795: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:07:37.371 INFO:tasks.rados.rados.0.smithi118.stdout:1795: finishing copy_from to smithi11881747-6 2024-03-20T08:07:37.371 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 793 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:07:37.371 INFO:tasks.rados.rados.0.smithi118.stdout:1795: done (1 left) 2024-03-20T08:07:37.371 INFO:tasks.rados.rados.0.smithi118.stdout:1796: done (0 left) 2024-03-20T08:07:37.372 INFO:tasks.rados.rados.0.smithi118.stdout:1797: snap_create 2024-03-20T08:07:37.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[95692]: osdmap e428: 8 total, 8 up, 8 in 2024-03-20T08:07:37.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:37.374 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:37.374 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:37.374 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:37.374 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:37.374 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:37.374 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[95692]: Upgrade: osd.4 is safe to restart 2024-03-20T08:07:37.374 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:37.371+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.132+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.148+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.166+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.182+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.199+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.215+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.246+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.263+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.277+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.375 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.293+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.309+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.327+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.344+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.360+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.376 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[93121]: osdmap e428: 8 total, 8 up, 8 in 2024-03-20T08:07:37.376 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:37.376 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:37.376 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:37.376 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:37.376 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:37.376 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:37.377 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-mon[93121]: Upgrade: osd.4 is safe to restart 2024-03-20T08:07:37.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:37 smithi196 ceph-mon[87080]: osdmap e428: 8 total, 8 up, 8 in 2024-03-20T08:07:37.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:37.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:37.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:37.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:37.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:37 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:37.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:37 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-20T08:07:37.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:37 smithi196 ceph-mon[87080]: Upgrade: osd.4 is safe to restart 2024-03-20T08:07:37.638 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:37.376+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:37.638 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:37.376+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:37.638 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:37.374+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:37.638 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.388+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.404+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.420+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.436+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.451+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.469+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.486+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.504+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.520+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.536+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.551+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.568+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.586+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.604+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.620+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.639 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.638+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.655+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.670+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.685+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.702+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.719+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.751+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.767+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.782+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.796+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.811+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.828+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.844+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.860+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:37.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.876+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.892+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.908+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.924+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.940+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.975+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:37.992+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.024+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.039+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.055+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.071+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.088+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.103+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.119+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.152 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.150+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.376 INFO:tasks.rados.rados.0.smithi118.stdout:1797: done (0 left) 2024-03-20T08:07:38.376 INFO:tasks.rados.rados.0.smithi118.stdout:1798: delete oid 46 current snap is 182 2024-03-20T08:07:38.382 INFO:tasks.rados.rados.0.smithi118.stdout:1798: done (0 left) 2024-03-20T08:07:38.382 INFO:tasks.rados.rados.0.smithi118.stdout:1799: rollback oid 6 current snap is 182 2024-03-20T08:07:38.382 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 161 2024-03-20T08:07:38.382 INFO:tasks.rados.rados.0.smithi118.stdout:1800: read oid 17 snap -1 2024-03-20T08:07:38.383 INFO:tasks.rados.rados.0.smithi118.stdout:1800: expect (ObjNum 552 snap 180 seq_num 552) 2024-03-20T08:07:38.383 INFO:tasks.rados.rados.0.smithi118.stdout:1801: snap_create 2024-03-20T08:07:38.385 INFO:tasks.rados.rados.0.smithi118.stdout:1799: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:07:38.386 INFO:tasks.rados.rados.0.smithi118.stdout:1799: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:07:38.386 INFO:tasks.rados.rados.0.smithi118.stdout:1799: finishing rollback tid 2 to smithi11881747-6 2024-03-20T08:07:38.386 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 1294 (ObjNum 485 snap 159 seq_num 485) dirty exists 2024-03-20T08:07:38.394 INFO:tasks.rados.rados.0.smithi118.stdout:1800: expect (ObjNum 552 snap 180 seq_num 552) 2024-03-20T08:07:38.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:38.385+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:38.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:38.373+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:38.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:38.381+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:38.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:38.381+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:38.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:38.381+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:38.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[95692]: osdmap e429: 8 total, 8 up, 8 in 2024-03-20T08:07:38.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:38.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:38.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:38.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:38.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[95692]: pgmap v321: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 497 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2024-03-20T08:07:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[93121]: osdmap e429: 8 total, 8 up, 8 in 2024-03-20T08:07:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-mon[93121]: pgmap v321: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 497 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2024-03-20T08:07:38.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.167+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.182+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.197+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.211+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.227+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.243+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.258+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.275+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.309+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.326+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.342+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.358+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.378+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.400+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.421+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:38 smithi196 ceph-mon[87080]: osdmap e429: 8 total, 8 up, 8 in 2024-03-20T08:07:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:38 smithi196 ceph-mon[87080]: pgmap v321: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 497 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 12 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2024-03-20T08:07:38.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.440+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.457+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.474+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.492+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.509+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.526+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.542+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.559+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.577+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.594+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.611+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.627+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.646+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.666+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.682+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.698+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.716+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.733+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.749+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.765+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.779+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.795+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.811+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.826+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.843+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.858+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.874+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.890+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.936 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.905+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.936 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.919+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:38.936 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.933+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.949+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.964+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.184 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.980+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:38.996+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.012+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.026+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.041+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.056+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.071+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.090+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.107+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.122+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.138+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.151+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.167+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.185 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.183+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.380 INFO:tasks.rados.rados.0.smithi118.stdout:1799: done (2 left) 2024-03-20T08:07:39.380 INFO:tasks.rados.rados.0.smithi118.stdout:1800: done (1 left) 2024-03-20T08:07:39.380 INFO:tasks.rados.rados.0.smithi118.stdout:1801: done (0 left) 2024-03-20T08:07:39.380 INFO:tasks.rados.rados.0.smithi118.stdout:1802: rollback oid 5 current snap is 183 2024-03-20T08:07:39.380 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 89 2024-03-20T08:07:39.381 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 183 2024-03-20T08:07:39.381 INFO:tasks.rados.rados.0.smithi118.stdout:1803: seq_num 559 ranges {0=565248,565248=573440,1138688=409600,1548288=622592} 2024-03-20T08:07:39.384 INFO:tasks.rados.rados.0.smithi118.stdout:1803: writing smithi11881747-46 from 0 to 565248 tid 1 2024-03-20T08:07:39.388 INFO:tasks.rados.rados.0.smithi118.stdout:1803: writing smithi11881747-46 from 565248 to 1138688 tid 2 2024-03-20T08:07:39.391 INFO:tasks.rados.rados.0.smithi118.stdout:1803: writing smithi11881747-46 from 1138688 to 1548288 tid 3 2024-03-20T08:07:39.395 INFO:tasks.rados.rados.0.smithi118.stdout:1803: writing smithi11881747-46 from 1548288 to 2170880 tid 4 2024-03-20T08:07:39.395 INFO:tasks.rados.rados.0.smithi118.stdout:1804: snap_create 2024-03-20T08:07:39.396 INFO:tasks.rados.rados.0.smithi118.stdout:1802: finishing rollback tid 0 to smithi11881747-5 2024-03-20T08:07:39.396 INFO:tasks.rados.rados.0.smithi118.stdout:1802: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:07:39.397 INFO:tasks.rados.rados.0.smithi118.stdout:1802: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:07:39.398 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 7064 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-03-20T08:07:39.400 INFO:tasks.rados.rados.0.smithi118.stdout:1803: finishing write tid 1 to smithi11881747-46 2024-03-20T08:07:39.400 INFO:tasks.rados.rados.0.smithi118.stdout:1803: finishing write tid 2 to smithi11881747-46 2024-03-20T08:07:39.400 INFO:tasks.rados.rados.0.smithi118.stdout:1803: finishing write tid 3 to smithi11881747-46 2024-03-20T08:07:39.400 INFO:tasks.rados.rados.0.smithi118.stdout:1803: finishing write tid 4 to smithi11881747-46 2024-03-20T08:07:39.400 INFO:tasks.rados.rados.0.smithi118.stdout:1803: finishing write tid 5 to smithi11881747-46 2024-03-20T08:07:39.402 INFO:tasks.rados.rados.0.smithi118.stdout:1803: finishing write tid 6 to smithi11881747-46 2024-03-20T08:07:39.402 INFO:tasks.rados.rados.0.smithi118.stdout:1803: oid 46 updating version 0 to 1297 2024-03-20T08:07:39.402 INFO:tasks.rados.rados.0.smithi118.stdout:1803: oid 46 updating version 1297 to 1298 2024-03-20T08:07:39.402 INFO:tasks.rados.rados.0.smithi118.stdout:1803: oid 46 updating version 1298 to 1299 2024-03-20T08:07:39.402 INFO:tasks.rados.rados.0.smithi118.stdout:1803: oid 46 version 1299 is already newer than 1296 2024-03-20T08:07:39.402 INFO:tasks.rados.rados.0.smithi118.stdout:1803: oid 46 version 1299 is already newer than 1295 2024-03-20T08:07:39.402 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 1299 (ObjNum 559 snap 183 seq_num 559) dirty exists 2024-03-20T08:07:39.402 INFO:tasks.rados.rados.0.smithi118.stdout:1803: left oid 46 (ObjNum 559 snap 183 seq_num 559) 2024-03-20T08:07:39.445 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:39.384+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:39.445 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:39.390+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:39.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:39.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: osdmap e430: 8 total, 8 up, 8 in 2024-03-20T08:07:39.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:39.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:39.445 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:39.446 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: Upgrade: Updating osd.4 2024-03-20T08:07:39.446 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:39.446 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T08:07:39.446 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:07:39.446 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[93121]: Deploying daemon osd.4 on smithi196 2024-03-20T08:07:39.446 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:39.379+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:39.446 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:39.396+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: osdmap e430: 8 total, 8 up, 8 in 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: Upgrade: Updating osd.4 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T08:07:39.447 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:07:39.448 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-mon[95692]: Deploying daemon osd.4 on smithi196 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.198+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.212+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.227+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.244+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.258+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.273+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.288+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.302+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.316+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.448 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.332+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.346+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.361+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.378+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.400+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.415+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.431+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.449 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.444+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: osdmap e430: 8 total, 8 up, 8 in 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: Upgrade: Updating osd.4 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:07:39.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:39 smithi196 ceph-mon[87080]: Deploying daemon osd.4 on smithi196 2024-03-20T08:07:39.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.460+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.475+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.490+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.505+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.700 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.519+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.535+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.549+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.565+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.581+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.596+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.610+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.625+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.640+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.654+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.669+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.683+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.701 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.699+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.956 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.956 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.729+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.956 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.744+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.956 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.758+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.956 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.772+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.787+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.802+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.817+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.832+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.848+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.863+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.877+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.892+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.908+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.923+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.941+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:39.957 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.955+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.970+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.985+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:39.999+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.015+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.030+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.045+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.210 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.059+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.090+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.119+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.150+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.165+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.194+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.211 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.209+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.386 INFO:tasks.rados.rados.0.smithi118.stdout:1802: done (2 left) 2024-03-20T08:07:40.387 INFO:tasks.rados.rados.0.smithi118.stdout:1803: done (1 left) 2024-03-20T08:07:40.387 INFO:tasks.rados.rados.0.smithi118.stdout:1804: done (0 left) 2024-03-20T08:07:40.387 INFO:tasks.rados.rados.0.smithi118.stdout:1805: snap_remove snap 179 2024-03-20T08:07:40.460 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:40.395+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:40.460 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:40.395+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:40.460 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:40.395+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:40.460 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:40.460 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[93121]: osdmap e431: 8 total, 8 up, 8 in 2024-03-20T08:07:40.460 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[93121]: pgmap v324: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 502 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[95692]: osdmap e431: 8 total, 8 up, 8 in 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:40.461 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-mon[95692]: pgmap v324: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 502 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-03-20T08:07:40.462 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:40.397+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:40.462 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:40.385+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:40.462 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.223+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.462 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.238+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.462 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.254+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.462 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.268+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.282+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.312+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.329+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.343+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.360+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.374+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.395+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.414+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.429+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.445+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.463 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.459+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:40 smithi196 ceph-mon[87080]: osdmap e431: 8 total, 8 up, 8 in 2024-03-20T08:07:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:40 smithi196 ceph-mon[87080]: pgmap v324: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 502 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 6.1 MiB/s rd, 2.9 MiB/s wr, 18 op/s 2024-03-20T08:07:40.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.475+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.491+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.506+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.522+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.537+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.551+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.567+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.722 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.583+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.597+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.613+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.628+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.644+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.660+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.677+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.691+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.706+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.752+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.768+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.782+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.799+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.814+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.830+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.846+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.859+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.874+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.889+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.905+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.921+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.937+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.953+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.968+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:40.984 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.982+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.236 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:40.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.014+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.030+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.061+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.078+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.094+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.111+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.126+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.143+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.159+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.175+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.191+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.206+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.237 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.220+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.238 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.235+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.390 INFO:tasks.rados.rados.0.smithi118.stdout:1805: done (0 left) 2024-03-20T08:07:41.390 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 184 2024-03-20T08:07:41.390 INFO:tasks.rados.rados.0.smithi118.stdout:1806: seq_num 560 ranges {0=581632,581632=458752,1040384=614400,1654784=532480} 2024-03-20T08:07:41.393 INFO:tasks.rados.rados.0.smithi118.stdout:1806: writing smithi11881747-49 from 0 to 581632 tid 1 2024-03-20T08:07:41.397 INFO:tasks.rados.rados.0.smithi118.stdout:1806: writing smithi11881747-49 from 581632 to 1040384 tid 2 2024-03-20T08:07:41.401 INFO:tasks.rados.rados.0.smithi118.stdout:1806: writing smithi11881747-49 from 1040384 to 1654784 tid 3 2024-03-20T08:07:41.405 INFO:tasks.rados.rados.0.smithi118.stdout:1806: writing smithi11881747-49 from 1654784 to 2187264 tid 4 2024-03-20T08:07:41.405 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 184 2024-03-20T08:07:41.405 INFO:tasks.rados.rados.0.smithi118.stdout:1807: seq_num 561 ranges {7782400=598016,8380416=745472,9125888=778240,9904128=81920} 2024-03-20T08:07:41.452 INFO:tasks.rados.rados.0.smithi118.stdout:1807: writing smithi11881747-25 from 7782400 to 8380416 tid 1 2024-03-20T08:07:41.457 INFO:tasks.rados.rados.0.smithi118.stdout:1807: writing smithi11881747-25 from 8380416 to 9125888 tid 2 2024-03-20T08:07:41.462 INFO:tasks.rados.rados.0.smithi118.stdout:1807: writing smithi11881747-25 from 9125888 to 9904128 tid 3 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1807: writing smithi11881747-25 from 9904128 to 9986048 tid 4 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1808: copy_from oid 30 from oid 13 current snap is 184 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: finishing write tid 1 to smithi11881747-49 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: finishing write tid 2 to smithi11881747-49 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: finishing write tid 3 to smithi11881747-49 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: finishing write tid 4 to smithi11881747-49 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: finishing write tid 5 to smithi11881747-49 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: finishing write tid 6 to smithi11881747-49 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: oid 49 updating version 0 to 7070 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: oid 49 updating version 7070 to 7073 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: oid 49 version 7073 is already newer than 7071 2024-03-20T08:07:41.464 INFO:tasks.rados.rados.0.smithi118.stdout:1806: oid 49 version 7073 is already newer than 7069 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:1806: oid 49 version 7073 is already newer than 7072 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 7073 (ObjNum 560 snap 184 seq_num 560) dirty exists 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:1806: left oid 49 (ObjNum 560 snap 184 seq_num 560) 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:1807: finishing write tid 1 to smithi11881747-25 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:1807: finishing write tid 2 to smithi11881747-25 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:1806: done (2 left) 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:1809: read oid 36 snap -1 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:1809: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 184 2024-03-20T08:07:41.465 INFO:tasks.rados.rados.0.smithi118.stdout:1810: seq_num 562 ranges {1318912=622592,1941504=630784,2572288=581632,3153920=393216} 2024-03-20T08:07:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:1810: writing smithi11881747-37 from 1318912 to 1941504 tid 1 2024-03-20T08:07:41.479 INFO:tasks.rados.rados.0.smithi118.stdout:1810: writing smithi11881747-37 from 1941504 to 2572288 tid 2 2024-03-20T08:07:41.483 INFO:tasks.rados.rados.0.smithi118.stdout:1810: writing smithi11881747-37 from 2572288 to 3153920 tid 3 2024-03-20T08:07:41.486 INFO:tasks.rados.rados.0.smithi118.stdout:1810: writing smithi11881747-37 from 3153920 to 3547136 tid 4 2024-03-20T08:07:41.486 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 184 2024-03-20T08:07:41.487 INFO:tasks.rados.rados.0.smithi118.stdout:1811: seq_num 563 ranges {0=638976,638976=565248,1204224=794624,1998848=245760} 2024-03-20T08:07:41.491 INFO:tasks.rados.rados.0.smithi118.stdout:1811: writing smithi11881747-39 from 0 to 638976 tid 1 2024-03-20T08:07:41.495 INFO:tasks.rados.rados.0.smithi118.stdout:1811: writing smithi11881747-39 from 638976 to 1204224 tid 2 2024-03-20T08:07:41.500 INFO:tasks.rados.rados.0.smithi118.stdout:1811: writing smithi11881747-39 from 1204224 to 1998848 tid 3 2024-03-20T08:07:41.502 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:41.395+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:41.502 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:41.396+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:1811: writing smithi11881747-39 from 1998848 to 2244608 tid 4 2024-03-20T08:07:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 184 2024-03-20T08:07:41.503 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:41.400+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:1812: seq_num 564 ranges {3424256=655360,4079616=442368,4521984=598016,5120000=565248} 2024-03-20T08:07:41.503 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:41.504 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-mon[93121]: osdmap e432: 8 total, 8 up, 8 in 2024-03-20T08:07:41.504 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:41.504 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:41.504 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:41.402+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:41.504 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:41.504 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-mon[95692]: osdmap e432: 8 total, 8 up, 8 in 2024-03-20T08:07:41.504 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:41.505 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.251+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.266+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.281+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.311+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.325+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.340+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.355+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.370+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.386+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.417+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.505 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.432+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.447+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.466+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.484+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.506 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.502+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.526 INFO:tasks.rados.rados.0.smithi118.stdout:1812: writing smithi11881747-27 from 3424256 to 4079616 tid 1 2024-03-20T08:07:41.528 INFO:tasks.rados.rados.0.smithi118.stdout:1812: writing smithi11881747-27 from 4079616 to 4521984 tid 2 2024-03-20T08:07:41.532 INFO:tasks.rados.rados.0.smithi118.stdout:1812: writing smithi11881747-27 from 4521984 to 5120000 tid 3 2024-03-20T08:07:41.536 INFO:tasks.rados.rados.0.smithi118.stdout:1812: writing smithi11881747-27 from 5120000 to 5685248 tid 4 2024-03-20T08:07:41.536 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 184 2024-03-20T08:07:41.536 INFO:tasks.rados.rados.0.smithi118.stdout:1809: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:07:41.541 INFO:tasks.rados.rados.0.smithi118.stdout:1807: finishing write tid 3 to smithi11881747-25 2024-03-20T08:07:41.541 INFO:tasks.rados.rados.0.smithi118.stdout:1807: finishing write tid 4 to smithi11881747-25 2024-03-20T08:07:41.541 INFO:tasks.rados.rados.0.smithi118.stdout:1807: finishing write tid 5 to smithi11881747-25 2024-03-20T08:07:41.541 INFO:tasks.rados.rados.0.smithi118.stdout:1807: finishing write tid 6 to smithi11881747-25 2024-03-20T08:07:41.541 INFO:tasks.rados.rados.0.smithi118.stdout:1807: oid 25 updating version 0 to 7067 2024-03-20T08:07:41.541 INFO:tasks.rados.rados.0.smithi118.stdout:1807: oid 25 updating version 7067 to 7068 2024-03-20T08:07:41.541 INFO:tasks.rados.rados.0.smithi118.stdout:1807: oid 25 updating version 7068 to 7069 2024-03-20T08:07:41.541 INFO:tasks.rados.rados.0.smithi118.stdout:1807: oid 25 version 7069 is already newer than 7065 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1807: oid 25 version 7069 is already newer than 7066 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 7069 (ObjNum 561 snap 184 seq_num 561) dirty exists 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1807: left oid 25 (ObjNum 561 snap 184 seq_num 561) 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1808: finishing copy_from racing read to smithi11881747-30 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1808: finishing copy_from to smithi11881747-30 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 1285 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1811: finishing write tid 1 to smithi11881747-39 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1811: finishing write tid 2 to smithi11881747-39 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1810: finishing write tid 1 to smithi11881747-37 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1811: finishing write tid 3 to smithi11881747-39 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1811: finishing write tid 4 to smithi11881747-39 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1811: finishing write tid 5 to smithi11881747-39 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1811: finishing write tid 6 to smithi11881747-39 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1811: oid 39 updating version 0 to 7074 2024-03-20T08:07:41.542 INFO:tasks.rados.rados.0.smithi118.stdout:1811: oid 39 updating version 7074 to 7078 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1811: oid 39 version 7078 is already newer than 7075 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1811: oid 39 version 7078 is already newer than 7077 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1811: oid 39 version 7078 is already newer than 7076 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7078 (ObjNum 563 snap 184 seq_num 563) dirty exists 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1811: left oid 39 (ObjNum 563 snap 184 seq_num 563) 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: finishing write tid 2 to smithi11881747-37 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: finishing write tid 3 to smithi11881747-37 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: finishing write tid 4 to smithi11881747-37 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: finishing write tid 5 to smithi11881747-37 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: finishing write tid 6 to smithi11881747-37 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: oid 37 updating version 0 to 9307 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: oid 37 updating version 9307 to 9309 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: oid 37 updating version 9309 to 9310 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: oid 37 version 9310 is already newer than 9308 2024-03-20T08:07:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:1810: oid 37 version 9310 is already newer than 9306 2024-03-20T08:07:41.544 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 9310 (ObjNum 562 snap 184 seq_num 562) dirty exists 2024-03-20T08:07:41.544 INFO:tasks.rados.rados.0.smithi118.stdout:1810: left oid 37 (ObjNum 562 snap 184 seq_num 562) 2024-03-20T08:07:41.544 INFO:tasks.rados.rados.0.smithi118.stdout:1812: finishing write tid 1 to smithi11881747-27 2024-03-20T08:07:41.544 INFO:tasks.rados.rados.0.smithi118.stdout:1812: finishing write tid 2 to smithi11881747-27 2024-03-20T08:07:41.544 INFO:tasks.rados.rados.0.smithi118.stdout:1812: finishing write tid 3 to smithi11881747-27 2024-03-20T08:07:41.544 INFO:tasks.rados.rados.0.smithi118.stdout:1812: finishing write tid 4 to smithi11881747-27 2024-03-20T08:07:41.544 INFO:tasks.rados.rados.0.smithi118.stdout:1813: seq_num 565 ranges {2400256=671744,3072000=729088,3801088=753664,4554752=122880} 2024-03-20T08:07:41.558 INFO:tasks.rados.rados.0.smithi118.stdout:1813: writing smithi11881747-4 from 2400256 to 3072000 tid 1 2024-03-20T08:07:41.562 INFO:tasks.rados.rados.0.smithi118.stdout:1813: writing smithi11881747-4 from 3072000 to 3801088 tid 2 2024-03-20T08:07:41.568 INFO:tasks.rados.rados.0.smithi118.stdout:1813: writing smithi11881747-4 from 3801088 to 4554752 tid 3 2024-03-20T08:07:41.569 INFO:tasks.rados.rados.0.smithi118.stdout:1813: writing smithi11881747-4 from 4554752 to 4677632 tid 4 2024-03-20T08:07:41.569 INFO:tasks.rados.rados.0.smithi118.stdout:1807: done (6 left) 2024-03-20T08:07:41.569 INFO:tasks.rados.rados.0.smithi118.stdout:1808: done (5 left) 2024-03-20T08:07:41.569 INFO:tasks.rados.rados.0.smithi118.stdout:1809: done (4 left) 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1810: done (3 left) 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1811: done (2 left) 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1814: delete oid 31 current snap is 184 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1812: finishing write tid 5 to smithi11881747-27 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1812: finishing write tid 6 to smithi11881747-27 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1812: oid 27 updating version 0 to 7080 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1812: oid 27 updating version 7080 to 7081 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1812: oid 27 version 7081 is already newer than 7079 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1812: oid 27 updating version 7081 to 7082 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:1812: oid 27 updating version 7082 to 7083 2024-03-20T08:07:41.570 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7083 (ObjNum 564 snap 184 seq_num 564) dirty exists 2024-03-20T08:07:41.571 INFO:tasks.rados.rados.0.smithi118.stdout:1812: left oid 27 (ObjNum 564 snap 184 seq_num 564) 2024-03-20T08:07:41.571 INFO:tasks.rados.rados.0.smithi118.stdout:1813: finishing write tid 1 to smithi11881747-4 2024-03-20T08:07:41.571 INFO:tasks.rados.rados.0.smithi118.stdout:1813: finishing write tid 2 to smithi11881747-4 2024-03-20T08:07:41.572 INFO:tasks.rados.rados.0.smithi118.stdout:1812: done (2 left) 2024-03-20T08:07:41.572 INFO:tasks.rados.rados.0.smithi118.stdout:1814: done (1 left) 2024-03-20T08:07:41.572 INFO:tasks.rados.rados.0.smithi118.stdout:1815: rollback oid 49 current snap is 184 2024-03-20T08:07:41.572 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 49 to 183 2024-03-20T08:07:41.572 INFO:tasks.rados.rados.0.smithi118.stdout:1816: snap_remove snap 161 2024-03-20T08:07:41.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset ed=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:41.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:41 smithi196 ceph-mon[87080]: osdmap e432: 8 total, 8 up, 8 in 2024-03-20T08:07:41.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:41.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.519+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.536+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.552+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.567+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.585+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.600+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.616+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.632+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.766 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.646+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.661+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.676+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.692+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.705+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.735+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.750+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:41.767 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.765+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.023 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.782+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.798+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.815+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.831+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.847+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.862+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.878+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.895+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.911+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.928+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.945+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.024 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.961+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.976+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:41.992+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.008+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.256 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:07:42.282 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.024+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.039+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.070+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.087+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.102+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.138+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.154+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.169+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.184+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.200+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.217+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.283 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.234+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.284 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.249+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.284 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.266+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.284 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.283+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.405 INFO:tasks.rados.rados.0.smithi118.stdout:1816: done (2 left) 2024-03-20T08:07:42.405 INFO:tasks.rados.rados.0.smithi118.stdout:1817: rmattr oid 25 current snap is 184 2024-03-20T08:07:42.405 INFO:tasks.rados.rados.0.smithi118.stdout:1813: finishing write tid 3 to smithi11881747-4 2024-03-20T08:07:42.405 INFO:tasks.rados.rados.0.smithi118.stdout:1813: finishing write tid 4 to smithi11881747-4 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: finishing write tid 5 to smithi11881747-4 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: finishing write tid 6 to smithi11881747-4 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: oid 4 updating version 0 to 1288 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: oid 4 updating version 1288 to 1289 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: oid 4 updating version 1289 to 1290 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: oid 4 version 1290 is already newer than 1287 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: oid 4 version 1290 is already newer than 1286 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 1290 (ObjNum 565 snap 184 seq_num 565) dirty exists 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: left oid 4 (ObjNum 565 snap 184 seq_num 565) 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1815: finishing rollback tid 0 to smithi11881747-49 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1815: finishing rollback tid 1 to smithi11881747-49 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 7085 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1813: done (2 left) 2024-03-20T08:07:42.406 INFO:tasks.rados.rados.0.smithi118.stdout:1815: done (1 left) 2024-03-20T08:07:42.407 INFO:tasks.rados.rados.0.smithi118.stdout:1818: snap_create 2024-03-20T08:07:42.413 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 7070 (ObjNum 561 snap 184 seq_num 561) dirty exists 2024-03-20T08:07:42.543 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:42.543 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[93121]: osdmap e433: 8 total, 8 up, 8 in 2024-03-20T08:07:42.543 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:42.543 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[93121]: pgmap v327: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 502 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 13 op/s 2024-03-20T08:07:42.543 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[93121]: from='client.54202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:42.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:42.406+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:42.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:42.406+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:42.544 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:42.406+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:42.544 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:42.544 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[95692]: osdmap e433: 8 total, 8 up, 8 in 2024-03-20T08:07:42.544 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:42.544 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[95692]: pgmap v327: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 502 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 13 op/s 2024-03-20T08:07:42.544 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-mon[95692]: from='client.54202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:42.545 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:42.414+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:42.545 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:42.402+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:42.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.301+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.317+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.332+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.364+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.379+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.399+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.427+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.444+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.459+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.478+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.494+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.512+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.529+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.546 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.543+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.575 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:42 smithi196 systemd[1]: Stopping Ceph osd.4 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:07:42.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:42.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-mon[87080]: osdmap e433: 8 total, 8 up, 8 in 2024-03-20T08:07:42.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:42.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-mon[87080]: pgmap v327: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 112 active+clean; 502 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 13 op/s 2024-03-20T08:07:42.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-mon[87080]: from='client.54202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:42.576 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:42.720 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:07:42.720 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (7m) 26s ago 8m 27.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:07:42.720 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (8m) 3m ago 8m 86.9M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:07:42.720 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (9m) 26s ago 9m 15.4M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:07:42.720 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (9m) 3m ago 9m 19.3M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:07:42.720 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (4m) 3m ago 12m 449M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (4m) 26s ago 14m 508M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (3m) 26s ago 14m 56.2M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (3m) 3m ago 13m 26.6M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (3m) 26s ago 13m 51.1M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (7m) 26s ago 7m 15.5M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (7m) 3m ago 7m 15.8M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (2m) 26s ago 12m 334M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (105s) 26s ago 12m 299M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (64s) 26s ago 11m 251M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (32s) 26s ago 11m 12.7M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (10m) 3m ago 10m 181M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7b658cecc8fb 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (10m) 3m ago 10m 133M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 7f7229013587 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (10m) 3m ago 10m 117M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (9m) 3m ago 9m 115M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:07:42.721 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (4m) 3m ago 8m 56.2M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:07:42.800 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.559+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.575+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.589+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.606+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.622+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.636+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.666+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.684+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.702+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.719+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.751+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.767+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.783+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.800+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:42.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[36567]: 2024-03-20T08:07:42.576+0000 7fc6a4bdc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:07:42.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[36567]: 2024-03-20T08:07:42.576+0000 7fc6a4bdc700 -1 osd.4 434 *** Got signal Terminated *** 2024-03-20T08:07:42.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[36567]: 2024-03-20T08:07:42.576+0000 7fc6a4bdc700 -1 osd.4 434 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:07:43.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.818+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.834+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.850+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.867+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.882+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.897+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.913+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.051 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.945+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.961+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.976+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:42.991+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.021+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.036+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.051+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.287 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:07:43.287 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:07:43.287 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:07:43.287 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:07:43.287 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:07:43.287 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 4, 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 4 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 6, 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 9 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:07:43.288 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:07:43.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.065+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.311 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.095+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.110+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.125+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.141+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.158+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.173+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.187+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.202+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.218+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.232+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.248+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.263+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.278+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.312 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.293+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.313 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.311+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.405 INFO:tasks.rados.rados.0.smithi118.stdout:1817: done (1 left) 2024-03-20T08:07:43.405 INFO:tasks.rados.rados.0.smithi118.stdout:1818: done (0 left) 2024-03-20T08:07:43.405 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 185 2024-03-20T08:07:43.405 INFO:tasks.rados.rados.0.smithi118.stdout:1819: seq_num 567 ranges {1826816=704512,2531328=499712,3031040=770048,3801088=335872} 2024-03-20T08:07:43.420 INFO:tasks.rados.rados.0.smithi118.stdout:1819: writing smithi11881747-32 from 1826816 to 2531328 tid 1 2024-03-20T08:07:43.423 INFO:tasks.rados.rados.0.smithi118.stdout:1819: writing smithi11881747-32 from 2531328 to 3031040 tid 2 2024-03-20T08:07:43.429 INFO:tasks.rados.rados.0.smithi118.stdout:1819: writing smithi11881747-32 from 3031040 to 3801088 tid 3 2024-03-20T08:07:43.432 INFO:tasks.rados.rados.0.smithi118.stdout:1819: writing smithi11881747-32 from 3801088 to 4136960 tid 4 2024-03-20T08:07:43.432 INFO:tasks.rados.rados.0.smithi118.stdout:1820: rollback oid 4 current snap is 185 2024-03-20T08:07:43.432 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 4 to 169 2024-03-20T08:07:43.432 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 185 2024-03-20T08:07:43.432 INFO:tasks.rados.rados.0.smithi118.stdout:1821: seq_num 568 ranges {2056192=720896,2777088=786432,3563520=581632,4145152=237568} 2024-03-20T08:07:43.447 INFO:tasks.rados.rados.0.smithi118.stdout:1821: writing smithi11881747-17 from 2056192 to 2777088 tid 1 2024-03-20T08:07:43.452 INFO:tasks.rados.rados.0.smithi118.stdout:1821: writing smithi11881747-17 from 2777088 to 3563520 tid 2 2024-03-20T08:07:43.456 INFO:tasks.rados.rados.0.smithi118.stdout:1821: writing smithi11881747-17 from 3563520 to 4145152 tid 3 2024-03-20T08:07:43.458 INFO:tasks.rados.rados.0.smithi118.stdout:1821: writing smithi11881747-17 from 4145152 to 4382720 tid 4 2024-03-20T08:07:43.458 INFO:tasks.rados.rados.0.smithi118.stdout:1822: rollback oid 34 current snap is 185 2024-03-20T08:07:43.458 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 34 to 158 2024-03-20T08:07:43.458 INFO:tasks.rados.rados.0.smithi118.stdout:1821: finishing write tid 1 to smithi11881747-17 2024-03-20T08:07:43.458 INFO:tasks.rados.rados.0.smithi118.stdout:1821: finishing write tid 2 to smithi11881747-17 2024-03-20T08:07:43.458 INFO:tasks.rados.rados.0.smithi118.stdout:1823: setattr oid 14 current snap is 185 2024-03-20T08:07:43.459 INFO:tasks.rados.rados.0.smithi118.stdout:1821: finishing write tid 3 to smithi11881747-17 2024-03-20T08:07:43.460 INFO:tasks.rados.rados.0.smithi118.stdout:1821: finishing write tid 4 to smithi11881747-17 2024-03-20T08:07:43.460 INFO:tasks.rados.rados.0.smithi118.stdout:1821: finishing write tid 5 to smithi11881747-17 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1821: finishing write tid 6 to smithi11881747-17 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1821: oid 17 updating version 0 to 7043 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1821: oid 17 version 7043 is already newer than 7042 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1821: oid 17 updating version 7043 to 7045 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1821: oid 17 updating version 7045 to 7046 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1821: oid 17 version 7046 is already newer than 7044 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7046 (ObjNum 568 snap 185 seq_num 568) dirty exists 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1821: left oid 17 (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1821: done (4 left) 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 185 2024-03-20T08:07:43.462 INFO:tasks.rados.rados.0.smithi118.stdout:1824: seq_num 569 ranges {0=737280,737280=671744,1409024=737280,2146304=196608} 2024-03-20T08:07:43.466 INFO:tasks.rados.rados.0.smithi118.stdout:1824: writing smithi11881747-23 from 0 to 737280 tid 1 2024-03-20T08:07:43.470 INFO:tasks.rados.rados.0.smithi118.stdout:1824: writing smithi11881747-23 from 737280 to 1409024 tid 2 2024-03-20T08:07:43.474 INFO:tasks.rados.rados.0.smithi118.stdout:1824: writing smithi11881747-23 from 1409024 to 2146304 tid 3 2024-03-20T08:07:43.476 INFO:tasks.rados.rados.0.smithi118.stdout:1824: writing smithi11881747-23 from 2146304 to 2342912 tid 4 2024-03-20T08:07:43.476 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 185 2024-03-20T08:07:43.476 INFO:tasks.rados.rados.0.smithi118.stdout:1825: seq_num 570 ranges {7454720=753664,8208384=548864,8757248=540672,9297920=516096} 2024-03-20T08:07:43.522 INFO:tasks.rados.rados.0.smithi118.stdout:1825: writing smithi11881747-15 from 7454720 to 8208384 tid 1 2024-03-20T08:07:43.526 INFO:tasks.rados.rados.0.smithi118.stdout:1825: writing smithi11881747-15 from 8208384 to 8757248 tid 2 2024-03-20T08:07:43.530 INFO:tasks.rados.rados.0.smithi118.stdout:1825: writing smithi11881747-15 from 8757248 to 9297920 tid 3 2024-03-20T08:07:43.534 INFO:tasks.rados.rados.0.smithi118.stdout:1825: writing smithi11881747-15 from 9297920 to 9814016 tid 4 2024-03-20T08:07:43.534 INFO:tasks.rados.rados.0.smithi118.stdout:1826: read oid 3 snap -1 2024-03-20T08:07:43.534 INFO:tasks.rados.rados.0.smithi118.stdout:1826: expect (ObjNum 524 snap 169 seq_num 524) 2024-03-20T08:07:43.534 INFO:tasks.rados.rados.0.smithi118.stdout:1827: rollback oid 6 current snap is 185 2024-03-20T08:07:43.534 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 180 2024-03-20T08:07:43.534 INFO:tasks.rados.rados.0.smithi118.stdout:1822: finishing rollback tid 1 to smithi11881747-34 2024-03-20T08:07:43.534 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 7044 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:07:43.534 INFO:tasks.rados.rados.0.smithi118.stdout:1825: finishing write tid 1 to smithi11881747-15 2024-03-20T08:07:43.535 INFO:tasks.rados.rados.0.smithi118.stdout:1825: finishing write tid 2 to smithi11881747-15 2024-03-20T08:07:43.535 INFO:tasks.rados.rados.0.smithi118.stdout:1822: done (7 left) 2024-03-20T08:07:43.535 INFO:tasks.rados.rados.0.smithi118.stdout:1828: snap_create 2024-03-20T08:07:43.535 INFO:tasks.rados.rados.0.smithi118.stdout:1825: finishing write tid 3 to smithi11881747-15 2024-03-20T08:07:43.537 INFO:tasks.rados.rados.0.smithi118.stdout:1825: finishing write tid 4 to smithi11881747-15 2024-03-20T08:07:43.538 INFO:tasks.rados.rados.0.smithi118.stdout:1825: finishing write tid 5 to smithi11881747-15 2024-03-20T08:07:43.538 INFO:tasks.rados.rados.0.smithi118.stdout:1827: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:07:43.538 INFO:tasks.rados.rados.0.smithi118.stdout:1827: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:1825: finishing write tid 6 to smithi11881747-15 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:1825: oid 15 updating version 0 to 1022 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:1825: oid 15 updating version 1022 to 1024 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:1825: oid 15 version 1024 is already newer than 1021 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:1825: oid 15 version 1024 is already newer than 1020 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:1825: oid 15 version 1024 is already newer than 1023 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 1024 (ObjNum 570 snap 185 seq_num 570) dirty exists 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:1825: left oid 15 (ObjNum 570 snap 185 seq_num 570) 2024-03-20T08:07:43.539 INFO:tasks.rados.rados.0.smithi118.stdout:1827: finishing rollback tid 2 to smithi11881747-6 2024-03-20T08:07:43.540 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 1302 (ObjNum 527 snap 170 seq_num 527) dirty exists 2024-03-20T08:07:43.555 INFO:tasks.rados.rados.0.smithi118.stdout:1826: expect (ObjNum 524 snap 169 seq_num 524) 2024-03-20T08:07:43.574 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:43.422+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset f3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: osdmap e434: 8 total, 8 up, 8 in 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: osd.4 marked itself down and dead 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: from='client.44222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:43.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/802461835' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset f3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: osdmap e434: 8 total, 8 up, 8 in 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: osd.4 marked itself down and dead 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: from='client.44222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:43.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/802461835' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.326+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.340+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.354+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.369+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.382+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.398+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.426+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.442+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.457+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.472+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.489+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.504+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.520+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.539+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.578 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.560+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.578 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.575+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.839 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:07:43] "GET /metrics HTTP/1.1" 200 36261 "" "Prometheus/2.43.0" 2024-03-20T08:07:43.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.591+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.608+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.621+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.636+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.666+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.681+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.697+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.730+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.746+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.760+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.840 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.775+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.791+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.806+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.823+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.841 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.840+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:43.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:43.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset f3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: osdmap e434: 8 total, 8 up, 8 in 2024-03-20T08:07:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:43.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset dc=[]:{dc=[dc,db]} for object 5:332bf1c7:::smithi11881747-32:9f 2024-03-20T08:07:43.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: osd.4 marked itself down and dead 2024-03-20T08:07:43.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: from='client.44222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:07:43.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:43.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:43 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/802461835' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:07:44.095 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.855+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.095 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.871+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.095 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.886+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.095 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.900+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.095 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.916+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.930+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.945+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.960+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.974+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:43.992+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.008+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.036+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.050+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.065+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.080+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.095+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:43 smithi196 podman[92291]: 2024-03-20 08:07:43.977205635 +0000 UTC m=+1.516690291 container died 7b658cecc8fb75bc6ac1a69e3c0df1057bae53fe16d55363a1c2af8544e47fca (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-78dbc53, ceph=True, org.label-schema.vendor=CentOS) 2024-03-20T08:07:44.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.110+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.124+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.138+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.153+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.168+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.183+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.199+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.354 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.214+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.230+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.245+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.260+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.275+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.307+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.322+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.354+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.405 INFO:tasks.rados.rados.0.smithi118.stdout:1825: done (7 left) 2024-03-20T08:07:44.405 INFO:tasks.rados.rados.0.smithi118.stdout:1826: done (6 left) 2024-03-20T08:07:44.405 INFO:tasks.rados.rados.0.smithi118.stdout:1827: done (5 left) 2024-03-20T08:07:44.405 INFO:tasks.rados.rados.0.smithi118.stdout:1828: done (4 left) 2024-03-20T08:07:44.405 INFO:tasks.rados.rados.0.smithi118.stdout:1829: snap_create 2024-03-20T08:07:44.418 INFO:tasks.rados.rados.0.smithi118.stdout:1820: finishing rollback tid 0 to smithi11881747-4 2024-03-20T08:07:44.420 INFO:tasks.rados.rados.0.smithi118.stdout:1820: finishing rollback tid 1 to smithi11881747-4 2024-03-20T08:07:44.421 INFO:tasks.rados.rados.0.smithi118.stdout:1820: finishing rollback tid 2 to smithi11881747-4 2024-03-20T08:07:44.422 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 1293 (ObjNum 506 snap 168 seq_num 506) dirty exists 2024-03-20T08:07:44.429 INFO:tasks.rados.rados.0.smithi118.stdout:1824: finishing write tid 1 to smithi11881747-23 2024-03-20T08:07:44.429 INFO:tasks.rados.rados.0.smithi118.stdout:1824: finishing write tid 2 to smithi11881747-23 2024-03-20T08:07:44.430 INFO:tasks.rados.rados.0.smithi118.stdout:1824: finishing write tid 3 to smithi11881747-23 2024-03-20T08:07:44.431 INFO:tasks.rados.rados.0.smithi118.stdout:1824: finishing write tid 4 to smithi11881747-23 2024-03-20T08:07:44.431 INFO:tasks.rados.rados.0.smithi118.stdout:1824: finishing write tid 5 to smithi11881747-23 2024-03-20T08:07:44.433 INFO:tasks.rados.rados.0.smithi118.stdout:1824: finishing write tid 6 to smithi11881747-23 2024-03-20T08:07:44.433 INFO:tasks.rados.rados.0.smithi118.stdout:1824: oid 23 updating version 0 to 1340 2024-03-20T08:07:44.433 INFO:tasks.rados.rados.0.smithi118.stdout:1824: oid 23 version 1340 is already newer than 1337 2024-03-20T08:07:44.433 INFO:tasks.rados.rados.0.smithi118.stdout:1824: oid 23 version 1340 is already newer than 1338 2024-03-20T08:07:44.433 INFO:tasks.rados.rados.0.smithi118.stdout:1824: oid 23 version 1340 is already newer than 1339 2024-03-20T08:07:44.434 INFO:tasks.rados.rados.0.smithi118.stdout:1824: oid 23 version 1340 is already newer than 1336 2024-03-20T08:07:44.434 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 1340 (ObjNum 569 snap 185 seq_num 569) dirty exists 2024-03-20T08:07:44.434 INFO:tasks.rados.rados.0.smithi118.stdout:1824: left oid 23 (ObjNum 569 snap 185 seq_num 569) 2024-03-20T08:07:44.451 INFO:tasks.rados.rados.0.smithi118.stdout:1819: finishing write tid 1 to smithi11881747-32 2024-03-20T08:07:44.451 INFO:tasks.rados.rados.0.smithi118.stdout:1819: finishing write tid 2 to smithi11881747-32 2024-03-20T08:07:44.451 INFO:tasks.rados.rados.0.smithi118.stdout:1819: finishing write tid 3 to smithi11881747-32 2024-03-20T08:07:44.452 INFO:tasks.rados.rados.0.smithi118.stdout:1819: finishing write tid 4 to smithi11881747-32 2024-03-20T08:07:44.452 INFO:tasks.rados.rados.0.smithi118.stdout:1819: finishing write tid 5 to smithi11881747-32 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:1819: finishing write tid 6 to smithi11881747-32 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:1819: oid 32 updating version 0 to 33860 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:1819: oid 32 updating version 33860 to 33862 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:1819: oid 32 updating version 33862 to 33863 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:1819: oid 32 version 33863 is already newer than 33861 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:1819: oid 32 version 33863 is already newer than 33859 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 33863 (ObjNum 567 snap 185 seq_num 567) dirty exists 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:1819: left oid 32 (ObjNum 567 snap 185 seq_num 567) 2024-03-20T08:07:44.462 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 33864 (ObjNum 1281 snap 0 seq_num 322603984) dirty exists 2024-03-20T08:07:44.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:44 smithi196 podman[92291]: 2024-03-20 08:07:44.366288563 +0000 UTC m=+1.905773214 container cleanup 7b658cecc8fb75bc6ac1a69e3c0df1057bae53fe16d55363a1c2af8544e47fca (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-78dbc53, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-20T08:07:44.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:44 smithi196 bash[92291]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4 2024-03-20T08:07:44.583 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset f3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:44.583 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:44 smithi196 ceph-mon[87080]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:07:44.583 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:44 smithi196 ceph-mon[87080]: osdmap e435: 8 total, 7 up, 8 in 2024-03-20T08:07:44.583 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:44.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:44 smithi196 ceph-mon[87080]: pgmap v330: 121 pgs: 16 peering, 1 active+clean+snaptrim_wait+snaptrim_error, 7 stale+active+clean, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 508 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 767 B/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:07:44.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:44 smithi196 ceph-mon[87080]: osdmap e436: 8 total, 7 up, 8 in 2024-03-20T08:07:44.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset f3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:44.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[95692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:07:44.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[95692]: osdmap e435: 8 total, 7 up, 8 in 2024-03-20T08:07:44.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:44.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[95692]: pgmap v330: 121 pgs: 16 peering, 1 active+clean+snaptrim_wait+snaptrim_error, 7 stale+active+clean, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 508 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 767 B/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:07:44.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[95692]: osdmap e436: 8 total, 7 up, 8 in 2024-03-20T08:07:44.620 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:44.407+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:44.620 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:44.407+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:44.620 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset f3=[]:{c7=[c5,c3,c2],da=[da,d7,cd,c8],e2=[e2,e1,e0,df,de,dd],e3=[e3],ed=[ed,eb,e9,e8,e5]} for object 5:02e2296c:::smithi11881747-49:b7 2024-03-20T08:07:44.620 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[93121]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:07:44.620 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[93121]: osdmap e435: 8 total, 7 up, 8 in 2024-03-20T08:07:44.620 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:44.620 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[93121]: pgmap v330: 121 pgs: 16 peering, 1 active+clean+snaptrim_wait+snaptrim_error, 7 stale+active+clean, 4 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 508 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 767 B/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:07:44.620 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-mon[93121]: osdmap e436: 8 total, 7 up, 8 in 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.369+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.384+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.400+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.435+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.452+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.468+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.482+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.497+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.512+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.527+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.541+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.621 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.556+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.622 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.622 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.586+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.622 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.603+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.622 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.619+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.637+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.666+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.681+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.697+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.712+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.728+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.743+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.759+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.774+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.789+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.804+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.820+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.838+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.873 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.853+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.873 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.872+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:44.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:44 smithi196 podman[92314]: 2024-03-20 08:07:44.584595787 +0000 UTC m=+0.603800214 container remove 7b658cecc8fb75bc6ac1a69e3c0df1057bae53fe16d55363a1c2af8544e47fca (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, ceph=True, RELEASE=reef-78dbc53, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-20T08:07:45.128 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.889+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.128 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.905+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.128 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.921+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.128 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.937+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.128 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.953+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.128 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.967+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.128 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.982+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.128 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:44.997+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.011+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.025+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.040+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.055+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.069+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.083+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.099+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.114+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:45 smithi196 podman[92406]: 2024-03-20 08:07:44.922828891 +0000 UTC m=+0.020104245 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:07:45.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:45 smithi196 podman[92406]: 2024-03-20 08:07:45.027340242 +0000 UTC m=+0.124615576 container create 26c8c487b912326f132ffafde2c26f1318d84d5ec113a1d86bbee32d6278fa24 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.128+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.143+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.159+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.175+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.189+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.204+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.219+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.232+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.247+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.262+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.276+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.309+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.324+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.340+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.355+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.370+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.385+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.406 INFO:tasks.rados.rados.0.smithi118.stdout:1819: done (4 left) 2024-03-20T08:07:45.406 INFO:tasks.rados.rados.0.smithi118.stdout:1820: done (3 left) 2024-03-20T08:07:45.406 INFO:tasks.rados.rados.0.smithi118.stdout:1823: done (2 left) 2024-03-20T08:07:45.406 INFO:tasks.rados.rados.0.smithi118.stdout:1824: done (1 left) 2024-03-20T08:07:45.406 INFO:tasks.rados.rados.0.smithi118.stdout:1829: done (0 left) 2024-03-20T08:07:45.406 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 187 2024-03-20T08:07:45.406 INFO:tasks.rados.rados.0.smithi118.stdout:1830: seq_num 571 ranges {4980736=770048,5750784=483328,6234112=753664,6987776=368640} 2024-03-20T08:07:45.438 INFO:tasks.rados.rados.0.smithi118.stdout:1830: writing smithi11881747-16 from 4980736 to 5750784 tid 1 2024-03-20T08:07:45.441 INFO:tasks.rados.rados.0.smithi118.stdout:1830: writing smithi11881747-16 from 5750784 to 6234112 tid 2 2024-03-20T08:07:45.446 INFO:tasks.rados.rados.0.smithi118.stdout:1830: writing smithi11881747-16 from 6234112 to 6987776 tid 3 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:1830: writing smithi11881747-16 from 6987776 to 7356416 tid 4 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:1831: copy_from oid 50 from oid 48 current snap is 187 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:1830: finishing write tid 1 to smithi11881747-16 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:1830: finishing write tid 2 to smithi11881747-16 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:1832: copy_from oid 35 from oid 37 current snap is 187 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:1833: rollback oid 8 current snap is 187 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 8 to 89 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 187 2024-03-20T08:07:45.450 INFO:tasks.rados.rados.0.smithi118.stdout:1834: seq_num 572 ranges {6414336=786432,7200768=770048,7970816=557056,8527872=278528} 2024-03-20T08:07:45.490 INFO:tasks.rados.rados.0.smithi118.stdout:1834: writing smithi11881747-12 from 6414336 to 7200768 tid 1 2024-03-20T08:07:45.495 INFO:tasks.rados.rados.0.smithi118.stdout:1834: writing smithi11881747-12 from 7200768 to 7970816 tid 2 2024-03-20T08:07:45.498 INFO:tasks.rados.rados.0.smithi118.stdout:1834: writing smithi11881747-12 from 7970816 to 8527872 tid 3 2024-03-20T08:07:45.500 INFO:tasks.rados.rados.0.smithi118.stdout:1834: writing smithi11881747-12 from 8527872 to 8806400 tid 4 2024-03-20T08:07:45.500 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 187 2024-03-20T08:07:45.500 INFO:tasks.rados.rados.0.smithi118.stdout:1830: finishing write tid 3 to smithi11881747-16 2024-03-20T08:07:45.500 INFO:tasks.rados.rados.0.smithi118.stdout:1830: finishing write tid 4 to smithi11881747-16 2024-03-20T08:07:45.500 INFO:tasks.rados.rados.0.smithi118.stdout:1835: seq_num 573 ranges {9986048=802816,10788864=655360,11444224=720896,12165120=229376} 2024-03-20T08:07:45.560 INFO:tasks.rados.rados.0.smithi118.stdout:1835: writing smithi11881747-25 from 9986048 to 10788864 tid 1 2024-03-20T08:07:45.564 INFO:tasks.rados.rados.0.smithi118.stdout:1835: writing smithi11881747-25 from 10788864 to 11444224 tid 2 2024-03-20T08:07:45.569 INFO:tasks.rados.rados.0.smithi118.stdout:1835: writing smithi11881747-25 from 11444224 to 12165120 tid 3 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1835: writing smithi11881747-25 from 12165120 to 12394496 tid 4 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1836: delete oid 41 current snap is 187 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1830: finishing write tid 5 to smithi11881747-16 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1830: finishing write tid 6 to smithi11881747-16 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1830: oid 16 updating version 0 to 1344 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1830: oid 16 updating version 1344 to 1345 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1830: oid 16 version 1345 is already newer than 1343 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1830: oid 16 version 1345 is already newer than 1342 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:1830: oid 16 updating version 1345 to 1346 2024-03-20T08:07:45.571 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 1346 (ObjNum 571 snap 187 seq_num 571) dirty exists 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:1830: left oid 16 (ObjNum 571 snap 187 seq_num 571) 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:1833: finishing rollback tid 1 to smithi11881747-8 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:1831: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:1833: finishing rollback tid 2 to smithi11881747-8 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 33866 (ObjNum 251 snap 89 seq_num 251) dirty exists 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:1831: finishing copy_from to smithi11881747-50 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 7042 (ObjNum 104 snap 23 seq_num 104) dirty exists 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:1832: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:1832: finishing copy_from to smithi11881747-35 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 9310 (ObjNum 562 snap 184 seq_num 562) dirty exists 2024-03-20T08:07:45.572 INFO:tasks.rados.rados.0.smithi118.stdout:1834: finishing write tid 1 to smithi11881747-12 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: finishing write tid 2 to smithi11881747-12 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: finishing write tid 3 to smithi11881747-12 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: finishing write tid 4 to smithi11881747-12 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: finishing write tid 5 to smithi11881747-12 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: finishing write tid 6 to smithi11881747-12 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: oid 12 updating version 0 to 17901 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: oid 12 version 17901 is already newer than 17900 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: oid 12 updating version 17901 to 17903 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: oid 12 version 17903 is already newer than 17902 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: oid 12 version 17903 is already newer than 17899 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 17903 (ObjNum 572 snap 187 seq_num 572) dirty exists 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1834: left oid 12 (ObjNum 572 snap 187 seq_num 572) 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1835: finishing write tid 1 to smithi11881747-25 2024-03-20T08:07:45.573 INFO:tasks.rados.rados.0.smithi118.stdout:1835: finishing write tid 2 to smithi11881747-25 2024-03-20T08:07:45.574 INFO:tasks.rados.rados.0.smithi118.stdout:1835: finishing write tid 3 to smithi11881747-25 2024-03-20T08:07:45.574 INFO:tasks.rados.rados.0.smithi118.stdout:1835: finishing write tid 4 to smithi11881747-25 2024-03-20T08:07:45.574 INFO:tasks.rados.rados.0.smithi118.stdout:1835: finishing write tid 5 to smithi11881747-25 2024-03-20T08:07:45.576 INFO:tasks.rados.rados.0.smithi118.stdout:1835: finishing write tid 6 to smithi11881747-25 2024-03-20T08:07:45.576 INFO:tasks.rados.rados.0.smithi118.stdout:1835: oid 25 updating version 0 to 7073 2024-03-20T08:07:45.576 INFO:tasks.rados.rados.0.smithi118.stdout:1835: oid 25 version 7073 is already newer than 7072 2024-03-20T08:07:45.576 INFO:tasks.rados.rados.0.smithi118.stdout:1835: oid 25 updating version 7073 to 7074 2024-03-20T08:07:45.576 INFO:tasks.rados.rados.0.smithi118.stdout:1835: oid 25 updating version 7074 to 7075 2024-03-20T08:07:45.576 INFO:tasks.rados.rados.0.smithi118.stdout:1835: oid 25 version 7075 is already newer than 7071 2024-03-20T08:07:45.577 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 7075 (ObjNum 573 snap 187 seq_num 573) dirty exists 2024-03-20T08:07:45.577 INFO:tasks.rados.rados.0.smithi118.stdout:1835: left oid 25 (ObjNum 573 snap 187 seq_num 573) 2024-03-20T08:07:45.578 INFO:tasks.rados.rados.0.smithi118.stdout:1830: done (6 left) 2024-03-20T08:07:45.578 INFO:tasks.rados.rados.0.smithi118.stdout:1831: done (5 left) 2024-03-20T08:07:45.578 INFO:tasks.rados.rados.0.smithi118.stdout:1832: done (4 left) 2024-03-20T08:07:45.578 INFO:tasks.rados.rados.0.smithi118.stdout:1833: done (3 left) 2024-03-20T08:07:45.578 INFO:tasks.rados.rados.0.smithi118.stdout:1834: done (2 left) 2024-03-20T08:07:45.578 INFO:tasks.rados.rados.0.smithi118.stdout:1835: done (1 left) 2024-03-20T08:07:45.578 INFO:tasks.rados.rados.0.smithi118.stdout:1836: done (0 left) 2024-03-20T08:07:45.579 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 187 2024-03-20T08:07:45.579 INFO:tasks.rados.rados.0.smithi118.stdout:1837: seq_num 575 ranges {4268032=32768} 2024-03-20T08:07:45.601 INFO:tasks.rados.rados.0.smithi118.stdout:1837: writing smithi11881747-29 from 4268032 to 4300800 tid 1 2024-03-20T08:07:45.601 INFO:tasks.rados.rados.0.smithi118.stdout:1838: rollback oid 46 current snap is 187 2024-03-20T08:07:45.602 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 172 2024-03-20T08:07:45.602 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 187 2024-03-20T08:07:45.602 INFO:tasks.rados.rados.0.smithi118.stdout:1839: seq_num 576 ranges {1802240=49152} 2024-03-20T08:07:45.612 INFO:tasks.rados.rados.0.smithi118.stdout:1839: writing smithi11881747-45 from 1802240 to 1851392 tid 1 2024-03-20T08:07:45.612 INFO:tasks.rados.rados.0.smithi118.stdout:1840: setattr oid 13 current snap is 187 2024-03-20T08:07:45.612 INFO:tasks.rados.rados.0.smithi118.stdout:1838: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:07:45.612 INFO:tasks.rados.rados.0.smithi118.stdout:1838: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1838: finishing rollback tid 2 to smithi11881747-46 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 9313 (ObjNum 470 snap 155 seq_num 470) dirty exists 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1837: finishing write tid 1 to smithi11881747-29 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1837: finishing write tid 2 to smithi11881747-29 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1837: finishing write tid 3 to smithi11881747-29 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1837: oid 29 updating version 0 to 1350 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1837: oid 29 updating version 1350 to 1351 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1351 (ObjNum 575 snap 187 seq_num 575) dirty exists 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1837: left oid 29 (ObjNum 575 snap 187 seq_num 575) 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1837: done (3 left) 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1838: done (2 left) 2024-03-20T08:07:45.613 INFO:tasks.rados.rados.0.smithi118.stdout:1841: setattr oid 17 current snap is 187 2024-03-20T08:07:45.614 INFO:tasks.rados.rados.0.smithi118.stdout:1842: copy_from oid 43 from oid 39 current snap is 187 2024-03-20T08:07:45.614 INFO:tasks.rados.rados.0.smithi118.stdout:1843: read oid 38 snap -1 2024-03-20T08:07:45.615 INFO:tasks.rados.rados.0.smithi118.stdout:1843: expect (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:07:45.615 INFO:tasks.rados.rados.0.smithi118.stdout:1844: rmattr oid 7 current snap is 187 2024-03-20T08:07:45.615 INFO:tasks.rados.rados.0.smithi118.stdout:1844: done (5 left) 2024-03-20T08:07:45.615 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 187 2024-03-20T08:07:45.615 INFO:tasks.rados.rados.0.smithi118.stdout:1845: seq_num 577 ranges {6586368=65536} 2024-03-20T08:07:45.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:45.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:45 smithi196 ceph-mon[87080]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-20T08:07:45.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:45 smithi196 ceph-mon[87080]: osdmap e437: 8 total, 7 up, 8 in 2024-03-20T08:07:45.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:45 smithi196 podman[92406]: 2024-03-20 08:07:45.280475846 +0000 UTC m=+0.377751202 container init 26c8c487b912326f132ffafde2c26f1318d84d5ec113a1d86bbee32d6278fa24 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-20T08:07:45.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:45 smithi196 podman[92406]: 2024-03-20 08:07:45.28691662 +0000 UTC m=+0.384191957 container start 26c8c487b912326f132ffafde2c26f1318d84d5ec113a1d86bbee32d6278fa24 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-20T08:07:45.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:45 smithi196 podman[92406]: 2024-03-20 08:07:45.388740342 +0000 UTC m=+0.486015674 container attach 26c8c487b912326f132ffafde2c26f1318d84d5ec113a1d86bbee32d6278fa24 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-20T08:07:45.651 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:45.410+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:45.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.399+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.420+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.438+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.462+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.481+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.524+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.651 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.539+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.652 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.556+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.652 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.574+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.652 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.592+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.652 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.610+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.652 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.628+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.652 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:45.652 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-mon[93121]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-20T08:07:45.652 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-mon[93121]: osdmap e437: 8 total, 7 up, 8 in 2024-03-20T08:07:45.652 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:45.653 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-mon[95692]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-20T08:07:45.653 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-mon[95692]: osdmap e437: 8 total, 7 up, 8 in 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:1845: writing smithi11881747-33 from 6586368 to 6651904 tid 1 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:1846: read oid 34 snap -1 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:1846: expect deleted 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:1847: read oid 50 snap -1 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:1847: expect (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:1848: copy_from oid 32 from oid 27 current snap is 187 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 7076 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:1840: done (8 left) 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 187 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7047 (ObjNum 568 snap 185 seq_num 568) dirty exists 2024-03-20T08:07:45.654 INFO:tasks.rados.rados.0.smithi118.stdout:1842: finishing copy_from racing read to smithi11881747-43 2024-03-20T08:07:45.655 INFO:tasks.rados.rados.0.smithi118.stdout:1842: finishing copy_from to smithi11881747-43 2024-03-20T08:07:45.655 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 7078 (ObjNum 563 snap 184 seq_num 563) dirty exists 2024-03-20T08:07:45.655 INFO:tasks.rados.rados.0.smithi118.stdout:1849: seq_num 578 ranges {9814016=81920} 2024-03-20T08:07:45.708 INFO:tasks.rados.rados.0.smithi118.stdout:1849: writing smithi11881747-15 from 9814016 to 9895936 tid 1 2024-03-20T08:07:45.708 INFO:tasks.rados.rados.0.smithi118.stdout:1841: done (8 left) 2024-03-20T08:07:45.708 INFO:tasks.rados.rados.0.smithi118.stdout:1842: done (7 left) 2024-03-20T08:07:45.708 INFO:tasks.rados.rados.0.smithi118.stdout:1850: copy_from oid 1 from oid 44 current snap is 187 2024-03-20T08:07:45.708 INFO:tasks.rados.rados.0.smithi118.stdout:1839: finishing write tid 1 to smithi11881747-45 2024-03-20T08:07:45.708 INFO:tasks.rados.rados.0.smithi118.stdout:1839: finishing write tid 2 to smithi11881747-45 2024-03-20T08:07:45.708 INFO:tasks.rados.rados.0.smithi118.stdout:1839: finishing write tid 3 to smithi11881747-45 2024-03-20T08:07:45.709 INFO:tasks.rados.rados.0.smithi118.stdout:1839: oid 45 updating version 0 to 10757 2024-03-20T08:07:45.709 INFO:tasks.rados.rados.0.smithi118.stdout:1839: oid 45 version 10757 is already newer than 10756 2024-03-20T08:07:45.709 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 10757 (ObjNum 576 snap 187 seq_num 576) dirty exists 2024-03-20T08:07:45.709 INFO:tasks.rados.rados.0.smithi118.stdout:1839: left oid 45 (ObjNum 576 snap 187 seq_num 576) 2024-03-20T08:07:45.709 INFO:tasks.rados.rados.0.smithi118.stdout:1839: done (7 left) 2024-03-20T08:07:45.709 INFO:tasks.rados.rados.0.smithi118.stdout:1851: delete oid 48 current snap is 187 2024-03-20T08:07:45.709 INFO:tasks.rados.rados.0.smithi118.stdout:1843: expect (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:07:45.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:45 smithi196 podman[92406]: 2024-03-20 08:07:45.63977106 +0000 UTC m=+0.737046410 container died 26c8c487b912326f132ffafde2c26f1318d84d5ec113a1d86bbee32d6278fa24 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-20T08:07:45.912 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.912 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.671+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.912 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.690+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.912 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.706+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.912 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.724+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.742+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.760+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.775+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.791+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.807+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.822+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.837+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.851+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.867+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.882+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.897+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:45.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.912+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.929+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.945+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.962+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.976+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:45.989+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.018+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.033+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.048+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.066+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.095+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.111+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.128+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.144+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.161+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:1843: done (7 left) 2024-03-20T08:07:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:1851: done (6 left) 2024-03-20T08:07:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:1852: rollback oid 28 current snap is 187 2024-03-20T08:07:46.198 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 28 to 175 2024-03-20T08:07:46.198 INFO:tasks.rados.rados.0.smithi118.stdout:1847: expect (ObjNum 104 snap 23 seq_num 104) 2024-03-20T08:07:46.260 INFO:tasks.rados.rados.0.smithi118.stdout:1845: finishing write tid 1 to smithi11881747-33 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1845: finishing write tid 2 to smithi11881747-33 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1845: finishing write tid 3 to smithi11881747-33 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1845: oid 33 updating version 0 to 13581 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1845: oid 33 updating version 13581 to 13582 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 13582 (ObjNum 577 snap 187 seq_num 577) dirty exists 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1845: left oid 33 (ObjNum 577 snap 187 seq_num 577) 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1848: finishing copy_from racing read to smithi11881747-32 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1848: finishing copy_from to smithi11881747-32 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 7083 (ObjNum 564 snap 184 seq_num 564) dirty exists 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1849: finishing write tid 1 to smithi11881747-15 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1849: finishing write tid 2 to smithi11881747-15 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1849: finishing write tid 3 to smithi11881747-15 2024-03-20T08:07:46.261 INFO:tasks.rados.rados.0.smithi118.stdout:1849: oid 15 updating version 0 to 1025 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1849: oid 15 updating version 1025 to 1026 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 1026 (ObjNum 578 snap 187 seq_num 578) dirty exists 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1849: left oid 15 (ObjNum 578 snap 187 seq_num 578) 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1850: finishing copy_from racing read to smithi11881747-1 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1850: finishing copy_from to smithi11881747-1 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 7058 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1845: done (6 left) 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1846: done (5 left) 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1847: done (4 left) 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1848: done (3 left) 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1849: done (2 left) 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1850: done (1 left) 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 187 2024-03-20T08:07:46.262 INFO:tasks.rados.rados.0.smithi118.stdout:1852: finishing rollback tid 0 to smithi11881747-28 2024-03-20T08:07:46.263 INFO:tasks.rados.rados.0.smithi118.stdout:1852: finishing rollback tid 1 to smithi11881747-28 2024-03-20T08:07:46.263 INFO:tasks.rados.rados.0.smithi118.stdout:1853: seq_num 580 ranges {6651904=114688} 2024-03-20T08:07:46.298 INFO:tasks.rados.rados.0.smithi118.stdout:1853: writing smithi11881747-33 from 6651904 to 6766592 tid 1 2024-03-20T08:07:46.298 INFO:tasks.rados.rados.0.smithi118.stdout:1854: snap_create 2024-03-20T08:07:46.298 INFO:tasks.rados.rados.0.smithi118.stdout:1852: finishing rollback tid 2 to smithi11881747-28 2024-03-20T08:07:46.298 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 1355 (ObjNum 446 snap 148 seq_num 446) dirty exists 2024-03-20T08:07:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:1853: finishing write tid 1 to smithi11881747-33 2024-03-20T08:07:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:1853: finishing write tid 2 to smithi11881747-33 2024-03-20T08:07:46.311 INFO:tasks.rados.rados.0.smithi118.stdout:1853: finishing write tid 3 to smithi11881747-33 2024-03-20T08:07:46.311 INFO:tasks.rados.rados.0.smithi118.stdout:1853: oid 33 updating version 0 to 13584 2024-03-20T08:07:46.311 INFO:tasks.rados.rados.0.smithi118.stdout:1853: oid 33 version 13584 is already newer than 13583 2024-03-20T08:07:46.311 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 13584 (ObjNum 580 snap 187 seq_num 580) dirty exists 2024-03-20T08:07:46.311 INFO:tasks.rados.rados.0.smithi118.stdout:1853: left oid 33 (ObjNum 580 snap 187 seq_num 580) 2024-03-20T08:07:46.428 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.176+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.193+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.211+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.226+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.242+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.258+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.274+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.290+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.307+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.322+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.351+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.367+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.382+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.429 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.398+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.430 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.414+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.433 INFO:tasks.rados.rados.0.smithi118.stdout:1852: done (2 left) 2024-03-20T08:07:46.433 INFO:tasks.rados.rados.0.smithi118.stdout:1853: done (1 left) 2024-03-20T08:07:46.433 INFO:tasks.rados.rados.0.smithi118.stdout:1854: done (0 left) 2024-03-20T08:07:46.433 INFO:tasks.rados.rados.0.smithi118.stdout:1855: read oid 25 snap -1 2024-03-20T08:07:46.433 INFO:tasks.rados.rados.0.smithi118.stdout:1855: expect (ObjNum 573 snap 187 seq_num 573) 2024-03-20T08:07:46.434 INFO:tasks.rados.rados.0.smithi118.stdout:1856: rollback oid 3 current snap is 188 2024-03-20T08:07:46.434 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 3 to 185 2024-03-20T08:07:46.434 INFO:tasks.rados.rados.0.smithi118.stdout:1857: snap_remove snap 186 2024-03-20T08:07:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:46 smithi196 ceph-mon[87080]: pgmap v333: 121 pgs: 4 active+undersized+degraded, 2 active+undersized+degraded+snaptrim_error, 7 active+undersized, 22 peering, 1 active+clean+snaptrim_wait+snaptrim_error, 2 stale+active+clean, 2 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 77 active+clean; 520 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 26 op/s; 108/987 objects degraded (10.942%) 2024-03-20T08:07:46.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:46 smithi196 podman[92451]: 2024-03-20 08:07:46.351682946 +0000 UTC m=+0.702334983 container remove 26c8c487b912326f132ffafde2c26f1318d84d5ec113a1d86bbee32d6278fa24 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:07:46.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:46 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-20T08:07:46.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:46 smithi196 systemd[1]: Stopped Ceph osd.4 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:07:46.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:46 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.4.service: Consumed 1min 2.656s CPU time. 2024-03-20T08:07:46.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:46 smithi196 systemd[1]: Starting Ceph osd.4 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:07:46.681 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:46.681 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-mon[93121]: pgmap v333: 121 pgs: 4 active+undersized+degraded, 2 active+undersized+degraded+snaptrim_error, 7 active+undersized, 22 peering, 1 active+clean+snaptrim_wait+snaptrim_error, 2 stale+active+clean, 2 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 77 active+clean; 520 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 26 op/s; 108/987 objects degraded (10.942%) 2024-03-20T08:07:46.681 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:46.681 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-mon[95692]: pgmap v333: 121 pgs: 4 active+undersized+degraded, 2 active+undersized+degraded+snaptrim_error, 7 active+undersized, 22 peering, 1 active+clean+snaptrim_wait+snaptrim_error, 2 stale+active+clean, 2 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 77 active+clean; 520 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 18 KiB/s rd, 6.1 MiB/s wr, 26 op/s; 108/987 objects degraded (10.942%) 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:46.438+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:46.438+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.432+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.459+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.479+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.494+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.509+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.524+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.538+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.682 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.553+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.683 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.570+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.683 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.586+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.683 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.601+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.683 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.618+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.683 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.636+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.683 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.650+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.683 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.665+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.683 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.681+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.936 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.696+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.936 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.712+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.730+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.746+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.761+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.776+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.791+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.806+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.820+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.835+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.849+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.864+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.879+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.892+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.907+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.937 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.922+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:46.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.936+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:46 smithi196 podman[92561]: 2024-03-20 08:07:46.727414623 +0000 UTC m=+0.019746088 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:07:47.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:46 smithi196 podman[92561]: 2024-03-20 08:07:46.869862876 +0000 UTC m=+0.162194334 container create 57e91ef7abcd5b95b6c8cf23292343dd1dc0442453660874888514ea39f5ae4b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-20T08:07:47.195 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.951+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.966+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.981+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:46.998+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.012+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.027+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.041+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.057+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.089+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.196 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.151+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.197 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.168+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.197 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.182+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.197 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.196+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 podman[92561]: 2024-03-20 08:07:47.131238987 +0000 UTC m=+0.423570447 container init 57e91ef7abcd5b95b6c8cf23292343dd1dc0442453660874888514ea39f5ae4b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-20T08:07:47.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 podman[92561]: 2024-03-20 08:07:47.137251806 +0000 UTC m=+0.429583266 container start 57e91ef7abcd5b95b6c8cf23292343dd1dc0442453660874888514ea39f5ae4b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-20T08:07:47.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 podman[92561]: 2024-03-20 08:07:47.239647987 +0000 UTC m=+0.531979463 container attach 57e91ef7abcd5b95b6c8cf23292343dd1dc0442453660874888514ea39f5ae4b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-20T08:07:47.441 INFO:tasks.rados.rados.0.smithi118.stdout:1857: done (2 left) 2024-03-20T08:07:47.442 INFO:tasks.rados.rados.0.smithi118.stdout:1856: finishing rollback tid 0 to smithi11881747-3 2024-03-20T08:07:47.442 INFO:tasks.rados.rados.0.smithi118.stdout:1856: finishing rollback tid 1 to smithi11881747-3 2024-03-20T08:07:47.442 INFO:tasks.rados.rados.0.smithi118.stdout:1856: finishing rollback tid 2 to smithi11881747-3 2024-03-20T08:07:47.442 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 13588 (ObjNum 524 snap 169 seq_num 524) dirty exists 2024-03-20T08:07:47.442 INFO:tasks.rados.rados.0.smithi118.stdout:1855: expect (ObjNum 573 snap 187 seq_num 573) 2024-03-20T08:07:47.458 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:47.442+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:47.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-mon[93121]: Health check failed: Degraded data redundancy: 108/987 objects degraded (10.942%), 6 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:47.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-mon[93121]: osdmap e438: 8 total, 7 up, 8 in 2024-03-20T08:07:47.458 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.210+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.226+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.240+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.257+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.270+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.285+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.299+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.314+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.328+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.460 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.343+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.460 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.358+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.460 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.372+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.460 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.388+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.460 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.403+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.460 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.419+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.460 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.437+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.460 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-mon[95692]: Health check failed: Degraded data redundancy: 108/987 objects degraded (10.942%), 6 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:47.460 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-mon[95692]: osdmap e438: 8 total, 7 up, 8 in 2024-03-20T08:07:47.460 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.459+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.478+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.493+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.507+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.524+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.539+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.556+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.587+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.716 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.604+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.620+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.636+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.652+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.668+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.684+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.701+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.716+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-mon[87080]: Health check failed: Degraded data redundancy: 108/987 objects degraded (10.942%), 6 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-mon[87080]: osdmap e438: 8 total, 7 up, 8 in 2024-03-20T08:07:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:47.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate[92576]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-20T08:07:47.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate[92576]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 bash[92561]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 bash[92561]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 bash[92561]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 bash[92561]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 bash[92561]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 bash[92561]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 bash[92561]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate[92576]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate[92576]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate[92576]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate[92576]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate[92576]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-20T08:07:47.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:47 smithi196 podman[92561]: 2024-03-20 08:07:47.656515794 +0000 UTC m=+0.948847247 container died 57e91ef7abcd5b95b6c8cf23292343dd1dc0442453660874888514ea39f5ae4b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:07:47.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.733+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.747+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.763+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.778+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.798+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.815+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.832+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.850+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.866+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.881+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.896+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.913+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.946+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.978 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.961+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:47.979 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.977+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.229 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:47.995+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.229 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.012+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.229 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.027+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.229 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.043+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.058+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.075+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.090+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.106+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.121+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.149+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.165+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.196+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.213+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.230 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.229+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.487 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-mon[93121]: osdmap e439: 8 total, 7 up, 8 in 2024-03-20T08:07:48.487 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:48.488 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-mon[93121]: pgmap v336: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 8 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 13 active+undersized, 22 peering, 1 active+clean+snaptrim_error, 2 active+clean+snaptrim, 70 active+clean; 538 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.7 MiB/s rd, 8.2 MiB/s wr, 38 op/s; 180/1017 objects degraded (17.699%) 2024-03-20T08:07:48.488 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.246+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.488 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.260+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.488 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.275+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.488 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.291+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.488 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.308+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.488 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.326+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.343+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.358+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.374+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.390+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.406+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.423+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.441+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.458+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.473+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.489 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-mon[95692]: osdmap e439: 8 total, 7 up, 8 in 2024-03-20T08:07:48.490 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:48.490 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-mon[95692]: pgmap v336: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 8 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 13 active+undersized, 22 peering, 1 active+clean+snaptrim_error, 2 active+clean+snaptrim, 70 active+clean; 538 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.7 MiB/s rd, 8.2 MiB/s wr, 38 op/s; 180/1017 objects degraded (17.699%) 2024-03-20T08:07:48.546 INFO:tasks.rados.rados.0.smithi118.stdout:1858: read oid 11 snap 164 2024-03-20T08:07:48.546 INFO:tasks.rados.rados.0.smithi118.stdout:1858: expect deleted 2024-03-20T08:07:48.546 INFO:tasks.rados.rados.0.smithi118.stdout:1855: done (2 left) 2024-03-20T08:07:48.546 INFO:tasks.rados.rados.0.smithi118.stdout:1856: done (1 left) 2024-03-20T08:07:48.546 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 188 2024-03-20T08:07:48.546 INFO:tasks.rados.rados.0.smithi118.stdout:1859: seq_num 581 ranges {3547136=139264} 2024-03-20T08:07:48.567 INFO:tasks.rados.rados.0.smithi118.stdout:1859: writing smithi11881747-35 from 3547136 to 3686400 tid 1 2024-03-20T08:07:48.567 INFO:tasks.rados.rados.0.smithi118.stdout:1860: rollback oid 1 current snap is 188 2024-03-20T08:07:48.567 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 1 to 185 2024-03-20T08:07:48.567 INFO:tasks.rados.rados.0.smithi118.stdout:1858: done (2 left) 2024-03-20T08:07:48.567 INFO:tasks.rados.rados.0.smithi118.stdout:1861: read oid 33 snap -1 2024-03-20T08:07:48.568 INFO:tasks.rados.rados.0.smithi118.stdout:1861: expect (ObjNum 580 snap 187 seq_num 580) 2024-03-20T08:07:48.568 INFO:tasks.rados.rados.0.smithi118.stdout:1862: read oid 12 snap -1 2024-03-20T08:07:48.568 INFO:tasks.rados.rados.0.smithi118.stdout:1862: expect (ObjNum 572 snap 187 seq_num 572) 2024-03-20T08:07:48.568 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 188 2024-03-20T08:07:48.568 INFO:tasks.rados.rados.0.smithi118.stdout:1863: seq_num 582 ranges {2916352=155648} 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1863: writing smithi11881747-30 from 2916352 to 3072000 tid 1 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1864: copy_from oid 31 from oid 26 current snap is 188 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1859: finishing write tid 1 to smithi11881747-35 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1860: finishing rollback tid 0 to smithi11881747-1 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1859: finishing write tid 2 to smithi11881747-35 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1860: finishing rollback tid 1 to smithi11881747-1 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1860: finishing rollback tid 2 to smithi11881747-1 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 7081 (ObjNum 521 snap 169 seq_num 521) dirty exists 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1859: finishing write tid 3 to smithi11881747-35 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1859: oid 35 updating version 0 to 9315 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:1859: oid 35 version 9315 is already newer than 9314 2024-03-20T08:07:48.588 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 9315 (ObjNum 581 snap 188 seq_num 581) dirty exists 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1859: left oid 35 (ObjNum 581 snap 188 seq_num 581) 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1859: done (5 left) 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1860: done (4 left) 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1865: rollback oid 43 current snap is 188 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 43 to 169 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1866: copy_from oid 27 from oid 38 current snap is 188 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1867: read oid 7 snap 164 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1867: expect deleted 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1868: read oid 4 snap -1 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1868: expect (ObjNum 506 snap 168 seq_num 506) 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 188 2024-03-20T08:07:48.589 INFO:tasks.rados.rados.0.smithi118.stdout:1869: seq_num 583 ranges {5914624=172032} 2024-03-20T08:07:48.622 INFO:tasks.rados.rados.0.smithi118.stdout:1869: writing smithi11881747-1 from 5914624 to 6086656 tid 1 2024-03-20T08:07:48.622 INFO:tasks.rados.rados.0.smithi118.stdout:1870: rollback oid 40 current snap is 188 2024-03-20T08:07:48.622 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 183 2024-03-20T08:07:48.622 INFO:tasks.rados.rados.0.smithi118.stdout:1863: finishing write tid 1 to smithi11881747-30 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:1864: finishing copy_from racing read to smithi11881747-31 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:1865: finishing rollback tid 0 to smithi11881747-43 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:1865: finishing rollback tid 1 to smithi11881747-43 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:1865: finishing rollback tid 2 to smithi11881747-43 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 7084 (ObjNum 519 snap 169 seq_num 519) dirty exists 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:1864: finishing copy_from to smithi11881747-31 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 7057 (ObjNum 449 snap 149 seq_num 449) dirty exists 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:1863: finishing write tid 2 to smithi11881747-30 2024-03-20T08:07:48.623 INFO:tasks.rados.rados.0.smithi118.stdout:1861: expect (ObjNum 580 snap 187 seq_num 580) 2024-03-20T08:07:48.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:48 smithi196 ceph-mon[87080]: osdmap e439: 8 total, 7 up, 8 in 2024-03-20T08:07:48.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:48.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:48 smithi196 ceph-mon[87080]: pgmap v336: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 8 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 13 active+undersized, 22 peering, 1 active+clean+snaptrim_error, 2 active+clean+snaptrim, 70 active+clean; 538 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 3.7 MiB/s rd, 8.2 MiB/s wr, 38 op/s; 180/1017 objects degraded (17.699%) 2024-03-20T08:07:48.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:48 smithi196 podman[92727]: 2024-03-20 08:07:48.310777022 +0000 UTC m=+0.649126370 container remove 57e91ef7abcd5b95b6c8cf23292343dd1dc0442453660874888514ea39f5ae4b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD) 2024-03-20T08:07:48.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:48 smithi196 podman[92792]: 2024-03-20 08:07:48.434826783 +0000 UTC m=+0.021341305 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:07:48.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:48 smithi196 podman[92792]: 2024-03-20 08:07:48.545516206 +0000 UTC m=+0.132030705 container create 4b7be73bd9c3abfafb87107c4d22d17a429c4dd24de26c7180be725804e04084 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.488+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.505+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.520+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.534+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.551+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.565+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.615+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.635+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.651+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.667+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.749 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.682+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.750 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.696+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.750 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.712+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.750 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.730+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:48.750 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.749+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.766+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.782+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.799+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.817+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.832+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.847+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.863+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.878+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.892+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.908+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.924+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.939+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.954+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.971+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:48.988+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.019+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.034+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.049+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.063+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.079+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.093+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.110+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.124+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.138+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.153+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.170+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.186+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.201+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.218+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.264 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.234+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.249+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.265 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.263+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.269 INFO:tasks.rados.rados.0.smithi118.stdout:1868: expect (ObjNum 506 snap 168 seq_num 506) 2024-03-20T08:07:49.377 INFO:tasks.rados.rados.0.smithi118.stdout:1863: finishing write tid 3 to smithi11881747-30 2024-03-20T08:07:49.377 INFO:tasks.rados.rados.0.smithi118.stdout:1863: oid 30 updating version 0 to 1295 2024-03-20T08:07:49.377 INFO:tasks.rados.rados.0.smithi118.stdout:1863: oid 30 version 1295 is already newer than 1294 2024-03-20T08:07:49.377 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 1295 (ObjNum 582 snap 188 seq_num 582) dirty exists 2024-03-20T08:07:49.377 INFO:tasks.rados.rados.0.smithi118.stdout:1863: left oid 30 (ObjNum 582 snap 188 seq_num 582) 2024-03-20T08:07:49.377 INFO:tasks.rados.rados.0.smithi118.stdout:1862: expect (ObjNum 572 snap 187 seq_num 572) 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.277+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.306+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.323+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.341+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.358+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.374+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.410+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.426+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.443+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.526 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.460+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.477+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.493+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.509+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.527 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.526+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:49 smithi196 podman[92792]: 2024-03-20 08:07:49.206693583 +0000 UTC m=+0.793208090 container init 4b7be73bd9c3abfafb87107c4d22d17a429c4dd24de26c7180be725804e04084 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-20T08:07:49.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:49 smithi196 podman[92792]: 2024-03-20 08:07:49.212805293 +0000 UTC m=+0.799319791 container start 4b7be73bd9c3abfafb87107c4d22d17a429c4dd24de26c7180be725804e04084 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:07:49.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:49 smithi196 ceph-osd[92829]: -- 172.21.15.196:0/1312481431 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e2117d3680 con 0x55e21179d000 2024-03-20T08:07:49.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:49 smithi196 bash[92792]: 4b7be73bd9c3abfafb87107c4d22d17a429c4dd24de26c7180be725804e04084 2024-03-20T08:07:49.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:49 smithi196 systemd[1]: Started Ceph osd.4 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:07:49.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.544+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.560+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.578+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.595+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.612+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.627+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.643+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.659+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.676+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.693+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.710+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.726+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.745+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.762+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:49.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.794+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.809+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.825+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.842+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.859+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.877+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.040 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.910+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.925+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.972+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:49.989+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.006+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.041 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.022+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.053 INFO:tasks.rados.rados.0.smithi118.stdout:1869: finishing write tid 1 to smithi11881747-1 2024-03-20T08:07:50.053 INFO:tasks.rados.rados.0.smithi118.stdout:1869: finishing write tid 2 to smithi11881747-1 2024-03-20T08:07:50.053 INFO:tasks.rados.rados.0.smithi118.stdout:1869: finishing write tid 3 to smithi11881747-1 2024-03-20T08:07:50.053 INFO:tasks.rados.rados.0.smithi118.stdout:1869: oid 1 updating version 0 to 7085 2024-03-20T08:07:50.053 INFO:tasks.rados.rados.0.smithi118.stdout:1869: oid 1 updating version 7085 to 7086 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 7086 (ObjNum 583 snap 188 seq_num 583) dirty exists 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1869: left oid 1 (ObjNum 583 snap 188 seq_num 583) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1866: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1861: done (9 left) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1862: done (8 left) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1863: done (7 left) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1864: done (6 left) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1865: done (5 left) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1867: done (4 left) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1868: done (3 left) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1869: done (2 left) 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 188 2024-03-20T08:07:50.054 INFO:tasks.rados.rados.0.smithi118.stdout:1871: seq_num 584 ranges {3547136=188416} 2024-03-20T08:07:50.074 INFO:tasks.rados.rados.0.smithi118.stdout:1871: writing smithi11881747-37 from 3547136 to 3735552 tid 1 2024-03-20T08:07:50.074 INFO:tasks.rados.rados.0.smithi118.stdout:1872: snap_create 2024-03-20T08:07:50.074 INFO:tasks.rados.rados.0.smithi118.stdout:1866: finishing copy_from to smithi11881747-27 2024-03-20T08:07:50.074 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7056 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:07:50.074 INFO:tasks.rados.rados.0.smithi118.stdout:1870: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:07:50.075 INFO:tasks.rados.rados.0.smithi118.stdout:1870: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:07:50.075 INFO:tasks.rados.rados.0.smithi118.stdout:1870: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:07:50.075 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 11809 (ObjNum 551 snap 180 seq_num 551) dirty exists 2024-03-20T08:07:50.082 INFO:tasks.rados.rados.0.smithi118.stdout:1871: finishing write tid 1 to smithi11881747-37 2024-03-20T08:07:50.089 INFO:tasks.rados.rados.0.smithi118.stdout:1871: finishing write tid 2 to smithi11881747-37 2024-03-20T08:07:50.090 INFO:tasks.rados.rados.0.smithi118.stdout:1871: finishing write tid 3 to smithi11881747-37 2024-03-20T08:07:50.090 INFO:tasks.rados.rados.0.smithi118.stdout:1871: oid 37 updating version 0 to 12311 2024-03-20T08:07:50.090 INFO:tasks.rados.rados.0.smithi118.stdout:1871: oid 37 updating version 12311 to 12312 2024-03-20T08:07:50.090 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 12312 (ObjNum 584 snap 188 seq_num 584) dirty exists 2024-03-20T08:07:50.090 INFO:tasks.rados.rados.0.smithi118.stdout:1871: left oid 37 (ObjNum 584 snap 188 seq_num 584) 2024-03-20T08:07:50.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:49.756+0000 7fa5fac127c0 -1 Falling back to public interface 2024-03-20T08:07:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.040+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.058+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.089+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.296 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.134+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.150+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.164+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.179+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.193+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.209+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.223+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.237+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.252+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.266+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.280+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.297 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.384 INFO:tasks.rados.rados.0.smithi118.stdout:1866: done (3 left) 2024-03-20T08:07:50.384 INFO:tasks.rados.rados.0.smithi118.stdout:1870: done (2 left) 2024-03-20T08:07:50.384 INFO:tasks.rados.rados.0.smithi118.stdout:1871: done (1 left) 2024-03-20T08:07:50.384 INFO:tasks.rados.rados.0.smithi118.stdout:1872: done (0 left) 2024-03-20T08:07:50.384 INFO:tasks.rados.rados.0.smithi118.stdout:1873: read oid 12 snap -1 2024-03-20T08:07:50.384 INFO:tasks.rados.rados.0.smithi118.stdout:1873: expect (ObjNum 572 snap 187 seq_num 572) 2024-03-20T08:07:50.384 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 189 2024-03-20T08:07:50.384 INFO:tasks.rados.rados.0.smithi118.stdout:1874: seq_num 585 ranges {0=204800} 2024-03-20T08:07:50.385 INFO:tasks.rados.rados.0.smithi118.stdout:1874: writing smithi11881747-24 from 0 to 204800 tid 1 2024-03-20T08:07:50.385 INFO:tasks.rados.rados.0.smithi118.stdout:1875: rollback oid 45 current snap is 189 2024-03-20T08:07:50.385 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 45 to 89 2024-03-20T08:07:50.385 INFO:tasks.rados.rados.0.smithi118.stdout:1876: snap_create 2024-03-20T08:07:50.388 INFO:tasks.rados.rados.0.smithi118.stdout:1874: finishing write tid 1 to smithi11881747-24 2024-03-20T08:07:50.388 INFO:tasks.rados.rados.0.smithi118.stdout:1874: finishing write tid 2 to smithi11881747-24 2024-03-20T08:07:50.390 INFO:tasks.rados.rados.0.smithi118.stdout:1874: finishing write tid 3 to smithi11881747-24 2024-03-20T08:07:50.390 INFO:tasks.rados.rados.0.smithi118.stdout:1874: oid 24 updating version 0 to 7065 2024-03-20T08:07:50.390 INFO:tasks.rados.rados.0.smithi118.stdout:1874: oid 24 updating version 7065 to 7066 2024-03-20T08:07:50.390 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 7066 (ObjNum 585 snap 189 seq_num 585) dirty exists 2024-03-20T08:07:50.390 INFO:tasks.rados.rados.0.smithi118.stdout:1874: left oid 24 (ObjNum 585 snap 189 seq_num 585) 2024-03-20T08:07:50.403 INFO:tasks.rados.rados.0.smithi118.stdout:1875: finishing rollback tid 0 to smithi11881747-45 2024-03-20T08:07:50.412 INFO:tasks.rados.rados.0.smithi118.stdout:1875: finishing rollback tid 1 to smithi11881747-45 2024-03-20T08:07:50.412 INFO:tasks.rados.rados.0.smithi118.stdout:1875: finishing rollback tid 2 to smithi11881747-45 2024-03-20T08:07:50.413 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 12428 (ObjNum 196 snap 64 seq_num 196) dirty exists 2024-03-20T08:07:50.434 INFO:tasks.rados.rados.0.smithi118.stdout:1873: expect (ObjNum 572 snap 187 seq_num 572) 2024-03-20T08:07:50.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:50.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:50.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:50.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[93121]: Health check update: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-03-20T08:07:50.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[93121]: pgmap v337: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 1 active+clean+snaptrim_error, 2 active+clean+snaptrim, 70 active+clean; 565 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 7.7 MiB/s rd, 8.0 MiB/s wr, 36 op/s; 215/1056 objects degraded (20.360%) 2024-03-20T08:07:50.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:50.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:50.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:50.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[95692]: Health check update: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-03-20T08:07:50.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-mon[95692]: pgmap v337: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 1 active+clean+snaptrim_error, 2 active+clean+snaptrim, 70 active+clean; 565 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 7.7 MiB/s rd, 8.0 MiB/s wr, 36 op/s; 215/1056 objects degraded (20.360%) 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:50.391+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:50.391+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.318+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.333+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.349+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.364+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.380+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.404+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.439+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.456+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.471+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.485+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.500+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.515+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.529+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.545+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.560+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:07:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:50 smithi196 ceph-mon[87080]: Health check update: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-03-20T08:07:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:50 smithi196 ceph-mon[87080]: pgmap v337: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 1 active+clean+snaptrim_error, 2 active+clean+snaptrim, 70 active+clean; 565 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 7.7 MiB/s rd, 8.0 MiB/s wr, 36 op/s; 215/1056 objects degraded (20.360%) 2024-03-20T08:07:50.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.576+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.592+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.609+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.625+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.639+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.655+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.670+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.687+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.704+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.720+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.751+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.768+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.781+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.797+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:50.814 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.812+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.830+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.846+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.863+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.879+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.895+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.913+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.074 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.931+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.949+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.965+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.982+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:50.998+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.014+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.028+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.044+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.060+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.337 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.074+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.337 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.089+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.337 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.136+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.152+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.168+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.182+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.199+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.214+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.229+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.245+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.261+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.276+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.307+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.322+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.387 INFO:tasks.rados.rados.0.smithi118.stdout:1873: done (3 left) 2024-03-20T08:07:51.387 INFO:tasks.rados.rados.0.smithi118.stdout:1874: done (2 left) 2024-03-20T08:07:51.387 INFO:tasks.rados.rados.0.smithi118.stdout:1875: done (1 left) 2024-03-20T08:07:51.387 INFO:tasks.rados.rados.0.smithi118.stdout:1876: done (0 left) 2024-03-20T08:07:51.387 INFO:tasks.rados.rados.0.smithi118.stdout:1877: rmattr oid 18 current snap is 190 2024-03-20T08:07:51.387 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 190 2024-03-20T08:07:51.387 INFO:tasks.rados.rados.0.smithi118.stdout:1878: seq_num 586 ranges {0=221184} 2024-03-20T08:07:51.388 INFO:tasks.rados.rados.0.smithi118.stdout:1878: writing smithi11881747-14 from 0 to 221184 tid 1 2024-03-20T08:07:51.388 INFO:tasks.rados.rados.0.smithi118.stdout:1879: snap_create 2024-03-20T08:07:51.389 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 7086 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:07:51.404 INFO:tasks.rados.rados.0.smithi118.stdout:1878: finishing write tid 1 to smithi11881747-14 2024-03-20T08:07:51.404 INFO:tasks.rados.rados.0.smithi118.stdout:1878: finishing write tid 2 to smithi11881747-14 2024-03-20T08:07:51.408 INFO:tasks.rados.rados.0.smithi118.stdout:1878: finishing write tid 3 to smithi11881747-14 2024-03-20T08:07:51.408 INFO:tasks.rados.rados.0.smithi118.stdout:1878: oid 14 updating version 0 to 33870 2024-03-20T08:07:51.408 INFO:tasks.rados.rados.0.smithi118.stdout:1878: oid 14 updating version 33870 to 33871 2024-03-20T08:07:51.408 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 33871 (ObjNum 586 snap 190 seq_num 586) dirty exists 2024-03-20T08:07:51.408 INFO:tasks.rados.rados.0.smithi118.stdout:1878: left oid 14 (ObjNum 586 snap 190 seq_num 586) 2024-03-20T08:07:51.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-mon[93121]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 3 pgs peering) 2024-03-20T08:07:51.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-mon[93121]: osdmap e440: 8 total, 7 up, 8 in 2024-03-20T08:07:51.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:51.599 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:51.394+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:51.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-mon[95692]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 3 pgs peering) 2024-03-20T08:07:51.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-mon[95692]: osdmap e440: 8 total, 7 up, 8 in 2024-03-20T08:07:51.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.353+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.369+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.385+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.405+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.421+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.435+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.450+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.464+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.479+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.494+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.509+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.525+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.540+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.555+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.569+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.584+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.599+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:51 smithi196 ceph-mon[87080]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 3 pgs peering) 2024-03-20T08:07:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:51 smithi196 ceph-mon[87080]: osdmap e440: 8 total, 7 up, 8 in 2024-03-20T08:07:51.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:51.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.614+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.629+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.644+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.660+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.674+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.690+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.867 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.705+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.719+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.752+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.768+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.783+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.799+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.816+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.832+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:51.868 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.848+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.867+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.882+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.897+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.915+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.930+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.948+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.964+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.978+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:51.992+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.022+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.037+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.053+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.068+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.083+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.098+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.114+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.129+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.144+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.158+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.172+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.188+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.203+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.219+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.237+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.253+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.268+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.285+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.304+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.318+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.380 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.333+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.381 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.381 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.365+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:1877: done (2 left) 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:1878: done (1 left) 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:1879: done (0 left) 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:1880: copy_from oid 34 from oid 41 current snap is 191 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:1881: rollback oid 2 current snap is 191 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 182 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:1882: rollback oid 26 current snap is 191 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 167 2024-03-20T08:07:52.391 INFO:tasks.rados.rados.0.smithi118.stdout:1883: read oid 9 snap -1 2024-03-20T08:07:52.392 INFO:tasks.rados.rados.0.smithi118.stdout:1883: expect (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:07:52.392 INFO:tasks.rados.rados.0.smithi118.stdout:1884: rollback oid 37 current snap is 191 2024-03-20T08:07:52.392 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 189 2024-03-20T08:07:52.392 INFO:tasks.rados.rados.0.smithi118.stdout:1885: delete oid 4 current snap is 191 2024-03-20T08:07:52.395 INFO:tasks.rados.rados.0.smithi118.stdout:1882: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:07:52.396 INFO:tasks.rados.rados.0.smithi118.stdout:1881: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:07:52.397 INFO:tasks.rados.rados.0.smithi118.stdout:1885: done (5 left) 2024-03-20T08:07:52.397 INFO:tasks.rados.rados.0.smithi118.stdout:1886: snap_create 2024-03-20T08:07:52.397 INFO:tasks.rados.rados.0.smithi118.stdout:1881: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:07:52.398 INFO:tasks.rados.rados.0.smithi118.stdout:1881: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:07:52.398 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 9318 (ObjNum 553 snap 181 seq_num 553) dirty exists 2024-03-20T08:07:52.399 INFO:tasks.rados.rados.0.smithi118.stdout:1880: finishing copy_from to smithi11881747-34 2024-03-20T08:07:52.399 INFO:tasks.rados.rados.0.smithi118.stdout:1880: got expected ENOENT (src dne) 2024-03-20T08:07:52.399 INFO:tasks.rados.rados.0.smithi118.stdout:1880: finishing copy_from racing read to smithi11881747-34 2024-03-20T08:07:52.400 INFO:tasks.rados.rados.0.smithi118.stdout:1882: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:07:52.400 INFO:tasks.rados.rados.0.smithi118.stdout:1882: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:07:52.400 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 7069 (ObjNum 449 snap 149 seq_num 449) dirty exists 2024-03-20T08:07:52.408 INFO:tasks.rados.rados.0.smithi118.stdout:1883: expect (ObjNum 471 snap 156 seq_num 471) 2024-03-20T08:07:52.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:52 smithi196 ceph-mon[87080]: osdmap e441: 8 total, 7 up, 8 in 2024-03-20T08:07:52.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:52.548 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:52 smithi196 ceph-mon[87080]: pgmap v340: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 565 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 23 op/s; 216/1059 objects degraded (20.397%) 2024-03-20T08:07:52.548 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:52.313Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:52.591 INFO:tasks.rados.rados.0.smithi118.stdout:1884: finishing rollback tid 0 to smithi11881747-37 2024-03-20T08:07:52.591 INFO:tasks.rados.rados.0.smithi118.stdout:1884: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:07:52.591 INFO:tasks.rados.rados.0.smithi118.stdout:1884: finishing rollback tid 2 to smithi11881747-37 2024-03-20T08:07:52.591 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 13140 (ObjNum 584 snap 188 seq_num 584) dirty exists 2024-03-20T08:07:52.631 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:52.397+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:52.632 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:52.397+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:52.632 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.380+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.632 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.403+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.632 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.422+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.632 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.439+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.632 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.455+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.474+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.492+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.507+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.524+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.540+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.556+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.570+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.589+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.603+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.617+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.634 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-mon[93121]: osdmap e441: 8 total, 7 up, 8 in 2024-03-20T08:07:52.634 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:52.634 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-mon[93121]: pgmap v340: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 565 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 23 op/s; 216/1059 objects degraded (20.397%) 2024-03-20T08:07:52.634 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-mon[95692]: osdmap e441: 8 total, 7 up, 8 in 2024-03-20T08:07:52.634 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:52.634 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-mon[95692]: pgmap v340: 121 pgs: 1 active+clean+snaptrim_error, 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 565 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 23 op/s; 216/1059 objects degraded (20.397%) 2024-03-20T08:07:52.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.632+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.647+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.662+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.677+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.693+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.893 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.710+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.726+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.741+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.756+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.772+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.787+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.801+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.817+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.833+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.847+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.862+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:52.894 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.876+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.146 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.146 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.910+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.146 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.925+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.146 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.942+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.146 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.959+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.146 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.979+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.146 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:52.995+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.011+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.025+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.040+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.055+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.071+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.086+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.102+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.117+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.132+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.147 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.146+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.392 INFO:tasks.rados.rados.0.smithi118.stdout:1880: done (5 left) 2024-03-20T08:07:53.392 INFO:tasks.rados.rados.0.smithi118.stdout:1881: done (4 left) 2024-03-20T08:07:53.392 INFO:tasks.rados.rados.0.smithi118.stdout:1882: done (3 left) 2024-03-20T08:07:53.393 INFO:tasks.rados.rados.0.smithi118.stdout:1883: done (2 left) 2024-03-20T08:07:53.393 INFO:tasks.rados.rados.0.smithi118.stdout:1884: done (1 left) 2024-03-20T08:07:53.393 INFO:tasks.rados.rados.0.smithi118.stdout:1886: done (0 left) 2024-03-20T08:07:53.393 INFO:tasks.rados.rados.0.smithi118.stdout:1887: copy_from oid 2 from oid 47 current snap is 192 2024-03-20T08:07:53.393 INFO:tasks.rados.rados.0.smithi118.stdout:1888: snap_remove snap 191 2024-03-20T08:07:53.396 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 216/1059 objects degraded (20.397%), 13 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:53.396 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-mon[93121]: osdmap e442: 8 total, 7 up, 8 in 2024-03-20T08:07:53.396 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:53.396 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:53.393+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:53.396 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 216/1059 objects degraded (20.397%), 13 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.162+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.178+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.193+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.210+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.227+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.244+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.262+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.281+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.299+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.317+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.335+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.352+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.397 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.367+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.398 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.384+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:53 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 216/1059 objects degraded (20.397%), 13 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:53.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:53 smithi196 ceph-mon[87080]: osdmap e442: 8 total, 7 up, 8 in 2024-03-20T08:07:53.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:53.663 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-mon[95692]: osdmap e442: 8 total, 7 up, 8 in 2024-03-20T08:07:53.663 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:53.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.406+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.424+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.441+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.459+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.475+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.490+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.506+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.520+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.536+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.551+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.567+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.585+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.601+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.615+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.632+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.647+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.664 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.663+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.665 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:07:53] "GET /metrics HTTP/1.1" 200 36261 "" "Prometheus/2.43.0" 2024-03-20T08:07:53.888 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:07:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:07:53.505Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=nodes name=CephNodeDiskspaceWarning index=4 msg="Evaluating rule failed" rule="alert: CephNodeDiskspaceWarning\nexpr: predict_linear(node_filesystem_free_bytes{device=~\"/.*\"}[2d], 3600 * 24 * 5)\n * on (instance) group_left (nodename) node_uname_info < 0\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.8.4\n severity: warning\n type: ceph_default\nannotations:\n description: Mountpoint {{ $labels.mountpoint }} on {{ $labels.nodename }} will\n be full in less than 5 days based on the 48 hour trailing fill rate.\n summary: Host filesystem free space is getting low\n" err="found duplicate series for the match group {instance=\"smithi118\"} on the right hand-side of the operation: [{__name__=\"node_uname_info\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}, {__name__=\"node_uname_info\", cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", domainname=\"(none)\", instance=\"smithi118\", job=\"node\", machine=\"x86_64\", nodename=\"smithi118\", release=\"5.14.0-430.el9.x86_64\", sysname=\"Linux\", version=\"#1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:07:53.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.679+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.696+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.710+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.727+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.744+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.761+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.795+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.813+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.830+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.844+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.861+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.878+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.908+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.925+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.181 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.940+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.956+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.971+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:53.986+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.020+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.036+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.056+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.073+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.089+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.104+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.120+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.182 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.137+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.183 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.152+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.183 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.166+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.336 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:54.095+0000 7fa5fac127c0 -1 osd.4 434 log_to_monitors true 2024-03-20T08:07:54.410 INFO:tasks.rados.rados.0.smithi118.stdout:1888: done (1 left) 2024-03-20T08:07:54.410 INFO:tasks.rados.rados.0.smithi118.stdout:1889: rollback oid 25 current snap is 192 2024-03-20T08:07:54.410 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 25 to 164 2024-03-20T08:07:54.411 INFO:tasks.rados.rados.0.smithi118.stdout:1890: rmattr oid 5 current snap is 192 2024-03-20T08:07:54.411 INFO:tasks.rados.rados.0.smithi118.stdout:1891: read oid 23 snap 184 2024-03-20T08:07:54.411 INFO:tasks.rados.rados.0.smithi118.stdout:1891: expect deleted 2024-03-20T08:07:54.411 INFO:tasks.rados.rados.0.smithi118.stdout:1892: delete oid 9 current snap is 192 2024-03-20T08:07:54.411 INFO:tasks.rados.rados.0.smithi118.stdout:1887: finishing copy_from to smithi11881747-2 2024-03-20T08:07:54.411 INFO:tasks.rados.rados.0.smithi118.stdout:1887: got expected ENOENT (src dne) 2024-03-20T08:07:54.411 INFO:tasks.rados.rados.0.smithi118.stdout:1887: finishing copy_from racing read to smithi11881747-2 2024-03-20T08:07:54.417 INFO:tasks.rados.rados.0.smithi118.stdout:1889: finishing rollback tid 0 to smithi11881747-25 2024-03-20T08:07:54.417 INFO:tasks.rados.rados.0.smithi118.stdout:1889: finishing rollback tid 1 to smithi11881747-25 2024-03-20T08:07:54.417 INFO:tasks.rados.rados.0.smithi118.stdout:1889: finishing rollback tid 2 to smithi11881747-25 2024-03-20T08:07:54.417 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 7079 (ObjNum 493 snap 163 seq_num 493) dirty exists 2024-03-20T08:07:54.419 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 7080 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-03-20T08:07:54.420 INFO:tasks.rados.rados.0.smithi118.stdout:1887: done (4 left) 2024-03-20T08:07:54.420 INFO:tasks.rados.rados.0.smithi118.stdout:1889: done (3 left) 2024-03-20T08:07:54.420 INFO:tasks.rados.rados.0.smithi118.stdout:1890: done (2 left) 2024-03-20T08:07:54.420 INFO:tasks.rados.rados.0.smithi118.stdout:1891: done (1 left) 2024-03-20T08:07:54.420 INFO:tasks.rados.rados.0.smithi118.stdout:1892: done (0 left) 2024-03-20T08:07:54.420 INFO:tasks.rados.rados.0.smithi118.stdout:1893: read oid 42 snap -1 2024-03-20T08:07:54.420 INFO:tasks.rados.rados.0.smithi118.stdout:1893: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:07:54.421 INFO:tasks.rados.rados.0.smithi118.stdout:1894: rollback oid 47 current snap is 192 2024-03-20T08:07:54.421 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 189 2024-03-20T08:07:54.421 INFO:tasks.rados.rados.0.smithi118.stdout:1895: snap_create 2024-03-20T08:07:54.423 INFO:tasks.rados.rados.0.smithi118.stdout:1894: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:07:54.423 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 1099 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:07:54.430 INFO:tasks.rados.rados.0.smithi118.stdout:1893: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:07:54.515 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:54.410+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:54.515 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:54.410+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:54.516 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[93121]: osdmap e443: 8 total, 7 up, 8 in 2024-03-20T08:07:54.516 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:54.516 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[93121]: from='osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T08:07:54.516 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[93121]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T08:07:54.516 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[93121]: pgmap v343: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 569 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 100 KiB/s wr, 7 op/s; 216/1074 objects degraded (20.112%) 2024-03-20T08:07:54.516 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[95692]: osdmap e443: 8 total, 7 up, 8 in 2024-03-20T08:07:54.517 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:54.517 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[95692]: from='osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T08:07:54.517 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[95692]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T08:07:54.517 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-mon[95692]: pgmap v343: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 569 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 100 KiB/s wr, 7 op/s; 216/1074 objects degraded (20.112%) 2024-03-20T08:07:54.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.181+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.210+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.226+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.241+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.270+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.517 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.284+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.300+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.315+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.331+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.347+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.360+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.375+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.390+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.407+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.518 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.516+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:54.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:54 smithi196 ceph-mon[87080]: osdmap e443: 8 total, 7 up, 8 in 2024-03-20T08:07:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:54 smithi196 ceph-mon[87080]: from='osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T08:07:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:54 smithi196 ceph-mon[87080]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-20T08:07:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:54 smithi196 ceph-mon[87080]: pgmap v343: 121 pgs: 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 1 active+clean+snaptrim_error, 10 active+undersized+degraded, 3 active+undersized+degraded+snaptrim_error, 33 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 569 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 26 MiB/s rd, 100 KiB/s wr, 7 op/s; 216/1074 objects degraded (20.112%) 2024-03-20T08:07:54.796 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.701+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.797+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.052 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.814+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.831+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.846+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.863+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.878+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.893+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.910+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.925+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.972+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:54.988+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.005+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.021+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.038+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.053 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.052+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.306 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.068+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.306 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.083+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.101+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.119+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.133+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.149+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.164+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.178+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.210+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.224+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.242+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.258+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.276+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.291+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.307 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.306+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:54.902+0000 7fa5e8062700 -1 osd.4 434 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T08:07:55.412 INFO:tasks.rados.rados.0.smithi118.stdout:1893: done (2 left) 2024-03-20T08:07:55.412 INFO:tasks.rados.rados.0.smithi118.stdout:1894: done (1 left) 2024-03-20T08:07:55.412 INFO:tasks.rados.rados.0.smithi118.stdout:1895: done (0 left) 2024-03-20T08:07:55.412 INFO:tasks.rados.rados.0.smithi118.stdout:1896: read oid 4 snap 164 2024-03-20T08:07:55.412 INFO:tasks.rados.rados.0.smithi118.stdout:1896: expect (ObjNum 496 snap 164 seq_num 496) 2024-03-20T08:07:55.412 INFO:tasks.rados.rados.0.smithi118.stdout:1897: snap_remove snap 189 2024-03-20T08:07:55.570 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:55.420+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:55.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[93121]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-20T08:07:55.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[93121]: osdmap e444: 8 total, 7 up, 8 in 2024-03-20T08:07:55.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[93121]: from='osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:07:55.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[93121]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:07:55.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:55.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:55.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:55.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[93121]: from='osd.4 ' entity='osd.4' 2024-03-20T08:07:55.571 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.321+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.571 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.336+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.353+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.367+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.383+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.398+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.424+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.445+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.465+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.481+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.496+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.510+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.525+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.541+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.556+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[95692]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-20T08:07:55.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[95692]: osdmap e444: 8 total, 7 up, 8 in 2024-03-20T08:07:55.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[95692]: from='osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:07:55.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[95692]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:07:55.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:55.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:55.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:55.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-mon[95692]: from='osd.4 ' entity='osd.4' 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.571+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.586+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.600+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.616+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.630+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.647+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.662+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.677+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.693+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.836 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.707+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.837 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.723+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.837 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.740+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.837 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.755+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.837 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.771+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.837 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.787+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.837 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.804+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.837 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.819+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.837 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.835+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:55.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:55 smithi196 ceph-mon[87080]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-20T08:07:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:55 smithi196 ceph-mon[87080]: osdmap e444: 8 total, 7 up, 8 in 2024-03-20T08:07:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:55 smithi196 ceph-mon[87080]: from='osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:07:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:55 smithi196 ceph-mon[87080]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:07:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:55 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:55 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:55.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:55 smithi196 ceph-mon[87080]: from='osd.4 ' entity='osd.4' 2024-03-20T08:07:56.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.852+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.867+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.884+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.901+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.918+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.933+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.948+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.963+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.977+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:55.991+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.023+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.039+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.053+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.068+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.083+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.097+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.111+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.126+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.142+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.157+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.172+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.187+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.351 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.201+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.217+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.232+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.246+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.260+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.274+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.290+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.304+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.319+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.336+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.351+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.413 INFO:tasks.rados.rados.0.smithi118.stdout:1897: done (1 left) 2024-03-20T08:07:56.413 INFO:tasks.rados.rados.0.smithi118.stdout:1898: setattr oid 49 current snap is 193 2024-03-20T08:07:56.415 INFO:tasks.rados.rados.0.smithi118.stdout:1899: snap_remove snap 158 2024-03-20T08:07:56.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[93121]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:07:56.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[93121]: osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599] boot 2024-03-20T08:07:56.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[93121]: osdmap e445: 8 total, 8 up, 8 in 2024-03-20T08:07:56.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:07:56.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[93121]: pgmap v346: 121 pgs: 1 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 8 active+undersized+degraded, 2 active+undersized+degraded+snaptrim_error, 30 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 569 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 96 KiB/s wr, 11 op/s; 151/1092 objects degraded (13.828%) 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[95692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[95692]: osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599] boot 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[95692]: osdmap e445: 8 total, 8 up, 8 in 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:56.608 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:56.609 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:56.609 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-mon[95692]: pgmap v346: 121 pgs: 1 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 8 active+undersized+degraded, 2 active+undersized+degraded+snaptrim_error, 30 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 569 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 96 KiB/s wr, 11 op/s; 151/1092 objects degraded (13.828%) 2024-03-20T08:07:56.609 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:56.418+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:56.609 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:56.418+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:56.609 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:56.510+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.367+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.383+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.399+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.435+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.457+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.475+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.494+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.511+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.528+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.544+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.561+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.610 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.575+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.591+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.607+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.725 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:56 smithi196 ceph-mon[87080]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:07:56.725 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:56 smithi196 ceph-mon[87080]: osd.4 [v2:172.21.15.196:6800/502714599,v1:172.21.15.196:6801/502714599] boot 2024-03-20T08:07:56.725 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:56 smithi196 ceph-mon[87080]: osdmap e445: 8 total, 8 up, 8 in 2024-03-20T08:07:56.726 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:56 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-20T08:07:56.726 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:56.726 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:56 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:56.726 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:56 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:56.726 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:56 smithi196 ceph-mon[87080]: pgmap v346: 121 pgs: 1 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 1 active+clean+snaptrim_wait, 1 active+clean+snaptrim_wait+snaptrim_error, 8 active+undersized+degraded, 2 active+undersized+degraded+snaptrim_error, 30 active+undersized, 2 active+clean+snaptrim, 70 active+clean; 569 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 20 MiB/s rd, 96 KiB/s wr, 11 op/s; 151/1092 objects degraded (13.828%) 2024-03-20T08:07:56.864 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:56.787+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:56.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.625+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.642+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.658+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.674+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.691+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.707+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.726+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.747+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.764+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.781+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.800+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.817+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.833+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.849+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:56.865 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.865+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.114 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:56.918+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:57.115 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.884+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.115 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.900+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.115 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.917+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.115 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.934+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.115 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.949+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.115 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.965+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.115 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.979+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:56.995+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.010+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.024+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.040+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.069+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.084+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.100+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.116 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.115+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.372 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.130+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.372 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.146+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.372 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.161+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.176+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.190+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.206+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.220+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.235+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.250+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.265+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.281+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.373 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.311+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.374 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.327+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.374 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.341+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.374 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.357+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.374 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.372+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.415 INFO:tasks.rados.rados.0.smithi118.stdout:1899: done (2 left) 2024-03-20T08:07:57.415 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 193 2024-03-20T08:07:57.415 INFO:tasks.rados.rados.0.smithi118.stdout:1900: seq_num 589 ranges {0=270336} 2024-03-20T08:07:57.416 INFO:tasks.rados.rados.0.smithi118.stdout:1900: writing smithi11881747-47 from 0 to 270336 tid 1 2024-03-20T08:07:57.417 INFO:tasks.rados.rados.0.smithi118.stdout:1901: copy_from oid 19 from oid 28 current snap is 193 2024-03-20T08:07:57.417 INFO:tasks.rados.rados.0.smithi118.stdout:1902: rollback oid 11 current snap is 193 2024-03-20T08:07:57.417 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 164 2024-03-20T08:07:57.419 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 193 2024-03-20T08:07:57.419 INFO:tasks.rados.rados.0.smithi118.stdout:1903: seq_num 590 ranges {4751360=286720} 2024-03-20T08:07:57.446 INFO:tasks.rados.rados.0.smithi118.stdout:1903: writing smithi11881747-7 from 4751360 to 5038080 tid 1 2024-03-20T08:07:57.446 INFO:tasks.rados.rados.0.smithi118.stdout:1904: read oid 42 snap -1 2024-03-20T08:07:57.446 INFO:tasks.rados.rados.0.smithi118.stdout:1904: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:07:57.446 INFO:tasks.rados.rados.0.smithi118.stdout:1905: rollback oid 5 current snap is 193 2024-03-20T08:07:57.446 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 172 2024-03-20T08:07:57.446 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 7087 (ObjNum 65 snap 0 seq_num 1752460628) dirty exists 2024-03-20T08:07:57.446 INFO:tasks.rados.rados.0.smithi118.stdout:1896: expect (ObjNum 496 snap 164 seq_num 496) 2024-03-20T08:07:57.491 INFO:tasks.rados.rados.0.smithi118.stdout:1904: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:07:57.635 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:57.413+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:57.635 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:57.413+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:57.635 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-mon[93121]: osdmap e446: 8 total, 8 up, 8 in 2024-03-20T08:07:57.635 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:57.635 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:57.635 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:57.636 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-mon[95692]: osdmap e446: 8 total, 8 up, 8 in 2024-03-20T08:07:57.636 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:57.636 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:57.636 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:57.636 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:57.425+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.386+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.402+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.459+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.471+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.491+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.509+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.525+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.543+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.564+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.583+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.601+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.637 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.617+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.638 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:57.428+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:57.667 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-mon[87080]: osdmap e446: 8 total, 8 up, 8 in 2024-03-20T08:07:57.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:57.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:57.668 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:57.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.460+0000 7fa5d583d700 -1 osd.4 pg_epoch: 447 pg[5.1s0( v 446'422 (0'0,446'422] local-lis/les=445/446 n=9 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [4,3,7]p4(0) r=0 lpr=445 crt=446'422 lcod 442'420 mlcod 442'420 active+clean+snaptrim trimq=[da~1] ps=82] update_snap_map remove_oid 5:8a455596:::smithi11881747-30:db failed with -2 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.468+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.468+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.480+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.490+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.493+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.496+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.503+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.509+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.513+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.516+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.531+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.534+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.537+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.544+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.568+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.595+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.599+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.633+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.642+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.646+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.665+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1903: finishing write tid 1 to smithi11881747-7 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1902: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 1276 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1905: finishing rollback tid 0 to smithi11881747-5 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1903: finishing write tid 2 to smithi11881747-7 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1905: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1905: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 7084 (ObjNum 495 snap 164 seq_num 495) dirty exists 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1900: finishing write tid 1 to smithi11881747-47 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1900: finishing write tid 2 to smithi11881747-47 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1903: finishing write tid 3 to smithi11881747-7 2024-03-20T08:07:57.680 INFO:tasks.rados.rados.0.smithi118.stdout:1903: oid 7 updating version 0 to 20790 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1903: oid 7 version 20790 is already newer than 20789 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 20790 (ObjNum 590 snap 193 seq_num 590) dirty exists 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1903: left oid 7 (ObjNum 590 snap 193 seq_num 590) 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1900: finishing write tid 3 to smithi11881747-47 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1900: oid 47 updating version 0 to 1358 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1900: oid 47 version 1358 is already newer than 1357 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 1358 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1900: left oid 47 (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1901: finishing copy_from racing read to smithi11881747-19 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1901: finishing copy_from to smithi11881747-19 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 1355 (ObjNum 446 snap 148 seq_num 446) dirty exists 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1896: done (7 left) 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1898: done (6 left) 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1900: done (5 left) 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1901: done (4 left) 2024-03-20T08:07:57.681 INFO:tasks.rados.rados.0.smithi118.stdout:1902: done (3 left) 2024-03-20T08:07:57.682 INFO:tasks.rados.rados.0.smithi118.stdout:1903: done (2 left) 2024-03-20T08:07:57.682 INFO:tasks.rados.rados.0.smithi118.stdout:1904: done (1 left) 2024-03-20T08:07:57.682 INFO:tasks.rados.rados.0.smithi118.stdout:1905: done (0 left) 2024-03-20T08:07:57.682 INFO:tasks.rados.rados.0.smithi118.stdout:1906: snap_remove snap 180 2024-03-20T08:07:57.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.635+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.652+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.669+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.686+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.702+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.718+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.735+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.750+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.768+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.784+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.801+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.817+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.834+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.850+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.867+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.884+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.902+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:57.921 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.672+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.703+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.710+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.922 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.717+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.923 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.720+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.923 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.724+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.923 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.923 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.731+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.923 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.738+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.742+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.745+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.749+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.760+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.763+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.767+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.774+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.788+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.792+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.802+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.809+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.824+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.828+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.831+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.847+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.850+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.854+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.858+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.862+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.866+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.873+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.880+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.888+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.899+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.903+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.918+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:57.928 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.920+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.936+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.953+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.970+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:57.987+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.003+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.019+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.037+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.055+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.071+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.088+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.123+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.140+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.938+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.964+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.971+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.975+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.983+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:57.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.002+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.018+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.022+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.031+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.040+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.071+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.080+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.084+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.100+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.106+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.113+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.127+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.132+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.137+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.146+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.167+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.157+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.174+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.193+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.211+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.229+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.247+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.265+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.283+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.302+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.320+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.355+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.372+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.388+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.406+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.182+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.188+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.195+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.202+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.209+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.226+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.235+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.240+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.244+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.261+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.296+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.301+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.329+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.333+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.338+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.348+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.353+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.358+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.367+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.372+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.382+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.392+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.397+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.415+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.420+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.430 INFO:tasks.rados.rados.0.smithi118.stdout:1906: done (0 left) 2024-03-20T08:07:58.430 INFO:tasks.rados.rados.0.smithi118.stdout:1907: delete oid 29 current snap is 193 2024-03-20T08:07:58.436 INFO:tasks.rados.rados.0.smithi118.stdout:1907: done (0 left) 2024-03-20T08:07:58.436 INFO:tasks.rados.rados.0.smithi118.stdout:1908: setattr oid 44 current snap is 193 2024-03-20T08:07:58.438 INFO:tasks.rados.rados.0.smithi118.stdout:1909: rollback oid 18 current snap is 193 2024-03-20T08:07:58.438 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 18 to 190 2024-03-20T08:07:58.439 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 193 2024-03-20T08:07:58.439 INFO:tasks.rados.rados.0.smithi118.stdout:1910: seq_num 592 ranges {0=319488} 2024-03-20T08:07:58.441 INFO:tasks.rados.rados.0.smithi118.stdout:1910: writing smithi11881747-48 from 0 to 319488 tid 1 2024-03-20T08:07:58.441 INFO:tasks.rados.rados.0.smithi118.stdout:1911: read oid 1 snap -1 2024-03-20T08:07:58.441 INFO:tasks.rados.rados.0.smithi118.stdout:1911: expect (ObjNum 583 snap 188 seq_num 583) 2024-03-20T08:07:58.442 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 193 2024-03-20T08:07:58.442 INFO:tasks.rados.rados.0.smithi118.stdout:1912: seq_num 593 ranges {9895936=335872} 2024-03-20T08:07:58.499 INFO:tasks.rados.rados.0.smithi118.stdout:1912: writing smithi11881747-15 from 9895936 to 10231808 tid 1 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:1909: finishing rollback tid 0 to smithi11881747-18 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:1909: finishing rollback tid 1 to smithi11881747-18 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 7070 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:1909: finishing rollback tid 2 to smithi11881747-18 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 7090 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:1910: finishing write tid 1 to smithi11881747-48 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:1910: finishing write tid 2 to smithi11881747-48 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:1910: finishing write tid 3 to smithi11881747-48 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:1910: oid 48 updating version 0 to 7069 2024-03-20T08:07:58.500 INFO:tasks.rados.rados.0.smithi118.stdout:1910: oid 48 updating version 7069 to 7070 2024-03-20T08:07:58.501 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 7070 (ObjNum 592 snap 193 seq_num 592) dirty exists 2024-03-20T08:07:58.501 INFO:tasks.rados.rados.0.smithi118.stdout:1910: left oid 48 (ObjNum 592 snap 193 seq_num 592) 2024-03-20T08:07:58.501 INFO:tasks.rados.rados.0.smithi118.stdout:1911: expect (ObjNum 583 snap 188 seq_num 583) 2024-03-20T08:07:58.681 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: osdmap e447: 8 total, 8 up, 8 in 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: pgmap v349: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+undersized+degraded, 1 active+undersized+degraded+snaptrim_error, 26 active+undersized, 2 active+clean+snaptrim, 77 active+clean; 569 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 186 KiB/s wr, 7 op/s; 78/1095 objects degraded (7.123%); 11 MiB/s, 3 objects/s recovering 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:58.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.433+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.451+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.492+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.497+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.504+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.531+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.537+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.547+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.552+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.570+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.580+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.597+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.602+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.617+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.631+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.639+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.662+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.669+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.686 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.675+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.686 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.686 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:58.432+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:58.686 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:58.437+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:58.687 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:58.437+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:07:58.687 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:58.437+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:58.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:58.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: osdmap e447: 8 total, 8 up, 8 in 2024-03-20T08:07:58.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:58.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:58.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:58.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:58.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:58.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: pgmap v349: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+undersized+degraded, 1 active+undersized+degraded+snaptrim_error, 26 active+undersized, 2 active+clean+snaptrim, 77 active+clean; 569 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 186 KiB/s wr, 7 op/s; 78/1095 objects degraded (7.123%); 11 MiB/s, 3 objects/s recovering 2024-03-20T08:07:58.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:58.688 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:58.688 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:58.427+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: osdmap e447: 8 total, 8 up, 8 in 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: pgmap v349: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+undersized+degraded, 1 active+undersized+degraded+snaptrim_error, 26 active+undersized, 2 active+clean+snaptrim, 77 active+clean; 569 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 2.2 MiB/s rd, 186 KiB/s wr, 7 op/s; 78/1095 objects degraded (7.123%); 11 MiB/s, 3 objects/s recovering 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:07:58.689 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.426+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.459+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.484+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.503+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.522+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.539+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.557+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.575+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.593+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.690 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.611+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.630+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.649+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.667+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.691 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.686+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.936 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.687+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.936 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.936 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.721+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.733+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.751+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.758+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.764+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.770+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.776+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.937 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.787+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.793+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.818+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.831+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.843+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.852+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.865+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.938 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.871+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.877+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.883+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.889+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.897+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.903+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.918+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.924+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.939 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:58.954 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.704+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.724+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.744+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.763+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.782+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.802+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.820+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.838+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.858+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.878+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.897+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.917+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:58.955 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.936+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1912: finishing write tid 1 to smithi11881747-15 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1912: finishing write tid 2 to smithi11881747-15 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1908: done (4 left) 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1909: done (3 left) 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1910: done (2 left) 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1911: done (1 left) 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1913: delete oid 40 current snap is 193 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1912: finishing write tid 3 to smithi11881747-15 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1912: oid 15 updating version 0 to 1357 2024-03-20T08:07:59.005 INFO:tasks.rados.rados.0.smithi118.stdout:1912: oid 15 version 1357 is already newer than 1356 2024-03-20T08:07:59.006 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 1357 (ObjNum 593 snap 193 seq_num 593) dirty exists 2024-03-20T08:07:59.006 INFO:tasks.rados.rados.0.smithi118.stdout:1912: left oid 15 (ObjNum 593 snap 193 seq_num 593) 2024-03-20T08:07:59.024 INFO:tasks.rados.rados.0.smithi118.stdout:1912: done (1 left) 2024-03-20T08:07:59.024 INFO:tasks.rados.rados.0.smithi118.stdout:1913: done (0 left) 2024-03-20T08:07:59.024 INFO:tasks.rados.rados.0.smithi118.stdout:1914: snap_create 2024-03-20T08:07:59.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.955+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.961+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.968+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.975+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.993+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:58.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.003+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.053+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.080+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.086+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.119+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.125+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.132+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.138+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.150+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.184+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.191+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.207 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.955+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.975+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:58.991+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.009+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.027+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.044+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.062+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.080+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.098+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.116+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.153+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.170+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.208 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.189+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.209 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.431 INFO:tasks.rados.rados.0.smithi118.stdout:1914: done (0 left) 2024-03-20T08:07:59.432 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 194 2024-03-20T08:07:59.432 INFO:tasks.rados.rados.0.smithi118.stdout:1915: seq_num 595 ranges {5169152=368640} 2024-03-20T08:07:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 78/1095 objects degraded (7.123%), 5 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-mon[87080]: osdmap e448: 8 total, 8 up, 8 in 2024-03-20T08:07:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:59.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.198+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.231+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.253+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.259+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.266+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.287+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.301+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.307+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.326+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.339+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.345+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.351+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.367+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.381+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.395+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.408+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.463 INFO:tasks.rados.rados.0.smithi118.stdout:1915: writing smithi11881747-3 from 5169152 to 5537792 tid 1 2024-03-20T08:07:59.463 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:59.439+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:59.463 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:07:59.443+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:59.464 INFO:tasks.rados.rados.0.smithi118.stdout:1916: rollback oid 11 current snap is 194 2024-03-20T08:07:59.465 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 183 2024-03-20T08:07:59.465 INFO:tasks.rados.rados.0.smithi118.stdout:1917: rollback oid 26 current snap is 194 2024-03-20T08:07:59.465 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 183 2024-03-20T08:07:59.465 INFO:tasks.rados.rados.0.smithi118.stdout:1918: rollback oid 19 current snap is 194 2024-03-20T08:07:59.465 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 19 to 175 2024-03-20T08:07:59.465 INFO:tasks.rados.rados.0.smithi118.stdout:1919: snap_create 2024-03-20T08:07:59.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.225+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.243+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.260+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.277+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.313+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.330+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.350+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.367+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.384+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.401+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.418+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.441+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 78/1095 objects degraded (7.123%), 5 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:59.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[93121]: osdmap e448: 8 total, 8 up, 8 in 2024-03-20T08:07:59.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:59.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:59.470 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 78/1095 objects degraded (7.123%), 5 pgs degraded (PG_DEGRADED) 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[95692]: osdmap e448: 8 total, 8 up, 8 in 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:07:59.471 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:07:59.472 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:07:59.434+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:07:59.472 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:07:59.446+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:07:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:1918: finishing rollback tid 0 to smithi11881747-19 2024-03-20T08:07:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:1916: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:07:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 1276 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-03-20T08:07:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:1918: finishing rollback tid 1 to smithi11881747-19 2024-03-20T08:07:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:1917: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:07:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:1918: finishing rollback tid 2 to smithi11881747-19 2024-03-20T08:07:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 1360 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:07:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:1915: finishing write tid 1 to smithi11881747-3 2024-03-20T08:07:59.477 INFO:tasks.rados.rados.0.smithi118.stdout:1915: finishing write tid 2 to smithi11881747-3 2024-03-20T08:07:59.477 INFO:tasks.rados.rados.0.smithi118.stdout:1917: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:07:59.477 INFO:tasks.rados.rados.0.smithi118.stdout:1917: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:07:59.477 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 7073 (ObjNum 449 snap 149 seq_num 449) dirty exists 2024-03-20T08:07:59.481 INFO:tasks.rados.rados.0.smithi118.stdout:1915: finishing write tid 3 to smithi11881747-3 2024-03-20T08:07:59.482 INFO:tasks.rados.rados.0.smithi118.stdout:1915: oid 3 updating version 0 to 13591 2024-03-20T08:07:59.482 INFO:tasks.rados.rados.0.smithi118.stdout:1915: oid 3 version 13591 is already newer than 13590 2024-03-20T08:07:59.482 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 13591 (ObjNum 595 snap 194 seq_num 595) dirty exists 2024-03-20T08:07:59.482 INFO:tasks.rados.rados.0.smithi118.stdout:1915: left oid 3 (ObjNum 595 snap 194 seq_num 595) 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.443+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.443+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.459+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.473+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.490+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.497+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.504+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.511+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.518+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.526+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.533+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.545+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.552+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.559+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.582+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.596+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.611+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.618+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.631+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.645+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.667+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.674+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.690+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.464+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.485+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.502+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.519+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.537+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.555+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.578+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.595+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.614+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.632+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.651+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.669+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.723 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.686+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.705+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.724 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.723+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.705+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.718+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.734+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.767+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.774+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.783+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.792+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.800+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.815+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.826+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.832+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.837+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.847+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.852+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.857+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.862+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.882+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.887+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.903+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.917+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.939+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.944+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:07:59.981 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.742+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.760+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.797+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.815+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.833+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.850+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.867+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.884+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.901+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.919+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.935+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.982 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.950+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:07:59.983 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.966+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.966+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.971+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.993+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:07:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:07:59.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.004+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.060+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.071+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.077+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.082+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.087+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.092+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.103+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.115+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.121+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.127+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.143+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.149+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.154+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.160+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.176+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.181+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.193+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.199+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.204+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.981+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:07:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:07:59.998+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.015+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.031+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.047+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.064+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.099+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.115+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.133+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.148+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.248 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.164+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.182+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.198+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.215+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.249 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.231+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.440 INFO:tasks.rados.rados.0.smithi118.stdout:1915: done (4 left) 2024-03-20T08:08:00.440 INFO:tasks.rados.rados.0.smithi118.stdout:1916: done (3 left) 2024-03-20T08:08:00.440 INFO:tasks.rados.rados.0.smithi118.stdout:1917: done (2 left) 2024-03-20T08:08:00.440 INFO:tasks.rados.rados.0.smithi118.stdout:1918: done (1 left) 2024-03-20T08:08:00.440 INFO:tasks.rados.rados.0.smithi118.stdout:1919: done (0 left) 2024-03-20T08:08:00.440 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 195 2024-03-20T08:08:00.441 INFO:tasks.rados.rados.0.smithi118.stdout:1920: seq_num 596 ranges {6414336=385024} 2024-03-20T08:08:00.461 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-mon[87080]: osdmap e449: 8 total, 8 up, 8 in 2024-03-20T08:08:00.461 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:00.461 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:00.461 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:00.461 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:00.461 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-mon[87080]: pgmap v352: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 105 active+clean; 561 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 5.2 MiB/s rd, 738 KiB/s wr, 16 op/s; 25 MiB/s, 8 objects/s recovering 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.211+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.217+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.249+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.261+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.267+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.278+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.299+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.322+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.355+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.360+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.366+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.384+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.395+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.407+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.424+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.431+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.439+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.443+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.454+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.478 INFO:tasks.rados.rados.0.smithi118.stdout:1920: writing smithi11881747-38 from 6414336 to 6799360 tid 1 2024-03-20T08:08:00.479 INFO:tasks.rados.rados.0.smithi118.stdout:1921: snap_create 2024-03-20T08:08:00.484 INFO:tasks.rados.rados.0.smithi118.stdout:1920: finishing write tid 1 to smithi11881747-38 2024-03-20T08:08:00.484 INFO:tasks.rados.rados.0.smithi118.stdout:1920: finishing write tid 2 to smithi11881747-38 2024-03-20T08:08:00.485 INFO:tasks.rados.rados.0.smithi118.stdout:1920: finishing write tid 3 to smithi11881747-38 2024-03-20T08:08:00.485 INFO:tasks.rados.rados.0.smithi118.stdout:1920: oid 38 updating version 0 to 7071 2024-03-20T08:08:00.485 INFO:tasks.rados.rados.0.smithi118.stdout:1920: oid 38 updating version 7071 to 7072 2024-03-20T08:08:00.485 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7072 (ObjNum 596 snap 195 seq_num 596) dirty exists 2024-03-20T08:08:00.485 INFO:tasks.rados.rados.0.smithi118.stdout:1920: left oid 38 (ObjNum 596 snap 195 seq_num 596) 2024-03-20T08:08:00.512 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:00.449+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:00.512 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:00.451+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:00.512 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:00.451+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.248+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.265+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.282+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.298+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.314+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.332+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.365+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.381+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.397+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.413+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.431+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.459+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.477+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.514 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.495+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[95692]: osdmap e449: 8 total, 8 up, 8 in 2024-03-20T08:08:00.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:00.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:00.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:00.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:00.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[95692]: pgmap v352: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 105 active+clean; 561 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 5.2 MiB/s rd, 738 KiB/s wr, 16 op/s; 25 MiB/s, 8 objects/s recovering 2024-03-20T08:08:00.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:08:00.428+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 6 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:00.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[93121]: osdmap e449: 8 total, 8 up, 8 in 2024-03-20T08:08:00.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:00.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:00.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:00.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:00.515 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-mon[93121]: pgmap v352: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim_error, 5 peering, 1 peering+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 105 active+clean; 561 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 5.2 MiB/s rd, 738 KiB/s wr, 16 op/s; 25 MiB/s, 8 objects/s recovering 2024-03-20T08:08:00.515 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:00.452+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:00.717 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.469+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.492+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.498+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.514+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.520+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.539+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.718 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.546+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.719 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.719 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.562+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.719 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.568+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.719 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.574+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.719 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.719 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.719 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.628+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.636+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.643+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.651+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.658+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.664+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.695+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.702+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.710+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.717+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.775 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:00.555+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:00.775 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.513+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.530+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.548+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.569+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.587+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.604+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.622+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.638+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.655+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.672+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.690+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.707+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.725+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.742+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.776 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.759+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.777 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.776+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:00.972 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.725+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.972 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.972 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.747+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.761+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.769+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.849+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.857+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.873+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.881+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.919+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.934+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.957+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.965+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:00.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.032 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.793+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.810+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.844+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.862+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.879+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.896+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.912+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.947+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.963+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.981+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.033 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:00.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.034 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.015+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:00.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.004+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.017+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.025+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.031+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.038+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.046+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.060+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.083+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.091+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.098+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.113+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.121+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.129+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.150+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.158+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.180+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.197+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.227+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.032+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.050+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.067+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.085+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.292 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.102+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.119+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.135+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.153+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.170+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.188+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.223+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.239+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.257+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.274+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.293 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.292+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.448 INFO:tasks.rados.rados.0.smithi118.stdout:1920: done (1 left) 2024-03-20T08:08:01.448 INFO:tasks.rados.rados.0.smithi118.stdout:1921: done (0 left) 2024-03-20T08:08:01.448 INFO:tasks.rados.rados.0.smithi118.stdout:1922: setattr oid 29 current snap is 196 2024-03-20T08:08:01.449 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 196 2024-03-20T08:08:01.449 INFO:tasks.rados.rados.0.smithi118.stdout:1923: seq_num 597 ranges {327680=401408} 2024-03-20T08:08:01.453 INFO:tasks.rados.rados.0.smithi118.stdout:1923: writing smithi11881747-31 from 327680 to 729088 tid 1 2024-03-20T08:08:01.453 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 196 2024-03-20T08:08:01.454 INFO:tasks.rados.rados.0.smithi118.stdout:1924: seq_num 598 ranges {3661824=417792} 2024-03-20T08:08:01.477 INFO:tasks.rados.rados.0.smithi118.stdout:1924: writing smithi11881747-46 from 3661824 to 4079616 tid 1 2024-03-20T08:08:01.477 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 196 2024-03-20T08:08:01.477 INFO:tasks.rados.rados.0.smithi118.stdout:1925: seq_num 599 ranges {221184=434176} 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:1925: writing smithi11881747-14 from 221184 to 655360 tid 1 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 196 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1361 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:1923: finishing write tid 1 to smithi11881747-31 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:1923: finishing write tid 2 to smithi11881747-31 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:1923: finishing write tid 3 to smithi11881747-31 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:1923: oid 31 updating version 0 to 7088 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:1923: oid 31 version 7088 is already newer than 7087 2024-03-20T08:08:01.482 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 7088 (ObjNum 597 snap 196 seq_num 597) dirty exists 2024-03-20T08:08:01.483 INFO:tasks.rados.rados.0.smithi118.stdout:1923: left oid 31 (ObjNum 597 snap 196 seq_num 597) 2024-03-20T08:08:01.483 INFO:tasks.rados.rados.0.smithi118.stdout:1926: seq_num 600 ranges {8806400=450560} 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 6 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 78/1095 objects degraded (7.123%), 5 pgs degraded) 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: osdmap e450: 8 total, 8 up, 8 in 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:01.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-20T08:08:01.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-mon[87080]: Upgrade: osd.5 is safe to restart 2024-03-20T08:08:01.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.237+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.245+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.253+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.261+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.285+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.294+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.302+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.328+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.337+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.356+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.406+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.419+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.449+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.449+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:01.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.467+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.535 INFO:tasks.rados.rados.0.smithi118.stdout:1926: writing smithi11881747-12 from 8806400 to 9256960 tid 1 2024-03-20T08:08:01.535 INFO:tasks.rados.rados.0.smithi118.stdout:1922: done (4 left) 2024-03-20T08:08:01.535 INFO:tasks.rados.rados.0.smithi118.stdout:1923: done (3 left) 2024-03-20T08:08:01.535 INFO:tasks.rados.rados.0.smithi118.stdout:1927: rmattr oid 2 current snap is 196 2024-03-20T08:08:01.535 INFO:tasks.rados.rados.0.smithi118.stdout:1924: finishing write tid 1 to smithi11881747-46 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1924: finishing write tid 2 to smithi11881747-46 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1924: finishing write tid 3 to smithi11881747-46 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1924: oid 46 updating version 0 to 9319 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1924: oid 46 updating version 9319 to 9320 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 9320 (ObjNum 598 snap 196 seq_num 598) dirty exists 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1924: left oid 46 (ObjNum 598 snap 196 seq_num 598) 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1925: finishing write tid 1 to smithi11881747-14 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1925: finishing write tid 2 to smithi11881747-14 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1925: finishing write tid 3 to smithi11881747-14 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1925: oid 14 updating version 0 to 33886 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1925: oid 14 version 33886 is already newer than 33885 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 33886 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1925: left oid 14 (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1924: done (3 left) 2024-03-20T08:08:01.536 INFO:tasks.rados.rados.0.smithi118.stdout:1925: done (2 left) 2024-03-20T08:08:01.537 INFO:tasks.rados.rados.0.smithi118.stdout:1928: setattr oid 4 current snap is 196 2024-03-20T08:08:01.537 INFO:tasks.rados.rados.0.smithi118.stdout:1929: rollback oid 40 current snap is 196 2024-03-20T08:08:01.537 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 187 2024-03-20T08:08:01.537 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 196 2024-03-20T08:08:01.537 INFO:tasks.rados.rados.0.smithi118.stdout:1930: seq_num 601 ranges {3866624=475136} 2024-03-20T08:08:01.559 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:01.452+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:01.559 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:01.456+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:01.559 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:01.454+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 6 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 78/1095 objects degraded (7.123%), 5 pgs degraded) 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: osdmap e450: 8 total, 8 up, 8 in 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.560 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-20T08:08:01.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[93121]: Upgrade: osd.5 is safe to restart 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.310+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.328+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.346+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.362+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.380+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.398+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.414+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.431+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.458+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.480+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.501+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.519+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.536+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.559+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.563 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:01.463+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:01.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:01.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 6 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:01.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 78/1095 objects degraded (7.123%), 5 pgs degraded) 2024-03-20T08:08:01.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: osdmap e450: 8 total, 8 up, 8 in 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:01.564 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-20T08:08:01.565 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-20T08:08:01.565 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-mon[95692]: Upgrade: osd.5 is safe to restart 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:1930: writing smithi11881747-8 from 3866624 to 4341760 tid 1 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:1931: rollback oid 29 current snap is 196 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 183 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 9321 (ObjNum 553 snap 181 seq_num 553) dirty exists 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 1297 (ObjNum 33 snap 0 seq_num 323093968) dirty exists 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:1926: finishing write tid 1 to smithi11881747-12 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:1927: done (5 left) 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:1928: done (4 left) 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:1932: copy_from oid 38 from oid 21 current snap is 196 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:1929: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:08:01.566 INFO:tasks.rados.rados.0.smithi118.stdout:1926: finishing write tid 2 to smithi11881747-12 2024-03-20T08:08:01.567 INFO:tasks.rados.rados.0.smithi118.stdout:1933: delete oid 42 current snap is 196 2024-03-20T08:08:01.567 INFO:tasks.rados.rados.0.smithi118.stdout:1929: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:08:01.567 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 16304 (ObjNum 551 snap 180 seq_num 551) dirty exists 2024-03-20T08:08:01.567 INFO:tasks.rados.rados.0.smithi118.stdout:1926: finishing write tid 3 to smithi11881747-12 2024-03-20T08:08:01.567 INFO:tasks.rados.rados.0.smithi118.stdout:1926: oid 12 updating version 0 to 21706 2024-03-20T08:08:01.567 INFO:tasks.rados.rados.0.smithi118.stdout:1926: oid 12 updating version 21706 to 21707 2024-03-20T08:08:01.567 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 21707 (ObjNum 600 snap 196 seq_num 600) dirty exists 2024-03-20T08:08:01.567 INFO:tasks.rados.rados.0.smithi118.stdout:1926: left oid 12 (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:08:01.568 INFO:tasks.rados.rados.0.smithi118.stdout:1926: done (5 left) 2024-03-20T08:08:01.568 INFO:tasks.rados.rados.0.smithi118.stdout:1929: done (4 left) 2024-03-20T08:08:01.568 INFO:tasks.rados.rados.0.smithi118.stdout:1933: done (3 left) 2024-03-20T08:08:01.568 INFO:tasks.rados.rados.0.smithi118.stdout:1934: rmattr oid 27 current snap is 196 2024-03-20T08:08:01.568 INFO:tasks.rados.rados.0.smithi118.stdout:1935: copy_from oid 17 from oid 9 current snap is 196 2024-03-20T08:08:01.568 INFO:tasks.rados.rados.0.smithi118.stdout:1936: setattr oid 30 current snap is 196 2024-03-20T08:08:01.569 INFO:tasks.rados.rados.0.smithi118.stdout:1935: finishing copy_from to smithi11881747-17 2024-03-20T08:08:01.569 INFO:tasks.rados.rados.0.smithi118.stdout:1935: got expected ENOENT (src dne) 2024-03-20T08:08:01.569 INFO:tasks.rados.rados.0.smithi118.stdout:1935: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:08:01.569 INFO:tasks.rados.rados.0.smithi118.stdout:1935: done (5 left) 2024-03-20T08:08:01.569 INFO:tasks.rados.rados.0.smithi118.stdout:1937: snap_create 2024-03-20T08:08:01.573 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7091 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:08:01.575 INFO:tasks.rados.rados.0.smithi118.stdout:1931: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:08:01.575 INFO:tasks.rados.rados.0.smithi118.stdout:1931: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:08:01.577 INFO:tasks.rados.rados.0.smithi118.stdout:1930: finishing write tid 1 to smithi11881747-8 2024-03-20T08:08:01.577 INFO:tasks.rados.rados.0.smithi118.stdout:1930: finishing write tid 2 to smithi11881747-8 2024-03-20T08:08:01.578 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 1298 (ObjNum 582 snap 188 seq_num 582) dirty exists 2024-03-20T08:08:01.580 INFO:tasks.rados.rados.0.smithi118.stdout:1931: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:08:01.580 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 1364 (ObjNum 547 snap 180 seq_num 547) dirty exists 2024-03-20T08:08:01.582 INFO:tasks.rados.rados.0.smithi118.stdout:1930: finishing write tid 3 to smithi11881747-8 2024-03-20T08:08:01.582 INFO:tasks.rados.rados.0.smithi118.stdout:1930: oid 8 updating version 0 to 33888 2024-03-20T08:08:01.582 INFO:tasks.rados.rados.0.smithi118.stdout:1930: oid 8 updating version 33888 to 33889 2024-03-20T08:08:01.582 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 33889 (ObjNum 601 snap 196 seq_num 601) dirty exists 2024-03-20T08:08:01.582 INFO:tasks.rados.rados.0.smithi118.stdout:1930: left oid 8 (ObjNum 601 snap 196 seq_num 601) 2024-03-20T08:08:01.598 INFO:tasks.rados.rados.0.smithi118.stdout:1932: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:08:01.600 INFO:tasks.rados.rados.0.smithi118.stdout:1932: finishing copy_from to smithi11881747-38 2024-03-20T08:08:01.601 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7062 (ObjNum 545 snap 179 seq_num 545) dirty exists 2024-03-20T08:08:01.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.493+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.501+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.518+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.544+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.595+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.616+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.651+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.676+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.684+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.691+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.697+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.718+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.738+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.811 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.584+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.811 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.606+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.811 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.623+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.641+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.660+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.675+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.692+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.709+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.726+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.743+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.760+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.793+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.813 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.811+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:01.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.760+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.773+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.788+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.804+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.822+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.830+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.884+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.901+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.909+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.943+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.964+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.987+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:01.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:01.994+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.845+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.069 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.864+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.882+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.900+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.916+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.933+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.949+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.966+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:01.984+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.001+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.019+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.035+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.053+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.069+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.001+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.016+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.037+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.051+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.058+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.065+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.072+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.079+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.087+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.108+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.116+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.171+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.250 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.193+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.251 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.251 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.209+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.251 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.251 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.251 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.234+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.251 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.241+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.251 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.248+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.087+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.103+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.118+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.338 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.152+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.169+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.185+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.201+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.219+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.235+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.252+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.268+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.286+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.302+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.318+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.339 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.335+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.456 INFO:tasks.rados.rados.0.smithi118.stdout:1930: done (5 left) 2024-03-20T08:08:02.456 INFO:tasks.rados.rados.0.smithi118.stdout:1931: done (4 left) 2024-03-20T08:08:02.456 INFO:tasks.rados.rados.0.smithi118.stdout:1932: done (3 left) 2024-03-20T08:08:02.456 INFO:tasks.rados.rados.0.smithi118.stdout:1934: done (2 left) 2024-03-20T08:08:02.457 INFO:tasks.rados.rados.0.smithi118.stdout:1936: done (1 left) 2024-03-20T08:08:02.457 INFO:tasks.rados.rados.0.smithi118.stdout:1937: done (0 left) 2024-03-20T08:08:02.457 INFO:tasks.rados.rados.0.smithi118.stdout:1938: delete oid 24 current snap is 197 2024-03-20T08:08:02.469 INFO:tasks.rados.rados.0.smithi118.stdout:1938: done (0 left) 2024-03-20T08:08:02.469 INFO:tasks.rados.rados.0.smithi118.stdout:1939: rmattr oid 14 current snap is 197 2024-03-20T08:08:02.469 INFO:tasks.rados.rados.0.smithi118.stdout:1940: delete oid 7 current snap is 197 2024-03-20T08:08:02.475 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 33893 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:08:02.482 INFO:tasks.rados.rados.0.smithi118.stdout:1939: done (1 left) 2024-03-20T08:08:02.482 INFO:tasks.rados.rados.0.smithi118.stdout:1940: done (0 left) 2024-03-20T08:08:02.483 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 197 2024-03-20T08:08:02.483 INFO:tasks.rados.rados.0.smithi118.stdout:1941: seq_num 605 ranges {0=540672} 2024-03-20T08:08:02.486 INFO:tasks.rados.rados.0.smithi118.stdout:1941: writing smithi11881747-4 from 0 to 540672 tid 1 2024-03-20T08:08:02.486 INFO:tasks.rados.rados.0.smithi118.stdout:1942: snap_create 2024-03-20T08:08:02.491 INFO:tasks.rados.rados.0.smithi118.stdout:1941: finishing write tid 1 to smithi11881747-4 2024-03-20T08:08:02.491 INFO:tasks.rados.rados.0.smithi118.stdout:1941: finishing write tid 2 to smithi11881747-4 2024-03-20T08:08:02.492 INFO:tasks.rados.rados.0.smithi118.stdout:1941: finishing write tid 3 to smithi11881747-4 2024-03-20T08:08:02.492 INFO:tasks.rados.rados.0.smithi118.stdout:1941: oid 4 updating version 0 to 1299 2024-03-20T08:08:02.492 INFO:tasks.rados.rados.0.smithi118.stdout:1941: oid 4 updating version 1299 to 1300 2024-03-20T08:08:02.493 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 1300 (ObjNum 605 snap 197 seq_num 605) dirty exists 2024-03-20T08:08:02.493 INFO:tasks.rados.rados.0.smithi118.stdout:1941: left oid 4 (ObjNum 605 snap 197 seq_num 605) 2024-03-20T08:08:02.505 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:08:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:08:02.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:02.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-mon[87080]: osdmap e451: 8 total, 8 up, 8 in 2024-03-20T08:08:02.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:02.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:02.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:02.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:02.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:02.506 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-mon[87080]: pgmap v355: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 565 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 7.2 MiB/s rd, 824 KiB/s wr, 17 op/s; 23 MiB/s, 7 objects/s recovering 2024-03-20T08:08:02.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.283+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.289+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.327+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.358+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.366+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.396+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.403+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.440+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.460+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.476+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.597 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:02.466+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:02.597 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:02.467+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:02.597 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:02.467+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:02.597 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:02.465+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:02.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:02.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[95692]: osdmap e451: 8 total, 8 up, 8 in 2024-03-20T08:08:02.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:02.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:02.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[95692]: pgmap v355: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 565 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 7.2 MiB/s rd, 824 KiB/s wr, 17 op/s; 23 MiB/s, 7 objects/s recovering 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[93121]: osdmap e451: 8 total, 8 up, 8 in 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset f7=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:02.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:02.600 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-mon[93121]: pgmap v355: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 565 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 7.2 MiB/s rd, 824 KiB/s wr, 17 op/s; 23 MiB/s, 7 objects/s recovering 2024-03-20T08:08:02.600 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:02.457+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:02.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.352+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.369+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.386+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.403+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.420+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.438+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.600 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.459+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.488+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.506+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.524+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.541+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.560+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.576+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.601 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.594+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.529+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.546+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.604+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.611+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.617+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.651+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.660+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.670+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.762+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:02.850 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.612+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.850 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.628+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.645+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.662+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.679+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.696+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.712+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.729+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.746+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.763+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.780+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.797+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.815+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.833+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:02.851 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.850+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.015 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.015 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.782+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.816+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.835+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.844+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.854+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.862+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.871+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.881+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.903+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.958+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.977+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:02.997+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.020 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.020 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.108 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.868+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.887+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.903+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.920+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.937+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.954+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.970+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:02.987+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.004+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.022+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.040+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.057+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.074+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.109 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.091+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.110 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.109+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.024+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.035+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.044+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.083+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.092+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.128+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.146+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.174+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.193+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.246+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.256+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.267+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.128+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.145+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.163+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.182+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.199+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.217+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.234+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.252+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.376 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.268+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.286+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.303+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.321+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.340+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.377 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.358+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.460 INFO:tasks.rados.rados.0.smithi118.stdout:1941: done (1 left) 2024-03-20T08:08:03.460 INFO:tasks.rados.rados.0.smithi118.stdout:1942: done (0 left) 2024-03-20T08:08:03.461 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 198 2024-03-20T08:08:03.461 INFO:tasks.rados.rados.0.smithi118.stdout:1943: seq_num 606 ranges {3440640=557056} 2024-03-20T08:08:03.483 INFO:tasks.rados.rados.0.smithi118.stdout:1943: writing smithi11881747-45 from 3440640 to 3997696 tid 1 2024-03-20T08:08:03.483 INFO:tasks.rados.rados.0.smithi118.stdout:1944: snap_remove snap 195 2024-03-20T08:08:03.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: osdmap e452: 8 total, 8 up, 8 in 2024-03-20T08:08:03.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:03.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:03.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:03.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:03.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:03.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: Upgrade: Updating osd.5 2024-03-20T08:08:03.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:03.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T08:08:03.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:03.524 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-mon[87080]: Deploying daemon osd.5 on smithi196 2024-03-20T08:08:03.524 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.524 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.287+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.524 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.296+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.345+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.374+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.384+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.441+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.462+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.468+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.468+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.479+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.504+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.523+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.640 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:03.466+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:03.641 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:03.468+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:03.641 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:03.463+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:03.641 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.376+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.641 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.394+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.412+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.428+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.445+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.467+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.488+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.505+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.522+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.541+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.558+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.575+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.592+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.608+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.624+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.641+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.643 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:03.470+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:03.643 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:08:03] "GET /metrics HTTP/1.1" 200 36135 "" "Prometheus/2.43.0" 2024-03-20T08:08:03.643 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: osdmap e452: 8 total, 8 up, 8 in 2024-03-20T08:08:03.643 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: Upgrade: Updating osd.5 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[95692]: Deploying daemon osd.5 on smithi196 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: osdmap e452: 8 total, 8 up, 8 in 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:03.644 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: Upgrade: Updating osd.5 2024-03-20T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-20T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-mon[93121]: Deploying daemon osd.5 on smithi196 2024-03-20T08:08:03.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.556+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.580+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.588+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.595+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.604+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.621+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.629+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.656+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.671+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.713+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.731+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.740+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.749+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.757+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.782+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:03.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.660+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.679+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.697+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.730+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.747+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.764+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.781+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.797+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.812+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.828+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.846+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.869+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:03.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.885+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.800+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.837+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.875+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.885+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.893+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.902+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.919+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.927+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.945+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.962+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.971+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:03.997+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.154 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.902+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.154 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.919+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.154 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.935+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.154 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.952+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.154 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.968+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.154 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:03.985+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.154 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.001+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.018+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.035+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.053+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.068+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.085+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.102+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.136+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.155 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.154+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.060+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.069+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.104+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.113+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.121+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.129+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.148+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.191+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.200+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.209+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.245+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.405 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.171+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.405 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.189+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.405 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.206+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.405 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.221+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.405 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.238+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.405 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.255+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.405 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.272+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.288+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.305+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.322+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.339+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.355+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.371+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.406 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.388+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.466 INFO:tasks.rados.rados.0.smithi118.stdout:1944: done (1 left) 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1945: copy_from oid 25 from oid 40 current snap is 198 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1943: finishing write tid 1 to smithi11881747-45 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1943: finishing write tid 2 to smithi11881747-45 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1943: finishing write tid 3 to smithi11881747-45 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1943: oid 45 updating version 0 to 16954 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1943: oid 45 updating version 16954 to 16955 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 16955 (ObjNum 606 snap 198 seq_num 606) dirty exists 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1943: left oid 45 (ObjNum 606 snap 198 seq_num 606) 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1943: done (1 left) 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 198 2024-03-20T08:08:04.467 INFO:tasks.rados.rados.0.smithi118.stdout:1946: seq_num 607 ranges {5685248=573440} 2024-03-20T08:08:04.502 INFO:tasks.rados.rados.0.smithi118.stdout:1946: writing smithi11881747-32 from 5685248 to 6258688 tid 1 2024-03-20T08:08:04.503 INFO:tasks.rados.rados.0.smithi118.stdout:1947: snap_remove snap 196 2024-03-20T08:08:04.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-mon[87080]: osdmap e453: 8 total, 8 up, 8 in 2024-03-20T08:08:04.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:04.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:04.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:04.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:04.541 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:04.541 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-mon[87080]: pgmap v358: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 579 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 15 op/s; 8.6 MiB/s, 2 objects/s recovering 2024-03-20T08:08:04.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.299+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.309+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.337+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.391+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.459+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.464+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:04.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.521+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.540+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.660 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:04.469+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:04.661 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:04.469+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:04.661 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:04.470+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:04.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[93121]: osdmap e453: 8 total, 8 up, 8 in 2024-03-20T08:08:04.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:04.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:04.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:04.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:04.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:04.661 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[93121]: pgmap v358: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 579 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 15 op/s; 8.6 MiB/s, 2 objects/s recovering 2024-03-20T08:08:04.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[95692]: osdmap e453: 8 total, 8 up, 8 in 2024-03-20T08:08:04.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:04.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:04.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:04.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:04.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:04.662 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-mon[95692]: pgmap v358: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 579 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 15 op/s; 8.6 MiB/s, 2 objects/s recovering 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.405+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.422+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.440+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.457+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.510+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.527+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.543+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.560+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.576+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.593+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.609+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.627+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.663 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.645+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.664 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:04.480+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:04.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.560+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.587+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.596+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.606+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.614+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.633+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.670+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.697+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.734+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.754+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.763+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.790+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.800+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:04.913 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:04.818+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:04.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.661+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.677+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.693+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.710+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.726+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.913 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.743+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.759+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.776+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.793+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.810+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.829+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.845+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.862+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.879+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.897+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:04.914 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.913+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.820+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.830+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.858+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.867+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.888+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.908+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.918+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.950+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.959+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.970+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:04.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.015+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.047+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.056+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.169 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.930+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.947+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.966+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.982+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:04.999+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.017+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.035+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.072+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.088+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.104+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.121+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.137+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.171 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.153+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.171 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.170+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.066+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.085+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.095+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.124+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.134+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.176+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.214+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.224+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.233+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.254+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.276+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.186+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.203+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.221+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.238+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.255+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.273+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.291+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.308+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.325+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.342+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.359+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.375+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.391+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.408+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:1945: finishing copy_from racing read to smithi11881747-25 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:1945: finishing copy_from to smithi11881747-25 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 16304 (ObjNum 551 snap 180 seq_num 551) dirty exists 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:1946: finishing write tid 1 to smithi11881747-32 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:1946: finishing write tid 2 to smithi11881747-32 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:1946: finishing write tid 3 to smithi11881747-32 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:1946: oid 32 updating version 0 to 33899 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:1946: oid 32 updating version 33899 to 33900 2024-03-20T08:08:05.472 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 33900 (ObjNum 607 snap 198 seq_num 607) dirty exists 2024-03-20T08:08:05.473 INFO:tasks.rados.rados.0.smithi118.stdout:1946: left oid 32 (ObjNum 607 snap 198 seq_num 607) 2024-03-20T08:08:05.473 INFO:tasks.rados.rados.0.smithi118.stdout:1945: done (2 left) 2024-03-20T08:08:05.473 INFO:tasks.rados.rados.0.smithi118.stdout:1946: done (1 left) 2024-03-20T08:08:05.473 INFO:tasks.rados.rados.0.smithi118.stdout:1947: done (0 left) 2024-03-20T08:08:05.473 INFO:tasks.rados.rados.0.smithi118.stdout:1948: delete oid 8 current snap is 198 2024-03-20T08:08:05.489 INFO:tasks.rados.rados.0.smithi118.stdout:1948: done (0 left) 2024-03-20T08:08:05.489 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 198 2024-03-20T08:08:05.489 INFO:tasks.rados.rados.0.smithi118.stdout:1949: seq_num 609 ranges {2342912=606208} 2024-03-20T08:08:05.506 INFO:tasks.rados.rados.0.smithi118.stdout:1949: writing smithi11881747-23 from 2342912 to 2949120 tid 1 2024-03-20T08:08:05.506 INFO:tasks.rados.rados.0.smithi118.stdout:1950: snap_create 2024-03-20T08:08:05.511 INFO:tasks.rados.rados.0.smithi118.stdout:1949: finishing write tid 1 to smithi11881747-23 2024-03-20T08:08:05.511 INFO:tasks.rados.rados.0.smithi118.stdout:1949: finishing write tid 2 to smithi11881747-23 2024-03-20T08:08:05.514 INFO:tasks.rados.rados.0.smithi118.stdout:1949: finishing write tid 3 to smithi11881747-23 2024-03-20T08:08:05.515 INFO:tasks.rados.rados.0.smithi118.stdout:1949: oid 23 updating version 0 to 1366 2024-03-20T08:08:05.515 INFO:tasks.rados.rados.0.smithi118.stdout:1949: oid 23 updating version 1366 to 1367 2024-03-20T08:08:05.515 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 1367 (ObjNum 609 snap 198 seq_num 609) dirty exists 2024-03-20T08:08:05.515 INFO:tasks.rados.rados.0.smithi118.stdout:1949: left oid 23 (ObjNum 609 snap 198 seq_num 609) 2024-03-20T08:08:05.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:05.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-mon[87080]: osdmap e454: 8 total, 8 up, 8 in 2024-03-20T08:08:05.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:05.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:05.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:05.562 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:05.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.310+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.341+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.361+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.382+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.409+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.430+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.563 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.441+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.478+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.478+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.480+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.493+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.529+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.562+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.677 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:05.475+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:05.678 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:05.482+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:05.678 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:05.483+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:05.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:05.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[95692]: osdmap e454: 8 total, 8 up, 8 in 2024-03-20T08:08:05.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:05.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:05.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:05.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:05.679 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:05.486+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:05.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:05.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[93121]: osdmap e454: 8 total, 8 up, 8 in 2024-03-20T08:08:05.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:05.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:05.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:05.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.425+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.442+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.461+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.483+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.503+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.521+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.538+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.555+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.589+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.607+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.680 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.625+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.681 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.642+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.681 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.661+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.681 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.678+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.822 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.822 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.822 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.822 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.823 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.823 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.630+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.823 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.665+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.677+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.711+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.722+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.801+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.822+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:05.933 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.695+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.711+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.728+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.746+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.763+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.780+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.798+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.818+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.834+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.851+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.867+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.884+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.900+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.916+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:05.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.934+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.078 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.833+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.844+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.864+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.901+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.961+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.971+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:05.993+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.004+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.036+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.047+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.078+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.951+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.969+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:05.985+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.002+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.018+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.034+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.050+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.066+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.082+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.100+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.116+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.133+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.201 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.150+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.167+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.183+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.202 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.200+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.098+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.120+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.139+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.239+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.248+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.259+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.270+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.310+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.218+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.236+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.252+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.270+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.286+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.304+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.321+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.338+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.355+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.455 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.371+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.389+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.405+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.421+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.437+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.456 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.455+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.479 INFO:tasks.rados.rados.0.smithi118.stdout:1949: done (1 left) 2024-03-20T08:08:06.479 INFO:tasks.rados.rados.0.smithi118.stdout:1950: done (0 left) 2024-03-20T08:08:06.479 INFO:tasks.rados.rados.0.smithi118.stdout:1951: read oid 33 snap -1 2024-03-20T08:08:06.479 INFO:tasks.rados.rados.0.smithi118.stdout:1951: expect (ObjNum 580 snap 187 seq_num 580) 2024-03-20T08:08:06.479 INFO:tasks.rados.rados.0.smithi118.stdout:1952: read oid 6 snap -1 2024-03-20T08:08:06.479 INFO:tasks.rados.rados.0.smithi118.stdout:1952: expect (ObjNum 527 snap 170 seq_num 527) 2024-03-20T08:08:06.479 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 199 2024-03-20T08:08:06.479 INFO:tasks.rados.rados.0.smithi118.stdout:1953: seq_num 610 ranges {6266880=622592} 2024-03-20T08:08:06.518 INFO:tasks.rados.rados.0.smithi118.stdout:1953: writing smithi11881747-2 from 6266880 to 6889472 tid 1 2024-03-20T08:08:06.518 INFO:tasks.rados.rados.0.smithi118.stdout:1954: delete oid 8 current snap is 199 2024-03-20T08:08:06.518 INFO:tasks.rados.rados.0.smithi118.stdout:1952: expect (ObjNum 527 snap 170 seq_num 527) 2024-03-20T08:08:06.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-mon[87080]: osdmap e455: 8 total, 8 up, 8 in 2024-03-20T08:08:06.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:06.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:06.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:06.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:06.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:06.590 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-mon[87080]: pgmap v361: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 583 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.8 MiB/s wr, 27 op/s 2024-03-20T08:08:06.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.341+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.353+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.404+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.435+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.446+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.457+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.468+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.488+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.489+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.519+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.546+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.604 INFO:tasks.rados.rados.0.smithi118.stdout:1951: expect (ObjNum 580 snap 187 seq_num 580) 2024-03-20T08:08:06.722 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[93121]: osdmap e455: 8 total, 8 up, 8 in 2024-03-20T08:08:06.722 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:06.722 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:06.722 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:06.723 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:06.723 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:06.723 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[93121]: pgmap v361: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 583 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.8 MiB/s wr, 27 op/s 2024-03-20T08:08:06.723 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:06.486+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:06.723 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:06.486+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:06.723 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:06.486+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:06.723 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:06.493+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:06.724 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:06.479+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:06.724 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[95692]: osdmap e455: 8 total, 8 up, 8 in 2024-03-20T08:08:06.724 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:06.724 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:06.724 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:06.724 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:06.724 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:06.724 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-mon[95692]: pgmap v361: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 5 active+clean+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 110 active+clean; 583 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.8 MiB/s wr, 27 op/s 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.475+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.501+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.520+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.538+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.556+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.574+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.591+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.609+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.625+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.644+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.661+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.679+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.697+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.725 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.722+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.600+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.622+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.633+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.643+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.654+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.691+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.710+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.721+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.733+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.801+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.823+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.847+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:06.990 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.739+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.990 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.756+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.990 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.774+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.990 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.792+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.991 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.810+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.991 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.991 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.846+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.991 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.863+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.991 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.880+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.991 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.899+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.916+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.935+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.953+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.970+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:06.992 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:06.990+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.106 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:06 smithi196 systemd[1]: Stopping Ceph osd.5 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:08:07.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.872+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.898+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.925+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.986+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:06.998+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.022+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.033+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.082+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.095+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.255 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.008+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.023+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.044+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.063+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.083+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.104+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.125+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.143+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.258 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.161+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.178+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.215+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.234+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.259 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.265 INFO:tasks.rados.rados.0.smithi118.stdout:1953: finishing write tid 1 to smithi11881747-2 2024-03-20T08:08:07.265 INFO:tasks.rados.rados.0.smithi118.stdout:1953: finishing write tid 2 to smithi11881747-2 2024-03-20T08:08:07.265 INFO:tasks.rados.rados.0.smithi118.stdout:1953: finishing write tid 3 to smithi11881747-2 2024-03-20T08:08:07.265 INFO:tasks.rados.rados.0.smithi118.stdout:1953: oid 2 updating version 0 to 9322 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:1953: oid 2 updating version 9322 to 9323 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 9323 (ObjNum 610 snap 199 seq_num 610) dirty exists 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:1953: left oid 2 (ObjNum 610 snap 199 seq_num 610) 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:1951: done (3 left) 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:1952: done (2 left) 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:1953: done (1 left) 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:1954: done (0 left) 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 199 2024-03-20T08:08:07.266 INFO:tasks.rados.rados.0.smithi118.stdout:1955: seq_num 612 ranges {0=655360} 2024-03-20T08:08:07.270 INFO:tasks.rados.rados.0.smithi118.stdout:1955: writing smithi11881747-42 from 0 to 655360 tid 1 2024-03-20T08:08:07.271 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 199 2024-03-20T08:08:07.271 INFO:tasks.rados.rados.0.smithi118.stdout:1956: seq_num 613 ranges {2244608=671744} 2024-03-20T08:08:07.287 INFO:tasks.rados.rados.0.smithi118.stdout:1956: writing smithi11881747-39 from 2244608 to 2916352 tid 1 2024-03-20T08:08:07.288 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 199 2024-03-20T08:08:07.288 INFO:tasks.rados.rados.0.smithi118.stdout:1955: finishing write tid 1 to smithi11881747-42 2024-03-20T08:08:07.288 INFO:tasks.rados.rados.0.smithi118.stdout:1955: finishing write tid 2 to smithi11881747-42 2024-03-20T08:08:07.288 INFO:tasks.rados.rados.0.smithi118.stdout:1955: finishing write tid 3 to smithi11881747-42 2024-03-20T08:08:07.289 INFO:tasks.rados.rados.0.smithi118.stdout:1955: oid 42 updating version 0 to 16306 2024-03-20T08:08:07.289 INFO:tasks.rados.rados.0.smithi118.stdout:1955: oid 42 version 16306 is already newer than 16305 2024-03-20T08:08:07.289 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 16306 (ObjNum 612 snap 199 seq_num 612) dirty exists 2024-03-20T08:08:07.289 INFO:tasks.rados.rados.0.smithi118.stdout:1955: left oid 42 (ObjNum 612 snap 199 seq_num 612) 2024-03-20T08:08:07.289 INFO:tasks.rados.rados.0.smithi118.stdout:1957: seq_num 614 ranges {2949120=688128} 2024-03-20T08:08:07.310 INFO:tasks.rados.rados.0.smithi118.stdout:1957: writing smithi11881747-23 from 2949120 to 3637248 tid 1 2024-03-20T08:08:07.310 INFO:tasks.rados.rados.0.smithi118.stdout:1955: done (2 left) 2024-03-20T08:08:07.310 INFO:tasks.rados.rados.0.smithi118.stdout:1958: delete oid 1 current snap is 199 2024-03-20T08:08:07.310 INFO:tasks.rados.rados.0.smithi118.stdout:1956: finishing write tid 1 to smithi11881747-39 2024-03-20T08:08:07.310 INFO:tasks.rados.rados.0.smithi118.stdout:1956: finishing write tid 2 to smithi11881747-39 2024-03-20T08:08:07.311 INFO:tasks.rados.rados.0.smithi118.stdout:1956: finishing write tid 3 to smithi11881747-39 2024-03-20T08:08:07.311 INFO:tasks.rados.rados.0.smithi118.stdout:1956: oid 39 updating version 0 to 7092 2024-03-20T08:08:07.311 INFO:tasks.rados.rados.0.smithi118.stdout:1956: oid 39 updating version 7092 to 7093 2024-03-20T08:08:07.311 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7093 (ObjNum 613 snap 199 seq_num 613) dirty exists 2024-03-20T08:08:07.311 INFO:tasks.rados.rados.0.smithi118.stdout:1956: left oid 39 (ObjNum 613 snap 199 seq_num 613) 2024-03-20T08:08:07.314 INFO:tasks.rados.rados.0.smithi118.stdout:1956: done (2 left) 2024-03-20T08:08:07.314 INFO:tasks.rados.rados.0.smithi118.stdout:1958: done (1 left) 2024-03-20T08:08:07.314 INFO:tasks.rados.rados.0.smithi118.stdout:1959: rmattr oid 7 current snap is 199 2024-03-20T08:08:07.314 INFO:tasks.rados.rados.0.smithi118.stdout:1959: done (1 left) 2024-03-20T08:08:07.314 INFO:tasks.rados.rados.0.smithi118.stdout:1960: read oid 29 snap -1 2024-03-20T08:08:07.315 INFO:tasks.rados.rados.0.smithi118.stdout:1960: expect (ObjNum 547 snap 180 seq_num 547) 2024-03-20T08:08:07.315 INFO:tasks.rados.rados.0.smithi118.stdout:1961: snap_create 2024-03-20T08:08:07.315 INFO:tasks.rados.rados.0.smithi118.stdout:1957: finishing write tid 1 to smithi11881747-23 2024-03-20T08:08:07.315 INFO:tasks.rados.rados.0.smithi118.stdout:1957: finishing write tid 2 to smithi11881747-23 2024-03-20T08:08:07.328 INFO:tasks.rados.rados.0.smithi118.stdout:1957: finishing write tid 3 to smithi11881747-23 2024-03-20T08:08:07.328 INFO:tasks.rados.rados.0.smithi118.stdout:1957: oid 23 updating version 0 to 1370 2024-03-20T08:08:07.328 INFO:tasks.rados.rados.0.smithi118.stdout:1957: oid 23 version 1370 is already newer than 1369 2024-03-20T08:08:07.328 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 1370 (ObjNum 614 snap 199 seq_num 614) dirty exists 2024-03-20T08:08:07.328 INFO:tasks.rados.rados.0.smithi118.stdout:1957: left oid 23 (ObjNum 614 snap 199 seq_num 614) 2024-03-20T08:08:07.340 INFO:tasks.rados.rados.0.smithi118.stdout:1960: expect (ObjNum 547 snap 180 seq_num 547) 2024-03-20T08:08:07.358 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[42600]: 2024-03-20T08:08:07.218+0000 7f481e73f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:08:07.358 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[42600]: 2024-03-20T08:08:07.218+0000 7f481e73f700 -1 osd.5 456 *** Got signal Terminated *** 2024-03-20T08:08:07.358 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[42600]: 2024-03-20T08:08:07.218+0000 7f481e73f700 -1 osd.5 456 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.122+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.156+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.167+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.189+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.213+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.226+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.251+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.273+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.311+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.323+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.509 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:07.498+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:07.509 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:07.499+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:07.509 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[93121]: osdmap e456: 8 total, 8 up, 8 in 2024-03-20T08:08:07.509 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:07.509 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:07.509 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:07.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:07.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:07.510 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[93121]: osd.5 marked itself down and dead 2024-03-20T08:08:07.510 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.277+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.510 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.298+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.510 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.321+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.510 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.341+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.510 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.362+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.510 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.381+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.510 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.402+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.426+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.444+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.463+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.511 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.482+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:07.511 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:07.488+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:07.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[95692]: osdmap e456: 8 total, 8 up, 8 in 2024-03-20T08:08:07.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:07.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:07.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:07.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:07.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:07.512 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-mon[95692]: osd.5 marked itself down and dead 2024-03-20T08:08:07.512 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:07.509+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:07.558 INFO:tasks.rados.rados.0.smithi118.stdout:1957: done (2 left) 2024-03-20T08:08:07.558 INFO:tasks.rados.rados.0.smithi118.stdout:1960: done (1 left) 2024-03-20T08:08:07.558 INFO:tasks.rados.rados.0.smithi118.stdout:1961: done (0 left) 2024-03-20T08:08:07.558 INFO:tasks.rados.rados.0.smithi118.stdout:1962: read oid 33 snap -1 2024-03-20T08:08:07.558 INFO:tasks.rados.rados.0.smithi118.stdout:1962: expect (ObjNum 580 snap 187 seq_num 580) 2024-03-20T08:08:07.558 INFO:tasks.rados.rados.0.smithi118.stdout:1963: snap_remove snap 167 2024-03-20T08:08:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-mon[87080]: osdmap e456: 8 total, 8 up, 8 in 2024-03-20T08:08:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-mon[87080]: osd.5 marked itself down and dead 2024-03-20T08:08:07.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.386+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.411+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.480+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.495+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.495+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:07.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:07.510+0000 7fa5d1034700 -1 osd.4 pg_epoch: 457 pg[5.as1( v 457'1372 (0'0,457'1372] local-lis/les=445/446 n=54 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [2,4,3]p2(0) r=1 lpr=445 luod=0'0 crt=456'1370 mlcod 456'1370 active mbc={} ps=38] update_snap_map remove_oid 5:5ee214c4:::smithi11881747-29:9e failed with -2 2024-03-20T08:08:07.780 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.510+0000 7f831c6aa700 -1 osd.2 pg_epoch: 457 pg[5.as0( v 457'1372 (0'0,457'1372] local-lis/les=445/446 n=54 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [2,4,3]p2(0) r=0 lpr=445 luod=456'1370 crt=456'1370 lcod 456'1369 mlcod 456'1369 active+clean+snaptrim trimq=48 ps=38] update_snap_map remove_oid 5:5ee214c4:::smithi11881747-29:9e failed with -2 2024-03-20T08:08:07.780 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:07.516+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:07.781 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:07.511+0000 7f59e562e700 -1 osd.3 pg_epoch: 457 pg[5.as2( v 457'1372 (0'0,457'1372] local-lis/les=445/446 n=54 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [2,4,3]p2(0) r=2 lpr=445 luod=0'0 crt=456'1370 mlcod 456'1370 active mbc={} ps=38] update_snap_map remove_oid 5:5ee214c4:::smithi11881747-29:9e failed with -2 2024-03-20T08:08:08.476 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:08 smithi196 podman[97467]: 2024-03-20 08:08:08.218955064 +0000 UTC m=+1.122853918 container died 7f7229013587a89522748cb37269dada14322cd2a5edfab77281b9dfead92fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=reef-78dbc53, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:08:08.494 INFO:tasks.rados.rados.0.smithi118.stdout:1963: done (1 left) 2024-03-20T08:08:08.494 INFO:tasks.rados.rados.0.smithi118.stdout:1964: snap_create 2024-03-20T08:08:08.542 INFO:tasks.rados.rados.0.smithi118.stdout:1962: expect (ObjNum 580 snap 187 seq_num 580) 2024-03-20T08:08:08.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:08.497+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:08.757 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:08.505+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:08.757 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:08.505+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:08.757 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:08.507+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:08.758 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:08:08.758 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: osdmap e457: 8 total, 7 up, 8 in 2024-03-20T08:08:08.758 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:08.759 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:08.759 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:08.759 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:08.759 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:08.759 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:08.759 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: pgmap v364: 121 pgs: 7 peering, 17 stale+active+clean, 6 active+clean+snaptrim_error, 2 active+clean+snaptrim, 89 active+clean; 584 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 7.9 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2024-03-20T08:08:08.760 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:08.760 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:08.494+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:08.760 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:08.510+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: osdmap e457: 8 total, 7 up, 8 in 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: pgmap v364: 121 pgs: 7 peering, 17 stale+active+clean, 6 active+clean+snaptrim_error, 2 active+clean+snaptrim, 89 active+clean; 584 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 7.9 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2024-03-20T08:08:08.761 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:08 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:08.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:08.496+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: osdmap e457: 8 total, 7 up, 8 in 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: pgmap v364: 121 pgs: 7 peering, 17 stale+active+clean, 6 active+clean+snaptrim_error, 2 active+clean+snaptrim, 89 active+clean; 584 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 7.9 MiB/s rd, 1.3 MiB/s wr, 21 op/s 2024-03-20T08:08:08.796 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:08 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:09.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:08 smithi196 podman[97467]: 2024-03-20 08:08:08.797440993 +0000 UTC m=+1.701339850 container cleanup 7f7229013587a89522748cb37269dada14322cd2a5edfab77281b9dfead92fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=reef-78dbc53, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:08:09.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:08 smithi196 bash[97467]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5 2024-03-20T08:08:09.493 INFO:tasks.rados.rados.0.smithi118.stdout:1962: done (1 left) 2024-03-20T08:08:09.493 INFO:tasks.rados.rados.0.smithi118.stdout:1964: done (0 left) 2024-03-20T08:08:09.493 INFO:tasks.rados.rados.0.smithi118.stdout:1965: snap_remove snap 190 2024-03-20T08:08:09.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:09 smithi196 podman[97488]: 2024-03-20 08:08:09.257316745 +0000 UTC m=+1.033335630 container remove 7f7229013587a89522748cb37269dada14322cd2a5edfab77281b9dfead92fa7 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53) 2024-03-20T08:08:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:09 smithi196 ceph-mon[87080]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-20T08:08:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:09 smithi196 ceph-mon[87080]: osdmap e458: 8 total, 7 up, 8 in 2024-03-20T08:08:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:09.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:09.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:09.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:09.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:09.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[93121]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-20T08:08:09.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[93121]: osdmap e458: 8 total, 7 up, 8 in 2024-03-20T08:08:09.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:09.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:09.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:09.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:09.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:09.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[95692]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-20T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[95692]: osdmap e458: 8 total, 7 up, 8 in 2024-03-20T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:09.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:09.927 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:09.494+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:10.005 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:09 smithi196 podman[97584]: 2024-03-20 08:08:09.56940828 +0000 UTC m=+0.018406022 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:08:10.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:10.016+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:10.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:10.016+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:10.259 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:10 smithi196 podman[97584]: 2024-03-20 08:08:10.133896724 +0000 UTC m=+0.582894459 container create 4860c5a7d55a72b3096c4450659b7120a52c9f525c3d8ef4f49d7dcce6ea9385 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-20T08:08:10.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:10.019+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:10.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:10.020+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:10.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:10.021+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:10.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:10.025+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:10.498 INFO:tasks.rados.rados.0.smithi118.stdout:1965: done (0 left) 2024-03-20T08:08:10.499 INFO:tasks.rados.rados.0.smithi118.stdout:1966: read oid 38 snap 184 2024-03-20T08:08:10.499 INFO:tasks.rados.rados.0.smithi118.stdout:1966: expect (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:08:10.499 INFO:tasks.rados.rados.0.smithi118.stdout:1967: snap_create 2024-03-20T08:08:10.510 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:10 smithi196 podman[97584]: 2024-03-20 08:08:10.378600554 +0000 UTC m=+0.827598289 container init 4860c5a7d55a72b3096c4450659b7120a52c9f525c3d8ef4f49d7dcce6ea9385 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-20T08:08:10.510 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:10 smithi196 podman[97584]: 2024-03-20 08:08:10.384469395 +0000 UTC m=+0.833467129 container start 4860c5a7d55a72b3096c4450659b7120a52c9f525c3d8ef4f49d7dcce6ea9385 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:08:10.510 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:10 smithi196 podman[97584]: 2024-03-20 08:08:10.444938228 +0000 UTC m=+0.893935963 container attach 4860c5a7d55a72b3096c4450659b7120a52c9f525c3d8ef4f49d7dcce6ea9385 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=) 2024-03-20T08:08:10.510 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-mon[87080]: osdmap e459: 8 total, 7 up, 8 in 2024-03-20T08:08:10.510 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:10.510 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:10.510 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:10.510 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:10.510 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:10.511 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-mon[87080]: pgmap v367: 121 pgs: 23 active+undersized, 6 active+undersized+degraded, 7 peering, 4 stale+active+clean, 6 active+clean+snaptrim_error, 1 active+clean+snaptrim, 74 active+clean; 583 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 9.3 MiB/s rd, 908 KiB/s wr, 9 op/s; 64/1170 objects degraded (5.470%) 2024-03-20T08:08:10.534 INFO:tasks.rados.rados.0.smithi118.stdout:1966: expect (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:08:10.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:10.508+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:10.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:10 smithi196 podman[97584]: 2024-03-20 08:08:10.689507417 +0000 UTC m=+1.138505155 container died 4860c5a7d55a72b3096c4450659b7120a52c9f525c3d8ef4f49d7dcce6ea9385 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:08:10.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:10.507+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:10.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:10.507+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:10.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:10.507+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:10.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:10.507+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[93121]: osdmap e459: 8 total, 7 up, 8 in 2024-03-20T08:08:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:10.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[93121]: pgmap v367: 121 pgs: 23 active+undersized, 6 active+undersized+degraded, 7 peering, 4 stale+active+clean, 6 active+clean+snaptrim_error, 1 active+clean+snaptrim, 74 active+clean; 583 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 9.3 MiB/s rd, 908 KiB/s wr, 9 op/s; 64/1170 objects degraded (5.470%) 2024-03-20T08:08:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[95692]: osdmap e459: 8 total, 7 up, 8 in 2024-03-20T08:08:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-mon[95692]: pgmap v367: 121 pgs: 23 active+undersized, 6 active+undersized+degraded, 7 peering, 4 stale+active+clean, 6 active+clean+snaptrim_error, 1 active+clean+snaptrim, 74 active+clean; 583 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 9.3 MiB/s rd, 908 KiB/s wr, 9 op/s; 64/1170 objects degraded (5.470%) 2024-03-20T08:08:10.925 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:10.512+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:10.925 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:10.503+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:1966: done (1 left) 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:1967: done (0 left) 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:1968: rollback oid 2 current snap is 202 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 164 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:1969: rollback oid 12 current snap is 202 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 12 to 200 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:1970: rmattr oid 9 current snap is 202 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:1970: done (2 left) 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 202 2024-03-20T08:08:11.499 INFO:tasks.rados.rados.0.smithi118.stdout:1971: seq_num 616 ranges {0=720896} 2024-03-20T08:08:11.500 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 podman[97630]: 2024-03-20 08:08:11.248389947 +0000 UTC m=+0.554025338 container remove 4860c5a7d55a72b3096c4450659b7120a52c9f525c3d8ef4f49d7dcce6ea9385 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-20T08:08:11.500 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-20T08:08:11.500 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 systemd[1]: Stopped Ceph osd.5 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:08:11.500 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.5.service: Consumed 1min 56.123s CPU time. 2024-03-20T08:08:11.500 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 systemd[1]: Starting Ceph osd.5 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:08:11.503 INFO:tasks.rados.rados.0.smithi118.stdout:1971: writing smithi11881747-34 from 0 to 720896 tid 1 2024-03-20T08:08:11.504 INFO:tasks.rados.rados.0.smithi118.stdout:1972: read oid 13 snap -1 2024-03-20T08:08:11.504 INFO:tasks.rados.rados.0.smithi118.stdout:1972: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:08:11.505 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 202 2024-03-20T08:08:11.505 INFO:tasks.rados.rados.0.smithi118.stdout:1973: seq_num 617 ranges {0=737280} 2024-03-20T08:08:11.508 INFO:tasks.rados.rados.0.smithi118.stdout:1973: writing smithi11881747-24 from 0 to 737280 tid 1 2024-03-20T08:08:11.509 INFO:tasks.rados.rados.0.smithi118.stdout:1974: setattr oid 9 current snap is 202 2024-03-20T08:08:11.509 INFO:tasks.rados.rados.0.smithi118.stdout:1968: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:08:11.510 INFO:tasks.rados.rados.0.smithi118.stdout:1968: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:08:11.510 INFO:tasks.rados.rados.0.smithi118.stdout:1968: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:08:11.510 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 9326 (ObjNum 299 snap 99 seq_num 299) dirty exists 2024-03-20T08:08:11.511 INFO:tasks.rados.rados.0.smithi118.stdout:1968: done (5 left) 2024-03-20T08:08:11.511 INFO:tasks.rados.rados.0.smithi118.stdout:1975: snap_remove snap 187 2024-03-20T08:08:11.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:11.512+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:11.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:11.512+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: Health check failed: Degraded data redundancy: 64/1170 objects degraded (5.470%), 6 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: osdmap e460: 8 total, 7 up, 8 in 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:11.810 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:11.811 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 podman[97737]: 2024-03-20 08:08:11.592310839 +0000 UTC m=+0.018864671 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:08:11.811 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 podman[97737]: 2024-03-20 08:08:11.708059357 +0000 UTC m=+0.134613188 container create 1310069f7ff83e5ec5e451564e3cabe26a8131968174e604b341c09f7555ee8b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:08:11.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:11.511+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:11.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:11.518+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:11.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:11.520+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:11.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:11.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: Health check failed: Degraded data redundancy: 64/1170 objects degraded (5.470%), 6 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:11.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: osdmap e460: 8 total, 7 up, 8 in 2024-03-20T08:08:11.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:11.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:11.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:11.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:11.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:11.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:11.925 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:11.498+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:11.926 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:11.518+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:11.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:11.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: Health check failed: Degraded data redundancy: 64/1170 objects degraded (5.470%), 6 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:11.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: osdmap e460: 8 total, 7 up, 8 in 2024-03-20T08:08:11.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:11.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:11.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:11.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:11.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:11.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:12.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 podman[97737]: 2024-03-20 08:08:11.927404042 +0000 UTC m=+0.353957875 container init 1310069f7ff83e5ec5e451564e3cabe26a8131968174e604b341c09f7555ee8b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-20T08:08:12.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:11 smithi196 podman[97737]: 2024-03-20 08:08:11.93358718 +0000 UTC m=+0.360141010 container start 1310069f7ff83e5ec5e451564e3cabe26a8131968174e604b341c09f7555ee8b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-03-20T08:08:12.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 podman[97737]: 2024-03-20 08:08:12.018967143 +0000 UTC m=+0.445520988 container attach 1310069f7ff83e5ec5e451564e3cabe26a8131968174e604b341c09f7555ee8b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:08:12.511 INFO:tasks.rados.rados.0.smithi118.stdout:1975: done (5 left) 2024-03-20T08:08:12.511 INFO:tasks.rados.rados.0.smithi118.stdout:1976: copy_from oid 23 from oid 6 current snap is 202 2024-03-20T08:08:12.511 INFO:tasks.rados.rados.0.smithi118.stdout:1973: finishing write tid 1 to smithi11881747-24 2024-03-20T08:08:12.511 INFO:tasks.rados.rados.0.smithi118.stdout:1971: finishing write tid 1 to smithi11881747-34 2024-03-20T08:08:12.511 INFO:tasks.rados.rados.0.smithi118.stdout:1971: finishing write tid 2 to smithi11881747-34 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1973: finishing write tid 2 to smithi11881747-24 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1969: finishing rollback tid 0 to smithi11881747-12 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1969: finishing rollback tid 1 to smithi11881747-12 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1969: finishing rollback tid 2 to smithi11881747-12 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 23061 (ObjNum 600 snap 196 seq_num 600) dirty exists 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1971: finishing write tid 3 to smithi11881747-34 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1971: oid 34 updating version 0 to 7050 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1971: oid 34 version 7050 is already newer than 7049 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 7050 (ObjNum 616 snap 202 seq_num 616) dirty exists 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1971: left oid 34 (ObjNum 616 snap 202 seq_num 616) 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1973: finishing write tid 3 to smithi11881747-24 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1973: oid 24 updating version 0 to 7076 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:1973: oid 24 version 7076 is already newer than 7075 2024-03-20T08:08:12.512 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 7076 (ObjNum 617 snap 202 seq_num 617) dirty exists 2024-03-20T08:08:12.513 INFO:tasks.rados.rados.0.smithi118.stdout:1973: left oid 24 (ObjNum 617 snap 202 seq_num 617) 2024-03-20T08:08:12.513 INFO:tasks.rados.rados.0.smithi118.stdout:1972: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:08:12.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:12.514+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:08:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:08:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-mon[87080]: osdmap e461: 8 total, 7 up, 8 in 2024-03-20T08:08:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-mon[87080]: pgmap v370: 121 pgs: 34 active+undersized, 8 active+undersized+degraded, 7 peering, 6 active+clean+snaptrim_error, 1 active+clean+snaptrim, 65 active+clean; 584 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.4 MiB/s rd, 476 KiB/s wr, 3 op/s; 78/1170 objects degraded (6.667%) 2024-03-20T08:08:12.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate[97752]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-20T08:08:12.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate[97752]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-20T08:08:12.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 bash[97737]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-20T08:08:12.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 bash[97737]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 bash[97737]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 bash[97737]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 bash[97737]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 bash[97737]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 bash[97737]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate[97752]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate[97752]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate[97752]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-5/block 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate[97752]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate[97752]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-20T08:08:12.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:12 smithi196 podman[97737]: 2024-03-20 08:08:12.39603209 +0000 UTC m=+0.822585920 container died 1310069f7ff83e5ec5e451564e3cabe26a8131968174e604b341c09f7555ee8b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-03-20T08:08:12.704 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 16307 (ObjNum 33 snap 0 seq_num 291747632) dirty exists 2024-03-20T08:08:12.704 INFO:tasks.rados.rados.0.smithi118.stdout:1969: done (5 left) 2024-03-20T08:08:12.704 INFO:tasks.rados.rados.0.smithi118.stdout:1971: done (4 left) 2024-03-20T08:08:12.704 INFO:tasks.rados.rados.0.smithi118.stdout:1972: done (3 left) 2024-03-20T08:08:12.704 INFO:tasks.rados.rados.0.smithi118.stdout:1973: done (2 left) 2024-03-20T08:08:12.704 INFO:tasks.rados.rados.0.smithi118.stdout:1974: done (1 left) 2024-03-20T08:08:12.704 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 202 2024-03-20T08:08:12.704 INFO:tasks.rados.rados.0.smithi118.stdout:1976: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:08:12.705 INFO:tasks.rados.rados.0.smithi118.stdout:1976: finishing copy_from to smithi11881747-23 2024-03-20T08:08:12.705 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 1302 (ObjNum 527 snap 170 seq_num 527) dirty exists 2024-03-20T08:08:12.705 INFO:tasks.rados.rados.0.smithi118.stdout:1977: seq_num 618 ranges {6766592=753664} 2024-03-20T08:08:12.746 INFO:tasks.rados.rados.0.smithi118.stdout:1977: writing smithi11881747-33 from 6766592 to 7520256 tid 1 2024-03-20T08:08:12.746 INFO:tasks.rados.rados.0.smithi118.stdout:1976: done (1 left) 2024-03-20T08:08:12.746 INFO:tasks.rados.rados.0.smithi118.stdout:1978: copy_from oid 3 from oid 24 current snap is 202 2024-03-20T08:08:12.747 INFO:tasks.rados.rados.0.smithi118.stdout:1979: delete oid 46 current snap is 202 2024-03-20T08:08:12.749 INFO:tasks.rados.rados.0.smithi118.stdout:1979: done (2 left) 2024-03-20T08:08:12.750 INFO:tasks.rados.rados.0.smithi118.stdout:1980: copy_from oid 47 from oid 14 current snap is 202 2024-03-20T08:08:12.750 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 202 2024-03-20T08:08:12.750 INFO:tasks.rados.rados.0.smithi118.stdout:1981: seq_num 620 ranges {327680=794624} 2024-03-20T08:08:12.757 INFO:tasks.rados.rados.0.smithi118.stdout:1981: writing smithi11881747-26 from 327680 to 1122304 tid 1 2024-03-20T08:08:12.757 INFO:tasks.rados.rados.0.smithi118.stdout:1982: delete oid 30 current snap is 202 2024-03-20T08:08:12.758 INFO:tasks.rados.rados.0.smithi118.stdout:1980: finishing copy_from racing read to smithi11881747-47 2024-03-20T08:08:12.758 INFO:tasks.rados.rados.0.smithi118.stdout:1980: finishing copy_from to smithi11881747-47 2024-03-20T08:08:12.758 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33893 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:08:12.758 INFO:tasks.rados.rados.0.smithi118.stdout:1977: finishing write tid 1 to smithi11881747-33 2024-03-20T08:08:12.759 INFO:tasks.rados.rados.0.smithi118.stdout:1977: finishing write tid 2 to smithi11881747-33 2024-03-20T08:08:12.761 INFO:tasks.rados.rados.0.smithi118.stdout:1980: done (4 left) 2024-03-20T08:08:12.761 INFO:tasks.rados.rados.0.smithi118.stdout:1982: done (3 left) 2024-03-20T08:08:12.761 INFO:tasks.rados.rados.0.smithi118.stdout:1983: snap_remove snap 194 2024-03-20T08:08:12.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:12.514+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:12.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:12.512+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:12.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:12.518+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:12.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:12.518+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:12.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:12.519+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:12.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:12.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[93121]: osdmap e461: 8 total, 7 up, 8 in 2024-03-20T08:08:12.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:12.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:12.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:12.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:12.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:12.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[93121]: pgmap v370: 121 pgs: 34 active+undersized, 8 active+undersized+degraded, 7 peering, 6 active+clean+snaptrim_error, 1 active+clean+snaptrim, 65 active+clean; 584 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.4 MiB/s rd, 476 KiB/s wr, 3 op/s; 78/1170 objects degraded (6.667%) 2024-03-20T08:08:12.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:12.517+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:12.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:12.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[95692]: osdmap e461: 8 total, 7 up, 8 in 2024-03-20T08:08:12.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:12.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:12.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:12.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:12.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:12.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:12 smithi118 ceph-mon[95692]: pgmap v370: 121 pgs: 34 active+undersized, 8 active+undersized+degraded, 7 peering, 6 active+clean+snaptrim_error, 1 active+clean+snaptrim, 65 active+clean; 584 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.4 MiB/s rd, 476 KiB/s wr, 3 op/s; 78/1170 objects degraded (6.667%) 2024-03-20T08:08:13.289 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:13 smithi196 podman[97888]: 2024-03-20 08:08:13.0148477 +0000 UTC m=+0.613420433 container remove 1310069f7ff83e5ec5e451564e3cabe26a8131968174e604b341c09f7555ee8b (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-20T08:08:13.289 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:13 smithi196 podman[97961]: 2024-03-20 08:08:13.128583359 +0000 UTC m=+0.017776825 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:08:13.289 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:13 smithi196 podman[97961]: 2024-03-20 08:08:13.290887506 +0000 UTC m=+0.180080980 container create 72b569d10541a50f36b451da230a9333e61de5701646e3ac26a95121d021ecab (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:08:13.519 INFO:tasks.rados.rados.0.smithi118.stdout:1983: done (3 left) 2024-03-20T08:08:13.519 INFO:tasks.rados.rados.0.smithi118.stdout:1984: read oid 18 snap -1 2024-03-20T08:08:13.519 INFO:tasks.rados.rados.0.smithi118.stdout:1984: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:08:13.519 INFO:tasks.rados.rados.0.smithi118.stdout:1985: snap_remove snap 185 2024-03-20T08:08:13.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:13.525+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:13.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:13.526+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: osdmap e462: 8 total, 7 up, 8 in 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:13.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:13.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:13 smithi196 podman[97961]: 2024-03-20 08:08:13.493064476 +0000 UTC m=+0.382257944 container init 72b569d10541a50f36b451da230a9333e61de5701646e3ac26a95121d021ecab (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:08:13.593 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:13 smithi196 podman[97961]: 2024-03-20 08:08:13.498316283 +0000 UTC m=+0.387509751 container start 72b569d10541a50f36b451da230a9333e61de5701646e3ac26a95121d021ecab (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-20T08:08:13.593 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:13 smithi196 ceph-osd[97999]: -- 172.21.15.196:0/1914905131 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55635287b680 con 0x556352844c00 2024-03-20T08:08:13.748 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:08:13.888 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:13 smithi196 bash[97961]: 72b569d10541a50f36b451da230a9333e61de5701646e3ac26a95121d021ecab 2024-03-20T08:08:13.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:13 smithi196 systemd[1]: Started Ceph osd.5 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:08:13.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:13.529+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:13.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:13.529+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:08:13] "GET /metrics HTTP/1.1" 200 36337 "" "Prometheus/2.43.0" 2024-03-20T08:08:13.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:13.530+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: osdmap e462: 8 total, 7 up, 8 in 2024-03-20T08:08:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:13.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:13.529+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:13.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: osdmap e462: 8 total, 7 up, 8 in 2024-03-20T08:08:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:13.925 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:13.518+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (7m) 58s ago 8m 27.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (8m) 19s ago 8m 88.8M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (9m) 58s ago 9m 15.4M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (9m) 19s ago 9m 21.0M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (4m) 19s ago 13m 452M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (5m) 58s ago 15m 508M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (4m) 58s ago 15m 56.2M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (3m) 19s ago 13m 45.5M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:08:14.166 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (4m) 58s ago 13m 51.1M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (8m) 58s ago 8m 15.5M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (8m) 19s ago 8m 16.0M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (2m) 58s ago 13m 334M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (2m) 58s ago 12m 299M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (96s) 58s ago 12m 251M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (63s) 58s ago 11m 12.7M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (25s) 19s ago 11m 14.9M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 starting - - - 2480M 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (10m) 19s ago 10m 236M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (10m) 19s ago 10m 223M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:08:14.167 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (4m) 19s ago 9m 60.7M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:08:14.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:14.054+0000 7ffa935427c0 -1 Falling back to public interface 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:1985: done (4 left) 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:1986: rollback oid 34 current snap is 202 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 34 to 188 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:1977: finishing write tid 3 to smithi11881747-33 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:1977: oid 33 updating version 0 to 13594 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:1977: oid 33 version 13594 is already newer than 13593 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 13594 (ObjNum 618 snap 202 seq_num 618) dirty exists 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:1977: left oid 33 (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:08:14.520 INFO:tasks.rados.rados.0.smithi118.stdout:1981: finishing write tid 1 to smithi11881747-26 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:1981: finishing write tid 2 to smithi11881747-26 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:1981: finishing write tid 3 to smithi11881747-26 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:1981: oid 26 updating version 0 to 7078 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:1981: oid 26 version 7078 is already newer than 7077 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 7078 (ObjNum 620 snap 202 seq_num 620) dirty exists 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:1981: left oid 26 (ObjNum 620 snap 202 seq_num 620) 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:1978: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:1978: finishing copy_from to smithi11881747-3 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 7076 (ObjNum 617 snap 202 seq_num 617) dirty exists 2024-03-20T08:08:14.521 INFO:tasks.rados.rados.0.smithi118.stdout:1984: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:08:14.580 INFO:tasks.rados.rados.0.smithi118.stdout:1986: finishing rollback tid 0 to smithi11881747-34 2024-03-20T08:08:14.580 INFO:tasks.rados.rados.0.smithi118.stdout:1986: finishing rollback tid 1 to smithi11881747-34 2024-03-20T08:08:14.580 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 7052 (ObjNum 161 snap 0 seq_num 0) dirty dne 2024-03-20T08:08:14.580 INFO:tasks.rados.rados.0.smithi118.stdout:1977: done (4 left) 2024-03-20T08:08:14.580 INFO:tasks.rados.rados.0.smithi118.stdout:1978: done (3 left) 2024-03-20T08:08:14.580 INFO:tasks.rados.rados.0.smithi118.stdout:1981: done (2 left) 2024-03-20T08:08:14.580 INFO:tasks.rados.rados.0.smithi118.stdout:1984: done (1 left) 2024-03-20T08:08:14.581 INFO:tasks.rados.rados.0.smithi118.stdout:1986: done (0 left) 2024-03-20T08:08:14.581 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 202 2024-03-20T08:08:14.581 INFO:tasks.rados.rados.0.smithi118.stdout:1987: seq_num 622 ranges {0=425984,425984=401408} 2024-03-20T08:08:14.582 INFO:tasks.rados.rados.0.smithi118.stdout:1987: writing smithi11881747-9 from 0 to 425984 tid 1 2024-03-20T08:08:14.585 INFO:tasks.rados.rados.0.smithi118.stdout:1987: writing smithi11881747-9 from 425984 to 827392 tid 2 2024-03-20T08:08:14.585 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 202 2024-03-20T08:08:14.585 INFO:tasks.rados.rados.0.smithi118.stdout:1988: seq_num 623 ranges {2916352=442368,3358720=401408} 2024-03-20T08:08:14.603 INFO:tasks.rados.rados.0.smithi118.stdout:1988: writing smithi11881747-39 from 2916352 to 3358720 tid 1 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1988: writing smithi11881747-39 from 3358720 to 3760128 tid 2 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1989: snap_create 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1987: finishing write tid 1 to smithi11881747-9 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1987: finishing write tid 2 to smithi11881747-9 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1987: finishing write tid 3 to smithi11881747-9 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1987: finishing write tid 4 to smithi11881747-9 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1987: oid 9 updating version 0 to 16309 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1987: oid 9 version 16309 is already newer than 16308 2024-03-20T08:08:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:1987: oid 9 updating version 16309 to 16310 2024-03-20T08:08:14.608 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 16310 (ObjNum 622 snap 202 seq_num 622) dirty exists 2024-03-20T08:08:14.608 INFO:tasks.rados.rados.0.smithi118.stdout:1987: left oid 9 (ObjNum 622 snap 202 seq_num 622) 2024-03-20T08:08:14.608 INFO:tasks.rados.rados.0.smithi118.stdout:1988: finishing write tid 1 to smithi11881747-39 2024-03-20T08:08:14.612 INFO:tasks.rados.rados.0.smithi118.stdout:1988: finishing write tid 2 to smithi11881747-39 2024-03-20T08:08:14.612 INFO:tasks.rados.rados.0.smithi118.stdout:1988: finishing write tid 3 to smithi11881747-39 2024-03-20T08:08:14.616 INFO:tasks.rados.rados.0.smithi118.stdout:1988: finishing write tid 4 to smithi11881747-39 2024-03-20T08:08:14.616 INFO:tasks.rados.rados.0.smithi118.stdout:1988: oid 39 updating version 0 to 7095 2024-03-20T08:08:14.616 INFO:tasks.rados.rados.0.smithi118.stdout:1988: oid 39 version 7095 is already newer than 7094 2024-03-20T08:08:14.616 INFO:tasks.rados.rados.0.smithi118.stdout:1988: oid 39 updating version 7095 to 7096 2024-03-20T08:08:14.616 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7096 (ObjNum 623 snap 202 seq_num 623) dirty exists 2024-03-20T08:08:14.616 INFO:tasks.rados.rados.0.smithi118.stdout:1988: left oid 39 (ObjNum 623 snap 202 seq_num 623) 2024-03-20T08:08:14.701 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:08:14.701 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:08:14.701 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:08:14.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:08:14.701 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:08:14.701 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:08:14.701 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2, 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 5 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 4, 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 10 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:08:14.702 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:08:14.865 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:14.520+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: osdmap e463: 8 total, 7 up, 8 in 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:14.866 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:14.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: from='client.34323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:14.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: from='client.34329 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:14.867 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:14 smithi196 ceph-mon[87080]: pgmap v373: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 586 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 9.4 MiB/s rd, 640 KiB/s wr, 4 op/s; 113/1179 objects degraded (9.584%) 2024-03-20T08:08:14.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:14.520+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:14.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:14.527+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:14.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:14.542+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:14.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:14.542+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:14.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:14.533+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:14.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:14.521+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:14.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: osdmap e463: 8 total, 7 up, 8 in 2024-03-20T08:08:14.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: from='client.34323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: from='client.34329 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[95692]: pgmap v373: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 586 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 9.4 MiB/s rd, 640 KiB/s wr, 4 op/s; 113/1179 objects degraded (9.584%) 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: osdmap e463: 8 total, 7 up, 8 in 2024-03-20T08:08:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: from='client.34323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: from='client.34329 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:14 smithi118 ceph-mon[93121]: pgmap v373: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 586 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 9.4 MiB/s rd, 640 KiB/s wr, 4 op/s; 113/1179 objects degraded (9.584%) 2024-03-20T08:08:15.542 INFO:tasks.rados.rados.0.smithi118.stdout:1987: done (2 left) 2024-03-20T08:08:15.542 INFO:tasks.rados.rados.0.smithi118.stdout:1988: done (1 left) 2024-03-20T08:08:15.543 INFO:tasks.rados.rados.0.smithi118.stdout:1989: done (0 left) 2024-03-20T08:08:15.543 INFO:tasks.rados.rados.0.smithi118.stdout:1990: delete oid 50 current snap is 203 2024-03-20T08:08:15.550 INFO:tasks.rados.rados.0.smithi118.stdout:1990: done (0 left) 2024-03-20T08:08:15.551 INFO:tasks.rados.rados.0.smithi118.stdout:1991: rmattr oid 47 current snap is 203 2024-03-20T08:08:15.551 INFO:tasks.rados.rados.0.smithi118.stdout:1992: delete oid 15 current snap is 203 2024-03-20T08:08:15.555 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33894 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:08:15.555 INFO:tasks.rados.rados.0.smithi118.stdout:1991: done (1 left) 2024-03-20T08:08:15.555 INFO:tasks.rados.rados.0.smithi118.stdout:1992: done (0 left) 2024-03-20T08:08:15.555 INFO:tasks.rados.rados.0.smithi118.stdout:1993: snap_remove snap 188 2024-03-20T08:08:15.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:15.547+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:15.858 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:15.547+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: osdmap e464: 8 total, 7 up, 8 in 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-20T08:08:15.858 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:15 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/3956568736' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:15.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:15.541+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:15.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:15.543+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:15.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:15.544+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: osdmap e464: 8 total, 7 up, 8 in 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-20T08:08:15.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3956568736' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: osdmap e464: 8 total, 7 up, 8 in 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-20T08:08:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/3956568736' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:15.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:15.548+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:15.925 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:15.550+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:16.546 INFO:tasks.rados.rados.0.smithi118.stdout:1993: done (0 left) 2024-03-20T08:08:16.547 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 203 2024-03-20T08:08:16.547 INFO:tasks.rados.rados.0.smithi118.stdout:1994: seq_num 626 ranges {1966080=491520,2457600=401408} 2024-03-20T08:08:16.561 INFO:tasks.rados.rados.0.smithi118.stdout:1994: writing smithi11881747-23 from 1966080 to 2457600 tid 1 2024-03-20T08:08:16.564 INFO:tasks.rados.rados.0.smithi118.stdout:1994: writing smithi11881747-23 from 2457600 to 2859008 tid 2 2024-03-20T08:08:16.564 INFO:tasks.rados.rados.0.smithi118.stdout:1995: read oid 15 snap -1 2024-03-20T08:08:16.564 INFO:tasks.rados.rados.0.smithi118.stdout:1995: expect deleted 2024-03-20T08:08:16.564 INFO:tasks.rados.rados.0.smithi118.stdout:1996: setattr oid 36 current snap is 203 2024-03-20T08:08:16.566 INFO:tasks.rados.rados.0.smithi118.stdout:1995: done (2 left) 2024-03-20T08:08:16.566 INFO:tasks.rados.rados.0.smithi118.stdout:1997: snap_create 2024-03-20T08:08:16.567 INFO:tasks.rados.rados.0.smithi118.stdout:1994: finishing write tid 1 to smithi11881747-23 2024-03-20T08:08:16.568 INFO:tasks.rados.rados.0.smithi118.stdout:1994: finishing write tid 2 to smithi11881747-23 2024-03-20T08:08:16.568 INFO:tasks.rados.rados.0.smithi118.stdout:1994: finishing write tid 3 to smithi11881747-23 2024-03-20T08:08:16.572 INFO:tasks.rados.rados.0.smithi118.stdout:1994: finishing write tid 4 to smithi11881747-23 2024-03-20T08:08:16.572 INFO:tasks.rados.rados.0.smithi118.stdout:1994: oid 23 updating version 0 to 33896 2024-03-20T08:08:16.572 INFO:tasks.rados.rados.0.smithi118.stdout:1994: oid 23 updating version 33896 to 33897 2024-03-20T08:08:16.572 INFO:tasks.rados.rados.0.smithi118.stdout:1994: oid 23 version 33897 is already newer than 33895 2024-03-20T08:08:16.572 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 33897 (ObjNum 626 snap 203 seq_num 626) dirty exists 2024-03-20T08:08:16.572 INFO:tasks.rados.rados.0.smithi118.stdout:1994: left oid 23 (ObjNum 626 snap 203 seq_num 626) 2024-03-20T08:08:16.572 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 7054 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-03-20T08:08:16.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:16.554+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-mon[87080]: osdmap e465: 8 total, 7 up, 8 in 2024-03-20T08:08:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:16.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:16.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:16 smithi196 ceph-mon[87080]: pgmap v376: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 589 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 15 op/s; 114/1188 objects degraded (9.596%) 2024-03-20T08:08:16.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:16.555+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:16.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:16.558+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:16.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:16.558+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:16.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:16.558+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:16.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:16.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[93121]: osdmap e465: 8 total, 7 up, 8 in 2024-03-20T08:08:16.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:16.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:16.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:16.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:16.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:16.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[93121]: pgmap v376: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 589 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 15 op/s; 114/1188 objects degraded (9.596%) 2024-03-20T08:08:16.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:16.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[95692]: osdmap e465: 8 total, 7 up, 8 in 2024-03-20T08:08:16.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:16.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:16.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:16.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:16.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:16.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-mon[95692]: pgmap v376: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 589 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 15 op/s; 114/1188 objects degraded (9.596%) 2024-03-20T08:08:16.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:16.556+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:16.925 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:16.548+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:17.553 INFO:tasks.rados.rados.0.smithi118.stdout:1994: done (2 left) 2024-03-20T08:08:17.553 INFO:tasks.rados.rados.0.smithi118.stdout:1996: done (1 left) 2024-03-20T08:08:17.553 INFO:tasks.rados.rados.0.smithi118.stdout:1997: done (0 left) 2024-03-20T08:08:17.553 INFO:tasks.rados.rados.0.smithi118.stdout:1998: delete oid 2 current snap is 204 2024-03-20T08:08:17.561 INFO:tasks.rados.rados.0.smithi118.stdout:1998: done (0 left) 2024-03-20T08:08:17.562 INFO:tasks.rados.rados.0.smithi118.stdout:1999: copy_from oid 29 from oid 30 current snap is 204 2024-03-20T08:08:17.562 INFO:tasks.rados.rados.0.smithi118.stdout:2000: read oid 1 snap 199 2024-03-20T08:08:17.562 INFO:tasks.rados.rados.0.smithi118.stdout:2000: expect deleted 2024-03-20T08:08:17.562 INFO:tasks.rados.rados.0.smithi118.stdout:2001: rollback oid 17 current snap is 204 2024-03-20T08:08:17.562 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 200 2024-03-20T08:08:17.562 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 204 2024-03-20T08:08:17.562 INFO:tasks.rados.rados.0.smithi118.stdout:2002: seq_num 628 ranges {0=524288,524288=401408} 2024-03-20T08:08:17.565 INFO:tasks.rados.rados.0.smithi118.stdout:2002: writing smithi11881747-49 from 0 to 524288 tid 1 2024-03-20T08:08:17.568 INFO:tasks.rados.rados.0.smithi118.stdout:2002: writing smithi11881747-49 from 524288 to 925696 tid 2 2024-03-20T08:08:17.568 INFO:tasks.rados.rados.0.smithi118.stdout:1999: finishing copy_from to smithi11881747-29 2024-03-20T08:08:17.568 INFO:tasks.rados.rados.0.smithi118.stdout:1999: got expected ENOENT (src dne) 2024-03-20T08:08:17.568 INFO:tasks.rados.rados.0.smithi118.stdout:2001: finishing rollback tid 0 to smithi11881747-17 2024-03-20T08:08:17.568 INFO:tasks.rados.rados.0.smithi118.stdout:1999: finishing copy_from racing read to smithi11881747-29 2024-03-20T08:08:17.568 INFO:tasks.rados.rados.0.smithi118.stdout:2001: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:08:17.568 INFO:tasks.rados.rados.0.smithi118.stdout:2001: finishing rollback tid 2 to smithi11881747-17 2024-03-20T08:08:17.569 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7050 (ObjNum 568 snap 185 seq_num 568) dirty exists 2024-03-20T08:08:17.569 INFO:tasks.rados.rados.0.smithi118.stdout:1999: done (3 left) 2024-03-20T08:08:17.569 INFO:tasks.rados.rados.0.smithi118.stdout:2000: done (2 left) 2024-03-20T08:08:17.569 INFO:tasks.rados.rados.0.smithi118.stdout:2001: done (1 left) 2024-03-20T08:08:17.569 INFO:tasks.rados.rados.0.smithi118.stdout:2003: setattr oid 32 current snap is 204 2024-03-20T08:08:17.571 INFO:tasks.rados.rados.0.smithi118.stdout:2004: read oid 4 snap 197 2024-03-20T08:08:17.571 INFO:tasks.rados.rados.0.smithi118.stdout:2004: expect (ObjNum 605 snap 197 seq_num 605) 2024-03-20T08:08:17.571 INFO:tasks.rados.rados.0.smithi118.stdout:2005: snap_create 2024-03-20T08:08:17.574 INFO:tasks.rados.rados.0.smithi118.stdout:2002: finishing write tid 1 to smithi11881747-49 2024-03-20T08:08:17.574 INFO:tasks.rados.rados.0.smithi118.stdout:2002: finishing write tid 2 to smithi11881747-49 2024-03-20T08:08:17.574 INFO:tasks.rados.rados.0.smithi118.stdout:2002: finishing write tid 3 to smithi11881747-49 2024-03-20T08:08:17.576 INFO:tasks.rados.rados.0.smithi118.stdout:2004: expect (ObjNum 605 snap 197 seq_num 605) 2024-03-20T08:08:17.596 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 33931 (ObjNum 607 snap 198 seq_num 607) dirty exists 2024-03-20T08:08:17.596 INFO:tasks.rados.rados.0.smithi118.stdout:2002: finishing write tid 4 to smithi11881747-49 2024-03-20T08:08:17.596 INFO:tasks.rados.rados.0.smithi118.stdout:2002: oid 49 updating version 0 to 7099 2024-03-20T08:08:17.596 INFO:tasks.rados.rados.0.smithi118.stdout:2002: oid 49 version 7099 is already newer than 7097 2024-03-20T08:08:17.596 INFO:tasks.rados.rados.0.smithi118.stdout:2002: oid 49 version 7099 is already newer than 7098 2024-03-20T08:08:17.596 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 7099 (ObjNum 628 snap 204 seq_num 628) dirty exists 2024-03-20T08:08:17.596 INFO:tasks.rados.rados.0.smithi118.stdout:2002: left oid 49 (ObjNum 628 snap 204 seq_num 628) 2024-03-20T08:08:17.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:17.557+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:17.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:17.557+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:17.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 114/1188 objects degraded (9.596%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:17.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-mon[87080]: osdmap e466: 8 total, 7 up, 8 in 2024-03-20T08:08:17.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:17.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:17.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:17.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:17.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:17.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:17.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:17.555+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:17.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:17.563+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:17.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:17.563+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:17.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 114/1188 objects degraded (9.596%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:17.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[95692]: osdmap e466: 8 total, 7 up, 8 in 2024-03-20T08:08:17.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:17.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:17.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:17.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:17.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:17.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:17.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:17.551+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 114/1188 objects degraded (9.596%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[93121]: osdmap e466: 8 total, 7 up, 8 in 2024-03-20T08:08:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:17.925 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:17.562+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:18.558 INFO:tasks.rados.rados.0.smithi118.stdout:2002: done (3 left) 2024-03-20T08:08:18.558 INFO:tasks.rados.rados.0.smithi118.stdout:2003: done (2 left) 2024-03-20T08:08:18.558 INFO:tasks.rados.rados.0.smithi118.stdout:2004: done (1 left) 2024-03-20T08:08:18.559 INFO:tasks.rados.rados.0.smithi118.stdout:2005: done (0 left) 2024-03-20T08:08:18.559 INFO:tasks.rados.rados.0.smithi118.stdout:2006: copy_from oid 21 from oid 7 current snap is 205 2024-03-20T08:08:18.559 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 205 2024-03-20T08:08:18.559 INFO:tasks.rados.rados.0.smithi118.stdout:2007: seq_num 629 ranges {3760128=540672,4300800=401408} 2024-03-20T08:08:18.583 INFO:tasks.rados.rados.0.smithi118.stdout:2007: writing smithi11881747-39 from 3760128 to 4300800 tid 1 2024-03-20T08:08:18.586 INFO:tasks.rados.rados.0.smithi118.stdout:2007: writing smithi11881747-39 from 4300800 to 4702208 tid 2 2024-03-20T08:08:18.586 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 205 2024-03-20T08:08:18.586 INFO:tasks.rados.rados.0.smithi118.stdout:2008: seq_num 630 ranges {7110656=557056,7667712=401408} 2024-03-20T08:08:18.629 INFO:tasks.rados.rados.0.smithi118.stdout:2008: writing smithi11881747-38 from 7110656 to 7667712 tid 1 2024-03-20T08:08:18.632 INFO:tasks.rados.rados.0.smithi118.stdout:2008: writing smithi11881747-38 from 7667712 to 8069120 tid 2 2024-03-20T08:08:18.632 INFO:tasks.rados.rados.0.smithi118.stdout:2009: copy_from oid 40 from oid 18 current snap is 205 2024-03-20T08:08:18.632 INFO:tasks.rados.rados.0.smithi118.stdout:2006: finishing copy_from to smithi11881747-21 2024-03-20T08:08:18.632 INFO:tasks.rados.rados.0.smithi118.stdout:2006: got expected ENOENT (src dne) 2024-03-20T08:08:18.632 INFO:tasks.rados.rados.0.smithi118.stdout:2006: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:08:18.632 INFO:tasks.rados.rados.0.smithi118.stdout:2007: finishing write tid 1 to smithi11881747-39 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2007: finishing write tid 2 to smithi11881747-39 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2007: finishing write tid 3 to smithi11881747-39 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2007: finishing write tid 4 to smithi11881747-39 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2007: oid 39 updating version 0 to 7100 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2007: oid 39 updating version 7100 to 7102 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2007: oid 39 version 7102 is already newer than 7101 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7102 (ObjNum 629 snap 205 seq_num 629) dirty exists 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2007: left oid 39 (ObjNum 629 snap 205 seq_num 629) 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2006: done (3 left) 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2007: done (2 left) 2024-03-20T08:08:18.633 INFO:tasks.rados.rados.0.smithi118.stdout:2010: delete oid 45 current snap is 205 2024-03-20T08:08:18.635 INFO:tasks.rados.rados.0.smithi118.stdout:2008: finishing write tid 1 to smithi11881747-38 2024-03-20T08:08:18.635 INFO:tasks.rados.rados.0.smithi118.stdout:2008: finishing write tid 2 to smithi11881747-38 2024-03-20T08:08:18.635 INFO:tasks.rados.rados.0.smithi118.stdout:2008: finishing write tid 3 to smithi11881747-38 2024-03-20T08:08:18.636 INFO:tasks.rados.rados.0.smithi118.stdout:2008: finishing write tid 4 to smithi11881747-38 2024-03-20T08:08:18.636 INFO:tasks.rados.rados.0.smithi118.stdout:2008: oid 38 updating version 0 to 7074 2024-03-20T08:08:18.637 INFO:tasks.rados.rados.0.smithi118.stdout:2008: oid 38 updating version 7074 to 7075 2024-03-20T08:08:18.637 INFO:tasks.rados.rados.0.smithi118.stdout:2008: oid 38 version 7075 is already newer than 7073 2024-03-20T08:08:18.637 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7075 (ObjNum 630 snap 205 seq_num 630) dirty exists 2024-03-20T08:08:18.637 INFO:tasks.rados.rados.0.smithi118.stdout:2008: left oid 38 (ObjNum 630 snap 205 seq_num 630) 2024-03-20T08:08:18.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:18.565+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-mon[87080]: osdmap e467: 8 total, 7 up, 8 in 2024-03-20T08:08:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:18 smithi196 ceph-mon[87080]: pgmap v379: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 592 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 16 op/s; 117/1206 objects degraded (9.701%) 2024-03-20T08:08:18.661 INFO:tasks.rados.rados.0.smithi118.stdout:2009: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:08:18.661 INFO:tasks.rados.rados.0.smithi118.stdout:2008: done (2 left) 2024-03-20T08:08:18.662 INFO:tasks.rados.rados.0.smithi118.stdout:2010: done (1 left) 2024-03-20T08:08:18.662 INFO:tasks.rados.rados.0.smithi118.stdout:2011: read oid 36 snap 183 2024-03-20T08:08:18.662 INFO:tasks.rados.rados.0.smithi118.stdout:2011: expect (ObjNum 322109744 snap 21898 seq_num 321955472) 2024-03-20T08:08:18.662 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 205 2024-03-20T08:08:18.662 INFO:tasks.rados.rados.0.smithi118.stdout:2012: seq_num 632 ranges {4702208=589824,5292032=401408} 2024-03-20T08:08:18.691 INFO:tasks.rados.rados.0.smithi118.stdout:2012: writing smithi11881747-39 from 4702208 to 5292032 tid 1 2024-03-20T08:08:18.693 INFO:tasks.rados.rados.0.smithi118.stdout:2012: writing smithi11881747-39 from 5292032 to 5693440 tid 2 2024-03-20T08:08:18.694 INFO:tasks.rados.rados.0.smithi118.stdout:2013: delete oid 46 current snap is 205 2024-03-20T08:08:18.694 INFO:tasks.rados.rados.0.smithi118.stdout:2011: expect (ObjNum 322109744 snap 21898 seq_num 321955472) 2024-03-20T08:08:18.697 INFO:tasks.rados.rados.0.smithi118.stdout:2009: finishing copy_from to smithi11881747-40 2024-03-20T08:08:18.697 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 7090 (ObjNum 374 snap 127 seq_num 374) dirty exists 2024-03-20T08:08:18.697 INFO:tasks.rados.rados.0.smithi118.stdout:2012: finishing write tid 1 to smithi11881747-39 2024-03-20T08:08:18.697 INFO:tasks.rados.rados.0.smithi118.stdout:2009: done (3 left) 2024-03-20T08:08:18.697 INFO:tasks.rados.rados.0.smithi118.stdout:2011: done (2 left) 2024-03-20T08:08:18.697 INFO:tasks.rados.rados.0.smithi118.stdout:2013: done (1 left) 2024-03-20T08:08:18.697 INFO:tasks.rados.rados.0.smithi118.stdout:2014: delete oid 8 current snap is 205 2024-03-20T08:08:18.697 INFO:tasks.rados.rados.0.smithi118.stdout:2012: finishing write tid 2 to smithi11881747-39 2024-03-20T08:08:18.698 INFO:tasks.rados.rados.0.smithi118.stdout:2012: finishing write tid 3 to smithi11881747-39 2024-03-20T08:08:18.698 INFO:tasks.rados.rados.0.smithi118.stdout:2012: finishing write tid 4 to smithi11881747-39 2024-03-20T08:08:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:2012: oid 39 updating version 0 to 7104 2024-03-20T08:08:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:2012: oid 39 version 7104 is already newer than 7103 2024-03-20T08:08:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:2012: oid 39 updating version 7104 to 7105 2024-03-20T08:08:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7105 (ObjNum 632 snap 205 seq_num 632) dirty exists 2024-03-20T08:08:18.699 INFO:tasks.rados.rados.0.smithi118.stdout:2012: left oid 39 (ObjNum 632 snap 205 seq_num 632) 2024-03-20T08:08:18.703 INFO:tasks.rados.rados.0.smithi118.stdout:2012: done (1 left) 2024-03-20T08:08:18.703 INFO:tasks.rados.rados.0.smithi118.stdout:2014: done (0 left) 2024-03-20T08:08:18.703 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 205 2024-03-20T08:08:18.703 INFO:tasks.rados.rados.0.smithi118.stdout:2015: seq_num 635 ranges {827392=638976,1466368=401408} 2024-03-20T08:08:18.711 INFO:tasks.rados.rados.0.smithi118.stdout:2015: writing smithi11881747-9 from 827392 to 1466368 tid 1 2024-03-20T08:08:18.714 INFO:tasks.rados.rados.0.smithi118.stdout:2015: writing smithi11881747-9 from 1466368 to 1867776 tid 2 2024-03-20T08:08:18.714 INFO:tasks.rados.rados.0.smithi118.stdout:2016: snap_remove snap 200 2024-03-20T08:08:18.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:18.567+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:18.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:18.567+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:18.835 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:18.567+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:18.835 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:18.563+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[95692]: osdmap e467: 8 total, 7 up, 8 in 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[95692]: pgmap v379: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 592 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 16 op/s; 117/1206 objects degraded (9.701%) 2024-03-20T08:08:18.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:18.837 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[93121]: osdmap e467: 8 total, 7 up, 8 in 2024-03-20T08:08:18.837 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:18.837 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:18.837 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:18.837 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:18.837 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:18.837 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-mon[93121]: pgmap v379: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 592 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 16 op/s; 117/1206 objects degraded (9.701%) 2024-03-20T08:08:18.837 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:18.558+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:18.838 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:18.573+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:19.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:19.085+0000 7ffa935427c0 -1 osd.5 456 log_to_monitors true 2024-03-20T08:08:19.603 INFO:tasks.rados.rados.0.smithi118.stdout:2016: done (1 left) 2024-03-20T08:08:19.603 INFO:tasks.rados.rados.0.smithi118.stdout:2017: snap_remove snap 192 2024-03-20T08:08:19.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:19.609+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:19.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:19.609+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:19.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:19.631+0000 7ffa8499a700 -1 osd.5 456 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T08:08:19.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:19.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: osdmap e468: 8 total, 7 up, 8 in 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: from='osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T08:08:19.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:19 smithi196 ceph-mon[87080]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T08:08:19.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:19.614+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:19.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:19.607+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:19.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:19.602+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:19.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:19.602+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:19.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:19.603+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:19.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:19.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: osdmap e468: 8 total, 7 up, 8 in 2024-03-20T08:08:19.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:19.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: from='osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[93121]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: osdmap e468: 8 total, 7 up, 8 in 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: from='osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T08:08:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:19 smithi118 ceph-mon[95692]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-20T08:08:20.607 INFO:tasks.rados.rados.0.smithi118.stdout:2015: finishing write tid 1 to smithi11881747-9 2024-03-20T08:08:20.607 INFO:tasks.rados.rados.0.smithi118.stdout:2015: finishing write tid 2 to smithi11881747-9 2024-03-20T08:08:20.607 INFO:tasks.rados.rados.0.smithi118.stdout:2015: finishing write tid 3 to smithi11881747-9 2024-03-20T08:08:20.607 INFO:tasks.rados.rados.0.smithi118.stdout:2015: finishing write tid 4 to smithi11881747-9 2024-03-20T08:08:20.607 INFO:tasks.rados.rados.0.smithi118.stdout:2015: oid 9 updating version 0 to 16312 2024-03-20T08:08:20.607 INFO:tasks.rados.rados.0.smithi118.stdout:2015: oid 9 version 16312 is already newer than 16311 2024-03-20T08:08:20.607 INFO:tasks.rados.rados.0.smithi118.stdout:2015: oid 9 updating version 16312 to 16313 2024-03-20T08:08:20.607 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 16313 (ObjNum 635 snap 205 seq_num 635) dirty exists 2024-03-20T08:08:20.608 INFO:tasks.rados.rados.0.smithi118.stdout:2015: left oid 9 (ObjNum 635 snap 205 seq_num 635) 2024-03-20T08:08:20.608 INFO:tasks.rados.rados.0.smithi118.stdout:2015: done (1 left) 2024-03-20T08:08:20.608 INFO:tasks.rados.rados.0.smithi118.stdout:2017: done (0 left) 2024-03-20T08:08:20.608 INFO:tasks.rados.rados.0.smithi118.stdout:2018: copy_from oid 15 from oid 20 current snap is 205 2024-03-20T08:08:20.608 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 205 2024-03-20T08:08:20.608 INFO:tasks.rados.rados.0.smithi118.stdout:2019: seq_num 636 ranges {1966080=655360,2621440=401408} 2024-03-20T08:08:20.623 INFO:tasks.rados.rados.0.smithi118.stdout:2019: writing smithi11881747-6 from 1966080 to 2621440 tid 1 2024-03-20T08:08:20.626 INFO:tasks.rados.rados.0.smithi118.stdout:2019: writing smithi11881747-6 from 2621440 to 3022848 tid 2 2024-03-20T08:08:20.626 INFO:tasks.rados.rados.0.smithi118.stdout:2020: read oid 46 snap -1 2024-03-20T08:08:20.626 INFO:tasks.rados.rados.0.smithi118.stdout:2020: expect deleted 2024-03-20T08:08:20.626 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 205 2024-03-20T08:08:20.626 INFO:tasks.rados.rados.0.smithi118.stdout:2021: seq_num 637 ranges {0=671744,671744=401408} 2024-03-20T08:08:20.630 INFO:tasks.rados.rados.0.smithi118.stdout:2021: writing smithi11881747-1 from 0 to 671744 tid 1 2024-03-20T08:08:20.633 INFO:tasks.rados.rados.0.smithi118.stdout:2021: writing smithi11881747-1 from 671744 to 1073152 tid 2 2024-03-20T08:08:20.633 INFO:tasks.rados.rados.0.smithi118.stdout:2022: rollback oid 34 current snap is 205 2024-03-20T08:08:20.633 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 34 to 202 2024-03-20T08:08:20.634 INFO:tasks.rados.rados.0.smithi118.stdout:2023: read oid 41 snap -1 2024-03-20T08:08:20.635 INFO:tasks.rados.rados.0.smithi118.stdout:2023: expect deleted 2024-03-20T08:08:20.635 INFO:tasks.rados.rados.0.smithi118.stdout:2024: rollback oid 2 current snap is 205 2024-03-20T08:08:20.635 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 204 2024-03-20T08:08:20.635 INFO:tasks.rados.rados.0.smithi118.stdout:2025: copy_from oid 11 from oid 12 current snap is 205 2024-03-20T08:08:20.635 INFO:tasks.rados.rados.0.smithi118.stdout:2026: read oid 28 snap -1 2024-03-20T08:08:20.635 INFO:tasks.rados.rados.0.smithi118.stdout:2026: expect (ObjNum 446 snap 148 seq_num 446) 2024-03-20T08:08:20.635 INFO:tasks.rados.rados.0.smithi118.stdout:2027: rollback oid 29 current snap is 205 2024-03-20T08:08:20.636 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 198 2024-03-20T08:08:20.636 INFO:tasks.rados.rados.0.smithi118.stdout:2028: snap_create 2024-03-20T08:08:20.639 INFO:tasks.rados.rados.0.smithi118.stdout:2019: finishing write tid 1 to smithi11881747-6 2024-03-20T08:08:20.639 INFO:tasks.rados.rados.0.smithi118.stdout:2019: finishing write tid 2 to smithi11881747-6 2024-03-20T08:08:20.639 INFO:tasks.rados.rados.0.smithi118.stdout:2019: finishing write tid 3 to smithi11881747-6 2024-03-20T08:08:20.639 INFO:tasks.rados.rados.0.smithi118.stdout:2024: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:08:20.639 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 9328 (ObjNum 65 snap 0 seq_num 1315398234) dirty dne 2024-03-20T08:08:20.640 INFO:tasks.rados.rados.0.smithi118.stdout:2027: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:08:20.641 INFO:tasks.rados.rados.0.smithi118.stdout:2019: finishing write tid 4 to smithi11881747-6 2024-03-20T08:08:20.641 INFO:tasks.rados.rados.0.smithi118.stdout:2019: oid 6 updating version 0 to 9329 2024-03-20T08:08:20.641 INFO:tasks.rados.rados.0.smithi118.stdout:2019: oid 6 updating version 9329 to 9331 2024-03-20T08:08:20.642 INFO:tasks.rados.rados.0.smithi118.stdout:2019: oid 6 version 9331 is already newer than 9330 2024-03-20T08:08:20.642 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 9331 (ObjNum 636 snap 205 seq_num 636) dirty exists 2024-03-20T08:08:20.642 INFO:tasks.rados.rados.0.smithi118.stdout:2019: left oid 6 (ObjNum 636 snap 205 seq_num 636) 2024-03-20T08:08:20.642 INFO:tasks.rados.rados.0.smithi118.stdout:2022: finishing rollback tid 1 to smithi11881747-34 2024-03-20T08:08:20.642 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 7052 (ObjNum 129 snap 0 seq_num 322202384) dirty dne 2024-03-20T08:08:20.643 INFO:tasks.rados.rados.0.smithi118.stdout:2026: expect (ObjNum 446 snap 148 seq_num 446) 2024-03-20T08:08:20.760 INFO:tasks.rados.rados.0.smithi118.stdout:2027: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:08:20.760 INFO:tasks.rados.rados.0.smithi118.stdout:2027: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:08:20.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33900 (ObjNum 547 snap 180 seq_num 547) dirty exists 2024-03-20T08:08:20.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:20.612+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-20T08:08:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: osdmap e469: 8 total, 7 up, 8 in 2024-03-20T08:08:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: from='osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:20 smithi196 ceph-mon[87080]: pgmap v382: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 595 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 407 KiB/s rd, 1.6 MiB/s wr, 10 op/s; 120/1224 objects degraded (9.804%) 2024-03-20T08:08:20.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:20.624+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:20.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:20.617+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:20.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:20.617+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:20.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:20.617+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: osdmap e469: 8 total, 7 up, 8 in 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: from='osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:20.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:20.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:20.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[93121]: pgmap v382: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 595 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 407 KiB/s rd, 1.6 MiB/s wr, 10 op/s; 120/1224 objects degraded (9.804%) 2024-03-20T08:08:20.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:20.607+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:20.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:20.629+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:20.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: osdmap e469: 8 total, 7 up, 8 in 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: from='osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:20 smithi118 ceph-mon[95692]: pgmap v382: 121 pgs: 1 active+clean+snaptrim, 38 active+undersized, 11 active+undersized+degraded, 6 active+clean+snaptrim_error, 65 active+clean; 595 MiB data, 4.7 GiB used, 710 GiB / 715 GiB avail; 407 KiB/s rd, 1.6 MiB/s wr, 10 op/s; 120/1224 objects degraded (9.804%) 2024-03-20T08:08:21.605 INFO:tasks.rados.rados.0.smithi118.stdout:2019: done (10 left) 2024-03-20T08:08:21.605 INFO:tasks.rados.rados.0.smithi118.stdout:2020: done (9 left) 2024-03-20T08:08:21.605 INFO:tasks.rados.rados.0.smithi118.stdout:2022: done (8 left) 2024-03-20T08:08:21.605 INFO:tasks.rados.rados.0.smithi118.stdout:2023: done (7 left) 2024-03-20T08:08:21.605 INFO:tasks.rados.rados.0.smithi118.stdout:2024: done (6 left) 2024-03-20T08:08:21.605 INFO:tasks.rados.rados.0.smithi118.stdout:2026: done (5 left) 2024-03-20T08:08:21.605 INFO:tasks.rados.rados.0.smithi118.stdout:2027: done (4 left) 2024-03-20T08:08:21.605 INFO:tasks.rados.rados.0.smithi118.stdout:2028: done (3 left) 2024-03-20T08:08:21.606 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 206 2024-03-20T08:08:21.606 INFO:tasks.rados.rados.0.smithi118.stdout:2029: seq_num 638 ranges {4268032=688128,4956160=401408} 2024-03-20T08:08:21.633 INFO:tasks.rados.rados.0.smithi118.stdout:2029: writing smithi11881747-29 from 4268032 to 4956160 tid 1 2024-03-20T08:08:21.637 INFO:tasks.rados.rados.0.smithi118.stdout:2029: writing smithi11881747-29 from 4956160 to 5357568 tid 2 2024-03-20T08:08:21.637 INFO:tasks.rados.rados.0.smithi118.stdout:2030: snap_create 2024-03-20T08:08:21.641 INFO:tasks.rados.rados.0.smithi118.stdout:2018: finishing copy_from to smithi11881747-15 2024-03-20T08:08:21.641 INFO:tasks.rados.rados.0.smithi118.stdout:2018: got expected ENOENT (src dne) 2024-03-20T08:08:21.642 INFO:tasks.rados.rados.0.smithi118.stdout:2018: finishing copy_from racing read to smithi11881747-15 2024-03-20T08:08:21.644 INFO:tasks.rados.rados.0.smithi118.stdout:2029: finishing write tid 1 to smithi11881747-29 2024-03-20T08:08:21.644 INFO:tasks.rados.rados.0.smithi118.stdout:2029: finishing write tid 2 to smithi11881747-29 2024-03-20T08:08:21.644 INFO:tasks.rados.rados.0.smithi118.stdout:2029: finishing write tid 3 to smithi11881747-29 2024-03-20T08:08:21.647 INFO:tasks.rados.rados.0.smithi118.stdout:2029: finishing write tid 4 to smithi11881747-29 2024-03-20T08:08:21.648 INFO:tasks.rados.rados.0.smithi118.stdout:2029: oid 29 updating version 0 to 33903 2024-03-20T08:08:21.648 INFO:tasks.rados.rados.0.smithi118.stdout:2029: oid 29 version 33903 is already newer than 33901 2024-03-20T08:08:21.648 INFO:tasks.rados.rados.0.smithi118.stdout:2029: oid 29 version 33903 is already newer than 33902 2024-03-20T08:08:21.648 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33903 (ObjNum 638 snap 206 seq_num 638) dirty exists 2024-03-20T08:08:21.648 INFO:tasks.rados.rados.0.smithi118.stdout:2029: left oid 29 (ObjNum 638 snap 206 seq_num 638) 2024-03-20T08:08:21.649 INFO:tasks.rados.rados.0.smithi118.stdout:2021: finishing write tid 1 to smithi11881747-1 2024-03-20T08:08:21.649 INFO:tasks.rados.rados.0.smithi118.stdout:2021: finishing write tid 2 to smithi11881747-1 2024-03-20T08:08:21.649 INFO:tasks.rados.rados.0.smithi118.stdout:2021: finishing write tid 3 to smithi11881747-1 2024-03-20T08:08:21.652 INFO:tasks.rados.rados.0.smithi118.stdout:2021: finishing write tid 4 to smithi11881747-1 2024-03-20T08:08:21.652 INFO:tasks.rados.rados.0.smithi118.stdout:2021: oid 1 updating version 0 to 7090 2024-03-20T08:08:21.652 INFO:tasks.rados.rados.0.smithi118.stdout:2021: oid 1 updating version 7090 to 7091 2024-03-20T08:08:21.652 INFO:tasks.rados.rados.0.smithi118.stdout:2021: oid 1 updating version 7091 to 7092 2024-03-20T08:08:21.652 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 7092 (ObjNum 637 snap 205 seq_num 637) dirty exists 2024-03-20T08:08:21.652 INFO:tasks.rados.rados.0.smithi118.stdout:2021: left oid 1 (ObjNum 637 snap 205 seq_num 637) 2024-03-20T08:08:21.681 INFO:tasks.rados.rados.0.smithi118.stdout:2025: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:08:21.684 INFO:tasks.rados.rados.0.smithi118.stdout:2025: finishing copy_from to smithi11881747-11 2024-03-20T08:08:21.684 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 23061 (ObjNum 600 snap 196 seq_num 600) dirty exists 2024-03-20T08:08:21.858 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:21.614+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:21.859 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:21.614+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:21.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.624+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 108=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7],108=[108,107,106,105,103,102,101]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:21.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.819+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:21.859 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.837+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:21.859 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:21.612+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693] boot 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: osdmap e470: 8 total, 8 up, 8 in 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:21.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[93121]: osdmap e471: 8 total, 8 up, 8 in 2024-03-20T08:08:21.861 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:21.606+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:21.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:08:21.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693] boot 2024-03-20T08:08:21.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: osdmap e470: 8 total, 8 up, 8 in 2024-03-20T08:08:21.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:08:21.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:21.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:21.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:21.861 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:21.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:21.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:21.862 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-mon[95692]: osdmap e471: 8 total, 8 up, 8 in 2024-03-20T08:08:21.864 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.613+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:21.864 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.613+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:21.864 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.646+0000 7fa5d2837700 -1 osd.4 pg_epoch: 471 pg[5.fs1( v 471'984 (0'0,471'984] local-lis/les=470/471 n=19 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [5,4,6]p5(0) r=1 lpr=470 luod=0'0 crt=456'981 mlcod 0'0 active mbc={} ps=86] update_snap_map remove_oid 5:f8faee4c:::smithi11881747-1:e3 failed with -2 2024-03-20T08:08:21.864 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:21.864 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:21.864 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:08:21.864 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: osd.5 [v2:172.21.15.196:6808/4232618693,v1:172.21.15.196:6809/4232618693] boot 2024-03-20T08:08:21.864 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: osdmap e470: 8 total, 8 up, 8 in 2024-03-20T08:08:21.864 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-20T08:08:21.865 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:21.865 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:21.865 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:21.865 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:21.865 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset ff=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:21.865 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:21.865 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-mon[87080]: osdmap e471: 8 total, 8 up, 8 in 2024-03-20T08:08:21.865 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:21.646+0000 7ffa6b167700 -1 osd.5 pg_epoch: 471 pg[5.fs0( v 471'984 (0'0,471'984] local-lis/les=470/471 n=19 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [5,4,6]p5(0) r=0 lpr=470 luod=456'981 crt=456'981 lcod 0'0 mlcod 0'0 active+clean+snaptrim trimq=[e3~1,f5~1,f7~2,fa~1,fc~1,fe~1,104~1] ps=86] update_snap_map remove_oid 5:f8faee4c:::smithi11881747-1:e3 failed with -2 2024-03-20T08:08:21.865 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:21.652+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:21.865 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:21.697+0000 7ffa71974700 -1 osd.5 pg_epoch: 471 pg[5.2s2( v 468'1004 (0'0,468'1004] local-lis/les=470/471 n=16 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [6,0,5]p6(0) r=2 lpr=470 luod=0'0 crt=468'1004 mlcod 0'0 active mbc={} ps=86] update_snap_map remove_oid 5:44a43f74:::smithi11881747-38:e4 failed with -2 2024-03-20T08:08:21.865 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:21.732+0000 7ffa6c96a700 -1 osd.5 pg_epoch: 471 pg[5.4s2( v 467'235 (0'0,467'235] local-lis/les=470/471 n=4 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [7,2,5]p7(0) r=2 lpr=470 luod=0'0 crt=467'235 mlcod 0'0 active mbc={} ps=87] update_snap_map remove_oid 5:27618338:::smithi11881747-17:f6 failed with -2 2024-03-20T08:08:22.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.877+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.890+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.903+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.916+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.944+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.974+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:21.987+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.001+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.055+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.069+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.081+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.095+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.125 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.858+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.125 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.879+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.125 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.899+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.918+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.937+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.957+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.975+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:21.993+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.012+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.031+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.049+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.068+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.087+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.107+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.126 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.124+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.375 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:08:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:08:22.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.148+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.188+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.241+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.267+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.294+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.308+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.322+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.385 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.144+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.162+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.181+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.199+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.218+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.237+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.274+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.293+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.311+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.330+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.367+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.386 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.384+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.628 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.416+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.428+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.455+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.469+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.549+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.588+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.616+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.632 INFO:tasks.rados.rados.0.smithi118.stdout:2018: done (4 left) 2024-03-20T08:08:22.632 INFO:tasks.rados.rados.0.smithi118.stdout:2021: done (3 left) 2024-03-20T08:08:22.632 INFO:tasks.rados.rados.0.smithi118.stdout:2025: done (2 left) 2024-03-20T08:08:22.632 INFO:tasks.rados.rados.0.smithi118.stdout:2029: done (1 left) 2024-03-20T08:08:22.632 INFO:tasks.rados.rados.0.smithi118.stdout:2030: done (0 left) 2024-03-20T08:08:22.632 INFO:tasks.rados.rados.0.smithi118.stdout:2031: rollback oid 27 current snap is 207 2024-03-20T08:08:22.632 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 201 2024-03-20T08:08:22.632 INFO:tasks.rados.rados.0.smithi118.stdout:2032: delete oid 39 current snap is 207 2024-03-20T08:08:22.644 INFO:tasks.rados.rados.0.smithi118.stdout:2031: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:08:22.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:22.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:22.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:22.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:22.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:22.645 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 108=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7],108=[108,107,106,105,103,102,101]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[93121]: pgmap v385: 121 pgs: 4 peering, 1 active+clean+snaptrim, 30 active+undersized, 10 active+undersized+degraded, 6 active+clean+snaptrim_error, 70 active+clean; 596 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 422 KiB/s rd, 1.2 MiB/s wr, 18 op/s; 118/1233 objects degraded (9.570%) 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 108=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7],108=[108,107,106,105,103,102,101]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:22.646 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:22.647 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-mon[95692]: pgmap v385: 121 pgs: 4 peering, 1 active+clean+snaptrim, 30 active+undersized, 10 active+undersized+degraded, 6 active+clean+snaptrim_error, 70 active+clean; 596 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 422 KiB/s rd, 1.2 MiB/s wr, 18 op/s; 118/1233 objects degraded (9.570%) 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.406+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.423+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.442+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.461+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.480+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.499+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.519+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.539+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.647 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.557+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.575+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.594+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.612+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.648 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.633+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:2031: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:2031: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7108 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:2031: done (1 left) 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:2032: done (0 left) 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:2033: rollback oid 50 current snap is 207 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 50 to 164 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:2034: copy_from oid 26 from oid 41 current snap is 207 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 207 2024-03-20T08:08:22.649 INFO:tasks.rados.rados.0.smithi118.stdout:2035: seq_num 640 ranges {0=729088,729088=401408} 2024-03-20T08:08:22.654 INFO:tasks.rados.rados.0.smithi118.stdout:2035: writing smithi11881747-20 from 0 to 729088 tid 1 2024-03-20T08:08:22.657 INFO:tasks.rados.rados.0.smithi118.stdout:2035: writing smithi11881747-20 from 729088 to 1130496 tid 2 2024-03-20T08:08:22.657 INFO:tasks.rados.rados.0.smithi118.stdout:2036: snap_create 2024-03-20T08:08:22.659 INFO:tasks.rados.rados.0.smithi118.stdout:2034: finishing copy_from to smithi11881747-26 2024-03-20T08:08:22.659 INFO:tasks.rados.rados.0.smithi118.stdout:2034: got expected ENOENT (src dne) 2024-03-20T08:08:22.660 INFO:tasks.rados.rados.0.smithi118.stdout:2034: finishing copy_from racing read to smithi11881747-26 2024-03-20T08:08:22.661 INFO:tasks.rados.rados.0.smithi118.stdout:2033: finishing rollback tid 1 to smithi11881747-50 2024-03-20T08:08:22.661 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 7079 (ObjNum 65 snap 0 seq_num 1500079211) dirty dne 2024-03-20T08:08:22.668 INFO:tasks.rados.rados.0.smithi118.stdout:2035: finishing write tid 1 to smithi11881747-20 2024-03-20T08:08:22.668 INFO:tasks.rados.rados.0.smithi118.stdout:2035: finishing write tid 2 to smithi11881747-20 2024-03-20T08:08:22.668 INFO:tasks.rados.rados.0.smithi118.stdout:2035: finishing write tid 3 to smithi11881747-20 2024-03-20T08:08:22.671 INFO:tasks.rados.rados.0.smithi118.stdout:2035: finishing write tid 4 to smithi11881747-20 2024-03-20T08:08:22.671 INFO:tasks.rados.rados.0.smithi118.stdout:2035: oid 20 updating version 0 to 7110 2024-03-20T08:08:22.671 INFO:tasks.rados.rados.0.smithi118.stdout:2035: oid 20 updating version 7110 to 7112 2024-03-20T08:08:22.671 INFO:tasks.rados.rados.0.smithi118.stdout:2035: oid 20 version 7112 is already newer than 7111 2024-03-20T08:08:22.671 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 7112 (ObjNum 640 snap 207 seq_num 640) dirty exists 2024-03-20T08:08:22.671 INFO:tasks.rados.rados.0.smithi118.stdout:2035: left oid 20 (ObjNum 640 snap 207 seq_num 640) 2024-03-20T08:08:22.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:22.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:22.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:22.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:22.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:22.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 108=[]:{97=[95],ba=[b4,b3],be=[be],d6=[d6,d4,d3,d1,cf,ce,cd,cb,ca,c9],db=[db,da,d9,d8,d7],e4=[e4,e3,e1,e0],ea=[e9,e8,e5],ed=[ed,eb],ef=[ef,ee],f6=[f6,f5,f4,f3,f2,f1,f0],fc=[fc,fa,f8,f7],108=[108,107,106,105,103,102,101]} for object 5:5ee214c4:::smithi11881747-29:9e 2024-03-20T08:08:22.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:22.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-mon[87080]: pgmap v385: 121 pgs: 4 peering, 1 active+clean+snaptrim, 30 active+undersized, 10 active+undersized+degraded, 6 active+clean+snaptrim_error, 70 active+clean; 596 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 422 KiB/s rd, 1.2 MiB/s wr, 18 op/s; 118/1233 objects degraded (9.570%) 2024-03-20T08:08:22.881 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:22.634+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.637+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.638+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.665+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.679+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.705+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.733+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.798+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.826+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.866+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:22.898 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:22.647+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:22.898 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:22.649+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:22.899 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:22.649+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:22.899 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:22.646+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:22.899 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:22.645+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:22.899 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.658+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.676+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.694+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.712+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.730+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.749+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.767+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.786+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.804+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.823+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.841+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.860+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.879+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:22.900 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.898+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.924+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.962+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.975+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:22.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.012+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.028+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.079+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.916+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.934+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.955+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.974+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.157 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:22.992+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.010+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.028+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.064+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.081+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.099+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.118+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.136+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.158 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.154+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.147+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.212+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.239+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.253+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.265+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.278+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.356+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.385+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.172+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.190+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.209+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.227+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.411 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.245+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.264+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.283+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.302+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.320+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.356+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.374+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.393+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.412 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.410+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.635 INFO:tasks.rados.rados.0.smithi118.stdout:2033: done (3 left) 2024-03-20T08:08:23.635 INFO:tasks.rados.rados.0.smithi118.stdout:2034: done (2 left) 2024-03-20T08:08:23.635 INFO:tasks.rados.rados.0.smithi118.stdout:2035: done (1 left) 2024-03-20T08:08:23.635 INFO:tasks.rados.rados.0.smithi118.stdout:2036: done (0 left) 2024-03-20T08:08:23.635 INFO:tasks.rados.rados.0.smithi118.stdout:2037: snap_remove snap 205 2024-03-20T08:08:23.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 118/1233 objects degraded (9.570%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:23.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-mon[87080]: osdmap e472: 8 total, 8 up, 8 in 2024-03-20T08:08:23.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:23.636 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:23.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:23.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:23.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:23.637 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:23.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.426+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.439+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.465+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.638 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.607+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.637+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.637+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 118/1233 objects degraded (9.570%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[93121]: osdmap e472: 8 total, 8 up, 8 in 2024-03-20T08:08:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:23.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:23.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:23.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:08:23] "GET /metrics HTTP/1.1" 200 36337 "" "Prometheus/2.43.0" 2024-03-20T08:08:23.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:23.641+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:23.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:23.641+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:23.674 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:23.647+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.428+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.446+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.465+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.502+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.521+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.540+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.558+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.576+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.595+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.613+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.636+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.658+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 118/1233 objects degraded (9.570%), 10 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:23.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[95692]: osdmap e472: 8 total, 8 up, 8 in 2024-03-20T08:08:23.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:23.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:23.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:23.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:23.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:23.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:23.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.670+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.683+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.697+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.754+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.782+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.796+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.808+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.823+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.880+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:23.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:23.643+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:23.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.677+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.696+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.714+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.732+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.751+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.772+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.791+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.934 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.810+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.835+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.855+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.876+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.895+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.913+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:23.935 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:23.790+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.895+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.924+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.938+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.951+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.977+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:23.993+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.007+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.020+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.058+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.096+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.198 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:08:23.989+0000 7ff9af1ba700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-03-20T08:08:24.198 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.932+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.198 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.951+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.198 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.970+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:23.990+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.009+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.031+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.049+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.067+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.086+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.105+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.125+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.143+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.160+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.179+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.197+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.188+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.214+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.227+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.240+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.254+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.266+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.304+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.369+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.382+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.396+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.458 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.216+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.458 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.235+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.458 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.253+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.272+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.309+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.328+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.348+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.366+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.384+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.402+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.420+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.439+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.459 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.457+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.640 INFO:tasks.rados.rados.0.smithi118.stdout:2037: done (0 left) 2024-03-20T08:08:24.640 INFO:tasks.rados.rados.0.smithi118.stdout:2038: read oid 42 snap 172 2024-03-20T08:08:24.640 INFO:tasks.rados.rados.0.smithi118.stdout:2038: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:08:24.640 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 208 2024-03-20T08:08:24.640 INFO:tasks.rados.rados.0.smithi118.stdout:2039: seq_num 641 ranges {1073152=745472,1818624=401408} 2024-03-20T08:08:24.650 INFO:tasks.rados.rados.0.smithi118.stdout:2039: writing smithi11881747-1 from 1073152 to 1818624 tid 1 2024-03-20T08:08:24.653 INFO:tasks.rados.rados.0.smithi118.stdout:2039: writing smithi11881747-1 from 1818624 to 2220032 tid 2 2024-03-20T08:08:24.653 INFO:tasks.rados.rados.0.smithi118.stdout:2040: snap_create 2024-03-20T08:08:24.657 INFO:tasks.rados.rados.0.smithi118.stdout:2039: finishing write tid 1 to smithi11881747-1 2024-03-20T08:08:24.658 INFO:tasks.rados.rados.0.smithi118.stdout:2039: finishing write tid 2 to smithi11881747-1 2024-03-20T08:08:24.659 INFO:tasks.rados.rados.0.smithi118.stdout:2039: finishing write tid 3 to smithi11881747-1 2024-03-20T08:08:24.660 INFO:tasks.rados.rados.0.smithi118.stdout:2039: finishing write tid 4 to smithi11881747-1 2024-03-20T08:08:24.661 INFO:tasks.rados.rados.0.smithi118.stdout:2039: oid 1 updating version 0 to 7093 2024-03-20T08:08:24.661 INFO:tasks.rados.rados.0.smithi118.stdout:2039: oid 1 updating version 7093 to 7094 2024-03-20T08:08:24.661 INFO:tasks.rados.rados.0.smithi118.stdout:2039: oid 1 updating version 7094 to 7095 2024-03-20T08:08:24.661 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 7095 (ObjNum 641 snap 208 seq_num 641) dirty exists 2024-03-20T08:08:24.661 INFO:tasks.rados.rados.0.smithi118.stdout:2039: left oid 1 (ObjNum 641 snap 208 seq_num 641) 2024-03-20T08:08:24.667 INFO:tasks.rados.rados.0.smithi118.stdout:2038: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: osdmap e473: 8 total, 8 up, 8 in 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:24.677 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.677 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.677 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.677 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:24.677 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:24.677 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-03-20T08:08:24.677 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-mon[87080]: pgmap v388: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 4 peering, 20 active+undersized, 3 active+undersized+degraded, 5 active+clean+snaptrim_error, 85 active+clean; 600 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 16 op/s; 54/1242 objects degraded (4.348%); 15 MiB/s, 2 objects/s recovering 2024-03-20T08:08:24.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:24.649+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.516+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.529+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.554+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.568+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.581+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.632+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.652+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.654+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.728 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:24.649+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:24.728 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:24.649+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:24.728 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:24.657+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:24.728 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: osdmap e473: 8 total, 8 up, 8 in 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.729 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:24.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:24.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-03-20T08:08:24.730 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[93121]: pgmap v388: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 4 peering, 20 active+undersized, 3 active+undersized+degraded, 5 active+clean+snaptrim_error, 85 active+clean; 600 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 16 op/s; 54/1242 objects degraded (4.348%); 15 MiB/s, 2 objects/s recovering 2024-03-20T08:08:24.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.477+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.495+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.513+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.532+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.549+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.730 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.568+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.586+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.606+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.625+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.645+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.670+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.688+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.731 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.707+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: osdmap e473: 8 total, 8 up, 8 in 2024-03-20T08:08:24.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:24.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:24.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:24.731 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: Upgrade: unsafe to stop osd(s) at this time (16 PGs are or would become offline) 2024-03-20T08:08:24.732 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-mon[95692]: pgmap v388: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 4 peering, 20 active+undersized, 3 active+undersized+degraded, 5 active+clean+snaptrim_error, 85 active+clean; 600 MiB data, 4.9 GiB used, 710 GiB / 715 GiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 16 op/s; 54/1242 objects degraded (4.348%); 15 MiB/s, 2 objects/s recovering 2024-03-20T08:08:24.733 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:24.637+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:24.733 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:24.659+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.691+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.705+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.719+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.733+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.787+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.812+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.826+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.894+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.933+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:24.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.727+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.752+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.771+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.791+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.813+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.834+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.855+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.986 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.876+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.895+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.913+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.932+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.950+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.967+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:24.987 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:24.985+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.958+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.971+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:24.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.025+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.038+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.051+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.065+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.079+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.093+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.120+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.147+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.161+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.174+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.244 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.041+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.058+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.077+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.096+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.117+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.135+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.153+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.171+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.188+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.206+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.225+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.245 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.243+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.226+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.251+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.329+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.354+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.367+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.381+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.407+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.420+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.448+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.262+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.280+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.299+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.318+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.338+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.356+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.373+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.392+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.410+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.429+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.445+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.463+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.481+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.499+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.658 INFO:tasks.rados.rados.0.smithi118.stdout:2038: done (2 left) 2024-03-20T08:08:25.728 INFO:tasks.rados.rados.0.smithi118.stdout:2039: done (1 left) 2024-03-20T08:08:25.728 INFO:tasks.rados.rados.0.smithi118.stdout:2040: done (0 left) 2024-03-20T08:08:25.728 INFO:tasks.rados.rados.0.smithi118.stdout:2041: read oid 48 snap -1 2024-03-20T08:08:25.728 INFO:tasks.rados.rados.0.smithi118.stdout:2041: expect (ObjNum 592 snap 193 seq_num 592) 2024-03-20T08:08:25.728 INFO:tasks.rados.rados.0.smithi118.stdout:2042: copy_from oid 1 from oid 22 current snap is 209 2024-03-20T08:08:25.728 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 5 current snap is 209 2024-03-20T08:08:25.728 INFO:tasks.rados.rados.0.smithi118.stdout:2043: seq_num 642 ranges {3252224=761856,4014080=401408} 2024-03-20T08:08:25.729 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:25.662+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 10b=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:25.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-mon[87080]: osdmap e474: 8 total, 8 up, 8 in 2024-03-20T08:08:25.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:25.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:25.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:25.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:25.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:25.730 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.460+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.538+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.663+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.663+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.689+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.733 INFO:tasks.rados.rados.0.smithi118.stdout:2043: writing smithi11881747-5 from 3252224 to 4014080 tid 1 2024-03-20T08:08:25.733 INFO:tasks.rados.rados.0.smithi118.stdout:2043: writing smithi11881747-5 from 4014080 to 4415488 tid 2 2024-03-20T08:08:25.733 INFO:tasks.rados.rados.0.smithi118.stdout:2044: read oid 3 snap -1 2024-03-20T08:08:25.733 INFO:tasks.rados.rados.0.smithi118.stdout:2044: expect (ObjNum 617 snap 202 seq_num 617) 2024-03-20T08:08:25.733 INFO:tasks.rados.rados.0.smithi118.stdout:2045: read oid 50 snap -1 2024-03-20T08:08:25.733 INFO:tasks.rados.rados.0.smithi118.stdout:2045: expect deleted 2024-03-20T08:08:25.733 INFO:tasks.rados.rados.0.smithi118.stdout:2046: snap_remove snap 207 2024-03-20T08:08:25.767 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:25.662+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:25.767 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:25.663+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:25.767 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[93121]: osdmap e474: 8 total, 8 up, 8 in 2024-03-20T08:08:25.767 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:25.767 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:25.768 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:25.768 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:25.768 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:25.768 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:25.768 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[95692]: osdmap e474: 8 total, 8 up, 8 in 2024-03-20T08:08:25.768 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:25.768 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 108=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:25.768 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:25.769 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:25.769 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:25.769 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:25.769 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:25.650+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:25.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.518+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.535+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.769 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.553+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.572+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.590+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.610+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.628+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.647+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.671+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.695+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.732+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.749+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:25.770 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:25.674+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.719+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.774+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.787+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.800+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.845+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.858+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.871+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.948+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:25.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.959+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.766+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.784+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.802+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.822+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.841+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.861+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.025 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.880+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.898+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.916+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.934+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.953+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.971+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:25.987+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.005+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.026 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.023+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.972+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.985+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:25.999+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.064+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.079+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.091+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.106+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.120+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.214 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.199+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.214 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.212+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.285 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.285 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.062+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.285 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.079+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.285 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.096+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.285 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.114+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.285 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.132+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.285 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.153+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.286 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.172+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.286 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.190+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.286 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.208+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.286 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.229+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.286 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.247+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.286 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.265+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.225+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.265+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.292+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.331+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.358+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.371+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.439+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.452+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.284+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.303+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.322+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.340+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.357+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.377+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.397+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.416+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.433+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.452+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.470+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.488+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.507+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.525+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.543+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.663 INFO:tasks.rados.rados.0.smithi118.stdout:2041: expect (ObjNum 592 snap 193 seq_num 592) 2024-03-20T08:08:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2042: finishing copy_from racing read to smithi11881747-1 2024-03-20T08:08:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2042: finishing copy_from to smithi11881747-1 2024-03-20T08:08:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 7067 (ObjNum 554 snap 181 seq_num 554) dirty exists 2024-03-20T08:08:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2041: done (5 left) 2024-03-20T08:08:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2042: done (4 left) 2024-03-20T08:08:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2046: done (3 left) 2024-03-20T08:08:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 209 2024-03-20T08:08:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2047: seq_num 643 ranges {2039808=778240,2818048=401408} 2024-03-20T08:08:26.692 INFO:tasks.rados.rados.0.smithi118.stdout:2047: writing smithi11881747-25 from 2039808 to 2818048 tid 1 2024-03-20T08:08:26.695 INFO:tasks.rados.rados.0.smithi118.stdout:2047: writing smithi11881747-25 from 2818048 to 3219456 tid 2 2024-03-20T08:08:26.695 INFO:tasks.rados.rados.0.smithi118.stdout:2048: read oid 15 snap -1 2024-03-20T08:08:26.695 INFO:tasks.rados.rados.0.smithi118.stdout:2048: expect deleted 2024-03-20T08:08:26.695 INFO:tasks.rados.rados.0.smithi118.stdout:2049: snap_remove snap 199 2024-03-20T08:08:26.723 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:26.723 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-mon[87080]: osdmap e475: 8 total, 8 up, 8 in 2024-03-20T08:08:26.723 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 10b=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:26.723 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:26.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:26.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:26.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:26.724 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-mon[87080]: pgmap v391: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 6 active+clean+snaptrim_error, 109 active+clean; 608 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 9 op/s; 18 MiB/s, 4 objects/s recovering 2024-03-20T08:08:26.724 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:26.663+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:26.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.477+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.490+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.504+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.517+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.544+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.557+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.636+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.664+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:26.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.669+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.683+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.724+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.800 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:26.665+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:26.800 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:26.665+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:26.800 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:26.670+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.562+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.581+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.599+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.616+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.634+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.655+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.683+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.703+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.722+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.743+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.762+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.780+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:08:26.647+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 7 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[93121]: osdmap e475: 8 total, 8 up, 8 in 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 10b=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:26.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[93121]: pgmap v391: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 6 active+clean+snaptrim_error, 109 active+clean; 608 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 9 op/s; 18 MiB/s, 4 objects/s recovering 2024-03-20T08:08:26.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:26.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[95692]: osdmap e475: 8 total, 8 up, 8 in 2024-03-20T08:08:26.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 10b=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:26.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:26.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:26.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:26.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:26.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-mon[95692]: pgmap v391: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 4 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 6 active+clean+snaptrim_error, 109 active+clean; 608 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 9 op/s; 18 MiB/s, 4 objects/s recovering 2024-03-20T08:08:26.804 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:26.667+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.737+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.780+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.792+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.821+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.833+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.846+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.981 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.981 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.981 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.981 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.924+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.981 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.938+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.981 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.981 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:26.981 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.056 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.799+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.817+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.836+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.853+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.871+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.889+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.907+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.925+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.943+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.961+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.979+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:26.998+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.016+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.057 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.035+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.058 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.055+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:26.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.006+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.018+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.044+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.084+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.110+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.122+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.138+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.163+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.191+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.234 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.316 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:27.288+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.073+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.093+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.110+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.129+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.147+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.167+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.185+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.203+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.221+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.241+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.258+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.317 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.278+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.318 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.297+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.318 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.316+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.246+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.313+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.325+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.338+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.378+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.445+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.470+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.482+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.583 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.336+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.355+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.375+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.395+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.414+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.433+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.452+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.471+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.489+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.506+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.526+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.545+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.563+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.584 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.582+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.666 INFO:tasks.rados.rados.0.smithi118.stdout:2049: done (5 left) 2024-03-20T08:08:27.666 INFO:tasks.rados.rados.0.smithi118.stdout:2050: snap_create 2024-03-20T08:08:27.666 INFO:tasks.rados.rados.0.smithi118.stdout:2044: expect (ObjNum 617 snap 202 seq_num 617) 2024-03-20T08:08:27.698 INFO:tasks.rados.rados.0.smithi118.stdout:2043: finishing write tid 1 to smithi11881747-5 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2043: finishing write tid 2 to smithi11881747-5 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2043: finishing write tid 3 to smithi11881747-5 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2043: finishing write tid 4 to smithi11881747-5 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2043: oid 5 updating version 0 to 23064 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2043: oid 5 version 23064 is already newer than 23062 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2043: oid 5 version 23064 is already newer than 23063 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 23064 (ObjNum 642 snap 209 seq_num 642) dirty exists 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2043: left oid 5 (ObjNum 642 snap 209 seq_num 642) 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2047: finishing write tid 1 to smithi11881747-25 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2047: finishing write tid 2 to smithi11881747-25 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2047: finishing write tid 3 to smithi11881747-25 2024-03-20T08:08:27.699 INFO:tasks.rados.rados.0.smithi118.stdout:2047: finishing write tid 4 to smithi11881747-25 2024-03-20T08:08:27.700 INFO:tasks.rados.rados.0.smithi118.stdout:2047: oid 25 updating version 0 to 23066 2024-03-20T08:08:27.700 INFO:tasks.rados.rados.0.smithi118.stdout:2047: oid 25 version 23066 is already newer than 23065 2024-03-20T08:08:27.700 INFO:tasks.rados.rados.0.smithi118.stdout:2047: oid 25 updating version 23066 to 23067 2024-03-20T08:08:27.700 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 23067 (ObjNum 643 snap 209 seq_num 643) dirty exists 2024-03-20T08:08:27.700 INFO:tasks.rados.rados.0.smithi118.stdout:2047: left oid 25 (ObjNum 643 snap 209 seq_num 643) 2024-03-20T08:08:27.745 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:27.674+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:27.745 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 7 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:27.745 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/1242 objects degraded (4.348%), 3 pgs degraded) 2024-03-20T08:08:27.746 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: osdmap e476: 8 total, 8 up, 8 in 2024-03-20T08:08:27.746 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:27.746 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:27.746 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:27.746 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:27.746 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:27.746 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:27.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.509+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.590+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.617+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.630+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.643+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.656+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.673+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.673+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.748 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.748 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.748 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.748 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.746+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:27.837 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:27.674+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:27.838 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:27.680+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:27.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.602+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.620+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.637+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.658+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.684+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.703+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.838 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.723+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.743+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.763+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.780+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.799+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.839 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.818+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:27.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 7 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:27.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/1242 objects degraded (4.348%), 3 pgs degraded) 2024-03-20T08:08:27.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: osdmap e476: 8 total, 8 up, 8 in 2024-03-20T08:08:27.840 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:27.840 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:27.840 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:27.840 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:27.840 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:27.840 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:27.841 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:27.678+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:27.841 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:27.668+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:27.841 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 7 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:27.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/1242 objects degraded (4.348%), 3 pgs degraded) 2024-03-20T08:08:27.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: osdmap e476: 8 total, 8 up, 8 in 2024-03-20T08:08:27.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:27.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:27.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:27.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:27.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:27.842 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:28.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.776+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.788+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.800+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.828+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.875+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.935+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.948+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.966+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.978+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:27.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.005+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.101 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.837+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.855+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.874+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.892+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.916+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.934+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.952+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.970+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:27.988+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.026+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.045+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.065+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.103 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.083+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.103 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.102+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.018+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.101+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.143+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.213+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.239+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.253+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.268+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.122+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.140+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.159+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.182+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.199+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.218+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.237+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.255+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.369 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.275+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.294+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.313+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.332+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.351+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.370 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.369+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.329+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.358+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.435+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.451+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.465+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.479+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.492+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.389+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.408+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.426+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.445+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.464+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.482+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.502+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.521+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.538+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.557+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.633 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.574+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.634 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.593+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.634 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.613+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.634 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.633+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:2043: done (5 left) 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:2044: done (4 left) 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:2045: done (3 left) 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:2047: done (2 left) 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:2048: done (1 left) 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:2050: done (0 left) 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:2051: rollback oid 39 current snap is 210 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 204 2024-03-20T08:08:28.676 INFO:tasks.rados.rados.0.smithi118.stdout:2052: snap_create 2024-03-20T08:08:28.737 INFO:tasks.rados.rados.0.smithi118.stdout:2051: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:08:28.744 INFO:tasks.rados.rados.0.smithi118.stdout:2051: finishing rollback tid 2 to smithi11881747-39 2024-03-20T08:08:28.744 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7114 (ObjNum 623 snap 202 seq_num 623) dirty exists 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.564+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.621+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.635+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.787 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.681+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.687+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.700+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.711+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.724+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.738+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.770+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:28.788 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: osdmap e477: 8 total, 8 up, 8 in 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: pgmap v394: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 608 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 6 op/s; 3.5 MiB/s, 1 objects/s recovering 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:28.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:28.789 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:28.690+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:28.901 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:28.684+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:28.901 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:28.684+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:28.901 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:28.684+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:28.901 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.652+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.676+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.696+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.712+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.730+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.750+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.769+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.786+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.807+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.826+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.845+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.902 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.864+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.903 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.883+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:28.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: osdmap e477: 8 total, 8 up, 8 in 2024-03-20T08:08:28.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:28.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:28.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:28.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:28.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:28.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:28.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: pgmap v394: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 608 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 6 op/s; 3.5 MiB/s, 1 objects/s recovering 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: osdmap e477: 8 total, 8 up, 8 in 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:28.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:28.905 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: pgmap v394: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 608 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 6 op/s; 3.5 MiB/s, 1 objects/s recovering 2024-03-20T08:08:28.905 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:28.905 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:28.905 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:28.674+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:28.905 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:28.740+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.787+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.820+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.882+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.912+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.976+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:28.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.003+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.017+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.046+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.167 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.901+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.920+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.939+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:28.994+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.013+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.032+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.051+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.071+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.089+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.106+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.125+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.168 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.147+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.074+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.089+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.104+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.131+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.158+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.246+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.288+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.167+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.185+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.223+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.242+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.264+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.282+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.299+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.318+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.337+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.355+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.374+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.409+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.313+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.326+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.340+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.370+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.384+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.556 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.675 INFO:tasks.rados.rados.0.smithi118.stdout:2051: done (1 left) 2024-03-20T08:08:29.675 INFO:tasks.rados.rados.0.smithi118.stdout:2052: done (0 left) 2024-03-20T08:08:29.675 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 211 2024-03-20T08:08:29.675 INFO:tasks.rados.rados.0.smithi118.stdout:2053: seq_num 644 ranges {1810432=794624,2605056=401408} 2024-03-20T08:08:29.685 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:29.672+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:29.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:29.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[95692]: osdmap e478: 8 total, 8 up, 8 in 2024-03-20T08:08:29.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:29.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:29.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:29.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:29.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:29.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:29.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[93121]: osdmap e478: 8 total, 8 up, 8 in 2024-03-20T08:08:29.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:29.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:29.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:29.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:29.687 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:29.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.428+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.445+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.687 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.465+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.486+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.504+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.524+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.542+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.561+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.580+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.600+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.619+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.640+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.658+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.688 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.685+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.690 INFO:tasks.rados.rados.0.smithi118.stdout:2053: writing smithi11881747-28 from 1810432 to 2605056 tid 1 2024-03-20T08:08:29.693 INFO:tasks.rados.rados.0.smithi118.stdout:2053: writing smithi11881747-28 from 2605056 to 3006464 tid 2 2024-03-20T08:08:29.693 INFO:tasks.rados.rados.0.smithi118.stdout:2054: snap_create 2024-03-20T08:08:29.699 INFO:tasks.rados.rados.0.smithi118.stdout:2053: finishing write tid 1 to smithi11881747-28 2024-03-20T08:08:29.701 INFO:tasks.rados.rados.0.smithi118.stdout:2053: finishing write tid 2 to smithi11881747-28 2024-03-20T08:08:29.703 INFO:tasks.rados.rados.0.smithi118.stdout:2053: finishing write tid 3 to smithi11881747-28 2024-03-20T08:08:29.705 INFO:tasks.rados.rados.0.smithi118.stdout:2053: finishing write tid 4 to smithi11881747-28 2024-03-20T08:08:29.705 INFO:tasks.rados.rados.0.smithi118.stdout:2053: oid 28 updating version 0 to 33904 2024-03-20T08:08:29.705 INFO:tasks.rados.rados.0.smithi118.stdout:2053: oid 28 updating version 33904 to 33906 2024-03-20T08:08:29.706 INFO:tasks.rados.rados.0.smithi118.stdout:2053: oid 28 version 33906 is already newer than 33905 2024-03-20T08:08:29.706 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 33906 (ObjNum 644 snap 211 seq_num 644) dirty exists 2024-03-20T08:08:29.706 INFO:tasks.rados.rados.0.smithi118.stdout:2053: left oid 28 (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:08:29.813 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:29.686+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:29.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:29.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-mon[87080]: osdmap e478: 8 total, 8 up, 8 in 2024-03-20T08:08:29.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:29.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:29.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:29.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:29.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:29.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.596+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.677+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.685+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.685+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.696+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.726+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.740+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.754+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:29.937 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:29.686+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:29.686+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.710+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.729+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.748+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.766+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.785+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.804+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.823+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.841+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.938 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.861+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.881+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.901+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.939 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.919+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:29.939 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:29.691+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:30.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.885+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.900+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.915+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.945+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:29.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.040+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.199 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.938+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.975+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:29.994+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.011+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.029+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.047+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.066+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.084+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.103+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.122+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.144+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.162+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.200 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.199+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.120+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.161+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.175+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.201+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.215+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.464 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.218+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.236+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.255+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.273+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.292+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.310+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.327+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.347+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.365+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.387+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.408+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.425+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.444+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.464+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.341+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.354+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.366+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.380+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.406+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.420+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.435+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.488+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.544+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.585+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.694 INFO:tasks.rados.rados.0.smithi118.stdout:2053: done (1 left) 2024-03-20T08:08:30.694 INFO:tasks.rados.rados.0.smithi118.stdout:2054: done (0 left) 2024-03-20T08:08:30.694 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 212 2024-03-20T08:08:30.694 INFO:tasks.rados.rados.0.smithi118.stdout:2055: seq_num 645 ranges {2088960=409600,2498560=778240,3276800=24576} 2024-03-20T08:08:30.708 INFO:tasks.rados.rados.0.smithi118.stdout:2055: writing smithi11881747-22 from 2088960 to 2498560 tid 1 2024-03-20T08:08:30.714 INFO:tasks.rados.rados.0.smithi118.stdout:2055: writing smithi11881747-22 from 2498560 to 3276800 tid 2 2024-03-20T08:08:30.715 INFO:tasks.rados.rados.0.smithi118.stdout:2055: writing smithi11881747-22 from 3276800 to 3301376 tid 3 2024-03-20T08:08:30.715 INFO:tasks.rados.rados.0.smithi118.stdout:2056: copy_from oid 50 from oid 15 current snap is 212 2024-03-20T08:08:30.715 INFO:tasks.rados.rados.0.smithi118.stdout:2055: finishing write tid 1 to smithi11881747-22 2024-03-20T08:08:30.715 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 212 2024-03-20T08:08:30.715 INFO:tasks.rados.rados.0.smithi118.stdout:2057: seq_num 646 ranges {0=425984,425984=655360,1081344=147456} 2024-03-20T08:08:30.717 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:30.696+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:30.717 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:30.696+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:30.717 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:30.697+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:30.717 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:30.698+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:30.717 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:30.687+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.485+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.506+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.526+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.544+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.562+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.580+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.599+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.617+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.634+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.652+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.670+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.692+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.719 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.717+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.719 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:30.719 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[95692]: osdmap e479: 8 total, 8 up, 8 in 2024-03-20T08:08:30.719 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:30.719 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:30.719 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:30.719 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:30.719 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[95692]: pgmap v397: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 615 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 775 KiB/s rd, 790 KiB/s wr, 3 op/s 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[93121]: osdmap e479: 8 total, 8 up, 8 in 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:30.720 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-mon[93121]: pgmap v397: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 615 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 775 KiB/s rd, 790 KiB/s wr, 3 op/s 2024-03-20T08:08:30.721 INFO:tasks.rados.rados.0.smithi118.stdout:2057: writing smithi11881747-8 from 0 to 425984 tid 1 2024-03-20T08:08:30.723 INFO:tasks.rados.rados.0.smithi118.stdout:2057: writing smithi11881747-8 from 425984 to 1081344 tid 2 2024-03-20T08:08:30.724 INFO:tasks.rados.rados.0.smithi118.stdout:2057: writing smithi11881747-8 from 1081344 to 1228800 tid 3 2024-03-20T08:08:30.725 INFO:tasks.rados.rados.0.smithi118.stdout:2058: copy_from oid 31 from oid 27 current snap is 212 2024-03-20T08:08:30.725 INFO:tasks.rados.rados.0.smithi118.stdout:2059: snap_create 2024-03-20T08:08:30.726 INFO:tasks.rados.rados.0.smithi118.stdout:2055: finishing write tid 2 to smithi11881747-22 2024-03-20T08:08:30.726 INFO:tasks.rados.rados.0.smithi118.stdout:2055: finishing write tid 3 to smithi11881747-22 2024-03-20T08:08:30.726 INFO:tasks.rados.rados.0.smithi118.stdout:2055: finishing write tid 4 to smithi11881747-22 2024-03-20T08:08:30.727 INFO:tasks.rados.rados.0.smithi118.stdout:2055: finishing write tid 5 to smithi11881747-22 2024-03-20T08:08:30.727 INFO:tasks.rados.rados.0.smithi118.stdout:2055: oid 22 updating version 0 to 7079 2024-03-20T08:08:30.728 INFO:tasks.rados.rados.0.smithi118.stdout:2055: oid 22 version 7079 is already newer than 7077 2024-03-20T08:08:30.728 INFO:tasks.rados.rados.0.smithi118.stdout:2055: oid 22 version 7079 is already newer than 7078 2024-03-20T08:08:30.728 INFO:tasks.rados.rados.0.smithi118.stdout:2055: oid 22 version 7079 is already newer than 7076 2024-03-20T08:08:30.728 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 7079 (ObjNum 645 snap 212 seq_num 645) dirty exists 2024-03-20T08:08:30.728 INFO:tasks.rados.rados.0.smithi118.stdout:2055: left oid 22 (ObjNum 645 snap 212 seq_num 645) 2024-03-20T08:08:30.732 INFO:tasks.rados.rados.0.smithi118.stdout:2056: finishing copy_from to smithi11881747-50 2024-03-20T08:08:30.733 INFO:tasks.rados.rados.0.smithi118.stdout:2056: got expected ENOENT (src dne) 2024-03-20T08:08:30.733 INFO:tasks.rados.rados.0.smithi118.stdout:2056: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:08:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:2057: finishing write tid 1 to smithi11881747-8 2024-03-20T08:08:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:2057: finishing write tid 2 to smithi11881747-8 2024-03-20T08:08:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:2057: finishing write tid 3 to smithi11881747-8 2024-03-20T08:08:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:2057: finishing write tid 4 to smithi11881747-8 2024-03-20T08:08:30.750 INFO:tasks.rados.rados.0.smithi118.stdout:2057: finishing write tid 5 to smithi11881747-8 2024-03-20T08:08:30.750 INFO:tasks.rados.rados.0.smithi118.stdout:2057: oid 8 updating version 0 to 33960 2024-03-20T08:08:30.750 INFO:tasks.rados.rados.0.smithi118.stdout:2057: oid 8 updating version 33960 to 33962 2024-03-20T08:08:30.750 INFO:tasks.rados.rados.0.smithi118.stdout:2057: oid 8 version 33962 is already newer than 33959 2024-03-20T08:08:30.750 INFO:tasks.rados.rados.0.smithi118.stdout:2057: oid 8 version 33962 is already newer than 33961 2024-03-20T08:08:30.750 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 33962 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:08:30.750 INFO:tasks.rados.rados.0.smithi118.stdout:2057: left oid 8 (ObjNum 646 snap 212 seq_num 646) 2024-03-20T08:08:30.771 INFO:tasks.rados.rados.0.smithi118.stdout:2058: finishing copy_from to smithi11881747-31 2024-03-20T08:08:30.771 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 7108 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:08:30.772 INFO:tasks.rados.rados.0.smithi118.stdout:2058: finishing copy_from racing read to smithi11881747-31 2024-03-20T08:08:30.847 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:30.695+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:30.847 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:30.848 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-mon[87080]: osdmap e479: 8 total, 8 up, 8 in 2024-03-20T08:08:30.848 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:30.848 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:30.848 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:30.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:30.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:30.849 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-mon[87080]: pgmap v397: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 615 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 775 KiB/s rd, 790 KiB/s wr, 3 op/s 2024-03-20T08:08:30.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.616+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.629+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.656+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.670+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.686+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.699+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:30.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.834+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.847+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:30.972 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.741+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.762+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.785+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.803+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.822+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.840+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.858+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.880+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.898+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.916+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.934+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.953+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:30.973 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.972+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.875+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.902+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.915+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.929+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.969+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:30.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.037+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.064+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.080+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.093+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.238 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:30.994+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.012+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.031+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.051+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.070+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.089+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.108+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.128+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.239 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.146+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.163+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.182+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.200+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.240 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.219+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.133+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.146+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.226+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.239+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.253+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.292+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.321+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.333+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.238+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.258+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.500 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.276+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.295+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.315+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.335+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.353+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.371+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.389+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.407+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.425+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.445+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.463+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.501 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.482+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.385+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.426+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.455+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.482+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.509+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.576+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.589+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.603+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.616+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.700 INFO:tasks.rados.rados.0.smithi118.stdout:2055: done (4 left) 2024-03-20T08:08:31.700 INFO:tasks.rados.rados.0.smithi118.stdout:2056: done (3 left) 2024-03-20T08:08:31.700 INFO:tasks.rados.rados.0.smithi118.stdout:2057: done (2 left) 2024-03-20T08:08:31.700 INFO:tasks.rados.rados.0.smithi118.stdout:2058: done (1 left) 2024-03-20T08:08:31.700 INFO:tasks.rados.rados.0.smithi118.stdout:2059: done (0 left) 2024-03-20T08:08:31.700 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 213 2024-03-20T08:08:31.700 INFO:tasks.rados.rados.0.smithi118.stdout:2060: seq_num 647 ranges {1499136=442368,1941504=589824,2531328=212992} 2024-03-20T08:08:31.711 INFO:tasks.rados.rados.0.smithi118.stdout:2060: writing smithi11881747-43 from 1499136 to 1941504 tid 1 2024-03-20T08:08:31.715 INFO:tasks.rados.rados.0.smithi118.stdout:2060: writing smithi11881747-43 from 1941504 to 2531328 tid 2 2024-03-20T08:08:31.717 INFO:tasks.rados.rados.0.smithi118.stdout:2060: writing smithi11881747-43 from 2531328 to 2744320 tid 3 2024-03-20T08:08:31.717 INFO:tasks.rados.rados.0.smithi118.stdout:2061: delete oid 18 current snap is 213 2024-03-20T08:08:31.717 INFO:tasks.rados.rados.0.smithi118.stdout:2060: finishing write tid 1 to smithi11881747-43 2024-03-20T08:08:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:2060: finishing write tid 2 to smithi11881747-43 2024-03-20T08:08:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:2060: finishing write tid 3 to smithi11881747-43 2024-03-20T08:08:31.722 INFO:tasks.rados.rados.0.smithi118.stdout:2060: finishing write tid 4 to smithi11881747-43 2024-03-20T08:08:31.726 INFO:tasks.rados.rados.0.smithi118.stdout:2061: done (1 left) 2024-03-20T08:08:31.726 INFO:tasks.rados.rados.0.smithi118.stdout:2062: read oid 13 snap -1 2024-03-20T08:08:31.726 INFO:tasks.rados.rados.0.smithi118.stdout:2062: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:08:31.726 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 213 2024-03-20T08:08:31.726 INFO:tasks.rados.rados.0.smithi118.stdout:2063: seq_num 649 ranges {3760128=475136,4235264=761856,4997120=40960} 2024-03-20T08:08:31.750 INFO:tasks.rados.rados.0.smithi118.stdout:2063: writing smithi11881747-39 from 3760128 to 4235264 tid 1 2024-03-20T08:08:31.755 INFO:tasks.rados.rados.0.smithi118.stdout:2063: writing smithi11881747-39 from 4235264 to 4997120 tid 2 2024-03-20T08:08:31.756 INFO:tasks.rados.rados.0.smithi118.stdout:2063: writing smithi11881747-39 from 4997120 to 5038080 tid 3 2024-03-20T08:08:31.756 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 213 2024-03-20T08:08:31.756 INFO:tasks.rados.rados.0.smithi118.stdout:2060: finishing write tid 5 to smithi11881747-43 2024-03-20T08:08:31.756 INFO:tasks.rados.rados.0.smithi118.stdout:2060: oid 43 updating version 0 to 7111 2024-03-20T08:08:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:2060: oid 43 updating version 7111 to 7112 2024-03-20T08:08:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:2060: oid 43 version 7112 is already newer than 7110 2024-03-20T08:08:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:2060: oid 43 version 7112 is already newer than 7109 2024-03-20T08:08:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 7112 (ObjNum 647 snap 213 seq_num 647) dirty exists 2024-03-20T08:08:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:2060: left oid 43 (ObjNum 647 snap 213 seq_num 647) 2024-03-20T08:08:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:2064: seq_num 650 ranges {0=491520,491520=647168,1138688=155648} 2024-03-20T08:08:31.759 INFO:tasks.rados.rados.0.smithi118.stdout:2064: writing smithi11881747-30 from 0 to 491520 tid 1 2024-03-20T08:08:31.764 INFO:tasks.rados.rados.0.smithi118.stdout:2064: writing smithi11881747-30 from 491520 to 1138688 tid 2 2024-03-20T08:08:31.766 INFO:tasks.rados.rados.0.smithi118.stdout:2064: writing smithi11881747-30 from 1138688 to 1294336 tid 3 2024-03-20T08:08:31.766 INFO:tasks.rados.rados.0.smithi118.stdout:2060: done (3 left) 2024-03-20T08:08:31.766 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 213 2024-03-20T08:08:31.766 INFO:tasks.rados.rados.0.smithi118.stdout:2065: seq_num 651 ranges {1867776=507904,2375680=532480,2908160=270336} 2024-03-20T08:08:31.770 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:31.705+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:31.770 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:31.705+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:31.771 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:31.712+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:31.771 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:31.701+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:31.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.500+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.520+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.538+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.556+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.575+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.594+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.611+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.629+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.647+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.665+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.683+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.702+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.726+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.746+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.764+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[95692]: osdmap e480: 8 total, 8 up, 8 in 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[93121]: osdmap e480: 8 total, 8 up, 8 in 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:31.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:31.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:31.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:31.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:31.779 INFO:tasks.rados.rados.0.smithi118.stdout:2065: writing smithi11881747-9 from 1867776 to 2375680 tid 1 2024-03-20T08:08:31.783 INFO:tasks.rados.rados.0.smithi118.stdout:2065: writing smithi11881747-9 from 2375680 to 2908160 tid 2 2024-03-20T08:08:31.786 INFO:tasks.rados.rados.0.smithi118.stdout:2065: writing smithi11881747-9 from 2908160 to 3178496 tid 3 2024-03-20T08:08:31.786 INFO:tasks.rados.rados.0.smithi118.stdout:2066: read oid 10 snap -1 2024-03-20T08:08:31.786 INFO:tasks.rados.rados.0.smithi118.stdout:2066: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:08:31.786 INFO:tasks.rados.rados.0.smithi118.stdout:2067: snap_remove snap 164 2024-03-20T08:08:31.873 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:31.705+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:31.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:31.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-mon[87080]: osdmap e480: 8 total, 8 up, 8 in 2024-03-20T08:08:31.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:31.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:31.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:31.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:31.874 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.630+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.644+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.659+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.685+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.703+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.703+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.819+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.846+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:31.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.874+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.035 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.788+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.035 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.809+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.830+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.849+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.868+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.886+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.903+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.922+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.941+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.960+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.979+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:31.997+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.016+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.035+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.887+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.900+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.913+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:31.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.053+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.066+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.091+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.054+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.073+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.092+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.110+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.128+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.149+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.168+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.186+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.298 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.204+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.222+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.240+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.259+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.299 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.277+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.384 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:08:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:08:32.384 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.384 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.184+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.200+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.213+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.237+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.290+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.329+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.371+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.298+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.318+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.337+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.356+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.375+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.395+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.413+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.432+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.450+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.469+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.562 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.488+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.507+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.525+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.544+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.563 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.561+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.636 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.636 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.636 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.424+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.636 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.636 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.636 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.464+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.636 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.636 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.488+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.540+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.552+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.621+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.637 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.636+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.708 INFO:tasks.rados.rados.0.smithi118.stdout:2062: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:08:32.827 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:32.713+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:32.827 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:32.713+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:32.828 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:32.714+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:32.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[95692]: osdmap e481: 8 total, 8 up, 8 in 2024-03-20T08:08:32.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:32.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:32.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:32.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:32.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:32.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:32.828 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[95692]: pgmap v400: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 617 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-03-20T08:08:32.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[93121]: osdmap e481: 8 total, 8 up, 8 in 2024-03-20T08:08:32.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:32.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:32.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:32.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:32.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:32.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:32.829 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-mon[93121]: pgmap v400: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 617 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-03-20T08:08:32.829 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:32.717+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:32.707+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.579+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.601+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.619+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.637+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.656+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.674+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.693+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.721+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.830 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.740+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.756+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.773+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.791+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.808+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.831 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.828+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:32.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-mon[87080]: osdmap e481: 8 total, 8 up, 8 in 2024-03-20T08:08:32.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:32.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 105=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:32.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 10c=[]:{9d=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c]} for object 5:f8faee4c:::smithi11881747-1:e3 2024-03-20T08:08:32.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:32.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:32.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:32.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-mon[87080]: pgmap v400: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 617 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 235 KiB/s rd, 1.3 MiB/s wr, 4 op/s 2024-03-20T08:08:32.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:32.704+0000 7ffa71974700 -1 osd.5 pg_epoch: 481 pg[5.2s2( v 480'1023 (0'0,480'1023] local-lis/les=470/471 n=16 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [6,0,5]p6(0) r=2 lpr=470 luod=0'0 crt=480'1023 mlcod 480'1023 active mbc={} ps=97] update_snap_map remove_oid 5:44a43f74:::smithi11881747-38:e0 failed with -2 2024-03-20T08:08:32.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:32.712+0000 7ffa6b167700 -1 osd.5 pg_epoch: 482 pg[5.fs0( v 482'1003 (0'0,482'1003] local-lis/les=470/471 n=21 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [5,4,6]p5(0) r=0 lpr=470 luod=481'1001 crt=481'1001 lcod 481'1000 mlcod 481'1000 active+clean+snaptrim trimq=[e0~1,e3~1,f5~1,f7~2,fa~1,fc~1,fe~1,103~2,109~1,10b~1] ps=86] update_snap_map remove_oid 5:f9443f50:::smithi11881747-10:e1 failed with -2 2024-03-20T08:08:32.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:32.722+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:32.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:32.730+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:32.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.662+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.676+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.714+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.769+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.793+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.805+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.828+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.841+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.865+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2063: finishing write tid 1 to smithi11881747-39 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2062: done (5 left) 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2067: done (4 left) 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2068: rollback oid 32 current snap is 213 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 32 to 209 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2064: finishing write tid 1 to smithi11881747-30 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2063: finishing write tid 2 to smithi11881747-39 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2063: finishing write tid 3 to smithi11881747-39 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2063: finishing write tid 4 to smithi11881747-39 2024-03-20T08:08:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:2064: finishing write tid 2 to smithi11881747-30 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2064: finishing write tid 3 to smithi11881747-30 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2064: finishing write tid 4 to smithi11881747-30 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2063: finishing write tid 5 to smithi11881747-39 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2063: oid 39 updating version 0 to 7116 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2063: oid 39 updating version 7116 to 7119 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2063: oid 39 version 7119 is already newer than 7118 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2063: oid 39 version 7119 is already newer than 7117 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7119 (ObjNum 649 snap 213 seq_num 649) dirty exists 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2063: left oid 39 (ObjNum 649 snap 213 seq_num 649) 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2064: finishing write tid 5 to smithi11881747-30 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2064: oid 30 updating version 0 to 1304 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2064: oid 30 version 1304 is already newer than 1303 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2064: oid 30 updating version 1304 to 1305 2024-03-20T08:08:32.923 INFO:tasks.rados.rados.0.smithi118.stdout:2064: oid 30 version 1305 is already newer than 1302 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 1305 (ObjNum 650 snap 213 seq_num 650) dirty exists 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2064: left oid 30 (ObjNum 650 snap 213 seq_num 650) 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: finishing write tid 1 to smithi11881747-9 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: finishing write tid 2 to smithi11881747-9 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: finishing write tid 3 to smithi11881747-9 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: finishing write tid 4 to smithi11881747-9 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: finishing write tid 5 to smithi11881747-9 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: oid 9 updating version 0 to 23068 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: oid 9 updating version 23068 to 23071 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: oid 9 version 23071 is already newer than 23069 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: oid 9 version 23071 is already newer than 23070 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 23071 (ObjNum 651 snap 213 seq_num 651) dirty exists 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2065: left oid 9 (ObjNum 651 snap 213 seq_num 651) 2024-03-20T08:08:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:2066: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.846+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.862+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.880+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.898+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.916+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.935+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.954+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.972+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.090 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:32.990+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.091 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.007+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.091 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.024+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.091 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.040+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.091 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.056+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.091 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.072+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.091 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.090+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.902+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.913+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.953+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:32.989+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.000+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.037+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.049+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.074+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.145+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.107+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.125+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.142+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.160+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.352 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.176+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.193+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.210+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.228+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.247+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.264+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.283+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.300+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.317+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.335+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.353 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.352+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.181+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.193+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.218+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.253+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.278+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.290+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.315+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.328+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.353+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.367+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.484 INFO:tasks.rados.rados.0.smithi118.stdout:2068: finishing rollback tid 0 to smithi11881747-32 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2068: finishing rollback tid 1 to smithi11881747-32 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2068: finishing rollback tid 2 to smithi11881747-32 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 33970 (ObjNum 607 snap 198 seq_num 607) dirty exists 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2063: done (4 left) 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2064: done (3 left) 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2065: done (2 left) 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2066: done (1 left) 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2068: done (0 left) 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2069: read oid 21 snap -1 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2069: expect (ObjNum 545 snap 179 seq_num 545) 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 213 2024-03-20T08:08:33.485 INFO:tasks.rados.rados.0.smithi118.stdout:2070: seq_num 652 ranges {3219456=524288,3743744=409600,4153344=393216} 2024-03-20T08:08:33.505 INFO:tasks.rados.rados.0.smithi118.stdout:2070: writing smithi11881747-25 from 3219456 to 3743744 tid 1 2024-03-20T08:08:33.508 INFO:tasks.rados.rados.0.smithi118.stdout:2070: writing smithi11881747-25 from 3743744 to 4153344 tid 2 2024-03-20T08:08:33.511 INFO:tasks.rados.rados.0.smithi118.stdout:2070: writing smithi11881747-25 from 4153344 to 4546560 tid 3 2024-03-20T08:08:33.511 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 213 2024-03-20T08:08:33.511 INFO:tasks.rados.rados.0.smithi118.stdout:2071: seq_num 653 ranges {3022848=540672,3563520=696320,4259840=106496} 2024-03-20T08:08:33.531 INFO:tasks.rados.rados.0.smithi118.stdout:2071: writing smithi11881747-6 from 3022848 to 3563520 tid 1 2024-03-20T08:08:33.535 INFO:tasks.rados.rados.0.smithi118.stdout:2071: writing smithi11881747-6 from 3563520 to 4259840 tid 2 2024-03-20T08:08:33.536 INFO:tasks.rados.rados.0.smithi118.stdout:2071: writing smithi11881747-6 from 4259840 to 4366336 tid 3 2024-03-20T08:08:33.537 INFO:tasks.rados.rados.0.smithi118.stdout:2070: finishing write tid 1 to smithi11881747-25 2024-03-20T08:08:33.537 INFO:tasks.rados.rados.0.smithi118.stdout:2070: finishing write tid 2 to smithi11881747-25 2024-03-20T08:08:33.537 INFO:tasks.rados.rados.0.smithi118.stdout:2070: finishing write tid 3 to smithi11881747-25 2024-03-20T08:08:33.537 INFO:tasks.rados.rados.0.smithi118.stdout:2070: finishing write tid 4 to smithi11881747-25 2024-03-20T08:08:33.538 INFO:tasks.rados.rados.0.smithi118.stdout:2070: finishing write tid 5 to smithi11881747-25 2024-03-20T08:08:33.538 INFO:tasks.rados.rados.0.smithi118.stdout:2070: oid 25 updating version 0 to 23075 2024-03-20T08:08:33.538 INFO:tasks.rados.rados.0.smithi118.stdout:2070: oid 25 version 23075 is already newer than 23072 2024-03-20T08:08:33.538 INFO:tasks.rados.rados.0.smithi118.stdout:2070: oid 25 version 23075 is already newer than 23074 2024-03-20T08:08:33.538 INFO:tasks.rados.rados.0.smithi118.stdout:2070: oid 25 version 23075 is already newer than 23073 2024-03-20T08:08:33.538 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 23075 (ObjNum 652 snap 213 seq_num 652) dirty exists 2024-03-20T08:08:33.538 INFO:tasks.rados.rados.0.smithi118.stdout:2070: left oid 25 (ObjNum 652 snap 213 seq_num 652) 2024-03-20T08:08:33.538 INFO:tasks.rados.rados.0.smithi118.stdout:2069: expect (ObjNum 545 snap 179 seq_num 545) 2024-03-20T08:08:33.605 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.370+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.388+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.407+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.425+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.443+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.459+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.476+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.495+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.519+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.537+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.555+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.606 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.571+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.607 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.588+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.607 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:08:33] "GET /metrics HTTP/1.1" 200 36348 "" "Prometheus/2.43.0" 2024-03-20T08:08:33.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.415+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.427+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.439+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.451+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.503+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.520+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.665 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.665 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.665 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.576+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.665 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.665 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.665 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.665 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.637+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.650+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.870 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.606+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.623+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.639+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.660+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.678+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.694+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.714+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.731+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.751+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.766+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.784+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.801+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.871 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.818+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.836+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.872 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.853+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:33.872 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: osdmap e482: 8 total, 8 up, 8 in 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: osdmap e482: 8 total, 8 up, 8 in 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:33.873 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:33.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:33.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:33.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:33.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:33.874 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: osdmap e482: 8 total, 8 up, 8 in 2024-03-20T08:08:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:33.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:33.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:33.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.687+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.713+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.778+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.790+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.801+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.888+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:33.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.915+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.033 INFO:tasks.rados.rados.0.smithi118.stdout:2071: finishing write tid 1 to smithi11881747-6 2024-03-20T08:08:34.033 INFO:tasks.rados.rados.0.smithi118.stdout:2069: done (2 left) 2024-03-20T08:08:34.033 INFO:tasks.rados.rados.0.smithi118.stdout:2070: done (1 left) 2024-03-20T08:08:34.033 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 213 2024-03-20T08:08:34.033 INFO:tasks.rados.rados.0.smithi118.stdout:2071: finishing write tid 2 to smithi11881747-6 2024-03-20T08:08:34.034 INFO:tasks.rados.rados.0.smithi118.stdout:2071: finishing write tid 3 to smithi11881747-6 2024-03-20T08:08:34.034 INFO:tasks.rados.rados.0.smithi118.stdout:2071: finishing write tid 4 to smithi11881747-6 2024-03-20T08:08:34.034 INFO:tasks.rados.rados.0.smithi118.stdout:2072: seq_num 654 ranges {2228224=557056,2785280=630784,3416064=172032} 2024-03-20T08:08:34.049 INFO:tasks.rados.rados.0.smithi118.stdout:2072: writing smithi11881747-44 from 2228224 to 2785280 tid 1 2024-03-20T08:08:34.052 INFO:tasks.rados.rados.0.smithi118.stdout:2072: writing smithi11881747-44 from 2785280 to 3416064 tid 2 2024-03-20T08:08:34.054 INFO:tasks.rados.rados.0.smithi118.stdout:2072: writing smithi11881747-44 from 3416064 to 3588096 tid 3 2024-03-20T08:08:34.054 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 213 2024-03-20T08:08:34.054 INFO:tasks.rados.rados.0.smithi118.stdout:2071: finishing write tid 5 to smithi11881747-6 2024-03-20T08:08:34.054 INFO:tasks.rados.rados.0.smithi118.stdout:2071: oid 6 updating version 0 to 9332 2024-03-20T08:08:34.054 INFO:tasks.rados.rados.0.smithi118.stdout:2071: oid 6 updating version 9332 to 9333 2024-03-20T08:08:34.054 INFO:tasks.rados.rados.0.smithi118.stdout:2071: oid 6 updating version 9333 to 9334 2024-03-20T08:08:34.054 INFO:tasks.rados.rados.0.smithi118.stdout:2071: oid 6 updating version 9334 to 9335 2024-03-20T08:08:34.054 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 9335 (ObjNum 653 snap 213 seq_num 653) dirty exists 2024-03-20T08:08:34.055 INFO:tasks.rados.rados.0.smithi118.stdout:2071: left oid 6 (ObjNum 653 snap 213 seq_num 653) 2024-03-20T08:08:34.055 INFO:tasks.rados.rados.0.smithi118.stdout:2073: seq_num 655 ranges {0=573440,573440=516096,1089536=286720} 2024-03-20T08:08:34.058 INFO:tasks.rados.rados.0.smithi118.stdout:2073: writing smithi11881747-45 from 0 to 573440 tid 1 2024-03-20T08:08:34.062 INFO:tasks.rados.rados.0.smithi118.stdout:2073: writing smithi11881747-45 from 573440 to 1089536 tid 2 2024-03-20T08:08:34.064 INFO:tasks.rados.rados.0.smithi118.stdout:2073: writing smithi11881747-45 from 1089536 to 1376256 tid 3 2024-03-20T08:08:34.064 INFO:tasks.rados.rados.0.smithi118.stdout:2071: done (2 left) 2024-03-20T08:08:34.064 INFO:tasks.rados.rados.0.smithi118.stdout:2074: copy_from oid 20 from oid 24 current snap is 213 2024-03-20T08:08:34.064 INFO:tasks.rados.rados.0.smithi118.stdout:2072: finishing write tid 1 to smithi11881747-44 2024-03-20T08:08:34.064 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 213 2024-03-20T08:08:34.064 INFO:tasks.rados.rados.0.smithi118.stdout:2075: seq_num 656 ranges {0=589824,589824=802816} 2024-03-20T08:08:34.067 INFO:tasks.rados.rados.0.smithi118.stdout:2075: writing smithi11881747-41 from 0 to 589824 tid 1 2024-03-20T08:08:34.074 INFO:tasks.rados.rados.0.smithi118.stdout:2075: writing smithi11881747-41 from 589824 to 1392640 tid 2 2024-03-20T08:08:34.074 INFO:tasks.rados.rados.0.smithi118.stdout:2076: read oid 23 snap -1 2024-03-20T08:08:34.074 INFO:tasks.rados.rados.0.smithi118.stdout:2076: expect (ObjNum 626 snap 203 seq_num 626) 2024-03-20T08:08:34.074 INFO:tasks.rados.rados.0.smithi118.stdout:2077: read oid 36 snap -1 2024-03-20T08:08:34.074 INFO:tasks.rados.rados.0.smithi118.stdout:2077: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:08:34.074 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 213 2024-03-20T08:08:34.074 INFO:tasks.rados.rados.0.smithi118.stdout:2078: seq_num 657 ranges {655360=606208,1261568=688128,1949696=114688} 2024-03-20T08:08:34.081 INFO:tasks.rados.rados.0.smithi118.stdout:2078: writing smithi11881747-47 from 655360 to 1261568 tid 1 2024-03-20T08:08:34.086 INFO:tasks.rados.rados.0.smithi118.stdout:2078: writing smithi11881747-47 from 1261568 to 1949696 tid 2 2024-03-20T08:08:34.087 INFO:tasks.rados.rados.0.smithi118.stdout:2078: writing smithi11881747-47 from 1949696 to 2064384 tid 3 2024-03-20T08:08:34.087 INFO:tasks.rados.rados.0.smithi118.stdout:2079: setattr oid 37 current snap is 213 2024-03-20T08:08:34.087 INFO:tasks.rados.rados.0.smithi118.stdout:2072: finishing write tid 2 to smithi11881747-44 2024-03-20T08:08:34.087 INFO:tasks.rados.rados.0.smithi118.stdout:2072: finishing write tid 3 to smithi11881747-44 2024-03-20T08:08:34.087 INFO:tasks.rados.rados.0.smithi118.stdout:2072: finishing write tid 4 to smithi11881747-44 2024-03-20T08:08:34.087 INFO:tasks.rados.rados.0.smithi118.stdout:2075: finishing write tid 1 to smithi11881747-41 2024-03-20T08:08:34.088 INFO:tasks.rados.rados.0.smithi118.stdout:2077: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2072: finishing write tid 5 to smithi11881747-44 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2072: oid 44 updating version 0 to 9656 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2072: oid 44 version 9656 is already newer than 9655 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2072: oid 44 version 9656 is already newer than 9654 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2072: oid 44 updating version 9656 to 9657 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 9657 (ObjNum 654 snap 213 seq_num 654) dirty exists 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2072: left oid 44 (ObjNum 654 snap 213 seq_num 654) 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2074: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2074: finishing copy_from to smithi11881747-20 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 7076 (ObjNum 617 snap 202 seq_num 617) dirty exists 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2075: finishing write tid 2 to smithi11881747-41 2024-03-20T08:08:34.096 INFO:tasks.rados.rados.0.smithi118.stdout:2075: finishing write tid 3 to smithi11881747-41 2024-03-20T08:08:34.097 INFO:tasks.rados.rados.0.smithi118.stdout:2073: finishing write tid 1 to smithi11881747-45 2024-03-20T08:08:34.097 INFO:tasks.rados.rados.0.smithi118.stdout:2073: finishing write tid 2 to smithi11881747-45 2024-03-20T08:08:34.097 INFO:tasks.rados.rados.0.smithi118.stdout:2073: finishing write tid 3 to smithi11881747-45 2024-03-20T08:08:34.097 INFO:tasks.rados.rados.0.smithi118.stdout:2073: finishing write tid 4 to smithi11881747-45 2024-03-20T08:08:34.097 INFO:tasks.rados.rados.0.smithi118.stdout:2072: done (7 left) 2024-03-20T08:08:34.097 INFO:tasks.rados.rados.0.smithi118.stdout:2074: done (6 left) 2024-03-20T08:08:34.097 INFO:tasks.rados.rados.0.smithi118.stdout:2077: done (5 left) 2024-03-20T08:08:34.097 INFO:tasks.rados.rados.0.smithi118.stdout:2080: snap_create 2024-03-20T08:08:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:2078: finishing write tid 1 to smithi11881747-47 2024-03-20T08:08:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:2075: finishing write tid 4 to smithi11881747-41 2024-03-20T08:08:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:2075: oid 41 updating version 0 to 33907 2024-03-20T08:08:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:2075: oid 41 updating version 33907 to 33909 2024-03-20T08:08:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:2075: oid 41 version 33909 is already newer than 33908 2024-03-20T08:08:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 33909 (ObjNum 656 snap 213 seq_num 656) dirty exists 2024-03-20T08:08:34.098 INFO:tasks.rados.rados.0.smithi118.stdout:2075: left oid 41 (ObjNum 656 snap 213 seq_num 656) 2024-03-20T08:08:34.100 INFO:tasks.rados.rados.0.smithi118.stdout:2076: expect (ObjNum 626 snap 203 seq_num 626) 2024-03-20T08:08:34.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.871+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.890+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.129 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.908+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.924+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.942+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.959+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.975+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:33.992+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.008+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.025+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.043+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.061+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.082+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.108+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.954+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.980+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:33.991+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.004+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.018+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.030+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.069+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.103+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.128+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.139+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.163+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.253 INFO:tasks.rados.rados.0.smithi118.stdout:2073: finishing write tid 5 to smithi11881747-45 2024-03-20T08:08:34.253 INFO:tasks.rados.rados.0.smithi118.stdout:2073: oid 45 updating version 0 to 27180 2024-03-20T08:08:34.253 INFO:tasks.rados.rados.0.smithi118.stdout:2073: oid 45 version 27180 is already newer than 27179 2024-03-20T08:08:34.253 INFO:tasks.rados.rados.0.smithi118.stdout:2073: oid 45 updating version 27180 to 27181 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2073: oid 45 version 27181 is already newer than 27178 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 27181 (ObjNum 655 snap 213 seq_num 655) dirty exists 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2073: left oid 45 (ObjNum 655 snap 213 seq_num 655) 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: finishing write tid 2 to smithi11881747-47 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: finishing write tid 3 to smithi11881747-47 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: finishing write tid 4 to smithi11881747-47 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: finishing write tid 5 to smithi11881747-47 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: oid 47 updating version 0 to 33913 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: oid 47 version 33913 is already newer than 33910 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: oid 47 version 33913 is already newer than 33912 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: oid 47 version 33913 is already newer than 33911 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33913 (ObjNum 657 snap 213 seq_num 657) dirty exists 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:2078: left oid 47 (ObjNum 657 snap 213 seq_num 657) 2024-03-20T08:08:34.254 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 27183 (ObjNum 584 snap 188 seq_num 584) dirty exists 2024-03-20T08:08:34.382 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.129+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.147+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.164+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.180+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.197+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.213+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.250+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.266+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.283+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.299+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.316+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.333+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.349+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.384 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.366+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.384 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.381+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.199+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.224+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.237+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.250+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.261+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.274+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.323+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.360+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.371+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.405+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.430 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.398+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.414+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.431+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.447+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.463+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.479+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.642 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.495+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.512+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.528+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.543+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.559+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.575+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.592+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.609+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.625+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.643 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.642+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.451+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.463+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.534+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.547+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.571+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.583+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.607+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.630+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.665+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.689 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.689 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.710 INFO:tasks.rados.rados.0.smithi118.stdout:2073: done (5 left) 2024-03-20T08:08:34.710 INFO:tasks.rados.rados.0.smithi118.stdout:2075: done (4 left) 2024-03-20T08:08:34.710 INFO:tasks.rados.rados.0.smithi118.stdout:2076: done (3 left) 2024-03-20T08:08:34.710 INFO:tasks.rados.rados.0.smithi118.stdout:2078: done (2 left) 2024-03-20T08:08:34.710 INFO:tasks.rados.rados.0.smithi118.stdout:2079: done (1 left) 2024-03-20T08:08:34.710 INFO:tasks.rados.rados.0.smithi118.stdout:2080: done (0 left) 2024-03-20T08:08:34.711 INFO:tasks.rados.rados.0.smithi118.stdout:2081: read oid 11 snap -1 2024-03-20T08:08:34.711 INFO:tasks.rados.rados.0.smithi118.stdout:2081: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:08:34.711 INFO:tasks.rados.rados.0.smithi118.stdout:2082: read oid 4 snap -1 2024-03-20T08:08:34.711 INFO:tasks.rados.rados.0.smithi118.stdout:2082: expect (ObjNum 605 snap 197 seq_num 605) 2024-03-20T08:08:34.711 INFO:tasks.rados.rados.0.smithi118.stdout:2083: delete oid 47 current snap is 214 2024-03-20T08:08:34.716 INFO:tasks.rados.rados.0.smithi118.stdout:2082: expect (ObjNum 605 snap 197 seq_num 605) 2024-03-20T08:08:34.741 INFO:tasks.rados.rados.0.smithi118.stdout:2082: done (2 left) 2024-03-20T08:08:34.741 INFO:tasks.rados.rados.0.smithi118.stdout:2083: done (1 left) 2024-03-20T08:08:34.742 INFO:tasks.rados.rados.0.smithi118.stdout:2084: snap_create 2024-03-20T08:08:34.760 INFO:tasks.rados.rados.0.smithi118.stdout:2081: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:08:34.909 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:34.715+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:34.910 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:34.722+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:34.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.658+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.675+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.691+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.709+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.731+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.754+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.773+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.791+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.813+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.833+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.857+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.874+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.891+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:34.911 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-mon[93121]: pgmap v402: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 617 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.5 MiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-03-20T08:08:34.911 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-mon[95692]: pgmap v402: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 617 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.5 MiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-03-20T08:08:34.912 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:34.730+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:34.912 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:34.708+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:34.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-mon[87080]: pgmap v402: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 6 active+clean+snaptrim_error, 108 active+clean; 617 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.5 MiB/s rd, 1.3 MiB/s wr, 6 op/s 2024-03-20T08:08:34.946 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:34.718+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:34.946 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:34.718+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.717+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.717+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.718+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.745+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.786+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.797+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.832+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.869+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.882+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:34.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.947+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.909+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.926+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.943+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.960+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.977+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:34.993+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.011+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.030+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.170 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.046+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.171 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.064+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.171 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.082+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.171 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.100+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.171 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.171 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.137+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.171 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.154+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.972+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:34.996+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.007+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.019+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.031+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.043+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.055+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.078+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.126+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.202 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.202 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.176+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.202 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.189+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.202 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.170+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.188+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.226+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.243+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.261+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.280+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.296+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.314+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.332+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.349+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.367+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.383+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.401+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.417+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.213+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.239+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.275+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.287+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.424+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.694 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.436+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.453+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.470+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.489+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.507+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.523+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.540+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.557+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.574+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.592+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.610+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.630+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.647+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.664+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.695 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.679+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.696 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.694+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.716 INFO:tasks.rados.rados.0.smithi118.stdout:2081: done (1 left) 2024-03-20T08:08:35.716 INFO:tasks.rados.rados.0.smithi118.stdout:2084: done (0 left) 2024-03-20T08:08:35.716 INFO:tasks.rados.rados.0.smithi118.stdout:2085: rollback oid 47 current snap is 215 2024-03-20T08:08:35.716 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 193 2024-03-20T08:08:35.716 INFO:tasks.rados.rados.0.smithi118.stdout:2086: read oid 15 snap -1 2024-03-20T08:08:35.716 INFO:tasks.rados.rados.0.smithi118.stdout:2086: expect deleted 2024-03-20T08:08:35.716 INFO:tasks.rados.rados.0.smithi118.stdout:2087: setattr oid 4 current snap is 215 2024-03-20T08:08:35.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:35.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-mon[87080]: osdmap e483: 8 total, 8 up, 8 in 2024-03-20T08:08:35.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:35.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:35.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:35.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:35.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:35.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:35.720 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 215 2024-03-20T08:08:35.720 INFO:tasks.rados.rados.0.smithi118.stdout:2088: seq_num 659 ranges {5357568=647168,6004736=450560,6455296=352256} 2024-03-20T08:08:35.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.475+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.720 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.515+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.589+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.627+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.665+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.689+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.701+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.716+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.752 INFO:tasks.rados.rados.0.smithi118.stdout:2088: writing smithi11881747-29 from 5357568 to 6004736 tid 1 2024-03-20T08:08:35.754 INFO:tasks.rados.rados.0.smithi118.stdout:2088: writing smithi11881747-29 from 6004736 to 6455296 tid 2 2024-03-20T08:08:35.758 INFO:tasks.rados.rados.0.smithi118.stdout:2088: writing smithi11881747-29 from 6455296 to 6807552 tid 3 2024-03-20T08:08:35.758 INFO:tasks.rados.rados.0.smithi118.stdout:2089: snap_remove snap 198 2024-03-20T08:08:35.949 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:35.724+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:35.949 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:35.727+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:35.949 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:35.727+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:35.950 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:35.716+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:35.950 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:35.950 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[93121]: osdmap e483: 8 total, 8 up, 8 in 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[95692]: osdmap e483: 8 total, 8 up, 8 in 2024-03-20T08:08:35.951 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:35.952 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:35.952 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:35.952 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:35.952 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:35.952 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:35.952 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:35.731+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.736+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.753+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.771+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.788+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.804+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.821+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.837+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.853+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.870+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.953 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.887+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.954 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.902+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.954 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.918+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.954 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.934+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.954 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.949+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:35.976 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:35.720+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:35.977 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:35.720+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:35.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.723+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:35.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.734+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.775+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.787+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.800+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.811+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.823+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.836+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.919+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:35.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.967+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:35.985+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.001+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.019+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.036+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.213 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.052+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.069+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.085+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.100+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.115+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.132+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.148+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.164+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.180+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.197+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.214 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.213+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:35.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.001+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.013+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.232 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.036+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.048+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.098+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.110+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.233 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.234 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.234 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.234 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.234 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.229+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.246+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.465 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.264+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.281+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.297+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.314+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.330+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.346+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.363+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.380+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.396+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.413+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.432+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.449+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.466 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.465+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.490 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.282+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.293+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.329+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.353+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.439+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.452+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.464+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.491+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.482+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.498+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.515+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.531+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.547+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.563+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.579+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.595+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.717 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.609+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.625+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.641+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.656+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.673+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.689+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.718 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.705+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.718 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:08:36.711+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 8 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:36.723 INFO:tasks.rados.rados.0.smithi118.stdout:2089: done (4 left) 2024-03-20T08:08:36.723 INFO:tasks.rados.rados.0.smithi118.stdout:2090: snap_create 2024-03-20T08:08:36.723 INFO:tasks.rados.rados.0.smithi118.stdout:2085: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:08:36.723 INFO:tasks.rados.rados.0.smithi118.stdout:2085: finishing rollback tid 2 to smithi11881747-47 2024-03-20T08:08:36.723 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33916 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:08:36.723 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 1306 (ObjNum 605 snap 197 seq_num 605) dirty exists 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: finishing write tid 1 to smithi11881747-29 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: finishing write tid 2 to smithi11881747-29 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: finishing write tid 3 to smithi11881747-29 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: finishing write tid 4 to smithi11881747-29 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: finishing write tid 5 to smithi11881747-29 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: oid 29 updating version 0 to 33919 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: oid 29 updating version 33919 to 33920 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: oid 29 version 33920 is already newer than 33918 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: oid 29 version 33920 is already newer than 33917 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33920 (ObjNum 659 snap 215 seq_num 659) dirty exists 2024-03-20T08:08:36.724 INFO:tasks.rados.rados.0.smithi118.stdout:2088: left oid 29 (ObjNum 659 snap 215 seq_num 659) 2024-03-20T08:08:36.766 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: osdmap e484: 8 total, 8 up, 8 in 2024-03-20T08:08:36.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:36.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:36.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:36.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:36.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:36.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:36.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:36.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-mon[87080]: pgmap v405: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 626 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 16 MiB/s rd, 4.2 MiB/s wr, 16 op/s 2024-03-20T08:08:36.767 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:36.726+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:36.727+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.514+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.526+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.538+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.573+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.584+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.620+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.633+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.645+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.658+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.669+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:36.974 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:36.731+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:36.974 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:36.725+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:36.974 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:36.725+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:36.974 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:36.726+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: osdmap e484: 8 total, 8 up, 8 in 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:36.975 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[95692]: pgmap v405: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 626 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 16 MiB/s rd, 4.2 MiB/s wr, 16 op/s 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: osdmap e484: 8 total, 8 up, 8 in 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:36.976 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-mon[93121]: pgmap v405: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 626 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 16 MiB/s rd, 4.2 MiB/s wr, 16 op/s 2024-03-20T08:08:36.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.724+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.742+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.976 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.756+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.773+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.791+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.808+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.824+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.841+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.857+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.874+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.892+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.907+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.925+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.941+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.958+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:36.977 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.974+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.717+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.768+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.768+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.811+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.862+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.875+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.887+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.922+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.946+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.957+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:36.991+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.232 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:36.991+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.232 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.007+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.023+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.039+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.054+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.071+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.087+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.102+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.118+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.133+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.149+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.165+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.181+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.199+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.214+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.233 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.232+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.282 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.282 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.051+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.282 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.063+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.282 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.282 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.089+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.282 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.127+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.139+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.150+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.162+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.174+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.198+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.209+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.222+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.258+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.283 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.284 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.282+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.249+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.266+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.281+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.498 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.298+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.314+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.330+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.347+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.363+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.383+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.399+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.417+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.434+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.450+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.465+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.482+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.499 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.498+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.330+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.353+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.365+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.388+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.400+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.459+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.483+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.530+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.725 INFO:tasks.rados.rados.0.smithi118.stdout:2085: done (4 left) 2024-03-20T08:08:37.725 INFO:tasks.rados.rados.0.smithi118.stdout:2086: done (3 left) 2024-03-20T08:08:37.726 INFO:tasks.rados.rados.0.smithi118.stdout:2087: done (2 left) 2024-03-20T08:08:37.726 INFO:tasks.rados.rados.0.smithi118.stdout:2088: done (1 left) 2024-03-20T08:08:37.726 INFO:tasks.rados.rados.0.smithi118.stdout:2090: done (0 left) 2024-03-20T08:08:37.726 INFO:tasks.rados.rados.0.smithi118.stdout:2091: delete oid 38 current snap is 216 2024-03-20T08:08:37.729 INFO:tasks.rados.rados.0.smithi118.stdout:2091: done (0 left) 2024-03-20T08:08:37.729 INFO:tasks.rados.rados.0.smithi118.stdout:2092: snap_remove snap 183 2024-03-20T08:08:37.752 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:37.730+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:37.752 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:37.730+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:37.752 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:37.731+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:37.752 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 8 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:37.753 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: osdmap e485: 8 total, 8 up, 8 in 2024-03-20T08:08:37.753 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:37.753 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:37.753 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:37.753 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:37.753 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:37.753 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:37.753 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:37.753 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:37.738+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:37.754 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:37.725+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 8 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: osdmap e485: 8 total, 8 up, 8 in 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:37.754 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.515+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.532+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.549+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.566+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.583+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.598+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.615+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.632+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.649+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.665+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.681+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.755 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.697+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.715+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.756 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.736+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:37.801 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:37.730+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:37.801 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:37.731+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:37.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 8 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:37.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: osdmap e485: 8 total, 8 up, 8 in 2024-03-20T08:08:37.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:37.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:37.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:37.802 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:37.802 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:37.802 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:37.802 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:37.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.590+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.650+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.674+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.686+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.709+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.723+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.732+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:37.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.801+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.753+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.770+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.787+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.803+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.819+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.837+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.854+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.870+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.887+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.904+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.004 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.922+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.939+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.957+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.972+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:37.988+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.005 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.061 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.061 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.824+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.061 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.836+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.848+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.859+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.870+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.918+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.975+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.062 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.987+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.063 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:37.999+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.063 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.063 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.063 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.063 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.049+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.063 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.260 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.019+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.035+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.051+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.068+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.084+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.100+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.261 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.117+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.134+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.150+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.165+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.181+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.195+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.262 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.212+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.228+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.244+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.263 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.260+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.145+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.194+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.206+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.231+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.266+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.278+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.302+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.314+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.519 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.277+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.293+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.310+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.326+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.342+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.360+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.376+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.520 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.408+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.424+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.439+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.457+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.472+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.521 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.488+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.522 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.504+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.339+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.351+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.376+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.399+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.447+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.472+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.484+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.521+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.544+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.557+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.572 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.737 INFO:tasks.rados.rados.0.smithi118.stdout:2092: done (0 left) 2024-03-20T08:08:38.737 INFO:tasks.rados.rados.0.smithi118.stdout:2093: rmattr oid 12 current snap is 216 2024-03-20T08:08:38.737 INFO:tasks.rados.rados.0.smithi118.stdout:2094: snap_remove snap 197 2024-03-20T08:08:38.776 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:38.739+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:38.776 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:38.739+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:38.776 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:38.734+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: osdmap e486: 8 total, 8 up, 8 in 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:38.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[93121]: pgmap v408: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 24 op/s 2024-03-20T08:08:38.777 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:38.740+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: osdmap e486: 8 total, 8 up, 8 in 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:38.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-mon[95692]: pgmap v408: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 24 op/s 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.519+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.536+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.551+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.567+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.583+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.600+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.615+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.631+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.648+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.663+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.679+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.696+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.779 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.713+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.780 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.732+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.780 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.760+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.780 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:38.827 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:38.735+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:38.827 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:38.735+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:38.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:38.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: osdmap e486: 8 total, 8 up, 8 in 2024-03-20T08:08:38.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:38.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:38.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:38.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:38.827 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:38.828 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:38.828 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-mon[87080]: pgmap v408: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 24 op/s 2024-03-20T08:08:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.580+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.604+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.628+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.640+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.711+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.723+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.738+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.738+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.767+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.779+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.792+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:38.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.792+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.808+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.823+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.840+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.855+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.872+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.889+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.905+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.921+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.938+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.954+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.969+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:38.986+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.003+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.019+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.037 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.036+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.852+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.864+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.890+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.904+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.916+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.927+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.939+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.964+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:38.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.022+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.035+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.049+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.062+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.089 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.053+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.070+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.087+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.104+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.120+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.136+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.153+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.288 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.170+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.188+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.205+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.223+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.239+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.255+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.272+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.289 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.288+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.101+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.128+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.156+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.169+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.213+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.228+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.284+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.337+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.350+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.305+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.322+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.339+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.358+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.375+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.392+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.409+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.544 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.427+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.443+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.458+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.474+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.492+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.509+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.545 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.527+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.375+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.447+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.488+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.501+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.541+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.582+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.595+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.733 INFO:tasks.rados.rados.0.smithi118.stdout:2094: done (1 left) 2024-03-20T08:08:39.733 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 216 2024-03-20T08:08:39.733 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 26767 (ObjNum 600 snap 196 seq_num 600) dirty exists 2024-03-20T08:08:39.733 INFO:tasks.rados.rados.0.smithi118.stdout:2095: seq_num 661 ranges {1376256=679936,2056192=622592,2678784=180224} 2024-03-20T08:08:39.745 INFO:tasks.rados.rados.0.smithi118.stdout:2095: writing smithi11881747-45 from 1376256 to 2056192 tid 1 2024-03-20T08:08:39.749 INFO:tasks.rados.rados.0.smithi118.stdout:2095: writing smithi11881747-45 from 2056192 to 2678784 tid 2 2024-03-20T08:08:39.751 INFO:tasks.rados.rados.0.smithi118.stdout:2095: writing smithi11881747-45 from 2678784 to 2859008 tid 3 2024-03-20T08:08:39.752 INFO:tasks.rados.rados.0.smithi118.stdout:2093: done (1 left) 2024-03-20T08:08:39.752 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 216 2024-03-20T08:08:39.752 INFO:tasks.rados.rados.0.smithi118.stdout:2096: seq_num 662 ranges {3735552=696320,4431872=557056,4988928=245760} 2024-03-20T08:08:39.777 INFO:tasks.rados.rados.0.smithi118.stdout:2096: writing smithi11881747-37 from 3735552 to 4431872 tid 1 2024-03-20T08:08:39.781 INFO:tasks.rados.rados.0.smithi118.stdout:2096: writing smithi11881747-37 from 4431872 to 4988928 tid 2 2024-03-20T08:08:39.783 INFO:tasks.rados.rados.0.smithi118.stdout:2096: writing smithi11881747-37 from 4988928 to 5234688 tid 3 2024-03-20T08:08:39.783 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 216 2024-03-20T08:08:39.783 INFO:tasks.rados.rados.0.smithi118.stdout:2095: finishing write tid 1 to smithi11881747-45 2024-03-20T08:08:39.783 INFO:tasks.rados.rados.0.smithi118.stdout:2095: finishing write tid 2 to smithi11881747-45 2024-03-20T08:08:39.784 INFO:tasks.rados.rados.0.smithi118.stdout:2095: finishing write tid 3 to smithi11881747-45 2024-03-20T08:08:39.784 INFO:tasks.rados.rados.0.smithi118.stdout:2095: finishing write tid 4 to smithi11881747-45 2024-03-20T08:08:39.784 INFO:tasks.rados.rados.0.smithi118.stdout:2097: seq_num 663 ranges {2088960=712704,2801664=442368,3244032=360448} 2024-03-20T08:08:39.799 INFO:tasks.rados.rados.0.smithi118.stdout:2097: writing smithi11881747-1 from 2088960 to 2801664 tid 1 2024-03-20T08:08:39.802 INFO:tasks.rados.rados.0.smithi118.stdout:2097: writing smithi11881747-1 from 2801664 to 3244032 tid 2 2024-03-20T08:08:39.805 INFO:tasks.rados.rados.0.smithi118.stdout:2097: writing smithi11881747-1 from 3244032 to 3604480 tid 3 2024-03-20T08:08:39.805 INFO:tasks.rados.rados.0.smithi118.stdout:2098: rollback oid 33 current snap is 216 2024-03-20T08:08:39.805 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 172 2024-03-20T08:08:39.805 INFO:tasks.rados.rados.0.smithi118.stdout:2096: finishing write tid 1 to smithi11881747-37 2024-03-20T08:08:39.805 INFO:tasks.rados.rados.0.smithi118.stdout:2095: finishing write tid 5 to smithi11881747-45 2024-03-20T08:08:39.805 INFO:tasks.rados.rados.0.smithi118.stdout:2095: oid 45 updating version 0 to 29005 2024-03-20T08:08:39.805 INFO:tasks.rados.rados.0.smithi118.stdout:2095: oid 45 updating version 29005 to 29008 2024-03-20T08:08:39.805 INFO:tasks.rados.rados.0.smithi118.stdout:2095: oid 45 version 29008 is already newer than 29006 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:2095: oid 45 version 29008 is already newer than 29007 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 29008 (ObjNum 661 snap 216 seq_num 661) dirty exists 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:2095: left oid 45 (ObjNum 661 snap 216 seq_num 661) 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:2096: finishing write tid 2 to smithi11881747-37 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:2095: done (3 left) 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:2099: rollback oid 6 current snap is 216 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 210 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:2100: copy_from oid 43 from oid 21 current snap is 216 2024-03-20T08:08:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:2101: read oid 17 snap -1 2024-03-20T08:08:39.807 INFO:tasks.rados.rados.0.smithi118.stdout:2101: expect (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:08:39.807 INFO:tasks.rados.rados.0.smithi118.stdout:2102: rollback oid 44 current snap is 216 2024-03-20T08:08:39.807 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 44 to 204 2024-03-20T08:08:39.807 INFO:tasks.rados.rados.0.smithi118.stdout:2103: copy_from oid 20 from oid 13 current snap is 216 2024-03-20T08:08:39.807 INFO:tasks.rados.rados.0.smithi118.stdout:2104: copy_from oid 4 from oid 47 current snap is 216 2024-03-20T08:08:39.808 INFO:tasks.rados.rados.0.smithi118.stdout:2105: read oid 10 snap 193 2024-03-20T08:08:39.808 INFO:tasks.rados.rados.0.smithi118.stdout:2105: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:08:39.808 INFO:tasks.rados.rados.0.smithi118.stdout:2106: snap_remove snap 208 2024-03-20T08:08:39.808 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:39.733+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:39.808 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:39.738+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:39.808 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:39.738+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:39.808 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:39.739+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:39.809 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:39.740+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:39.809 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:39.809 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:39.809 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:39.809 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: osdmap e487: 8 total, 8 up, 8 in 2024-03-20T08:08:39.809 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:39.809 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[93121]: Upgrade: osd.6 is safe to restart 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: osdmap e487: 8 total, 8 up, 8 in 2024-03-20T08:08:39.810 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:39.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:39.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:39.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:39.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:39.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:39.811 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-mon[95692]: Upgrade: osd.6 is safe to restart 2024-03-20T08:08:39.811 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.544+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.811 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.561+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.811 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.577+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.811 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.595+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.811 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.611+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.629+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.647+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.663+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.681+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.697+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.712+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.727+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.752+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.771+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.789+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.812 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.808+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:39.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.638+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.703+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.726+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.742+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.748+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.762+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.775+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.791+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:39.868 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:39.868 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:39.868 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:39.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: osdmap e487: 8 total, 8 up, 8 in 2024-03-20T08:08:39.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:39.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:39.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:39.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:39.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:39.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["6"], "max": 16}]: dispatch 2024-03-20T08:08:39.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-mon[87080]: Upgrade: osd.6 is safe to restart 2024-03-20T08:08:39.869 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:39.737+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:39.869 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:39.737+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:40.058 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.844+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.058 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.868+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.058 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.886+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.058 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.904+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.058 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.919+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.059 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.936+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.059 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.954+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.059 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.970+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.059 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:39.987+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.059 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.004+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.059 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.059 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.040+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.059 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.058+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.125 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.925+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.938+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.950+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:39.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.042+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.112+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.125+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.076+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.093+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.108+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.125+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.142+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.160+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.177+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.315 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.197+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.215+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.230+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.248+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.265+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.281+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.298+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.316 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.315+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.155+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.224+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.247+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.308+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.321+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.362+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.571 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.333+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.352+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.368+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.385+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.402+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.419+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.436+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.451+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.469+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.486+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.504+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.521+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.572 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.538+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.555+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.573 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.571+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.414+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.426+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.438+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.450+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.485+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.543+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.568+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.616+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2106: done (10 left) 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 216 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2097: finishing write tid 1 to smithi11881747-1 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2097: finishing write tid 2 to smithi11881747-1 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2096: finishing write tid 3 to smithi11881747-37 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2096: finishing write tid 4 to smithi11881747-37 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2096: finishing write tid 5 to smithi11881747-37 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2096: oid 37 updating version 0 to 29013 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2096: oid 37 version 29013 is already newer than 29011 2024-03-20T08:08:40.759 INFO:tasks.rados.rados.0.smithi118.stdout:2096: oid 37 version 29013 is already newer than 29012 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2096: oid 37 version 29013 is already newer than 29010 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 29013 (ObjNum 662 snap 216 seq_num 662) dirty exists 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2096: left oid 37 (ObjNum 662 snap 216 seq_num 662) 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2097: finishing write tid 3 to smithi11881747-1 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2097: finishing write tid 4 to smithi11881747-1 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2098: finishing rollback tid 0 to smithi11881747-33 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2103: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2104: finishing copy_from racing read to smithi11881747-4 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2099: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2099: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2099: finishing rollback tid 2 to smithi11881747-6 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 9338 (ObjNum 636 snap 205 seq_num 636) dirty exists 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2102: finishing rollback tid 0 to smithi11881747-44 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2104: finishing copy_from to smithi11881747-4 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 33916 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:08:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:2103: finishing copy_from to smithi11881747-20 2024-03-20T08:08:40.761 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 7076 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:08:40.761 INFO:tasks.rados.rados.0.smithi118.stdout:2098: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:08:40.761 INFO:tasks.rados.rados.0.smithi118.stdout:2102: finishing rollback tid 1 to smithi11881747-44 2024-03-20T08:08:40.761 INFO:tasks.rados.rados.0.smithi118.stdout:2102: finishing rollback tid 2 to smithi11881747-44 2024-03-20T08:08:40.761 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 11487 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:08:40.761 INFO:tasks.rados.rados.0.smithi118.stdout:2098: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:08:40.761 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 13600 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:08:40.761 INFO:tasks.rados.rados.0.smithi118.stdout:2101: expect (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:08:40.833 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:40.764+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:40.833 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:40.765+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.588+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.603+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.621+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.637+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.653+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.671+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.687+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.703+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.720+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.834 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.737+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.835 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.755+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.835 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.777+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.835 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.794+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.835 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.812+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:40.835 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: osdmap e488: 8 total, 8 up, 8 in 2024-03-20T08:08:40.835 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:40.835 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:40.835 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:40.835 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:40.835 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:40.835 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[93121]: pgmap v411: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: osdmap e488: 8 total, 8 up, 8 in 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:40.836 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:40.837 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-mon[95692]: pgmap v411: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-03-20T08:08:40.837 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:40.765+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:40.837 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:40.756+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:40.896 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:40.759+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:40.896 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:40.761+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],113=[113,112,111]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: osdmap e488: 8 total, 8 up, 8 in 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:40.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-mon[87080]: pgmap v411: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-03-20T08:08:40.898 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.898 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.898 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.898 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.678+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.898 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.898 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.702+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.898 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.713+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.898 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.737+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.749+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.764+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.764+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.809+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.822+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.848+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.860+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.899 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.900 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.900 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:40.971 INFO:tasks.rados.rados.0.smithi118.stdout:2105: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:08:41.098 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.833+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.851+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.868+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.885+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.904+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.922+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.940+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.976+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:40.993+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.009+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.026+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.099 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.044+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.062+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.100 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.079+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.950+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.976+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:40.988+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.013+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.040+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.053+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.079+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.093+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.106+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.119+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.133+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.360 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.099+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.360 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.117+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.360 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.135+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.360 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.152+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.170+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.187+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.205+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.222+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.239+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.255+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.272+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.290+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.307+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.325+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.342+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.361 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.360+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.159+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.218+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.231+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.296+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.307+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.418 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.418 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.418 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.347+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.418 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.418 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.418 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.389+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.418 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.403+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.418 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.529 INFO:tasks.rados.rados.0.smithi118.stdout:2100: finishing copy_from racing read to smithi11881747-43 2024-03-20T08:08:41.529 INFO:tasks.rados.rados.0.smithi118.stdout:2097: finishing write tid 5 to smithi11881747-1 2024-03-20T08:08:41.529 INFO:tasks.rados.rados.0.smithi118.stdout:2097: oid 1 updating version 0 to 7114 2024-03-20T08:08:41.530 INFO:tasks.rados.rados.0.smithi118.stdout:2097: oid 1 updating version 7114 to 7115 2024-03-20T08:08:41.530 INFO:tasks.rados.rados.0.smithi118.stdout:2097: oid 1 updating version 7115 to 7116 2024-03-20T08:08:41.530 INFO:tasks.rados.rados.0.smithi118.stdout:2097: oid 1 version 7116 is already newer than 7113 2024-03-20T08:08:41.530 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 7116 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:08:41.530 INFO:tasks.rados.rados.0.smithi118.stdout:2097: left oid 1 (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:08:41.530 INFO:tasks.rados.rados.0.smithi118.stdout:2100: finishing copy_from to smithi11881747-43 2024-03-20T08:08:41.530 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 7062 (ObjNum 545 snap 179 seq_num 545) dirty exists 2024-03-20T08:08:41.530 INFO:tasks.rados.rados.0.smithi118.stdout:2107: seq_num 664 ranges {6414336=729088,7143424=729088,7872512=73728} 2024-03-20T08:08:41.569 INFO:tasks.rados.rados.0.smithi118.stdout:2107: writing smithi11881747-27 from 6414336 to 7143424 tid 1 2024-03-20T08:08:41.574 INFO:tasks.rados.rados.0.smithi118.stdout:2107: writing smithi11881747-27 from 7143424 to 7872512 tid 2 2024-03-20T08:08:41.575 INFO:tasks.rados.rados.0.smithi118.stdout:2107: writing smithi11881747-27 from 7872512 to 7946240 tid 3 2024-03-20T08:08:41.575 INFO:tasks.rados.rados.0.smithi118.stdout:2096: done (10 left) 2024-03-20T08:08:41.575 INFO:tasks.rados.rados.0.smithi118.stdout:2097: done (9 left) 2024-03-20T08:08:41.575 INFO:tasks.rados.rados.0.smithi118.stdout:2098: done (8 left) 2024-03-20T08:08:41.576 INFO:tasks.rados.rados.0.smithi118.stdout:2099: done (7 left) 2024-03-20T08:08:41.576 INFO:tasks.rados.rados.0.smithi118.stdout:2100: done (6 left) 2024-03-20T08:08:41.576 INFO:tasks.rados.rados.0.smithi118.stdout:2101: done (5 left) 2024-03-20T08:08:41.576 INFO:tasks.rados.rados.0.smithi118.stdout:2102: done (4 left) 2024-03-20T08:08:41.576 INFO:tasks.rados.rados.0.smithi118.stdout:2103: done (3 left) 2024-03-20T08:08:41.576 INFO:tasks.rados.rados.0.smithi118.stdout:2104: done (2 left) 2024-03-20T08:08:41.576 INFO:tasks.rados.rados.0.smithi118.stdout:2105: done (1 left) 2024-03-20T08:08:41.576 INFO:tasks.rados.rados.0.smithi118.stdout:2108: setattr oid 21 current snap is 216 2024-03-20T08:08:41.577 INFO:tasks.rados.rados.0.smithi118.stdout:2107: finishing write tid 1 to smithi11881747-27 2024-03-20T08:08:41.577 INFO:tasks.rados.rados.0.smithi118.stdout:2109: snap_create 2024-03-20T08:08:41.580 INFO:tasks.rados.rados.0.smithi118.stdout:2107: finishing write tid 2 to smithi11881747-27 2024-03-20T08:08:41.582 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 7081 (ObjNum 545 snap 179 seq_num 545) dirty exists 2024-03-20T08:08:41.583 INFO:tasks.rados.rados.0.smithi118.stdout:2107: finishing write tid 3 to smithi11881747-27 2024-03-20T08:08:41.585 INFO:tasks.rados.rados.0.smithi118.stdout:2107: finishing write tid 4 to smithi11881747-27 2024-03-20T08:08:41.590 INFO:tasks.rados.rados.0.smithi118.stdout:2107: finishing write tid 5 to smithi11881747-27 2024-03-20T08:08:41.590 INFO:tasks.rados.rados.0.smithi118.stdout:2107: oid 27 updating version 0 to 7123 2024-03-20T08:08:41.590 INFO:tasks.rados.rados.0.smithi118.stdout:2107: oid 27 version 7123 is already newer than 7120 2024-03-20T08:08:41.590 INFO:tasks.rados.rados.0.smithi118.stdout:2107: oid 27 version 7123 is already newer than 7121 2024-03-20T08:08:41.590 INFO:tasks.rados.rados.0.smithi118.stdout:2107: oid 27 version 7123 is already newer than 7122 2024-03-20T08:08:41.590 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 7123 (ObjNum 664 snap 216 seq_num 664) dirty exists 2024-03-20T08:08:41.591 INFO:tasks.rados.rados.0.smithi118.stdout:2107: left oid 27 (ObjNum 664 snap 216 seq_num 664) 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.377+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.393+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.410+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.428+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.445+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.464+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.480+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.497+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.515+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.534+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.552+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.624 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.570+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.589+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.606+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.625 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.624+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.430+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.443+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.471+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.497+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.538+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.581+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.667+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.876 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: osdmap e489: 8 total, 8 up, 8 in 2024-03-20T08:08:41.876 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:41.876 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:41.876 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],113=[113,112,111]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: Upgrade: Updating osd.6 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[93121]: Deploying daemon osd.6 on smithi196 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: osdmap e489: 8 total, 8 up, 8 in 2024-03-20T08:08:41.877 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],113=[113,112,111]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: Upgrade: Updating osd.6 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:41.878 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-mon[95692]: Deploying daemon osd.6 on smithi196 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.641+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.658+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.675+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.692+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.708+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.723+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.739+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.758+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.775+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.793+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.811+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.879 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.827+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.880 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.844+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.880 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.861+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.880 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.877+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: osdmap e489: 8 total, 8 up, 8 in 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],113=[113,112,111]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:41.923 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: Upgrade: Updating osd.6 2024-03-20T08:08:41.924 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:41.924 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-20T08:08:41.924 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:08:41.924 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-mon[87080]: Deploying daemon osd.6 on smithi196 2024-03-20T08:08:41.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.679+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.751+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.765+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.802+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.826+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.860+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.873+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.899+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:41.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.031 INFO:tasks.rados.rados.0.smithi118.stdout:2107: done (2 left) 2024-03-20T08:08:42.031 INFO:tasks.rados.rados.0.smithi118.stdout:2108: done (1 left) 2024-03-20T08:08:42.031 INFO:tasks.rados.rados.0.smithi118.stdout:2109: done (0 left) 2024-03-20T08:08:42.032 INFO:tasks.rados.rados.0.smithi118.stdout:2110: setattr oid 1 current snap is 217 2024-03-20T08:08:42.033 INFO:tasks.rados.rados.0.smithi118.stdout:2111: rmattr oid 4 current snap is 217 2024-03-20T08:08:42.034 INFO:tasks.rados.rados.0.smithi118.stdout:2111: done (1 left) 2024-03-20T08:08:42.034 INFO:tasks.rados.rados.0.smithi118.stdout:2112: copy_from oid 15 from oid 24 current snap is 217 2024-03-20T08:08:42.034 INFO:tasks.rados.rados.0.smithi118.stdout:2113: snap_remove snap 89 2024-03-20T08:08:42.137 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:42.040+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:42.137 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:42.040+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:42.138 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:42.041+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:42.138 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:42.030+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:42.139 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:42.043+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:42.139 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.895+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.139 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.911+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.139 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.929+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.139 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.945+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.960+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.978+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:41.994+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.009+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.026+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.051+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.140 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.069+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.088+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.104+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.121+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.141 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.138+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.185 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:42.038+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:42.185 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:42.040+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],113=[113,112,111]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.966+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.978+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:41.990+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.026+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.041+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.046+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.074+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.124+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.159+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.187 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.389 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.155+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.389 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.171+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.389 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.188+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.389 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.205+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.389 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.222+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.389 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.239+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.389 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.256+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.389 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.272+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.390 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.289+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.390 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.306+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.390 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.323+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.390 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.340+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.390 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.356+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.390 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.372+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.444 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:08:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.197+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.209+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.221+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.267+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.302+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.326+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.361+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.395+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.407+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.445+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.389+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.653 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.406+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.424+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.442+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.458+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.474+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.492+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.508+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.524+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.538+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.555+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.571+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.588+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.604+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.654 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.621+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.655 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.637+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.458+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.469+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.494+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.505+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.529+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.542+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.555+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.592+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.604+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.617+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.628+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.639+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.651+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.675+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.701 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.687+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.653+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.669+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.909 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.686+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.701+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.717+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.733+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.749+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.766+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.781+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.797+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.813+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.828+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.845+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.860+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.877+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.910 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.893+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.909+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:42.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.711+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.783+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.818+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.830+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.856+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.869+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:42.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.047 INFO:tasks.rados.rados.0.smithi118.stdout:2113: done (2 left) 2024-03-20T08:08:43.047 INFO:tasks.rados.rados.0.smithi118.stdout:2114: setattr oid 39 current snap is 217 2024-03-20T08:08:43.047 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 7117 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:08:43.047 INFO:tasks.rados.rados.0.smithi118.stdout:2112: finishing copy_from racing read to smithi11881747-15 2024-03-20T08:08:43.047 INFO:tasks.rados.rados.0.smithi118.stdout:2112: finishing copy_from to smithi11881747-15 2024-03-20T08:08:43.047 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 7076 (ObjNum 617 snap 202 seq_num 617) dirty exists 2024-03-20T08:08:43.049 INFO:tasks.rados.rados.0.smithi118.stdout:2110: done (2 left) 2024-03-20T08:08:43.049 INFO:tasks.rados.rados.0.smithi118.stdout:2112: done (1 left) 2024-03-20T08:08:43.049 INFO:tasks.rados.rados.0.smithi118.stdout:2115: read oid 46 snap 184 2024-03-20T08:08:43.049 INFO:tasks.rados.rados.0.smithi118.stdout:2115: expect (ObjNum 559 snap 183 seq_num 559) 2024-03-20T08:08:43.049 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 217 2024-03-20T08:08:43.049 INFO:tasks.rados.rados.0.smithi118.stdout:2116: seq_num 665 ranges {3301376=745472,4046848=606208,4653056=196608} 2024-03-20T08:08:43.073 INFO:tasks.rados.rados.0.smithi118.stdout:2116: writing smithi11881747-22 from 3301376 to 4046848 tid 1 2024-03-20T08:08:43.077 INFO:tasks.rados.rados.0.smithi118.stdout:2116: writing smithi11881747-22 from 4046848 to 4653056 tid 2 2024-03-20T08:08:43.079 INFO:tasks.rados.rados.0.smithi118.stdout:2116: writing smithi11881747-22 from 4653056 to 4849664 tid 3 2024-03-20T08:08:43.079 INFO:tasks.rados.rados.0.smithi118.stdout:2117: copy_from oid 27 from oid 33 current snap is 217 2024-03-20T08:08:43.079 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7124 (ObjNum 649 snap 213 seq_num 649) dirty exists 2024-03-20T08:08:43.079 INFO:tasks.rados.rados.0.smithi118.stdout:2115: expect (ObjNum 559 snap 183 seq_num 559) 2024-03-20T08:08:43.142 INFO:tasks.rados.rados.0.smithi118.stdout:2116: finishing write tid 1 to smithi11881747-22 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2114: done (3 left) 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2115: done (2 left) 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2118: rollback oid 39 current snap is 217 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 204 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: finishing write tid 2 to smithi11881747-22 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: finishing write tid 3 to smithi11881747-22 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: finishing write tid 4 to smithi11881747-22 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: finishing write tid 5 to smithi11881747-22 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: oid 22 updating version 0 to 7082 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: oid 22 updating version 7082 to 7084 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: oid 22 updating version 7084 to 7085 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: oid 22 version 7085 is already newer than 7083 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 7085 (ObjNum 665 snap 217 seq_num 665) dirty exists 2024-03-20T08:08:43.143 INFO:tasks.rados.rados.0.smithi118.stdout:2116: left oid 22 (ObjNum 665 snap 217 seq_num 665) 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2117: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2117: finishing copy_from to smithi11881747-27 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 13600 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2116: done (2 left) 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2117: done (1 left) 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2119: read oid 12 snap -1 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2119: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2120: read oid 4 snap 216 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2120: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2121: read oid 43 snap -1 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2121: expect (ObjNum 545 snap 179 seq_num 545) 2024-03-20T08:08:43.144 INFO:tasks.rados.rados.0.smithi118.stdout:2122: snap_remove snap 203 2024-03-20T08:08:43.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:43.041+0000 7f0ceec5d700 -1 osd.0 pg_epoch: 490 pg[5.2s1( v 491'1053 (0'0,491'1053] local-lis/les=470/471 n=14 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [6,0,5]p6(0) r=1 lpr=470 luod=0'0 crt=489'1051 lcod 489'1051 mlcod 489'1051 active mbc={} ps=102] update_snap_map remove_oid 5:4cd24aac:::smithi11881747-48:9c failed with -2 2024-03-20T08:08:43.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:43.041+0000 7f0cedc5b700 -1 osd.0 pg_epoch: 491 pg[5.cs2( v 491'33988 (380'27302,491'33988] local-lis/les=445/446 n=23 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [1,4,0]p1(0) r=2 lpr=445 luod=0'0 crt=490'33986 lcod 490'33986 mlcod 490'33986 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~2,a9~1,ab~1,af~1,b1~1,b5~1,b8~2]] update_snap_map remove_oid 5:335100af:::smithi11881747-8:9c failed with -2 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:43.043+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:43.047+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.926+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.941+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.957+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.974+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:42.991+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:43.007+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:43.022+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:43.048+0000 7f8316e9f700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:43.075+0000 7f831aea7700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9d after trimming was started 2024-03-20T08:08:43.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:43.075+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:43.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:43.077+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:43.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:43.082+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:43.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:43.088+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: osdmap e490: 8 total, 8 up, 8 in 2024-03-20T08:08:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],113=[113,112,111]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[93121]: pgmap v414: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 406 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: osdmap e490: 8 total, 8 up, 8 in 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],113=[113,112,111]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:43.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:43.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-mon[95692]: pgmap v414: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 406 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-20T08:08:43.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:43.043+0000 7f59e9e37700 -1 osd.3 pg_epoch: 491 pg[5.1s1( v 488'440 (0'0,488'440] local-lis/les=445/446 n=13 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [4,3,7]p4(0) r=1 lpr=445 luod=0'0 crt=488'440 mlcod 488'440 active mbc={} ps=82] update_snap_map remove_oid 5:89304740:::smithi11881747-4:9c failed with -2 2024-03-20T08:08:43.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:43.050+0000 7f59ea638700 -1 osd.3 pg_epoch: 491 pg[5.0s0( v 491'1206 (0'0,491'1206] local-lis/les=445/446 n=43 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [3,1,4]p3(0) r=0 lpr=445 crt=491'1206 lcod 491'1204 mlcod 491'1204 active+clean+snaptrim trimq=45 ps=58] update_snap_map remove_oid 5:07779a33:::smithi11881747-18:9c failed with -2 2024-03-20T08:08:43.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:43.057+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:43.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:43.057+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:43.178 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:43.041+0000 7fc1f9639700 -1 osd.1 pg_epoch: 491 pg[5.cs0( v 491'33988 (380'27302,491'33988] local-lis/les=445/446 n=23 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [1,4,0]p1(0) r=0 lpr=445 luod=490'33986 crt=490'33986 lcod 489'33984 mlcod 489'33984 active+clean+snaptrim trimq=85 ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~2,a9~1,ab~1,af~1,b1~1,b5~1,b8~2]] update_snap_map remove_oid 5:335100af:::smithi11881747-8:9c failed with -2 2024-03-20T08:08:43.178 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:43.053+0000 7fc1fb63d700 -1 osd.1 pg_epoch: 491 pg[5.0s1( v 491'1206 (0'0,491'1206] local-lis/les=445/446 n=43 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [3,1,4]p3(0) r=1 lpr=445 luod=0'0 crt=491'1206 mlcod 491'1206 active mbc={} ps=58] update_snap_map remove_oid 5:07779a33:::smithi11881747-18:9c failed with -2 2024-03-20T08:08:43.178 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:43.059+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:43.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:02e2296c:::smithi11881747-49:b7 repair needed (no obc) 2024-03-20T08:08:43.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: osdmap e490: 8 total, 8 up, 8 in 2024-03-20T08:08:43.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{95=[95],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:43.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:43.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{98=[95],f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],113=[113,112,111]} for object 5:f468b815:::smithi11881747-43:e3 2024-03-20T08:08:43.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:43.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{f3=[f3,f2,f1,f0,eb,e8,e5,e3],f7=[f7,f5,f4],ff=[ff,fe,fd,fc,fa,f8],105=[105,104,103,102,101]} for object 5:8a455596:::smithi11881747-30:db 2024-03-20T08:08:43.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:332bf1c7:::smithi11881747-32:9f repair needed (no obc) 2024-03-20T08:08:43.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-mon[87080]: pgmap v414: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 631 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 406 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-20T08:08:43.208 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:43.041+0000 7ffa6d96c700 -1 osd.5 pg_epoch: 490 pg[5.2s2( v 491'1053 (0'0,491'1053] local-lis/les=470/471 n=14 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [6,0,5]p6(0) r=2 lpr=470 luod=0'0 crt=489'1051 lcod 489'1051 mlcod 489'1051 active mbc={} ps=102] update_snap_map remove_oid 5:4cd24aac:::smithi11881747-48:9c failed with -2 2024-03-20T08:08:43.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:43.043+0000 7ffa6f16f700 -1 osd.5 pg_epoch: 491 pg[5.fs0( v 490'1014 (0'0,490'1014] local-lis/les=470/471 n=23 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [5,4,6]p5(0) r=0 lpr=470 crt=490'1014 lcod 488'1012 mlcod 488'1012 active+clean+snaptrim trimq=[95~1,e0~1,e3~1,f3~1,f5~1,f7~2,fa~1,fc~1,fe~1,101~4,109~1,10b~2] ps=86] update_snap_map remove_oid 5:f8faee4c:::smithi11881747-1:9d failed with -2 2024-03-20T08:08:43.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:43.052+0000 7ffa71974700 -1 osd.5 pg_epoch: 491 pg[5.2s2( v 491'1055 (0'0,491'1055] local-lis/les=470/471 n=13 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [6,0,5]p6(0) r=2 lpr=470 luod=0'0 crt=491'1055 mlcod 491'1055 active mbc={} ps=102] update_snap_map remove_oid 5:4acd7ed0:::smithi11881747-22:a5 failed with -2 2024-03-20T08:08:43.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:43.056+0000 7ffa70972700 -1 osd.5 pg_epoch: 491 pg[5.4s2( v 488'247 (0'0,488'247] local-lis/les=470/471 n=3 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [7,2,5]p7(0) r=2 lpr=470 luod=0'0 crt=488'247 mlcod 488'247 active mbc={} ps=102] update_snap_map remove_oid 5:27618338:::smithi11881747-17:a5 failed with -2 2024-03-20T08:08:43.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:43.057+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c],113=[113,112,111,110,10f,10e,10d],114=[114]} for object 5:f8faee4c:::smithi11881747-1:9d 2024-03-20T08:08:43.209 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:43.063+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:43.209 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.966+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.209 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:42.993+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.033+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.041+0000 7fa5d0032700 -1 osd.4 pg_epoch: 491 pg[5.cs1( v 491'33988 (380'27302,491'33988] local-lis/les=445/446 n=23 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [1,4,0]p1(0) r=1 lpr=445 luod=0'0 crt=490'33986 lcod 490'33986 mlcod 490'33986 active mbc={} ps=[32~1,50~1,65~1,6b~1,83~1,89~3,91~1,a3~2,a9~1,ab~1,af~1,b1~1,b5~1,b8~2]] update_snap_map remove_oid 5:335100af:::smithi11881747-8:9c failed with -2 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.042+0000 7fa5d1835700 -1 osd.4 pg_epoch: 491 pg[5.1s0( v 488'440 (0'0,488'440] local-lis/les=445/446 n=13 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [4,3,7]p4(0) r=0 lpr=445 crt=488'440 lcod 484'438 mlcod 484'438 active+clean+snaptrim trimq=[95~1,da~1,e0~1,e3~1,f0~1,f3~1,f5~1,f7~2,fa~1,fc~1,fe~7,109~1,10b~2] ps=82] update_snap_map remove_oid 5:89304740:::smithi11881747-4:9c failed with -2 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.043+0000 7fa5ce82f700 -1 osd.4 pg_epoch: 491 pg[5.fs1( v 490'1014 (0'0,490'1014] local-lis/les=470/471 n=23 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [5,4,6]p5(0) r=1 lpr=470 luod=0'0 crt=490'1014 mlcod 490'1014 active mbc={} ps=86] update_snap_map remove_oid 5:f8faee4c:::smithi11881747-1:9d failed with -2 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.051+0000 7fa5d603e700 -1 osd.4 pg_epoch: 491 pg[5.0s2( v 491'1206 (0'0,491'1206] local-lis/les=445/446 n=43 ec=73/73 lis/c=445/445 les/c/f=446/446/0 sis=445) [3,1,4]p3(0) r=2 lpr=445 luod=0'0 crt=491'1206 mlcod 491'1206 active mbc={} ps=58] update_snap_map remove_oid 5:07779a33:::smithi11881747-18:9c failed with -2 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.052+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.052+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:43.210 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.079+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.120+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.131+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.141+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.199+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.225+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.233+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.240+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.256+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.296+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.463 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.353+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.362+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.427+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.435+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.442+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.464 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.462+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.470+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.478+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.494+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.518+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.529+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.540+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.571+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.607+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.617+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.636+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.645+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.721+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:08:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:08:43] "GET /metrics HTTP/1.1" 200 36335 "" "Prometheus/2.43.0" 2024-03-20T08:08:43.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.741+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.776+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.787+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.816+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.828+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.858+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.866+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.888+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.904+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.912+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.919+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.927+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.957+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.964+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:43.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.973+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.042 INFO:tasks.rados.rados.0.smithi118.stdout:2122: done (4 left) 2024-03-20T08:08:44.042 INFO:tasks.rados.rados.0.smithi118.stdout:2123: read oid 46 snap 206 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2123: expect deleted 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2124: copy_from oid 35 from oid 5 current snap is 217 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2118: finishing rollback tid 0 to smithi11881747-39 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2125: read oid 11 snap -1 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2125: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2126: rmattr oid 6 current snap is 217 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2126: done (7 left) 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2127: rmattr oid 24 current snap is 217 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2127: done (7 left) 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2128: copy_from oid 34 from oid 28 current snap is 217 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2118: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2118: finishing rollback tid 2 to smithi11881747-39 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 13603 (ObjNum 623 snap 202 seq_num 623) dirty exists 2024-03-20T08:08:44.043 INFO:tasks.rados.rados.0.smithi118.stdout:2120: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:08:44.053 INFO:tasks.rados.rados.0.smithi118.stdout:2121: expect (ObjNum 545 snap 179 seq_num 545) 2024-03-20T08:08:44.225 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:44.046+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:44.226 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:44.046+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c],113=[113,112,111,110,10f,10e,10d],114=[114]} for object 5:f8faee4c:::smithi11881747-1:9d 2024-03-20T08:08:44.226 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:44.226 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: osdmap e491: 8 total, 8 up, 8 in 2024-03-20T08:08:44.226 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:44.226 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:44.226 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:44.226 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c],113=[113,112,111,110,10f,10e,10d],114=[114]} for object 5:f8faee4c:::smithi11881747-1:9d 2024-03-20T08:08:44.226 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:44.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:44.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:44.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:44.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:44.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:44.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:44.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:43.997+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.013+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.020+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.038+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.047+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.074+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.100+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.139+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.180+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.195+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.203+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.230 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.226+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:44.040+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:44.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:44.042+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:08:44.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:44.042+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:44.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:44.042+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:44.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:44.048+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:44.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:44.049+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:44.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:44.050+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:44.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:44.050+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:08:44.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:44.050+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:44.424 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:44.048+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: osdmap e491: 8 total, 8 up, 8 in 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c],113=[113,112,111,110,10f,10e,10d],114=[114]} for object 5:f8faee4c:::smithi11881747-1:9d 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: osdmap e491: 8 total, 8 up, 8 in 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c],113=[113,112,111,110,10f,10e,10d],114=[114]} for object 5:f8faee4c:::smithi11881747-1:9d 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:44.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:44.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:44.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:44.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:44.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:44.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:44.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.234+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.250+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.265+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.274+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.297+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.327+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.334+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.372+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.380+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.444+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.484 INFO:tasks.rados.rados.0.smithi118.stdout:2119: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.504+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.520+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.543+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.558+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.576+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.617+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.630+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.638+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.648+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.659+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.670+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.697+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.721+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.729+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.983 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.983 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.748+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.775+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.797+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.826+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.835+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.845+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.875+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.984 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.885+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.913+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.932+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.961+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.971+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:44.985 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.983+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.082 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: osdmap e492: 8 total, 8 up, 8 in 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c],113=[113,112,111,110,10f,10e,10d],114=[114]} for object 5:f8faee4c:::smithi11881747-1:9d 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:45.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:45.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:45.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:45.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-mon[87080]: pgmap v417: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 620 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 13 op/s 2024-03-20T08:08:45.237 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:44 smithi196 systemd[1]: Stopping Ceph osd.6 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:08:45.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:44.994+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.049+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.070+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.096+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.168+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.188+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.212+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.228+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.236+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.258 INFO:tasks.rados.rados.0.smithi118.stdout:2118: done (7 left) 2024-03-20T08:08:45.258 INFO:tasks.rados.rados.0.smithi118.stdout:2119: done (6 left) 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2120: done (5 left) 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2121: done (4 left) 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2123: done (3 left) 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2129: read oid 27 snap -1 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2129: expect (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2130: copy_from oid 39 from oid 17 current snap is 217 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2128: finishing copy_from racing read to smithi11881747-34 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2128: finishing copy_from to smithi11881747-34 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 33906 (ObjNum 644 snap 211 seq_num 644) dirty exists 2024-03-20T08:08:45.260 INFO:tasks.rados.rados.0.smithi118.stdout:2125: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:08:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: osdmap e492: 8 total, 8 up, 8 in 2024-03-20T08:08:45.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c],113=[113,112,111,110,10f,10e,10d],114=[114]} for object 5:f8faee4c:::smithi11881747-1:9d 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[93121]: pgmap v417: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 620 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 13 op/s 2024-03-20T08:08:45.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: osdmap e492: 8 total, 8 up, 8 in 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{f6=[f5,f4,f3,f2,f1,eb,e8,e5],f7=[f7],102=[102,101,fe,fd,fc,fa,f8],10b=[10b,10a],10c=[10c],113=[113,112,111,110,10f,10e,10d],114=[114]} for object 5:f8faee4c:::smithi11881747-1:9d 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:45 smithi118 ceph-mon[95692]: pgmap v417: 121 pgs: 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 108 active+clean; 620 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 4.7 MiB/s rd, 2.1 MiB/s wr, 13 op/s 2024-03-20T08:08:45.437 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:08:45.437 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (8m) 89s ago 9m 27.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (9m) 26s ago 9m 89.2M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (10m) 89s ago 10m 15.4M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (10m) 26s ago 10m 21.2M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (5m) 26s ago 13m 452M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (5m) 89s ago 15m 508M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (4m) 89s ago 16m 56.2M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (4m) 26s ago 14m 47.8M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (4m) 89s ago 14m 51.1M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (8m) 89s ago 8m 15.5M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (8m) 26s ago 8m 15.1M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (3m) 89s ago 13m 334M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:08:45.438 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (2m) 89s ago 13m 299M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:08:45.439 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (2m) 89s ago 12m 251M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:08:45.439 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (94s) 89s ago 12m 12.7M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:08:45.439 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (56s) 26s ago 11m 197M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:08:45.439 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (32s) 26s ago 11m 12.4M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 72b569d10541 2024-03-20T08:08:45.439 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (11m) 26s ago 11m 251M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 399890989a91 2024-03-20T08:08:45.439 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (10m) 26s ago 10m 198M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:08:45.439 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (5m) 26s ago 9m 60.7M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:08:45.490 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[48573]: 2024-03-20T08:08:45.314+0000 7f97f4627700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:08:45.490 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[48573]: 2024-03-20T08:08:45.314+0000 7f97f4627700 -1 osd.6 492 *** Got signal Terminated *** 2024-03-20T08:08:45.490 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[48573]: 2024-03-20T08:08:45.314+0000 7f97f4627700 -1 osd.6 492 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.260+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.296+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.315+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.491 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.354+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.378+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.440+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.490+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.531+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.539+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.546+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.556+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.571+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.585+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.601+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.616+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.635+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.651+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.675+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.691+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.713+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.720+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.728+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.736+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.743+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.892 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:08:45.892 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:08:45.892 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:08:45.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:08:45.892 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:08:45.892 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:08:45.892 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:08:45.893 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:08:45.893 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2, 2024-03-20T08:08:45.893 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 6 2024-03-20T08:08:45.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:08:45.893 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:08:45.893 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:08:45.893 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:08:45.894 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:08:45.894 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 4, 2024-03-20T08:08:45.894 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 11 2024-03-20T08:08:45.894 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:08:45.894 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:08:45.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.752+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.801+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.833+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.841+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.849+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.857+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:45.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.873+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.880+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.887+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.906+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.913+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.935+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.944+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.983+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.990+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:45.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:2124: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:2124: finishing copy_from to smithi11881747-35 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 23064 (ObjNum 642 snap 209 seq_num 642) dirty exists 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:2124: done (4 left) 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:2125: done (3 left) 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:2128: done (2 left) 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:2131: rmattr oid 3 current snap is 217 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:2131: done (2 left) 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 217 2024-03-20T08:08:46.029 INFO:tasks.rados.rados.0.smithi118.stdout:2129: expect (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:08:46.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:46.053+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:46.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-mon[93121]: from='client.44264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:46.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-mon[93121]: osd.6 marked itself down and dead 2024-03-20T08:08:46.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/2107317336' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:46.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:46.055+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:46.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:46.049+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:46.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:46.049+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:46.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:46.055+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:46.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:46.056+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:46.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:46.058+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:46.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:46.058+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:46.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-mon[95692]: from='client.44264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:46.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-mon[95692]: osd.6 marked itself down and dead 2024-03-20T08:08:46.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:46 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/2107317336' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:46.250 INFO:tasks.rados.rados.0.smithi118.stdout:2130: finishing copy_from racing read to smithi11881747-39 2024-03-20T08:08:46.250 INFO:tasks.rados.rados.0.smithi118.stdout:2130: finishing copy_from to smithi11881747-39 2024-03-20T08:08:46.250 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 7050 (ObjNum 568 snap 185 seq_num 568) dirty exists 2024-03-20T08:08:46.250 INFO:tasks.rados.rados.0.smithi118.stdout:2132: seq_num 666 ranges {6258688=761856,7020544=491520,7512064=311296} 2024-03-20T08:08:46.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-mon[87080]: from='client.44264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:46.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-mon[87080]: osd.6 marked itself down and dead 2024-03-20T08:08:46.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/2107317336' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:08:46.251 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:46.056+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.007+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.050+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.058+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.058+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:46.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.083+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.091+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.100+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.122+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.152+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.159+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.167+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.190+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.236+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.251+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.289 INFO:tasks.rados.rados.0.smithi118.stdout:2132: writing smithi11881747-32 from 6258688 to 7020544 tid 1 2024-03-20T08:08:46.292 INFO:tasks.rados.rados.0.smithi118.stdout:2132: writing smithi11881747-32 from 7020544 to 7512064 tid 2 2024-03-20T08:08:46.295 INFO:tasks.rados.rados.0.smithi118.stdout:2132: writing smithi11881747-32 from 7512064 to 7823360 tid 3 2024-03-20T08:08:46.295 INFO:tasks.rados.rados.0.smithi118.stdout:2129: done (2 left) 2024-03-20T08:08:46.295 INFO:tasks.rados.rados.0.smithi118.stdout:2130: done (1 left) 2024-03-20T08:08:46.295 INFO:tasks.rados.rados.0.smithi118.stdout:2133: rollback oid 22 current snap is 217 2024-03-20T08:08:46.295 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 22 to 169 2024-03-20T08:08:46.295 INFO:tasks.rados.rados.0.smithi118.stdout:2134: copy_from oid 33 from oid 31 current snap is 217 2024-03-20T08:08:46.295 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 217 2024-03-20T08:08:46.295 INFO:tasks.rados.rados.0.smithi118.stdout:2135: seq_num 667 ranges {0=778240,778240=778240,1556480=24576} 2024-03-20T08:08:46.300 INFO:tasks.rados.rados.0.smithi118.stdout:2135: writing smithi11881747-36 from 0 to 778240 tid 1 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:2135: writing smithi11881747-36 from 778240 to 1556480 tid 2 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:2135: writing smithi11881747-36 from 1556480 to 1581056 tid 3 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:2136: read oid 20 snap -1 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:2136: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 217 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:2132: finishing write tid 1 to smithi11881747-32 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:2132: finishing write tid 2 to smithi11881747-32 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:2132: finishing write tid 3 to smithi11881747-32 2024-03-20T08:08:46.306 INFO:tasks.rados.rados.0.smithi118.stdout:2137: seq_num 668 ranges {4382720=794624,5177344=663552,5840896=139264} 2024-03-20T08:08:46.334 INFO:tasks.rados.rados.0.smithi118.stdout:2137: writing smithi11881747-39 from 4382720 to 5177344 tid 1 2024-03-20T08:08:46.338 INFO:tasks.rados.rados.0.smithi118.stdout:2137: writing smithi11881747-39 from 5177344 to 5840896 tid 2 2024-03-20T08:08:46.339 INFO:tasks.rados.rados.0.smithi118.stdout:2137: writing smithi11881747-39 from 5840896 to 5980160 tid 3 2024-03-20T08:08:46.339 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 217 2024-03-20T08:08:46.339 INFO:tasks.rados.rados.0.smithi118.stdout:2132: finishing write tid 4 to smithi11881747-32 2024-03-20T08:08:46.339 INFO:tasks.rados.rados.0.smithi118.stdout:2138: seq_num 669 ranges {0=409600,409600=548864,958464=655360} 2024-03-20T08:08:46.342 INFO:tasks.rados.rados.0.smithi118.stdout:2138: writing smithi11881747-18 from 0 to 409600 tid 1 2024-03-20T08:08:46.345 INFO:tasks.rados.rados.0.smithi118.stdout:2138: writing smithi11881747-18 from 409600 to 958464 tid 2 2024-03-20T08:08:46.350 INFO:tasks.rados.rados.0.smithi118.stdout:2138: writing smithi11881747-18 from 958464 to 1613824 tid 3 2024-03-20T08:08:46.350 INFO:tasks.rados.rados.0.smithi118.stdout:2139: delete oid 17 current snap is 217 2024-03-20T08:08:46.350 INFO:tasks.rados.rados.0.smithi118.stdout:2132: finishing write tid 5 to smithi11881747-32 2024-03-20T08:08:46.350 INFO:tasks.rados.rados.0.smithi118.stdout:2132: oid 32 updating version 0 to 33995 2024-03-20T08:08:46.350 INFO:tasks.rados.rados.0.smithi118.stdout:2132: oid 32 version 33995 is already newer than 33992 2024-03-20T08:08:46.351 INFO:tasks.rados.rados.0.smithi118.stdout:2132: oid 32 version 33995 is already newer than 33994 2024-03-20T08:08:46.351 INFO:tasks.rados.rados.0.smithi118.stdout:2132: oid 32 version 33995 is already newer than 33993 2024-03-20T08:08:46.351 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 33995 (ObjNum 666 snap 217 seq_num 666) dirty exists 2024-03-20T08:08:46.351 INFO:tasks.rados.rados.0.smithi118.stdout:2132: left oid 32 (ObjNum 666 snap 217 seq_num 666) 2024-03-20T08:08:46.351 INFO:tasks.rados.rados.0.smithi118.stdout:2136: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.268+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.276+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.284+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.301+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.323+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.331+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.342+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.388+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.396+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.419+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.428+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.435+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.471+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.479+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.528 INFO:tasks.rados.rados.0.smithi118.stdout:2137: finishing write tid 1 to smithi11881747-39 2024-03-20T08:08:46.528 INFO:tasks.rados.rados.0.smithi118.stdout:2137: finishing write tid 2 to smithi11881747-39 2024-03-20T08:08:46.528 INFO:tasks.rados.rados.0.smithi118.stdout:2137: finishing write tid 3 to smithi11881747-39 2024-03-20T08:08:46.528 INFO:tasks.rados.rados.0.smithi118.stdout:2132: done (7 left) 2024-03-20T08:08:46.528 INFO:tasks.rados.rados.0.smithi118.stdout:2136: done (6 left) 2024-03-20T08:08:46.528 INFO:tasks.rados.rados.0.smithi118.stdout:2139: done (5 left) 2024-03-20T08:08:46.528 INFO:tasks.rados.rados.0.smithi118.stdout:2140: read oid 37 snap -1 2024-03-20T08:08:46.528 INFO:tasks.rados.rados.0.smithi118.stdout:2140: expect (ObjNum 662 snap 216 seq_num 662) 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2141: snap_create 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2137: finishing write tid 4 to smithi11881747-39 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2137: finishing write tid 5 to smithi11881747-39 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2137: oid 39 updating version 0 to 13606 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2137: oid 39 updating version 13606 to 13607 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2137: oid 39 version 13607 is already newer than 13604 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2137: oid 39 version 13607 is already newer than 13605 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 13607 (ObjNum 668 snap 217 seq_num 668) dirty exists 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2137: left oid 39 (ObjNum 668 snap 217 seq_num 668) 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2138: finishing write tid 1 to smithi11881747-18 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2138: finishing write tid 2 to smithi11881747-18 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2138: finishing write tid 3 to smithi11881747-18 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2138: finishing write tid 4 to smithi11881747-18 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2138: finishing write tid 5 to smithi11881747-18 2024-03-20T08:08:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:2138: oid 18 updating version 0 to 13609 2024-03-20T08:08:46.530 INFO:tasks.rados.rados.0.smithi118.stdout:2138: oid 18 version 13609 is already newer than 13608 2024-03-20T08:08:46.530 INFO:tasks.rados.rados.0.smithi118.stdout:2138: oid 18 updating version 13609 to 13610 2024-03-20T08:08:46.530 INFO:tasks.rados.rados.0.smithi118.stdout:2138: oid 18 updating version 13610 to 13611 2024-03-20T08:08:46.530 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 13611 (ObjNum 669 snap 217 seq_num 669) dirty exists 2024-03-20T08:08:46.530 INFO:tasks.rados.rados.0.smithi118.stdout:2138: left oid 18 (ObjNum 669 snap 217 seq_num 669) 2024-03-20T08:08:46.545 INFO:tasks.rados.rados.0.smithi118.stdout:2140: expect (ObjNum 662 snap 216 seq_num 662) 2024-03-20T08:08:46.754 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:46 smithi196 podman[102652]: 2024-03-20 08:08:46.602056062 +0000 UTC m=+1.441956999 container died 399890989a91821d42a996c85c5638245664f3536a05f75bb8fab9f15d20f097 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=reef-78dbc53, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.510+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.518+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.526+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.535+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.547+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.558+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.568+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.600+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.631+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.654+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.661+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.669+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.676+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.684+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.691+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.731+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.747+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:46.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.764+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.780+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.789+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.805+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.816+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.824+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.841+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.849+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.857+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.879+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.887+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.936+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.944+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.952+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:46.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:2137: done (6 left) 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:2138: done (5 left) 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:2140: done (4 left) 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:2141: done (3 left) 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:2142: rollback oid 17 current snap is 218 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 212 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:2143: rollback oid 3 current snap is 218 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 3 to 169 2024-03-20T08:08:47.047 INFO:tasks.rados.rados.0.smithi118.stdout:2144: snap_create 2024-03-20T08:08:47.056 INFO:tasks.rados.rados.0.smithi118.stdout:2142: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:08:47.061 INFO:tasks.rados.rados.0.smithi118.stdout:2142: finishing rollback tid 2 to smithi11881747-17 2024-03-20T08:08:47.061 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7053 (ObjNum 568 snap 185 seq_num 568) dirty exists 2024-03-20T08:08:47.069 INFO:tasks.rados.rados.0.smithi118.stdout:2133: finishing rollback tid 0 to smithi11881747-22 2024-03-20T08:08:47.069 INFO:tasks.rados.rados.0.smithi118.stdout:2133: finishing rollback tid 1 to smithi11881747-22 2024-03-20T08:08:47.070 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 7087 (ObjNum 33 snap 0 seq_num 322727376) dirty dne 2024-03-20T08:08:47.071 INFO:tasks.rados.rados.0.smithi118.stdout:2143: finishing rollback tid 0 to smithi11881747-3 2024-03-20T08:08:47.071 INFO:tasks.rados.rados.0.smithi118.stdout:2143: finishing rollback tid 1 to smithi11881747-3 2024-03-20T08:08:47.072 INFO:tasks.rados.rados.0.smithi118.stdout:2143: finishing rollback tid 2 to smithi11881747-3 2024-03-20T08:08:47.072 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 13606 (ObjNum 410 snap 136 seq_num 410) dirty exists 2024-03-20T08:08:47.077 INFO:tasks.rados.rados.0.smithi118.stdout:2135: finishing write tid 1 to smithi11881747-36 2024-03-20T08:08:47.077 INFO:tasks.rados.rados.0.smithi118.stdout:2135: finishing write tid 2 to smithi11881747-36 2024-03-20T08:08:47.077 INFO:tasks.rados.rados.0.smithi118.stdout:2135: finishing write tid 3 to smithi11881747-36 2024-03-20T08:08:47.078 INFO:tasks.rados.rados.0.smithi118.stdout:2135: finishing write tid 4 to smithi11881747-36 2024-03-20T08:08:47.082 INFO:tasks.rados.rados.0.smithi118.stdout:2135: finishing write tid 5 to smithi11881747-36 2024-03-20T08:08:47.082 INFO:tasks.rados.rados.0.smithi118.stdout:2135: oid 36 updating version 0 to 33907 2024-03-20T08:08:47.082 INFO:tasks.rados.rados.0.smithi118.stdout:2135: oid 36 updating version 33907 to 33908 2024-03-20T08:08:47.082 INFO:tasks.rados.rados.0.smithi118.stdout:2135: oid 36 updating version 33908 to 33910 2024-03-20T08:08:47.082 INFO:tasks.rados.rados.0.smithi118.stdout:2135: oid 36 version 33910 is already newer than 33909 2024-03-20T08:08:47.082 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 33910 (ObjNum 667 snap 217 seq_num 667) dirty exists 2024-03-20T08:08:47.082 INFO:tasks.rados.rados.0.smithi118.stdout:2135: left oid 36 (ObjNum 667 snap 217 seq_num 667) 2024-03-20T08:08:47.094 INFO:tasks.rados.rados.0.smithi118.stdout:2134: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:08:47.098 INFO:tasks.rados.rados.0.smithi118.stdout:2134: finishing copy_from to smithi11881747-33 2024-03-20T08:08:47.098 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 7108 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:08:47.258 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:47.054+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:47.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:47.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:08:47.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: osdmap e493: 8 total, 7 up, 8 in 2024-03-20T08:08:47.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:47.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:47.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:47.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:47.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:47.260 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:47.260 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:47.260 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:47.260 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:47.260 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: pgmap v419: 121 pgs: 14 stale+active+clean, 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 94 active+clean; 630 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 20 MiB/s rd, 7.0 MiB/s wr, 32 op/s 2024-03-20T08:08:47.260 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-mon[87080]: osdmap e494: 8 total, 7 up, 8 in 2024-03-20T08:08:47.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.022+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.030+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.038+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.046+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.055+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.079+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.091+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.127+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.150+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.165+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.180+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.189+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.204+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.236+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.259+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:47.053+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:47.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:47.051+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:47.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:47.055+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:47.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:47.049+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:47.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:47.049+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:47.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:47.053+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:47.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:47.067+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:08:47.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:47.067+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: osdmap e493: 8 total, 7 up, 8 in 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: pgmap v419: 121 pgs: 14 stale+active+clean, 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 94 active+clean; 630 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 20 MiB/s rd, 7.0 MiB/s wr, 32 op/s 2024-03-20T08:08:47.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[93121]: osdmap e494: 8 total, 7 up, 8 in 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: osdmap e493: 8 total, 7 up, 8 in 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 110=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:47.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:47.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: pgmap v419: 121 pgs: 14 stale+active+clean, 1 active+clean+snaptrim_wait+snaptrim_error, 2 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 7 active+clean+snaptrim_error, 94 active+clean; 630 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 20 MiB/s rd, 7.0 MiB/s wr, 32 op/s 2024-03-20T08:08:47.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:47 smithi118 ceph-mon[95692]: osdmap e494: 8 total, 7 up, 8 in 2024-03-20T08:08:47.509 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:47 smithi196 podman[102652]: 2024-03-20 08:08:47.361758773 +0000 UTC m=+2.201659726 container cleanup 399890989a91821d42a996c85c5638245664f3536a05f75bb8fab9f15d20f097 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-20T08:08:47.510 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:47 smithi196 bash[102652]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6 2024-03-20T08:08:47.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.267+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.282+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.321+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.351+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.368+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.377+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.423+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.432+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.466+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.492+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.556+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.564+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.579+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.587+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.617+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.633+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.648+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.656+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.671+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.678+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.746+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:47.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.049 INFO:tasks.rados.rados.0.smithi118.stdout:2133: done (5 left) 2024-03-20T08:08:48.049 INFO:tasks.rados.rados.0.smithi118.stdout:2134: done (4 left) 2024-03-20T08:08:48.049 INFO:tasks.rados.rados.0.smithi118.stdout:2135: done (3 left) 2024-03-20T08:08:48.049 INFO:tasks.rados.rados.0.smithi118.stdout:2142: done (2 left) 2024-03-20T08:08:48.049 INFO:tasks.rados.rados.0.smithi118.stdout:2143: done (1 left) 2024-03-20T08:08:48.049 INFO:tasks.rados.rados.0.smithi118.stdout:2144: done (0 left) 2024-03-20T08:08:48.050 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 219 2024-03-20T08:08:48.050 INFO:tasks.rados.rados.0.smithi118.stdout:2145: seq_num 671 ranges {0=442368,442368=761856,1204224=442368} 2024-03-20T08:08:48.051 INFO:tasks.rados.rados.0.smithi118.stdout:2145: writing smithi11881747-50 from 0 to 442368 tid 1 2024-03-20T08:08:48.055 INFO:tasks.rados.rados.0.smithi118.stdout:2145: writing smithi11881747-50 from 442368 to 1204224 tid 2 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.769+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.792+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.800+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.815+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.831+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.847+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.854+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.862+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.878+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.885+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.893+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:2145: writing smithi11881747-50 from 1204224 to 1646592 tid 3 2024-03-20T08:08:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:2146: read oid 28 snap 216 2024-03-20T08:08:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:2146: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:08:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:2147: copy_from oid 5 from oid 49 current snap is 219 2024-03-20T08:08:48.058 INFO:tasks.rados.rados.0.smithi118.stdout:2148: snap_remove snap 214 2024-03-20T08:08:48.310 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:48 smithi196 podman[102674]: 2024-03-20 08:08:48.289263034 +0000 UTC m=+1.688746400 container remove 399890989a91821d42a996c85c5638245664f3536a05f75bb8fab9f15d20f097 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=reef-78dbc53, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-03-20T08:08:48.310 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:48.066+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:48.311 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-mon[87080]: osdmap e495: 8 total, 7 up, 8 in 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.901+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:47.908+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.061+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.061+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.092+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.100+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.138+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.145+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.160+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.176+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.184+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.200+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.208+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.215+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.240+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.247+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.255+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.270+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.286+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.295+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:48.062+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:48.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:48.065+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:48.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:48.051+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:48.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:48.051+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:48.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:48.048+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:48.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:48.048+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:48.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:48.048+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:48.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:48.048+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:48.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:48.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:48.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:48.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[93121]: osdmap e495: 8 total, 7 up, 8 in 2024-03-20T08:08:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:48.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:48.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:48.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:48 smithi118 ceph-mon[95692]: osdmap e495: 8 total, 7 up, 8 in 2024-03-20T08:08:48.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.340+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.352+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.432+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.450+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.496+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.521+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.545+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.556+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.827 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:48 smithi196 podman[102767]: 2024-03-20 08:08:48.582852545 +0000 UTC m=+0.022829736 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:08:48.827 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:48 smithi196 podman[102767]: 2024-03-20 08:08:48.707076352 +0000 UTC m=+0.147053517 container create 4d297e6e430792a9567a8c28d45d2a7a479cd7ef91bf01bbaf18db1267cd99bd (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-20T08:08:48.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.591+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.600+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.614+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.622+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.637+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.690+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.698+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.724+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.757+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.773+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.780+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.788+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:48.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.050 INFO:tasks.rados.rados.0.smithi118.stdout:2148: done (3 left) 2024-03-20T08:08:49.051 INFO:tasks.rados.rados.0.smithi118.stdout:2149: read oid 35 snap 213 2024-03-20T08:08:49.051 INFO:tasks.rados.rados.0.smithi118.stdout:2149: expect (ObjNum 581 snap 188 seq_num 581) 2024-03-20T08:08:49.051 INFO:tasks.rados.rados.0.smithi118.stdout:2150: rollback oid 11 current snap is 219 2024-03-20T08:08:49.051 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 201 2024-03-20T08:08:49.051 INFO:tasks.rados.rados.0.smithi118.stdout:2147: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:08:49.051 INFO:tasks.rados.rados.0.smithi118.stdout:2146: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:08:49.084 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:49.084 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:49.084 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:49.084 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:49.084 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:49.084 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:49.084 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:49.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:49.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:49.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: pgmap v422: 121 pgs: 5 active+undersized, 1 active+undersized+degraded, 14 stale+active+clean, 1 active+clean+snaptrim, 11 active+clean+snaptrim_error, 89 active+clean; 627 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 30 MiB/s rd, 8.1 MiB/s wr, 32 op/s; 1/1278 objects degraded (0.078%) 2024-03-20T08:08:49.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-mon[87080]: osdmap e496: 8 total, 7 up, 8 in 2024-03-20T08:08:49.085 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:49.060+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:49.085 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:48 smithi196 podman[102767]: 2024-03-20 08:08:48.96890423 +0000 UTC m=+0.408881396 container init 4d297e6e430792a9567a8c28d45d2a7a479cd7ef91bf01bbaf18db1267cd99bd (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-20T08:08:49.085 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:48 smithi196 podman[102767]: 2024-03-20 08:08:48.975779347 +0000 UTC m=+0.415756514 container start 4d297e6e430792a9567a8c28d45d2a7a479cd7ef91bf01bbaf18db1267cd99bd (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.838+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.871+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.879+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.886+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.895+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.902+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.925+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.932+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.972+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.983+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:48.994+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.021+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.063+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:49.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.218 INFO:tasks.rados.rados.0.smithi118.stdout:2147: finishing copy_from to smithi11881747-5 2024-03-20T08:08:49.218 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 7099 (ObjNum 628 snap 204 seq_num 628) dirty exists 2024-03-20T08:08:49.218 INFO:tasks.rados.rados.0.smithi118.stdout:2145: finishing write tid 1 to smithi11881747-50 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: finishing write tid 2 to smithi11881747-50 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: finishing write tid 3 to smithi11881747-50 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: finishing write tid 4 to smithi11881747-50 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: finishing write tid 5 to smithi11881747-50 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: oid 50 updating version 0 to 14722 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: oid 50 updating version 14722 to 14725 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: oid 50 version 14725 is already newer than 14724 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: oid 50 version 14725 is already newer than 14723 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 14725 (ObjNum 671 snap 219 seq_num 671) dirty exists 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: left oid 50 (ObjNum 671 snap 219 seq_num 671) 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2145: done (4 left) 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2146: done (3 left) 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2147: done (2 left) 2024-03-20T08:08:49.219 INFO:tasks.rados.rados.0.smithi118.stdout:2151: read oid 2 snap 210 2024-03-20T08:08:49.220 INFO:tasks.rados.rados.0.smithi118.stdout:2151: expect deleted 2024-03-20T08:08:49.220 INFO:tasks.rados.rados.0.smithi118.stdout:2152: snap_create 2024-03-20T08:08:49.220 INFO:tasks.rados.rados.0.smithi118.stdout:2150: finishing rollback tid 0 to smithi11881747-11 2024-03-20T08:08:49.220 INFO:tasks.rados.rados.0.smithi118.stdout:2150: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:08:49.220 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 23077 (ObjNum 33 snap 0 seq_num 322714448) dirty dne 2024-03-20T08:08:49.220 INFO:tasks.rados.rados.0.smithi118.stdout:2149: expect (ObjNum 581 snap 188 seq_num 581) 2024-03-20T08:08:49.336 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:49 smithi196 podman[102767]: 2024-03-20 08:08:49.310982092 +0000 UTC m=+0.750959262 container attach 4d297e6e430792a9567a8c28d45d2a7a479cd7ef91bf01bbaf18db1267cd99bd (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True) 2024-03-20T08:08:49.336 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:49 smithi196 podman[102767]: 2024-03-20 08:08:49.329666514 +0000 UTC m=+0.769643680 container died 4d297e6e430792a9567a8c28d45d2a7a479cd7ef91bf01bbaf18db1267cd99bd (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-20T08:08:49.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.103+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.122+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.153+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.164+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.202+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.211+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.289+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.326+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.337+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:49.058+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:49.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:49.057+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:49.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:49.052+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:49.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:49.049+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:49.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:49.049+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:49.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:49.050+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:49.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:49.050+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:08:49.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:49.050+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:08:49.069+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 11 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:49.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: pgmap v422: 121 pgs: 5 active+undersized, 1 active+undersized+degraded, 14 stale+active+clean, 1 active+clean+snaptrim, 11 active+clean+snaptrim_error, 89 active+clean; 627 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 30 MiB/s rd, 8.1 MiB/s wr, 32 op/s; 1/1278 objects degraded (0.078%) 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[93121]: osdmap e496: 8 total, 7 up, 8 in 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: pgmap v422: 121 pgs: 5 active+undersized, 1 active+undersized+degraded, 14 stale+active+clean, 1 active+clean+snaptrim, 11 active+clean+snaptrim_error, 89 active+clean; 627 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 30 MiB/s rd, 8.1 MiB/s wr, 32 op/s; 1/1278 objects degraded (0.078%) 2024-03-20T08:08:49.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:49 smithi118 ceph-mon[95692]: osdmap e496: 8 total, 7 up, 8 in 2024-03-20T08:08:49.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.360+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.368+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.414+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.421+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.444+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.460+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.489+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.504+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.519+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.547+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.556+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.564+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.573+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.597+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.604+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.627+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.643+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.651+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.660+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.669+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.723+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.754+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.762+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.793+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.801+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.809+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.834+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:49.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.082 INFO:tasks.rados.rados.0.smithi118.stdout:2149: done (3 left) 2024-03-20T08:08:50.082 INFO:tasks.rados.rados.0.smithi118.stdout:2150: done (2 left) 2024-03-20T08:08:50.082 INFO:tasks.rados.rados.0.smithi118.stdout:2151: done (1 left) 2024-03-20T08:08:50.082 INFO:tasks.rados.rados.0.smithi118.stdout:2152: done (0 left) 2024-03-20T08:08:50.082 INFO:tasks.rados.rados.0.smithi118.stdout:2153: snap_create 2024-03-20T08:08:50.096 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:49 smithi196 podman[102813]: 2024-03-20 08:08:49.982498891 +0000 UTC m=+0.642762663 container remove 4d297e6e430792a9567a8c28d45d2a7a479cd7ef91bf01bbaf18db1267cd99bd (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:08:50.097 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:49 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.6.service: Deactivated successfully. 2024-03-20T08:08:50.097 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:49 smithi196 systemd[1]: Stopped Ceph osd.6 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:08:50.097 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:49 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.6.service: Consumed 20.600s CPU time. 2024-03-20T08:08:50.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:50.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:50.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:50.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:50.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:50.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:50.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:50.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:50.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:50.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 11 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:50.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-mon[87080]: Health check failed: Degraded data redundancy: 1/1278 objects degraded (0.078%), 1 pg degraded (PG_DEGRADED) 2024-03-20T08:08:50.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.850+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.858+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.876+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.884+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.899+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.915+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.923+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.938+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.961+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.983+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:49.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.022+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.031+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.089+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.347 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:50.145+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:50.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.105+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.113+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.128+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.135+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.144+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.144+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.156+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.164+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.209+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.219+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.227+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.236+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.253+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.281+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.328+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.337+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:50.081+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:50.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:50.142+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:50.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:50.082+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:50.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:50.082+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:50.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:50.142+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:50.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:50.142+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:50.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:50.143+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:50.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:50.143+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 11 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[93121]: Health check failed: Degraded data redundancy: 1/1278 objects degraded (0.078%), 1 pg degraded (PG_DEGRADED) 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 11 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:08:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:50 smithi118 ceph-mon[95692]: Health check failed: Degraded data redundancy: 1/1278 objects degraded (0.078%), 1 pg degraded (PG_DEGRADED) 2024-03-20T08:08:50.601 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:50 smithi196 systemd[1]: Starting Ceph osd.6 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:08:50.602 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.358+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.602 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.369+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.602 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.379+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.602 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.602 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.602 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.414+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.602 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.424+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.459+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.484+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.507+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.519+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.547+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.556+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.603 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.571+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.602+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.858 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:50 smithi196 podman[102920]: 2024-03-20 08:08:50.520378059 +0000 UTC m=+0.020661878 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:08:50.858 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:50 smithi196 podman[102920]: 2024-03-20 08:08:50.64316768 +0000 UTC m=+0.143451460 container create 792882856c835cbbd88b926cf11ea69cec73c41355fc1700ed36236dd5d6ae01 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.633+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.693+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.701+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.708+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.716+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.724+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.747+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.814+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.828+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.836+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.844+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.851+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:50.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.858+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.086 INFO:tasks.rados.rados.0.smithi118.stdout:2153: done (0 left) 2024-03-20T08:08:51.086 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 221 2024-03-20T08:08:51.086 INFO:tasks.rados.rados.0.smithi118.stdout:2154: seq_num 672 ranges {1646592=458752,2105344=647168,2752512=466944,3219456=90112} 2024-03-20T08:08:51.097 INFO:tasks.rados.rados.0.smithi118.stdout:2154: writing smithi11881747-50 from 1646592 to 2105344 tid 1 2024-03-20T08:08:51.101 INFO:tasks.rados.rados.0.smithi118.stdout:2154: writing smithi11881747-50 from 2105344 to 2752512 tid 2 2024-03-20T08:08:51.103 INFO:tasks.rados.rados.0.smithi118.stdout:2154: writing smithi11881747-50 from 2752512 to 3219456 tid 3 2024-03-20T08:08:51.104 INFO:tasks.rados.rados.0.smithi118.stdout:2154: writing smithi11881747-50 from 3219456 to 3309568 tid 4 2024-03-20T08:08:51.104 INFO:tasks.rados.rados.0.smithi118.stdout:2155: rmattr oid 14 current snap is 221 2024-03-20T08:08:51.104 INFO:tasks.rados.rados.0.smithi118.stdout:2156: read oid 45 snap -1 2024-03-20T08:08:51.104 INFO:tasks.rados.rados.0.smithi118.stdout:2156: expect (ObjNum 661 snap 216 seq_num 661) 2024-03-20T08:08:51.104 INFO:tasks.rados.rados.0.smithi118.stdout:2157: snap_create 2024-03-20T08:08:51.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:51.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: osdmap e497: 8 total, 7 up, 8 in 2024-03-20T08:08:51.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:51.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:51.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:51.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:51.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:51.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:51.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-mon[87080]: pgmap v425: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 32 active+undersized, 6 active+undersized+degraded, 3 stale+active+clean, 10 active+clean+snaptrim_error, 68 active+clean; 637 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 19 MiB/s rd, 4.1 MiB/s wr, 12 op/s; 54/1281 objects degraded (4.215%) 2024-03-20T08:08:51.109 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:50 smithi196 podman[102920]: 2024-03-20 08:08:50.884151597 +0000 UTC m=+0.384435396 container init 792882856c835cbbd88b926cf11ea69cec73c41355fc1700ed36236dd5d6ae01 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-20T08:08:51.109 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:50 smithi196 podman[102920]: 2024-03-20 08:08:50.890484541 +0000 UTC m=+0.390768322 container start 792882856c835cbbd88b926cf11ea69cec73c41355fc1700ed36236dd5d6ae01 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-20T08:08:51.109 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:50 smithi196 podman[102920]: 2024-03-20 08:08:50.986802407 +0000 UTC m=+0.487086202 container attach 792882856c835cbbd88b926cf11ea69cec73c41355fc1700ed36236dd5d6ae01 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.866+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.874+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.882+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.893+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.902+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.110 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.954+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:50.990+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.022+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.032+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.050+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.101+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.104+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:51.112 INFO:tasks.rados.rados.0.smithi118.stdout:2154: finishing write tid 1 to smithi11881747-50 2024-03-20T08:08:51.113 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 33997 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:08:51.117 INFO:tasks.rados.rados.0.smithi118.stdout:2156: expect (ObjNum 661 snap 216 seq_num 661) 2024-03-20T08:08:51.219 INFO:tasks.rados.rados.0.smithi118.stdout:2154: finishing write tid 2 to smithi11881747-50 2024-03-20T08:08:51.219 INFO:tasks.rados.rados.0.smithi118.stdout:2154: finishing write tid 3 to smithi11881747-50 2024-03-20T08:08:51.219 INFO:tasks.rados.rados.0.smithi118.stdout:2154: finishing write tid 4 to smithi11881747-50 2024-03-20T08:08:51.219 INFO:tasks.rados.rados.0.smithi118.stdout:2154: finishing write tid 5 to smithi11881747-50 2024-03-20T08:08:51.219 INFO:tasks.rados.rados.0.smithi118.stdout:2154: finishing write tid 6 to smithi11881747-50 2024-03-20T08:08:51.219 INFO:tasks.rados.rados.0.smithi118.stdout:2154: oid 50 updating version 0 to 16079 2024-03-20T08:08:51.219 INFO:tasks.rados.rados.0.smithi118.stdout:2154: oid 50 version 16079 is already newer than 16077 2024-03-20T08:08:51.219 INFO:tasks.rados.rados.0.smithi118.stdout:2154: oid 50 updating version 16079 to 16081 2024-03-20T08:08:51.220 INFO:tasks.rados.rados.0.smithi118.stdout:2154: oid 50 version 16081 is already newer than 16078 2024-03-20T08:08:51.220 INFO:tasks.rados.rados.0.smithi118.stdout:2154: oid 50 version 16081 is already newer than 16080 2024-03-20T08:08:51.220 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 16081 (ObjNum 672 snap 221 seq_num 672) dirty exists 2024-03-20T08:08:51.220 INFO:tasks.rados.rados.0.smithi118.stdout:2154: left oid 50 (ObjNum 672 snap 221 seq_num 672) 2024-03-20T08:08:51.364 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:51.108+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:51.364 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.130+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.141+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.150+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.161+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.185+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.195+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.215+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.225+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.249+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.259+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.302+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.334+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.345+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:51.109+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:51.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:51.088+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:51.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:51.114+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:51.108+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:51.108+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:51.109+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:08:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:51.109+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:51.110+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: osdmap e497: 8 total, 7 up, 8 in 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[95692]: pgmap v425: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 32 active+undersized, 6 active+undersized+degraded, 3 stale+active+clean, 10 active+clean+snaptrim_error, 68 active+clean; 637 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 19 MiB/s rd, 4.1 MiB/s wr, 12 op/s; 54/1281 objects degraded (4.215%) 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: osdmap e497: 8 total, 7 up, 8 in 2024-03-20T08:08:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:51 smithi118 ceph-mon[93121]: pgmap v425: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 32 active+undersized, 6 active+undersized+degraded, 3 stale+active+clean, 10 active+clean+snaptrim_error, 68 active+clean; 637 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 19 MiB/s rd, 4.1 MiB/s wr, 12 op/s; 54/1281 objects degraded (4.215%) 2024-03-20T08:08:51.621 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate[102935]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-20T08:08:51.621 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate[102935]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-20T08:08:51.621 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 bash[102920]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-20T08:08:51.621 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 bash[102920]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-6 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-20T08:08:51.621 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 bash[102920]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-20T08:08:51.621 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 bash[102920]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-20T08:08:51.621 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 bash[102920]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 bash[102920]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 bash[102920]: --> ceph-volume raw activate successful for osd ID: 6 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate[102935]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate[102935]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate[102935]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-6/block 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate[102935]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate[102935]: --> ceph-volume raw activate successful for osd ID: 6 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:51 smithi196 podman[102920]: 2024-03-20 08:08:51.458136629 +0000 UTC m=+0.958420436 container died 792882856c835cbbd88b926cf11ea69cec73c41355fc1700ed36236dd5d6ae01 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-20T08:08:51.622 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.386+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.395+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.417+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.443+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.456+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.491+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.507+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.530+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.537+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.576+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.584+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.599+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.621+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.630+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.645+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.669+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.684+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.691+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.716+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.750+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.758+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.789+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.796+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.804+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.819+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.845+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.854+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:51.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.091 INFO:tasks.rados.rados.0.smithi118.stdout:2154: done (3 left) 2024-03-20T08:08:52.091 INFO:tasks.rados.rados.0.smithi118.stdout:2155: done (2 left) 2024-03-20T08:08:52.091 INFO:tasks.rados.rados.0.smithi118.stdout:2156: done (1 left) 2024-03-20T08:08:52.091 INFO:tasks.rados.rados.0.smithi118.stdout:2157: done (0 left) 2024-03-20T08:08:52.091 INFO:tasks.rados.rados.0.smithi118.stdout:2158: read oid 48 snap -1 2024-03-20T08:08:52.091 INFO:tasks.rados.rados.0.smithi118.stdout:2158: expect (ObjNum 592 snap 193 seq_num 592) 2024-03-20T08:08:52.091 INFO:tasks.rados.rados.0.smithi118.stdout:2159: read oid 22 snap -1 2024-03-20T08:08:52.091 INFO:tasks.rados.rados.0.smithi118.stdout:2159: expect deleted 2024-03-20T08:08:52.092 INFO:tasks.rados.rados.0.smithi118.stdout:2160: delete oid 49 current snap is 222 2024-03-20T08:08:52.094 INFO:tasks.rados.rados.0.smithi118.stdout:2158: expect (ObjNum 592 snap 193 seq_num 592) 2024-03-20T08:08:52.104 INFO:tasks.rados.rados.0.smithi118.stdout:2158: done (2 left) 2024-03-20T08:08:52.104 INFO:tasks.rados.rados.0.smithi118.stdout:2159: done (1 left) 2024-03-20T08:08:52.104 INFO:tasks.rados.rados.0.smithi118.stdout:2160: done (0 left) 2024-03-20T08:08:52.104 INFO:tasks.rados.rados.0.smithi118.stdout:2161: read oid 11 snap -1 2024-03-20T08:08:52.104 INFO:tasks.rados.rados.0.smithi118.stdout:2161: expect deleted 2024-03-20T08:08:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:2162: snap_remove snap 182 2024-03-20T08:08:52.127 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:52.095+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: osdmap e498: 8 total, 7 up, 8 in 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:52.128 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.918+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.933+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.948+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.972+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:51.995+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.018+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.034+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.042+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.057+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.064+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.080+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.089+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.097+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.097+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.128+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.384 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:52 smithi196 podman[103081]: 2024-03-20 08:08:52.133626815 +0000 UTC m=+0.669735365 container remove 792882856c835cbbd88b926cf11ea69cec73c41355fc1700ed36236dd5d6ae01 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-activate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-20T08:08:52.384 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:52 smithi196 podman[103150]: 2024-03-20 08:08:52.26262324 +0000 UTC m=+0.020929611 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:08:52.385 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:52 smithi196 podman[103150]: 2024-03-20 08:08:52.376377707 +0000 UTC m=+0.134684065 container create dca658056148404fad6de448d9f93df055d145dfd4dc6408106b6d61e2f7145a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS) 2024-03-20T08:08:52.385 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:08:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:08:52.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.139+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.150+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.174+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.185+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.197+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.234+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.260+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.271+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.287+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.328+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.344+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.359+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.367+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.385+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:52.091+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:52.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:52.096+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:52.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:52.096+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:52.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:52.098+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: osdmap e498: 8 total, 7 up, 8 in 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: osdmap e498: 8 total, 7 up, 8 in 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:52.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:52.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:52 smithi196 podman[103150]: 2024-03-20 08:08:52.620947891 +0000 UTC m=+0.379254258 container init dca658056148404fad6de448d9f93df055d145dfd4dc6408106b6d61e2f7145a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-20T08:08:52.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:52 smithi196 podman[103150]: 2024-03-20 08:08:52.627086234 +0000 UTC m=+0.385392593 container start dca658056148404fad6de448d9f93df055d145dfd4dc6408106b6d61e2f7145a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-20T08:08:52.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.402+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.409+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.441+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.480+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.490+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.555+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.571+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.581+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.597+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.620+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.642 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.631+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.894 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-osd[103188]: -- 172.21.15.196:0/581588193 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5607b81d7680 con 0x5607b81a0c00 2024-03-20T08:08:52.894 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:52 smithi196 bash[103150]: dca658056148404fad6de448d9f93df055d145dfd4dc6408106b6d61e2f7145a 2024-03-20T08:08:52.895 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:52 smithi196 systemd[1]: Started Ceph osd.6 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:08:52.895 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.895 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.895 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.661+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.895 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.669+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.895 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.684+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.709+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.717+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.725+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.733+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.797+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.818+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.847+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:52.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.895+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.092 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:52.706+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:53.092 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:52.706+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:53.092 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:52.706+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:53.092 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:52.706+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:53.097 INFO:tasks.rados.rados.0.smithi118.stdout:2162: done (1 left) 2024-03-20T08:08:53.098 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 222 2024-03-20T08:08:53.098 INFO:tasks.rados.rados.0.smithi118.stdout:2163: seq_num 674 ranges {0=491520,491520=417792,909312=483328,1392640=303104} 2024-03-20T08:08:53.100 INFO:tasks.rados.rados.0.smithi118.stdout:2163: writing smithi11881747-46 from 0 to 491520 tid 1 2024-03-20T08:08:53.102 INFO:tasks.rados.rados.0.smithi118.stdout:2163: writing smithi11881747-46 from 491520 to 909312 tid 2 2024-03-20T08:08:53.105 INFO:tasks.rados.rados.0.smithi118.stdout:2163: writing smithi11881747-46 from 909312 to 1392640 tid 3 2024-03-20T08:08:53.107 INFO:tasks.rados.rados.0.smithi118.stdout:2163: writing smithi11881747-46 from 1392640 to 1695744 tid 4 2024-03-20T08:08:53.107 INFO:tasks.rados.rados.0.smithi118.stdout:2161: done (1 left) 2024-03-20T08:08:53.108 INFO:tasks.rados.rados.0.smithi118.stdout:2164: copy_from oid 50 from oid 1 current snap is 222 2024-03-20T08:08:53.108 INFO:tasks.rados.rados.0.smithi118.stdout:2165: snap_create 2024-03-20T08:08:53.112 INFO:tasks.rados.rados.0.smithi118.stdout:2163: finishing write tid 1 to smithi11881747-46 2024-03-20T08:08:53.112 INFO:tasks.rados.rados.0.smithi118.stdout:2163: finishing write tid 2 to smithi11881747-46 2024-03-20T08:08:53.114 INFO:tasks.rados.rados.0.smithi118.stdout:2163: finishing write tid 3 to smithi11881747-46 2024-03-20T08:08:53.114 INFO:tasks.rados.rados.0.smithi118.stdout:2163: finishing write tid 4 to smithi11881747-46 2024-03-20T08:08:53.114 INFO:tasks.rados.rados.0.smithi118.stdout:2163: finishing write tid 5 to smithi11881747-46 2024-03-20T08:08:53.117 INFO:tasks.rados.rados.0.smithi118.stdout:2163: finishing write tid 6 to smithi11881747-46 2024-03-20T08:08:53.117 INFO:tasks.rados.rados.0.smithi118.stdout:2163: oid 46 updating version 0 to 23067 2024-03-20T08:08:53.117 INFO:tasks.rados.rados.0.smithi118.stdout:2163: oid 46 updating version 23067 to 23069 2024-03-20T08:08:53.117 INFO:tasks.rados.rados.0.smithi118.stdout:2163: oid 46 version 23069 is already newer than 23068 2024-03-20T08:08:53.117 INFO:tasks.rados.rados.0.smithi118.stdout:2163: oid 46 version 23069 is already newer than 23065 2024-03-20T08:08:53.117 INFO:tasks.rados.rados.0.smithi118.stdout:2163: oid 46 version 23069 is already newer than 23066 2024-03-20T08:08:53.117 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 23069 (ObjNum 674 snap 222 seq_num 674) dirty exists 2024-03-20T08:08:53.117 INFO:tasks.rados.rados.0.smithi118.stdout:2163: left oid 46 (ObjNum 674 snap 222 seq_num 674) 2024-03-20T08:08:53.130 INFO:tasks.rados.rados.0.smithi118.stdout:2164: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:08:53.136 INFO:tasks.rados.rados.0.smithi118.stdout:2164: finishing copy_from to smithi11881747-50 2024-03-20T08:08:53.136 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 7117 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:08:53.153 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:53.108+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: osdmap e499: 8 total, 7 up, 8 in 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: pgmap v428: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 637 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.8 MiB/s wr, 11 op/s; 90/1293 objects degraded (6.961%) 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:53.154 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:53.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:53.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:53.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:53.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:53.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:53.155 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.155 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.917+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.155 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.928+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.155 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.939+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.155 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.964+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.986+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:52.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.007+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.018+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.040+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.051+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.091+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.156 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.111+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:53.157 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.132+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.157 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.157 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.415 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:08:53.167+0000 7fafcc9627c0 -1 Falling back to public interface 2024-03-20T08:08:53.415 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.163+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.415 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.415 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.189+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.415 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.416 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.210+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.416 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.219+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.416 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.228+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.416 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.239+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.416 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.416 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.416 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.267+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.416 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.287+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.308+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.329+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.355+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.366+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.378+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.417 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.414+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:53.116+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: osdmap e499: 8 total, 7 up, 8 in 2024-03-20T08:08:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:53.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: pgmap v428: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 637 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.8 MiB/s wr, 11 op/s; 90/1293 objects degraded (6.961%) 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:53.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:53.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:53.094+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:53.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:53.097+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:53.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:53.097+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:53.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:53.097+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:08:53.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:53.097+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:53.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:53.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:53.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: osdmap e499: 8 total, 7 up, 8 in 2024-03-20T08:08:53.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:53.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:53.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: pgmap v428: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 637 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 8.1 MiB/s rd, 2.8 MiB/s wr, 11 op/s; 90/1293 objects degraded (6.961%) 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:08:53.425 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:53.095+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:53.426 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:53.110+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:53.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.463+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.485+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.511+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.550+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.668 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.558+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.581+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.621+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.631+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.647+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.656+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.666+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:08:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:08:53] "GET /metrics HTTP/1.1" 200 36335 "" "Prometheus/2.43.0" 2024-03-20T08:08:53.929 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.929 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.733+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.749+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.757+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.773+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.816+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.826+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.836+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.846+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.855+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.864+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.903+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.915+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:53.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.099 INFO:tasks.rados.rados.0.smithi118.stdout:2163: done (2 left) 2024-03-20T08:08:54.100 INFO:tasks.rados.rados.0.smithi118.stdout:2164: done (1 left) 2024-03-20T08:08:54.100 INFO:tasks.rados.rados.0.smithi118.stdout:2165: done (0 left) 2024-03-20T08:08:54.100 INFO:tasks.rados.rados.0.smithi118.stdout:2166: copy_from oid 24 from oid 14 current snap is 223 2024-03-20T08:08:54.100 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 223 2024-03-20T08:08:54.100 INFO:tasks.rados.rados.0.smithi118.stdout:2167: seq_num 675 ranges {6045696=507904,6553600=704512,7258112=499712} 2024-03-20T08:08:54.137 INFO:tasks.rados.rados.0.smithi118.stdout:2167: writing smithi11881747-10 from 6045696 to 6553600 tid 1 2024-03-20T08:08:54.140 INFO:tasks.rados.rados.0.smithi118.stdout:2167: writing smithi11881747-10 from 6553600 to 7258112 tid 2 2024-03-20T08:08:54.143 INFO:tasks.rados.rados.0.smithi118.stdout:2167: writing smithi11881747-10 from 7258112 to 7757824 tid 3 2024-03-20T08:08:54.144 INFO:tasks.rados.rados.0.smithi118.stdout:2168: snap_remove snap 217 2024-03-20T08:08:54.187 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:54.106+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:54.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:54.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:54.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:54.188 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: osdmap e500: 8 total, 7 up, 8 in 2024-03-20T08:08:54.188 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:54.188 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:54.188 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:54.188 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:54.188 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:54.188 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:54.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.939+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.973+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:53.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.001+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.025+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.048+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.056+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.064+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.071+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.089+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.103+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.104+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.112+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.125+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.190 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.154+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.162+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.178+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.186+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:54.107+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:54.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:54.112+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:54.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:54.099+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:54.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:54.099+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:54.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:54.098+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:54.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:54.099+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:54.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:54.100+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:54.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:54.100+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: osdmap e500: 8 total, 7 up, 8 in 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: osdmap e500: 8 total, 7 up, 8 in 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:54.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:54.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.195+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.203+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.211+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.219+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.226+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.234+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.263+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.288+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.309+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.331+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.342+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.353+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.365+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.379+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.391+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.415+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.427+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.438+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.451+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.463+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.485+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.697 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.519+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.543+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.590+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.664+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.672+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.721+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.729+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.760+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.822+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.831+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.852+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.874+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.886+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.936+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:54.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:2168: done (2 left) 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:2169: read oid 36 snap -1 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:2169: expect (ObjNum 667 snap 217 seq_num 667) 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:2170: snap_create 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:2166: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:2166: finishing copy_from to smithi11881747-24 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 33997 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:2167: finishing write tid 1 to smithi11881747-10 2024-03-20T08:08:55.106 INFO:tasks.rados.rados.0.smithi118.stdout:2167: finishing write tid 2 to smithi11881747-10 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:2167: finishing write tid 3 to smithi11881747-10 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:2167: finishing write tid 4 to smithi11881747-10 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:2167: finishing write tid 5 to smithi11881747-10 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:2167: oid 10 updating version 0 to 7120 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:2167: oid 10 version 7120 is already newer than 7118 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:2167: oid 10 version 7120 is already newer than 7119 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:2167: oid 10 updating version 7120 to 7121 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 7121 (ObjNum 675 snap 223 seq_num 675) dirty exists 2024-03-20T08:08:55.107 INFO:tasks.rados.rados.0.smithi118.stdout:2167: left oid 10 (ObjNum 675 snap 223 seq_num 675) 2024-03-20T08:08:55.112 INFO:tasks.rados.rados.0.smithi118.stdout:2169: expect (ObjNum 667 snap 217 seq_num 667) 2024-03-20T08:08:55.197 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:55.113+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:55.197 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:55.197 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:55.197 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: osdmap e501: 8 total, 7 up, 8 in 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: pgmap v431: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 639 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 7 op/s; 90/1293 objects degraded (6.961%) 2024-03-20T08:08:55.198 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 90/1293 objects degraded (6.961%), 9 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:55.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.958+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.969+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:54.995+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.012+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.020+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.056+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.065+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.074+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.082+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.098+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.109+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.113+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.132+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.140+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.148+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.165+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.181+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.201 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:55.108+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:55.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:55.292+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:55.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:55.111+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: osdmap e501: 8 total, 7 up, 8 in 2024-03-20T08:08:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:55.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:55.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:55.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:55.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: pgmap v431: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 639 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 7 op/s; 90/1293 objects degraded (6.961%) 2024-03-20T08:08:55.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 90/1293 objects degraded (6.961%), 9 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:55.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:55.189+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:55.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:55.189+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:55.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:55.189+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:08:55.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:55.189+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:55.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:55.189+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: osdmap e501: 8 total, 7 up, 8 in 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: pgmap v431: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 1 active+clean+snaptrim, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 639 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 4.2 MiB/s rd, 1.0 MiB/s wr, 7 op/s; 90/1293 objects degraded (6.961%) 2024-03-20T08:08:55.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:55 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 90/1293 objects degraded (6.961%), 9 pgs degraded (PG_DEGRADED) 2024-03-20T08:08:55.454 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.206+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.454 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.454 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.221+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.454 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.454 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.237+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.253+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.284+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.292+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.301+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.310+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.355+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.368+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.455 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.456 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.456 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.404+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.456 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.456 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.427+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.456 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.439+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.456 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.713 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.713 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.498+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.516+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.547+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.599+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.622+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.716 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.716 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.645+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.716 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.716 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.669+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.716 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.716 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.717 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.717 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.726+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.737+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.748+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.775+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.787+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.800+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.811+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.830+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.850+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.870+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.879+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.891+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.913+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.933+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.944+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:55.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.105 INFO:tasks.rados.rados.0.smithi118.stdout:2166: done (3 left) 2024-03-20T08:08:56.105 INFO:tasks.rados.rados.0.smithi118.stdout:2167: done (2 left) 2024-03-20T08:08:56.105 INFO:tasks.rados.rados.0.smithi118.stdout:2169: done (1 left) 2024-03-20T08:08:56.105 INFO:tasks.rados.rados.0.smithi118.stdout:2170: done (0 left) 2024-03-20T08:08:56.105 INFO:tasks.rados.rados.0.smithi118.stdout:2171: snap_create 2024-03-20T08:08:56.223 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:56.109+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:56.223 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: osdmap e502: 8 total, 7 up, 8 in 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:56.224 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:56.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:55.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.020+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.042+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.064+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.074+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.095+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.106+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.106+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.140+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.152+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.163+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.223+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:56.106+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:56.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: osdmap e502: 8 total, 7 up, 8 in 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: osdmap e502: 8 total, 7 up, 8 in 2024-03-20T08:08:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:56.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.237+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.250+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.275+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.286+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.372+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.385+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.397+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.441+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.490+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.503+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.516+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.543+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.555+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.583+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.597+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.640+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.654+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.703+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.714+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.741+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:56.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:56.425+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:56.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:56.424+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:56.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:56.424+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:56.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:56.423+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:56.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:56.425+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:56.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:56.426+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:56.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:56.426+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:56.999 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:08:56.821+0000 7fafcc9627c0 -1 osd.6 492 log_to_monitors true 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.831+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.847+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.856+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.871+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.879+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.904+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.913+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.945+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.961+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.968+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.991+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:56.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.116 INFO:tasks.rados.rados.0.smithi118.stdout:2171: done (0 left) 2024-03-20T08:08:57.116 INFO:tasks.rados.rados.0.smithi118.stdout:2172: delete oid 9 current snap is 225 2024-03-20T08:08:57.122 INFO:tasks.rados.rados.0.smithi118.stdout:2172: done (0 left) 2024-03-20T08:08:57.122 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 225 2024-03-20T08:08:57.122 INFO:tasks.rados.rados.0.smithi118.stdout:2173: seq_num 677 ranges {4415488=540672,4956160=516096,5472256=655360,6127616=32768} 2024-03-20T08:08:57.148 INFO:tasks.rados.rados.0.smithi118.stdout:2173: writing smithi11881747-35 from 4415488 to 4956160 tid 1 2024-03-20T08:08:57.151 INFO:tasks.rados.rados.0.smithi118.stdout:2173: writing smithi11881747-35 from 4956160 to 5472256 tid 2 2024-03-20T08:08:57.155 INFO:tasks.rados.rados.0.smithi118.stdout:2173: writing smithi11881747-35 from 5472256 to 6127616 tid 3 2024-03-20T08:08:57.156 INFO:tasks.rados.rados.0.smithi118.stdout:2173: writing smithi11881747-35 from 6127616 to 6160384 tid 4 2024-03-20T08:08:57.156 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 225 2024-03-20T08:08:57.156 INFO:tasks.rados.rados.0.smithi118.stdout:2174: seq_num 678 ranges {3022848=565248,3588096=802816,4390912=401408} 2024-03-20T08:08:57.175 INFO:tasks.rados.rados.0.smithi118.stdout:2174: writing smithi11881747-6 from 3022848 to 3588096 tid 1 2024-03-20T08:08:57.179 INFO:tasks.rados.rados.0.smithi118.stdout:2174: writing smithi11881747-6 from 3588096 to 4390912 tid 2 2024-03-20T08:08:57.181 INFO:tasks.rados.rados.0.smithi118.stdout:2174: writing smithi11881747-6 from 4390912 to 4792320 tid 3 2024-03-20T08:08:57.181 INFO:tasks.rados.rados.0.smithi118.stdout:2175: delete oid 37 current snap is 225 2024-03-20T08:08:57.184 INFO:tasks.rados.rados.0.smithi118.stdout:2173: finishing write tid 1 to smithi11881747-35 2024-03-20T08:08:57.189 INFO:tasks.rados.rados.0.smithi118.stdout:2175: done (2 left) 2024-03-20T08:08:57.189 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 225 2024-03-20T08:08:57.189 INFO:tasks.rados.rados.0.smithi118.stdout:2176: seq_num 680 ranges {0=598016,598016=573440,1171456=434176,1605632=196608} 2024-03-20T08:08:57.193 INFO:tasks.rados.rados.0.smithi118.stdout:2176: writing smithi11881747-2 from 0 to 598016 tid 1 2024-03-20T08:08:57.196 INFO:tasks.rados.rados.0.smithi118.stdout:2176: writing smithi11881747-2 from 598016 to 1171456 tid 2 2024-03-20T08:08:57.198 INFO:tasks.rados.rados.0.smithi118.stdout:2176: writing smithi11881747-2 from 1171456 to 1605632 tid 3 2024-03-20T08:08:57.200 INFO:tasks.rados.rados.0.smithi118.stdout:2176: writing smithi11881747-2 from 1605632 to 1802240 tid 4 2024-03-20T08:08:57.200 INFO:tasks.rados.rados.0.smithi118.stdout:2177: copy_from oid 43 from oid 28 current snap is 225 2024-03-20T08:08:57.200 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 225 2024-03-20T08:08:57.200 INFO:tasks.rados.rados.0.smithi118.stdout:2178: seq_num 681 ranges {1695744=614400,2310144=458752,2768896=638976,3407872=106496} 2024-03-20T08:08:57.212 INFO:tasks.rados.rados.0.smithi118.stdout:2178: writing smithi11881747-46 from 1695744 to 2310144 tid 1 2024-03-20T08:08:57.215 INFO:tasks.rados.rados.0.smithi118.stdout:2178: writing smithi11881747-46 from 2310144 to 2768896 tid 2 2024-03-20T08:08:57.218 INFO:tasks.rados.rados.0.smithi118.stdout:2178: writing smithi11881747-46 from 2768896 to 3407872 tid 3 2024-03-20T08:08:57.220 INFO:tasks.rados.rados.0.smithi118.stdout:2178: writing smithi11881747-46 from 3407872 to 3514368 tid 4 2024-03-20T08:08:57.220 INFO:tasks.rados.rados.0.smithi118.stdout:2179: copy_from oid 40 from oid 16 current snap is 225 2024-03-20T08:08:57.220 INFO:tasks.rados.rados.0.smithi118.stdout:2173: finishing write tid 2 to smithi11881747-35 2024-03-20T08:08:57.220 INFO:tasks.rados.rados.0.smithi118.stdout:2173: finishing write tid 3 to smithi11881747-35 2024-03-20T08:08:57.220 INFO:tasks.rados.rados.0.smithi118.stdout:2173: finishing write tid 4 to smithi11881747-35 2024-03-20T08:08:57.220 INFO:tasks.rados.rados.0.smithi118.stdout:2180: delete oid 42 current snap is 225 2024-03-20T08:08:57.220 INFO:tasks.rados.rados.0.smithi118.stdout:2173: finishing write tid 5 to smithi11881747-35 2024-03-20T08:08:57.220 INFO:tasks.rados.rados.0.smithi118.stdout:2173: finishing write tid 6 to smithi11881747-35 2024-03-20T08:08:57.221 INFO:tasks.rados.rados.0.smithi118.stdout:2173: oid 35 updating version 0 to 23070 2024-03-20T08:08:57.221 INFO:tasks.rados.rados.0.smithi118.stdout:2173: oid 35 updating version 23070 to 23073 2024-03-20T08:08:57.221 INFO:tasks.rados.rados.0.smithi118.stdout:2173: oid 35 version 23073 is already newer than 23071 2024-03-20T08:08:57.221 INFO:tasks.rados.rados.0.smithi118.stdout:2173: oid 35 updating version 23073 to 23074 2024-03-20T08:08:57.221 INFO:tasks.rados.rados.0.smithi118.stdout:2173: oid 35 version 23074 is already newer than 23072 2024-03-20T08:08:57.221 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 23074 (ObjNum 677 snap 225 seq_num 677) dirty exists 2024-03-20T08:08:57.221 INFO:tasks.rados.rados.0.smithi118.stdout:2173: left oid 35 (ObjNum 677 snap 225 seq_num 677) 2024-03-20T08:08:57.221 INFO:tasks.rados.rados.0.smithi118.stdout:2177: finishing copy_from racing read to smithi11881747-43 2024-03-20T08:08:57.232 INFO:tasks.rados.rados.0.smithi118.stdout:2174: finishing write tid 1 to smithi11881747-6 2024-03-20T08:08:57.232 INFO:tasks.rados.rados.0.smithi118.stdout:2174: finishing write tid 2 to smithi11881747-6 2024-03-20T08:08:57.232 INFO:tasks.rados.rados.0.smithi118.stdout:2174: finishing write tid 3 to smithi11881747-6 2024-03-20T08:08:57.232 INFO:tasks.rados.rados.0.smithi118.stdout:2174: finishing write tid 4 to smithi11881747-6 2024-03-20T08:08:57.236 INFO:tasks.rados.rados.0.smithi118.stdout:2174: finishing write tid 5 to smithi11881747-6 2024-03-20T08:08:57.236 INFO:tasks.rados.rados.0.smithi118.stdout:2174: oid 6 updating version 0 to 23078 2024-03-20T08:08:57.236 INFO:tasks.rados.rados.0.smithi118.stdout:2174: oid 6 version 23078 is already newer than 23075 2024-03-20T08:08:57.237 INFO:tasks.rados.rados.0.smithi118.stdout:2174: oid 6 version 23078 is already newer than 23077 2024-03-20T08:08:57.237 INFO:tasks.rados.rados.0.smithi118.stdout:2174: oid 6 version 23078 is already newer than 23076 2024-03-20T08:08:57.237 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 23078 (ObjNum 678 snap 225 seq_num 678) dirty exists 2024-03-20T08:08:57.237 INFO:tasks.rados.rados.0.smithi118.stdout:2174: left oid 6 (ObjNum 678 snap 225 seq_num 678) 2024-03-20T08:08:57.255 INFO:tasks.rados.rados.0.smithi118.stdout:2176: finishing write tid 1 to smithi11881747-2 2024-03-20T08:08:57.255 INFO:tasks.rados.rados.0.smithi118.stdout:2176: finishing write tid 2 to smithi11881747-2 2024-03-20T08:08:57.255 INFO:tasks.rados.rados.0.smithi118.stdout:2176: finishing write tid 3 to smithi11881747-2 2024-03-20T08:08:57.255 INFO:tasks.rados.rados.0.smithi118.stdout:2176: finishing write tid 4 to smithi11881747-2 2024-03-20T08:08:57.255 INFO:tasks.rados.rados.0.smithi118.stdout:2176: finishing write tid 5 to smithi11881747-2 2024-03-20T08:08:57.255 INFO:tasks.rados.rados.0.smithi118.stdout:2178: finishing write tid 1 to smithi11881747-46 2024-03-20T08:08:57.256 INFO:tasks.rados.rados.0.smithi118.stdout:2176: finishing write tid 6 to smithi11881747-2 2024-03-20T08:08:57.256 INFO:tasks.rados.rados.0.smithi118.stdout:2176: oid 2 updating version 0 to 23080 2024-03-20T08:08:57.256 INFO:tasks.rados.rados.0.smithi118.stdout:2176: oid 2 updating version 23080 to 23083 2024-03-20T08:08:57.256 INFO:tasks.rados.rados.0.smithi118.stdout:2176: oid 2 version 23083 is already newer than 23081 2024-03-20T08:08:57.256 INFO:tasks.rados.rados.0.smithi118.stdout:2176: oid 2 version 23083 is already newer than 23082 2024-03-20T08:08:57.256 INFO:tasks.rados.rados.0.smithi118.stdout:2176: oid 2 version 23083 is already newer than 23079 2024-03-20T08:08:57.256 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 23083 (ObjNum 680 snap 225 seq_num 680) dirty exists 2024-03-20T08:08:57.256 INFO:tasks.rados.rados.0.smithi118.stdout:2176: left oid 2 (ObjNum 680 snap 225 seq_num 680) 2024-03-20T08:08:57.261 INFO:tasks.rados.rados.0.smithi118.stdout:2179: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:08:57.264 INFO:tasks.rados.rados.0.smithi118.stdout:2179: finishing copy_from to smithi11881747-40 2024-03-20T08:08:57.265 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 1346 (ObjNum 571 snap 187 seq_num 571) dirty exists 2024-03-20T08:08:57.275 INFO:tasks.rados.rados.0.smithi118.stdout:2178: finishing write tid 2 to smithi11881747-46 2024-03-20T08:08:57.275 INFO:tasks.rados.rados.0.smithi118.stdout:2178: finishing write tid 3 to smithi11881747-46 2024-03-20T08:08:57.275 INFO:tasks.rados.rados.0.smithi118.stdout:2178: finishing write tid 4 to smithi11881747-46 2024-03-20T08:08:57.275 INFO:tasks.rados.rados.0.smithi118.stdout:2178: finishing write tid 5 to smithi11881747-46 2024-03-20T08:08:57.276 INFO:tasks.rados.rados.0.smithi118.stdout:2178: finishing write tid 6 to smithi11881747-46 2024-03-20T08:08:57.276 INFO:tasks.rados.rados.0.smithi118.stdout:2178: oid 46 updating version 0 to 23088 2024-03-20T08:08:57.276 INFO:tasks.rados.rados.0.smithi118.stdout:2178: oid 46 version 23088 is already newer than 23084 2024-03-20T08:08:57.277 INFO:tasks.rados.rados.0.smithi118.stdout:2178: oid 46 version 23088 is already newer than 23087 2024-03-20T08:08:57.277 INFO:tasks.rados.rados.0.smithi118.stdout:2178: oid 46 version 23088 is already newer than 23085 2024-03-20T08:08:57.277 INFO:tasks.rados.rados.0.smithi118.stdout:2178: oid 46 version 23088 is already newer than 23086 2024-03-20T08:08:57.277 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 23088 (ObjNum 681 snap 225 seq_num 681) dirty exists 2024-03-20T08:08:57.277 INFO:tasks.rados.rados.0.smithi118.stdout:2178: left oid 46 (ObjNum 681 snap 225 seq_num 681) 2024-03-20T08:08:57.310 INFO:tasks.rados.rados.0.smithi118.stdout:2177: finishing copy_from to smithi11881747-43 2024-03-20T08:08:57.310 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 33906 (ObjNum 644 snap 211 seq_num 644) dirty exists 2024-03-20T08:08:57.332 INFO:tasks.rados.rados.0.smithi118.stdout:2173: done (6 left) 2024-03-20T08:08:57.332 INFO:tasks.rados.rados.0.smithi118.stdout:2174: done (5 left) 2024-03-20T08:08:57.332 INFO:tasks.rados.rados.0.smithi118.stdout:2176: done (4 left) 2024-03-20T08:08:57.332 INFO:tasks.rados.rados.0.smithi118.stdout:2177: done (3 left) 2024-03-20T08:08:57.332 INFO:tasks.rados.rados.0.smithi118.stdout:2178: done (2 left) 2024-03-20T08:08:57.332 INFO:tasks.rados.rados.0.smithi118.stdout:2179: done (1 left) 2024-03-20T08:08:57.333 INFO:tasks.rados.rados.0.smithi118.stdout:2180: done (0 left) 2024-03-20T08:08:57.333 INFO:tasks.rados.rados.0.smithi118.stdout:2181: snap_remove snap 201 2024-03-20T08:08:57.335 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:57.138+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:57.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:57.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: osdmap e503: 8 total, 7 up, 8 in 2024-03-20T08:08:57.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:57.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:57.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:57.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: pgmap v434: 121 pgs: 1 active+clean+snaptrim, 2 active+undersized+degraded+snaptrim_error, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 646 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 9 op/s; 91/1308 objects degraded (6.957%) 2024-03-20T08:08:57.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:57.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:57.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:57.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:57.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:57.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: from='osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T08:08:57.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-mon[87080]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T08:08:57.336 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.007+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.336 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.015+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.031+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.038+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.123+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:57.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.131+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.184+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:57.120+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:57.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:57.117+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:57.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:57.122+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:57.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: osdmap e503: 8 total, 7 up, 8 in 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: pgmap v434: 121 pgs: 1 active+clean+snaptrim, 2 active+undersized+degraded+snaptrim_error, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 646 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 9 op/s; 91/1308 objects degraded (6.957%) 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: from='osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T08:08:57.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[95692]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T08:08:57.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:57.189+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:57.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:57.189+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:57.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:57.190+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:08:57.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:57.190+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:57.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:57.190+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:57.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:57.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: osdmap e503: 8 total, 7 up, 8 in 2024-03-20T08:08:57.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:57.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:57.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:57.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: pgmap v434: 121 pgs: 1 active+clean+snaptrim, 2 active+undersized+degraded+snaptrim_error, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 646 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 5.6 MiB/s rd, 2.2 MiB/s wr, 9 op/s; 91/1308 objects degraded (6.957%) 2024-03-20T08:08:57.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:57.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:57.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:57.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:57.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:57.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: from='osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T08:08:57.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:57 smithi118 ceph-mon[93121]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.427+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.492+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.504+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.538+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.552+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.579+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.845 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:08:57.590+0000 7fafb9db2700 -1 osd.6 492 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T08:08:57.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.592+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.606+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.658+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.689+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.696+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.724+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.741+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.848 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.749+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.848 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.757+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.848 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.848 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.848 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.848 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.794+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.848 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.802+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.848 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.818+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.826+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.834+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:57.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.845+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.897+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.905+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.913+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.929+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.950+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.957+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.973+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.980+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:57.996+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.044+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.055+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.100+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.123 INFO:tasks.rados.rados.0.smithi118.stdout:2181: done (0 left) 2024-03-20T08:08:58.123 INFO:tasks.rados.rados.0.smithi118.stdout:2182: snap_remove snap 172 2024-03-20T08:08:58.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:58.144+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:58.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:58.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-20T08:08:58.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: osdmap e504: 8 total, 7 up, 8 in 2024-03-20T08:08:58.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: from='osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:58.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:58.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:58.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:58.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:58.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:58.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:58.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:58.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:58.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:58.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:58.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-mon[87080]: from='osd.6 ' entity='osd.6' 2024-03-20T08:08:58.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.146+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.146+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.163+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.175+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.198+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.245+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.258+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.270+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.284+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.365+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:58.127+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:58.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:58.130+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:58.130+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:58.131+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:58.131+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:58.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:58.132+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:58.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:58.122+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:08:58.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:58.122+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: osdmap e504: 8 total, 7 up, 8 in 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: from='osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:58.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[95692]: from='osd.6 ' entity='osd.6' 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: osdmap e504: 8 total, 7 up, 8 in 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: from='osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:58.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:58.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:58.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:58.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:58.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:58.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:58.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:58.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:58.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:58 smithi118 ceph-mon[93121]: from='osd.6 ' entity='osd.6' 2024-03-20T08:08:58.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.390+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.427+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.441+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.455+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.509+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.534+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.548+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.587+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.601+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.623+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.635+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.711+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.877 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.724+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.748+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.758+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.821+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.834+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.846+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:58.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.877+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.125 INFO:tasks.rados.rados.0.smithi118.stdout:2182: done (0 left) 2024-03-20T08:08:59.125 INFO:tasks.rados.rados.0.smithi118.stdout:2183: copy_from oid 11 from oid 40 current snap is 225 2024-03-20T08:08:59.125 INFO:tasks.rados.rados.0.smithi118.stdout:2184: read oid 22 snap -1 2024-03-20T08:08:59.125 INFO:tasks.rados.rados.0.smithi118.stdout:2184: expect deleted 2024-03-20T08:08:59.125 INFO:tasks.rados.rados.0.smithi118.stdout:2185: setattr oid 29 current snap is 225 2024-03-20T08:08:59.126 INFO:tasks.rados.rados.0.smithi118.stdout:2186: snap_remove snap 181 2024-03-20T08:08:59.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.889+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.900+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.925+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.935+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.957+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:58.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.057+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.082+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.092+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.103+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.116+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.136 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:08:59.136 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496] boot 2024-03-20T08:08:59.136 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: osdmap e505: 8 total, 8 up, 8 in 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: pgmap v437: 121 pgs: 1 active+clean+snaptrim, 2 active+undersized+degraded+snaptrim_error, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 653 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 7 op/s; 91/1314 objects degraded (6.925%) 2024-03-20T08:08:59.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:59.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:59.132+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:59.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.130+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:59.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.202+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.218+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.234+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.250+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.267+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.284+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.325+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.374+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:08:59.129+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:59.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:08:59.124+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:59.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:59.122+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:59.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:08:59.299+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:08:59.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:59.132+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:59.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:59.132+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:59.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:59.132+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:08:59.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:59.132+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:59.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:08:59.134+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:59.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:08:59.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496] boot 2024-03-20T08:08:59.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: osdmap e505: 8 total, 8 up, 8 in 2024-03-20T08:08:59.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:59.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:08:59.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:59.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:59.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:59.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:59.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:59.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:59.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:59.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:59.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: pgmap v437: 121 pgs: 1 active+clean+snaptrim, 2 active+undersized+degraded+snaptrim_error, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 653 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 7 op/s; 91/1314 objects degraded (6.925%) 2024-03-20T08:08:59.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: osd.6 [v2:172.21.15.196:6816/812001496,v1:172.21.15.196:6817/812001496] boot 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: osdmap e505: 8 total, 8 up, 8 in 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:08:59.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: pgmap v437: 121 pgs: 1 active+clean+snaptrim, 2 active+undersized+degraded+snaptrim_error, 34 active+undersized, 7 active+undersized+degraded, 9 active+clean+snaptrim_error, 68 active+clean; 653 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 7 op/s; 91/1314 objects degraded (6.925%) 2024-03-20T08:08:59.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:08:59.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:08:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:08:59.651 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:08:59.439+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:08:59.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.443+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.472+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.533+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.547+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.651+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.694+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.708+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.763+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.808+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.836+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.864+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:08:59.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.130 INFO:tasks.rados.rados.0.smithi118.stdout:2186: done (3 left) 2024-03-20T08:09:00.130 INFO:tasks.rados.rados.0.smithi118.stdout:2187: read oid 42 snap 204 2024-03-20T08:09:00.130 INFO:tasks.rados.rados.0.smithi118.stdout:2187: expect (ObjNum 612 snap 199 seq_num 612) 2024-03-20T08:09:00.130 INFO:tasks.rados.rados.0.smithi118.stdout:2188: rmattr oid 3 current snap is 225 2024-03-20T08:09:00.130 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33921 (ObjNum 659 snap 215 seq_num 659) dirty exists 2024-03-20T08:09:00.130 INFO:tasks.rados.rados.0.smithi118.stdout:2183: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:09:00.130 INFO:tasks.rados.rados.0.smithi118.stdout:2183: finishing copy_from to smithi11881747-11 2024-03-20T08:09:00.130 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 1346 (ObjNum 571 snap 187 seq_num 571) dirty exists 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:2183: done (4 left) 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:2184: done (3 left) 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:2185: done (2 left) 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:2188: done (1 left) 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:2189: read oid 22 snap -1 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:2189: expect deleted 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:2190: setattr oid 36 current snap is 225 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 225 2024-03-20T08:09:00.131 INFO:tasks.rados.rados.0.smithi118.stdout:2191: seq_num 683 ranges {319488=647168,966656=630784,1597440=573440} 2024-03-20T08:09:00.137 INFO:tasks.rados.rados.0.smithi118.stdout:2191: writing smithi11881747-48 from 319488 to 966656 tid 1 2024-03-20T08:09:00.141 INFO:tasks.rados.rados.0.smithi118.stdout:2191: writing smithi11881747-48 from 966656 to 1597440 tid 2 2024-03-20T08:09:00.145 INFO:tasks.rados.rados.0.smithi118.stdout:2191: writing smithi11881747-48 from 1597440 to 2170880 tid 3 2024-03-20T08:09:00.145 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 225 2024-03-20T08:09:00.145 INFO:tasks.rados.rados.0.smithi118.stdout:2187: expect (ObjNum 612 snap 199 seq_num 612) 2024-03-20T08:09:00.156 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:09:00.143+0000 7fafa758d700 -1 osd.6 pg_epoch: 507 pg[5.9s1( v 507'1085 (0'0,507'1085] local-lis/les=505/506 n=23 ec=73/73 lis/c=505/505 les/c/f=506/506/0 sis=505) [7,6,1]p7(0) r=1 lpr=505 luod=0'0 crt=507'1083 lcod 507'1083 mlcod 507'1083 active mbc={} ps=109] update_snap_map remove_oid 5:9d5f254b:::smithi11881747-6:f1 failed with -2 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: osdmap e506: 8 total, 8 up, 8 in 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:00.157 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:00.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:00.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:00.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:00.158 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 91/1314 objects degraded (6.925%), 9 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:00.158 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:00.141+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:00.158 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:00.141+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.935+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:08:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:08:59.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.048+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.089+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.115+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.133+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.143+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:00.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.143+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:00.160 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.166 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 33911 (ObjNum 667 snap 217 seq_num 667) dirty exists 2024-03-20T08:09:00.166 INFO:tasks.rados.rados.0.smithi118.stdout:2191: finishing write tid 1 to smithi11881747-48 2024-03-20T08:09:00.166 INFO:tasks.rados.rados.0.smithi118.stdout:2191: finishing write tid 2 to smithi11881747-48 2024-03-20T08:09:00.166 INFO:tasks.rados.rados.0.smithi118.stdout:2191: finishing write tid 3 to smithi11881747-48 2024-03-20T08:09:00.166 INFO:tasks.rados.rados.0.smithi118.stdout:2191: finishing write tid 4 to smithi11881747-48 2024-03-20T08:09:00.166 INFO:tasks.rados.rados.0.smithi118.stdout:2192: seq_num 684 ranges {3006464=663552,3670016=507904,4177920=409600,4587520=286720} 2024-03-20T08:09:00.186 INFO:tasks.rados.rados.0.smithi118.stdout:2192: writing smithi11881747-43 from 3006464 to 3670016 tid 1 2024-03-20T08:09:00.189 INFO:tasks.rados.rados.0.smithi118.stdout:2192: writing smithi11881747-43 from 3670016 to 4177920 tid 2 2024-03-20T08:09:00.192 INFO:tasks.rados.rados.0.smithi118.stdout:2192: writing smithi11881747-43 from 4177920 to 4587520 tid 3 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2192: writing smithi11881747-43 from 4587520 to 4874240 tid 4 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2187: done (4 left) 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2189: done (3 left) 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2190: done (2 left) 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 225 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2191: finishing write tid 5 to smithi11881747-48 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2191: oid 48 updating version 0 to 7091 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2191: oid 48 version 7091 is already newer than 7090 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2191: oid 48 version 7091 is already newer than 7089 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:2191: oid 48 version 7091 is already newer than 7088 2024-03-20T08:09:00.195 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 7091 (ObjNum 683 snap 225 seq_num 683) dirty exists 2024-03-20T08:09:00.196 INFO:tasks.rados.rados.0.smithi118.stdout:2191: left oid 48 (ObjNum 683 snap 225 seq_num 683) 2024-03-20T08:09:00.196 INFO:tasks.rados.rados.0.smithi118.stdout:2193: seq_num 685 ranges {1802240=679936,2482176=442368,2924544=622592,3547136=139264} 2024-03-20T08:09:00.208 INFO:tasks.rados.rados.0.smithi118.stdout:2193: writing smithi11881747-2 from 1802240 to 2482176 tid 1 2024-03-20T08:09:00.211 INFO:tasks.rados.rados.0.smithi118.stdout:2193: writing smithi11881747-2 from 2482176 to 2924544 tid 2 2024-03-20T08:09:00.214 INFO:tasks.rados.rados.0.smithi118.stdout:2193: writing smithi11881747-2 from 2924544 to 3547136 tid 3 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2193: writing smithi11881747-2 from 3547136 to 3686400 tid 4 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2191: done (2 left) 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2194: delete oid 33 current snap is 225 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2192: finishing write tid 1 to smithi11881747-43 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2192: finishing write tid 2 to smithi11881747-43 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2192: finishing write tid 3 to smithi11881747-43 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2192: finishing write tid 4 to smithi11881747-43 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2192: finishing write tid 5 to smithi11881747-43 2024-03-20T08:09:00.216 INFO:tasks.rados.rados.0.smithi118.stdout:2192: finishing write tid 6 to smithi11881747-43 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:2192: oid 43 updating version 0 to 33907 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:2192: oid 43 updating version 33907 to 33908 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:2192: oid 43 updating version 33908 to 33910 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:2192: oid 43 updating version 33910 to 33911 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:2192: oid 43 version 33911 is already newer than 33909 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 33911 (ObjNum 684 snap 225 seq_num 684) dirty exists 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:2192: left oid 43 (ObjNum 684 snap 225 seq_num 684) 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:2193: finishing write tid 1 to smithi11881747-2 2024-03-20T08:09:00.217 INFO:tasks.rados.rados.0.smithi118.stdout:2193: finishing write tid 2 to smithi11881747-2 2024-03-20T08:09:00.221 INFO:tasks.rados.rados.0.smithi118.stdout:2193: finishing write tid 3 to smithi11881747-2 2024-03-20T08:09:00.221 INFO:tasks.rados.rados.0.smithi118.stdout:2193: finishing write tid 4 to smithi11881747-2 2024-03-20T08:09:00.222 INFO:tasks.rados.rados.0.smithi118.stdout:2193: finishing write tid 5 to smithi11881747-2 2024-03-20T08:09:00.225 INFO:tasks.rados.rados.0.smithi118.stdout:2193: finishing write tid 6 to smithi11881747-2 2024-03-20T08:09:00.225 INFO:tasks.rados.rados.0.smithi118.stdout:2193: oid 2 updating version 0 to 23091 2024-03-20T08:09:00.225 INFO:tasks.rados.rados.0.smithi118.stdout:2193: oid 2 updating version 23091 to 23092 2024-03-20T08:09:00.225 INFO:tasks.rados.rados.0.smithi118.stdout:2193: oid 2 version 23092 is already newer than 23090 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:2193: oid 2 version 23092 is already newer than 23089 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:2193: oid 2 updating version 23092 to 23093 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 23093 (ObjNum 685 snap 225 seq_num 685) dirty exists 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:2193: left oid 2 (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:2192: done (2 left) 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:2193: done (1 left) 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:2194: done (0 left) 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:2195: copy_from oid 17 from oid 37 current snap is 225 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 225 2024-03-20T08:09:00.226 INFO:tasks.rados.rados.0.smithi118.stdout:2196: seq_num 687 ranges {655360=712704,1368064=614400,1982464=589824} 2024-03-20T08:09:00.233 INFO:tasks.rados.rados.0.smithi118.stdout:2196: writing smithi11881747-24 from 655360 to 1368064 tid 1 2024-03-20T08:09:00.237 INFO:tasks.rados.rados.0.smithi118.stdout:2196: writing smithi11881747-24 from 1368064 to 1982464 tid 2 2024-03-20T08:09:00.241 INFO:tasks.rados.rados.0.smithi118.stdout:2196: writing smithi11881747-24 from 1982464 to 2572288 tid 3 2024-03-20T08:09:00.241 INFO:tasks.rados.rados.0.smithi118.stdout:2197: delete oid 1 current snap is 225 2024-03-20T08:09:00.241 INFO:tasks.rados.rados.0.smithi118.stdout:2195: finishing copy_from to smithi11881747-17 2024-03-20T08:09:00.242 INFO:tasks.rados.rados.0.smithi118.stdout:2195: got expected ENOENT (src dne) 2024-03-20T08:09:00.242 INFO:tasks.rados.rados.0.smithi118.stdout:2195: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:09:00.246 INFO:tasks.rados.rados.0.smithi118.stdout:2195: done (2 left) 2024-03-20T08:09:00.246 INFO:tasks.rados.rados.0.smithi118.stdout:2197: done (1 left) 2024-03-20T08:09:00.246 INFO:tasks.rados.rados.0.smithi118.stdout:2198: rmattr oid 27 current snap is 225 2024-03-20T08:09:00.246 INFO:tasks.rados.rados.0.smithi118.stdout:2198: done (1 left) 2024-03-20T08:09:00.246 INFO:tasks.rados.rados.0.smithi118.stdout:2199: snap_create 2024-03-20T08:09:00.258 INFO:tasks.rados.rados.0.smithi118.stdout:2196: finishing write tid 1 to smithi11881747-24 2024-03-20T08:09:00.259 INFO:tasks.rados.rados.0.smithi118.stdout:2196: finishing write tid 2 to smithi11881747-24 2024-03-20T08:09:00.259 INFO:tasks.rados.rados.0.smithi118.stdout:2196: finishing write tid 3 to smithi11881747-24 2024-03-20T08:09:00.259 INFO:tasks.rados.rados.0.smithi118.stdout:2196: finishing write tid 4 to smithi11881747-24 2024-03-20T08:09:00.270 INFO:tasks.rados.rados.0.smithi118.stdout:2196: finishing write tid 5 to smithi11881747-24 2024-03-20T08:09:00.271 INFO:tasks.rados.rados.0.smithi118.stdout:2196: oid 24 updating version 0 to 33999 2024-03-20T08:09:00.271 INFO:tasks.rados.rados.0.smithi118.stdout:2196: oid 24 updating version 33999 to 34000 2024-03-20T08:09:00.271 INFO:tasks.rados.rados.0.smithi118.stdout:2196: oid 24 updating version 34000 to 34001 2024-03-20T08:09:00.271 INFO:tasks.rados.rados.0.smithi118.stdout:2196: oid 24 version 34001 is already newer than 33998 2024-03-20T08:09:00.271 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 34001 (ObjNum 687 snap 225 seq_num 687) dirty exists 2024-03-20T08:09:00.271 INFO:tasks.rados.rados.0.smithi118.stdout:2196: left oid 24 (ObjNum 687 snap 225 seq_num 687) 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.274+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.304+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.419 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.334+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.348+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.378+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.392+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.419+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:00.136+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:00.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:00.142+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:00.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:00.142+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:00.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:00.134+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:00.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:00.131+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:00.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:00.131+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:00.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:00.136+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:00.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:00.137+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:00.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:00.137+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:00.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:00.137+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: osdmap e506: 8 total, 8 up, 8 in 2024-03-20T08:09:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:00.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 91/1314 objects degraded (6.925%), 9 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:00.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: osdmap e506: 8 total, 8 up, 8 in 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:00.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:00.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:00.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:00.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:00 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 91/1314 objects degraded (6.925%), 9 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.433+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.447+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.460+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.473+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.514+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.541+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.568+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.596+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.640+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.654+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.669+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.754+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.780+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.793+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.822+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.850+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.915+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:00.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.134 INFO:tasks.rados.rados.0.smithi118.stdout:2196: done (1 left) 2024-03-20T08:09:01.134 INFO:tasks.rados.rados.0.smithi118.stdout:2199: done (0 left) 2024-03-20T08:09:01.134 INFO:tasks.rados.rados.0.smithi118.stdout:2200: rollback oid 36 current snap is 226 2024-03-20T08:09:01.134 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 36 to 224 2024-03-20T08:09:01.134 INFO:tasks.rados.rados.0.smithi118.stdout:2201: delete oid 18 current snap is 226 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:2201: done (1 left) 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:2202: read oid 49 snap -1 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:2202: expect deleted 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:2203: copy_from oid 32 from oid 1 current snap is 226 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:2204: rollback oid 43 current snap is 226 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 43 to 221 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:2205: rollback oid 41 current snap is 226 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 41 to 206 2024-03-20T08:09:01.143 INFO:tasks.rados.rados.0.smithi118.stdout:2206: snap_create 2024-03-20T08:09:01.147 INFO:tasks.rados.rados.0.smithi118.stdout:2200: finishing rollback tid 0 to smithi11881747-36 2024-03-20T08:09:01.147 INFO:tasks.rados.rados.0.smithi118.stdout:2200: finishing rollback tid 1 to smithi11881747-36 2024-03-20T08:09:01.147 INFO:tasks.rados.rados.0.smithi118.stdout:2203: finishing copy_from to smithi11881747-32 2024-03-20T08:09:01.147 INFO:tasks.rados.rados.0.smithi118.stdout:2203: got expected ENOENT (src dne) 2024-03-20T08:09:01.147 INFO:tasks.rados.rados.0.smithi118.stdout:2205: finishing rollback tid 0 to smithi11881747-41 2024-03-20T08:09:01.147 INFO:tasks.rados.rados.0.smithi118.stdout:2203: finishing copy_from racing read to smithi11881747-32 2024-03-20T08:09:01.147 INFO:tasks.rados.rados.0.smithi118.stdout:2205: finishing rollback tid 1 to smithi11881747-41 2024-03-20T08:09:01.148 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 33923 (ObjNum 65 snap 0 seq_num 322988944) dirty dne 2024-03-20T08:09:01.148 INFO:tasks.rados.rados.0.smithi118.stdout:2204: finishing rollback tid 0 to smithi11881747-43 2024-03-20T08:09:01.148 INFO:tasks.rados.rados.0.smithi118.stdout:2200: finishing rollback tid 2 to smithi11881747-36 2024-03-20T08:09:01.148 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 33914 (ObjNum 667 snap 217 seq_num 667) dirty exists 2024-03-20T08:09:01.148 INFO:tasks.rados.rados.0.smithi118.stdout:2204: finishing rollback tid 1 to smithi11881747-43 2024-03-20T08:09:01.149 INFO:tasks.rados.rados.0.smithi118.stdout:2204: finishing rollback tid 2 to smithi11881747-43 2024-03-20T08:09:01.149 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 33915 (ObjNum 545 snap 179 seq_num 545) dirty exists 2024-03-20T08:09:01.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: osdmap e507: 8 total, 8 up, 8 in 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:01.203 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:01.204 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-mon[87080]: pgmap v440: 121 pgs: 1 active+clean+snaptrim, 1 active+recovering+degraded, 1 active+undersized+degraded+snaptrim_error, 16 active+undersized, 4 active+undersized+degraded, 10 active+clean+snaptrim_error, 88 active+clean; 663 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 767 B/s rd, 4.2 MiB/s wr, 6 op/s; 45/1326 objects degraded (3.394%); 10 MiB/s, 1 objects/s recovering 2024-03-20T08:09:01.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.969+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.983+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:00.996+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.036+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.048+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.062+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.101+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.135+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.203+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.206 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:01.140+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:01.206 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:01.140+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:01.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:01.135+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:01.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:01.135+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:01.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:01.147+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:01.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:01.147+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:01.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:01.147+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:01.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:01.147+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:01.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:01.147+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: osdmap e507: 8 total, 8 up, 8 in 2024-03-20T08:09:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:01.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[95692]: pgmap v440: 121 pgs: 1 active+clean+snaptrim, 1 active+recovering+degraded, 1 active+undersized+degraded+snaptrim_error, 16 active+undersized, 4 active+undersized+degraded, 10 active+clean+snaptrim_error, 88 active+clean; 663 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 767 B/s rd, 4.2 MiB/s wr, 6 op/s; 45/1326 objects degraded (3.394%); 10 MiB/s, 1 objects/s recovering 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: osdmap e507: 8 total, 8 up, 8 in 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 114=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:01.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-mon[93121]: pgmap v440: 121 pgs: 1 active+clean+snaptrim, 1 active+recovering+degraded, 1 active+undersized+degraded+snaptrim_error, 16 active+undersized, 4 active+undersized+degraded, 10 active+clean+snaptrim_error, 88 active+clean; 663 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 767 B/s rd, 4.2 MiB/s wr, 6 op/s; 45/1326 objects degraded (3.394%); 10 MiB/s, 1 objects/s recovering 2024-03-20T08:09:01.425 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:01.141+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:01.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:01.140+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.216+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.322+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.348+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.415+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.428+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.440+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.727 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.479+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.727 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.492+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.727 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.505+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.727 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.519+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.727 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.727 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.544+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.727 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.727 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.571+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.584+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.666+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.695+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.710+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.741+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.754+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.782+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.794+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.808+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.834+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.848+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.862+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.875+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.915+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.927+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.970+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:01.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.138 INFO:tasks.rados.rados.0.smithi118.stdout:2200: done (5 left) 2024-03-20T08:09:02.138 INFO:tasks.rados.rados.0.smithi118.stdout:2202: done (4 left) 2024-03-20T08:09:02.138 INFO:tasks.rados.rados.0.smithi118.stdout:2203: done (3 left) 2024-03-20T08:09:02.138 INFO:tasks.rados.rados.0.smithi118.stdout:2204: done (2 left) 2024-03-20T08:09:02.139 INFO:tasks.rados.rados.0.smithi118.stdout:2205: done (1 left) 2024-03-20T08:09:02.139 INFO:tasks.rados.rados.0.smithi118.stdout:2206: done (0 left) 2024-03-20T08:09:02.139 INFO:tasks.rados.rados.0.smithi118.stdout:2207: read oid 42 snap 175 2024-03-20T08:09:02.139 INFO:tasks.rados.rados.0.smithi118.stdout:2207: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:09:02.139 INFO:tasks.rados.rados.0.smithi118.stdout:2208: snap_remove snap 223 2024-03-20T08:09:02.236 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:02.148+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:02.236 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:02.148+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:02.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: osdmap e508: 8 total, 8 up, 8 in 2024-03-20T08:09:02.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:02.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:02.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:02.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:02.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:02.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:02.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:02.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:02.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:02.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:02.237 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:02.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:01.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.013+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.026+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.053+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.080+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.106+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.118+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.147+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.147+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.179+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.193+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.222+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:02.224+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:02.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:02.224+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:02.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:02.225+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:02.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:02.225+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:02.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:02.141+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:02.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:02.141+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:02.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:02.149+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:02.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:02.139+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:02.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:02.139+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:02.424 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:02.145+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:02.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: osdmap e508: 8 total, 8 up, 8 in 2024-03-20T08:09:02.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:02.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:02.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:02.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:02.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:02.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: osdmap e508: 8 total, 8 up, 8 in 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:02.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:02.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:02.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:02.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:02.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:02.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:02.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.309+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.323+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.352+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.366+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.497 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.497 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.497 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.409+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.497 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.497 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.438+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.497 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.498 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.498 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.498 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.494+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.499 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:09:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:09:02.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.535+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.550+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.639+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.667+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.708+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.723+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.737+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:02.755 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.767+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.796+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.810+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.867+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.881+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.014 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.014 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.014 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.955+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.014 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.014 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:02.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.014 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.144 INFO:tasks.rados.rados.0.smithi118.stdout:2208: done (1 left) 2024-03-20T08:09:03.144 INFO:tasks.rados.rados.0.smithi118.stdout:2209: rollback oid 38 current snap is 227 2024-03-20T08:09:03.144 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 38 to 211 2024-03-20T08:09:03.145 INFO:tasks.rados.rados.0.smithi118.stdout:2207: expect (ObjNum 326 snap 106 seq_num 326) 2024-03-20T08:09:03.271 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:03.151+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:03.271 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:03.151+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:03.271 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: osdmap e509: 8 total, 8 up, 8 in 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:03.272 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-mon[87080]: pgmap v443: 121 pgs: 2 active+clean+snaptrim, 1 active+recovering+degraded, 3 active+undersized, 11 active+clean+snaptrim_error, 104 active+clean; 672 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 481 KiB/s rd, 4.6 MiB/s wr, 14 op/s; 3/1338 objects degraded (0.224%); 10 MiB/s, 2 objects/s recovering 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.055+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.083+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.126+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.140+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.153+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.158+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.209+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.234+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.298 INFO:tasks.rados.rados.0.smithi118.stdout:2209: finishing rollback tid 1 to smithi11881747-38 2024-03-20T08:09:03.298 INFO:tasks.rados.rados.0.smithi118.stdout:2209: finishing rollback tid 2 to smithi11881747-38 2024-03-20T08:09:03.298 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 7093 (ObjNum 630 snap 205 seq_num 630) dirty exists 2024-03-20T08:09:03.299 INFO:tasks.rados.rados.0.smithi118.stdout:2207: done (1 left) 2024-03-20T08:09:03.299 INFO:tasks.rados.rados.0.smithi118.stdout:2209: done (0 left) 2024-03-20T08:09:03.299 INFO:tasks.rados.rados.0.smithi118.stdout:2210: snap_remove snap 224 2024-03-20T08:09:03.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:03.150+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:03.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:03.146+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:03.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:03.148+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: osdmap e509: 8 total, 8 up, 8 in 2024-03-20T08:09:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:03.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[93121]: pgmap v443: 121 pgs: 2 active+clean+snaptrim, 1 active+recovering+degraded, 3 active+undersized, 11 active+clean+snaptrim_error, 104 active+clean; 672 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 481 KiB/s rd, 4.6 MiB/s wr, 14 op/s; 3/1338 objects degraded (0.224%); 10 MiB/s, 2 objects/s recovering 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: osdmap e509: 8 total, 8 up, 8 in 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:03.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:03.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:03.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:03.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:03.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:03.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:03.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:03.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:03.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-mon[95692]: pgmap v443: 121 pgs: 2 active+clean+snaptrim, 1 active+recovering+degraded, 3 active+undersized, 11 active+clean+snaptrim_error, 104 active+clean; 672 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 481 KiB/s rd, 4.6 MiB/s wr, 14 op/s; 3/1338 objects degraded (0.224%); 10 MiB/s, 2 objects/s recovering 2024-03-20T08:09:03.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.284+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.297+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.309+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.322+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.438+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.451+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.489+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.590+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.629+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.679+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.745+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.758+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:03.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:09:03] "GET /metrics HTTP/1.1" 200 36187 "" "Prometheus/2.43.0" 2024-03-20T08:09:04.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.796+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.809+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.845+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.858+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.952+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.965+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.978+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:03.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.031+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.149 INFO:tasks.rados.rados.0.smithi118.stdout:2210: done (0 left) 2024-03-20T08:09:04.150 INFO:tasks.rados.rados.0.smithi118.stdout:2211: copy_from oid 33 from oid 21 current snap is 227 2024-03-20T08:09:04.150 INFO:tasks.rados.rados.0.smithi118.stdout:2212: snap_remove snap 220 2024-03-20T08:09:04.228 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:04.150+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:04.229 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:04.150+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:04.229 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:04.151+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:04.229 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: osdmap e510: 8 total, 8 up, 8 in 2024-03-20T08:09:04.229 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:04.229 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:04.229 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:04.229 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:04.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:04.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:04.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:04.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:09:04.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.230 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:04.156+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:03.944+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:03.944+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:03.945+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:03.946+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:03.946+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:04.152+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:04.153+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:04.153+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:04.231 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:04.153+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:04.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: osdmap e510: 8 total, 8 up, 8 in 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:09:04.232 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.233 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:03.945+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:04.233 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:09:04.228+0000 7ff9af1ba700 -1 mgr.server reply reply (16) Device or resource busy unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-03-20T08:09:04.301 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: osdmap e510: 8 total, 8 up, 8 in 2024-03-20T08:09:04.301 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:04.301 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:04.301 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:04.301 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:04.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:04.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:04.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:04.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:09:04.302 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:04.302 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:04.153+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:04.303 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:04.153+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:04.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.059+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.070+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.084+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.121+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.148+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.158+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.158+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.167+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.181+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.199+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.226+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.276+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.289+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.301+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.316+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.355+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.367+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.405+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.431+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.444+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.457+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.480+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.492+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.504+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.517+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.529+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.555 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:04.307+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:04.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:04.307+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.618+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.631+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.656+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.669+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.731+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.769+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.782+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:04.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.806+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.818+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.830+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.879+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.065 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.938+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.975+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:04.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.001+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.053+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.065+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.155 INFO:tasks.rados.rados.0.smithi118.stdout:2212: done (1 left) 2024-03-20T08:09:05.155 INFO:tasks.rados.rados.0.smithi118.stdout:2213: delete oid 11 current snap is 227 2024-03-20T08:09:05.155 INFO:tasks.rados.rados.0.smithi118.stdout:2211: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:09:05.155 INFO:tasks.rados.rados.0.smithi118.stdout:2211: finishing copy_from to smithi11881747-33 2024-03-20T08:09:05.155 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 7081 (ObjNum 545 snap 179 seq_num 545) dirty exists 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2211: done (1 left) 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2213: done (0 left) 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2214: read oid 47 snap -1 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2214: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2215: read oid 2 snap -1 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2215: expect (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2216: read oid 42 snap -1 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2216: expect deleted 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2217: copy_from oid 7 from oid 24 current snap is 227 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2218: copy_from oid 1 from oid 14 current snap is 227 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 227 2024-03-20T08:09:05.160 INFO:tasks.rados.rados.0.smithi118.stdout:2219: seq_num 691 ranges {7110656=778240,7888896=548864,8437760=614400,9052160=40960} 2024-03-20T08:09:05.203 INFO:tasks.rados.rados.0.smithi118.stdout:2219: writing smithi11881747-33 from 7110656 to 7888896 tid 1 2024-03-20T08:09:05.207 INFO:tasks.rados.rados.0.smithi118.stdout:2219: writing smithi11881747-33 from 7888896 to 8437760 tid 2 2024-03-20T08:09:05.211 INFO:tasks.rados.rados.0.smithi118.stdout:2219: writing smithi11881747-33 from 8437760 to 9052160 tid 3 2024-03-20T08:09:05.211 INFO:tasks.rados.rados.0.smithi118.stdout:2219: writing smithi11881747-33 from 9052160 to 9093120 tid 4 2024-03-20T08:09:05.211 INFO:tasks.rados.rados.0.smithi118.stdout:2220: read oid 41 snap -1 2024-03-20T08:09:05.211 INFO:tasks.rados.rados.0.smithi118.stdout:2220: expect deleted 2024-03-20T08:09:05.211 INFO:tasks.rados.rados.0.smithi118.stdout:2214: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:09:05.222 INFO:tasks.rados.rados.0.smithi118.stdout:2218: finishing copy_from racing read to smithi11881747-1 2024-03-20T08:09:05.222 INFO:tasks.rados.rados.0.smithi118.stdout:2218: finishing copy_from to smithi11881747-1 2024-03-20T08:09:05.222 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 33997 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:09:05.222 INFO:tasks.rados.rados.0.smithi118.stdout:2215: expect (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: osdmap e511: 8 total, 8 up, 8 in 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:05.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: pgmap v446: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 669 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 481 KiB/s rd, 2.6 MiB/s wr, 16 op/s; 2.0 MiB/s, 1 objects/s recovering 2024-03-20T08:09:05.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:05.321 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 3/1338 objects degraded (0.224%), 1 pg degraded (PG_DEGRADED) 2024-03-20T08:09:05.321 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.321 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.089+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.321 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.321 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.113+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.321 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.126+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.321 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.139+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.321 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.152+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.163+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.178+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.191+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.283+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.308+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.323 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:05.160+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:05.323 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:05.161+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2217: finishing copy_from racing read to smithi11881747-7 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2217: finishing copy_from to smithi11881747-7 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 34001 (ObjNum 687 snap 225 seq_num 687) dirty exists 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2214: done (6 left) 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2215: done (5 left) 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2216: done (4 left) 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2217: done (3 left) 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2218: done (2 left) 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2221: rollback oid 45 current snap is 227 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 45 to 184 2024-03-20T08:09:05.374 INFO:tasks.rados.rados.0.smithi118.stdout:2219: finishing write tid 1 to smithi11881747-33 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: finishing write tid 2 to smithi11881747-33 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: finishing write tid 3 to smithi11881747-33 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: finishing write tid 4 to smithi11881747-33 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: finishing write tid 5 to smithi11881747-33 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: finishing write tid 6 to smithi11881747-33 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: oid 33 updating version 0 to 13615 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: oid 33 version 13615 is already newer than 13613 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: oid 33 version 13615 is already newer than 13614 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: oid 33 updating version 13615 to 13616 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: oid 33 version 13616 is already newer than 13612 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 13616 (ObjNum 691 snap 227 seq_num 691) dirty exists 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: left oid 33 (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2219: done (2 left) 2024-03-20T08:09:05.375 INFO:tasks.rados.rados.0.smithi118.stdout:2222: delete oid 24 current snap is 227 2024-03-20T08:09:05.383 INFO:tasks.rados.rados.0.smithi118.stdout:2220: done (2 left) 2024-03-20T08:09:05.384 INFO:tasks.rados.rados.0.smithi118.stdout:2222: done (1 left) 2024-03-20T08:09:05.384 INFO:tasks.rados.rados.0.smithi118.stdout:2223: snap_create 2024-03-20T08:09:05.386 INFO:tasks.rados.rados.0.smithi118.stdout:2221: finishing rollback tid 0 to smithi11881747-45 2024-03-20T08:09:05.387 INFO:tasks.rados.rados.0.smithi118.stdout:2221: finishing rollback tid 1 to smithi11881747-45 2024-03-20T08:09:05.387 INFO:tasks.rados.rados.0.smithi118.stdout:2221: finishing rollback tid 2 to smithi11881747-45 2024-03-20T08:09:05.387 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 30065 (ObjNum 537 snap 175 seq_num 537) dirty exists 2024-03-20T08:09:05.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:05.154+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:05.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:05.158+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:05.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:05.160+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:05.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:05.160+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:05.164+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:05.165+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:05.166+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:05.166+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:05.167+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: osdmap e511: 8 total, 8 up, 8 in 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: pgmap v446: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 669 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 481 KiB/s rd, 2.6 MiB/s wr, 16 op/s; 2.0 MiB/s, 1 objects/s recovering 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 3/1338 objects degraded (0.224%), 1 pg degraded (PG_DEGRADED) 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: osdmap e511: 8 total, 8 up, 8 in 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-03-20T08:09:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: pgmap v446: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 669 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 481 KiB/s rd, 2.6 MiB/s wr, 16 op/s; 2.0 MiB/s, 1 objects/s recovering 2024-03-20T08:09:05.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:05.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:05 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 3/1338 objects degraded (0.224%), 1 pg degraded (PG_DEGRADED) 2024-03-20T08:09:05.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.331+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.381+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.407+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.432+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.444+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.457+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.471+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.482+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.518+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.530+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.566+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.640+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.724+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.736+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.748+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.762+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.774+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.800+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.824+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:05.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.835+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.873+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.898+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.936+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.961+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.973+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.986+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:05.999+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.011+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.034+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.095 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.082+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.160 INFO:tasks.rados.rados.0.smithi118.stdout:2221: done (1 left) 2024-03-20T08:09:06.160 INFO:tasks.rados.rados.0.smithi118.stdout:2223: done (0 left) 2024-03-20T08:09:06.160 INFO:tasks.rados.rados.0.smithi118.stdout:2224: snap_create 2024-03-20T08:09:06.354 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:06.165+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:06.355 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:06.166+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:06.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/1338 objects degraded (0.224%), 1 pg degraded) 2024-03-20T08:09:06.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:06.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: osdmap e512: 8 total, 8 up, 8 in 2024-03-20T08:09:06.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:06.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:06.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:06.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:06.356 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:06.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.155+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.168+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.168+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.258+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.282+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.357 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.306+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.330+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:06.226+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:06.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:06.225+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:06.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:06.225+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:06.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:06.164+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:06.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:06.164+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:06.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:06.164+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:06.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:06.227+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:06.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:06.227+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:06.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:06.227+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:06.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:06.228+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/1338 objects degraded (0.224%), 1 pg degraded) 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: osdmap e512: 8 total, 8 up, 8 in 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/1338 objects degraded (0.224%), 1 pg degraded) 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: osdmap e512: 8 total, 8 up, 8 in 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:06.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 100=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.368+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.392+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.427+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.439+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.451+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.462+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.511+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.535+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.560+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.585+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.664+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.701+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.714+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.872 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.738+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.750+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.761+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.773+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.797+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.822+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.845+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.858+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:06.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.125 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.125 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.125 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.908+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.958+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.969+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:06.993+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.004+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.016+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.040+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.065+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.126 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.113+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.125+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.167 INFO:tasks.rados.rados.0.smithi118.stdout:2224: done (0 left) 2024-03-20T08:09:07.167 INFO:tasks.rados.rados.0.smithi118.stdout:2225: rmattr oid 40 current snap is 229 2024-03-20T08:09:07.167 INFO:tasks.rados.rados.0.smithi118.stdout:2225: done (0 left) 2024-03-20T08:09:07.167 INFO:tasks.rados.rados.0.smithi118.stdout:2226: rmattr oid 32 current snap is 229 2024-03-20T08:09:07.167 INFO:tasks.rados.rados.0.smithi118.stdout:2227: rollback oid 27 current snap is 229 2024-03-20T08:09:07.168 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 225 2024-03-20T08:09:07.168 INFO:tasks.rados.rados.0.smithi118.stdout:2228: snap_create 2024-03-20T08:09:07.171 INFO:tasks.rados.rados.0.smithi118.stdout:2227: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:09:07.171 INFO:tasks.rados.rados.0.smithi118.stdout:2227: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:09:07.171 INFO:tasks.rados.rados.0.smithi118.stdout:2227: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:09:07.171 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 13616 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:09:07.174 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 33999 (ObjNum 666 snap 217 seq_num 666) dirty exists 2024-03-20T08:09:07.377 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:07.174+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:07.377 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:07.174+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: osdmap e513: 8 total, 8 up, 8 in 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:07.378 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:07.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:07.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:07.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:07.379 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-mon[87080]: pgmap v449: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 679 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 13 op/s; 1.7 MiB/s, 0 objects/s recovering 2024-03-20T08:09:07.379 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.379 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.150+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.379 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.165+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.379 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.171+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:07.379 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.206+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.268+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.293+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.329+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.341+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.353+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.380 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.365+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.381 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.378+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:07.173+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:07.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:07.174+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:07.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: osdmap e513: 8 total, 8 up, 8 in 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[93121]: pgmap v449: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 679 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 13 op/s; 1.7 MiB/s, 0 objects/s recovering 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: osdmap e513: 8 total, 8 up, 8 in 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:07.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-mon[95692]: pgmap v449: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 679 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 13 op/s; 1.7 MiB/s, 0 objects/s recovering 2024-03-20T08:09:07.425 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:07.166+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:07.425 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:07.174+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:07.633 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.633 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.403+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.633 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.415+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.633 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.428+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.633 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.633 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.489+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.584+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.634 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.596+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.635 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.635 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.621+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.635 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.633+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.645+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.716+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.728+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.754+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.779+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.803+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.828+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.852+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:07.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.902+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.915+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.927+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.939+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:07.988+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.037+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.085+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.098+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.148 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.122+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.149 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.136+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.149 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:2226: done (2 left) 2024-03-20T08:09:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:2227: done (1 left) 2024-03-20T08:09:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:2228: done (0 left) 2024-03-20T08:09:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:2229: rollback oid 31 current snap is 230 2024-03-20T08:09:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 221 2024-03-20T08:09:08.167 INFO:tasks.rados.rados.0.smithi118.stdout:2230: snap_remove snap 193 2024-03-20T08:09:08.406 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:08.179+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:08.407 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:08.181+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:08.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:08.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-mon[87080]: osdmap e514: 8 total, 8 up, 8 in 2024-03-20T08:09:08.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:08.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:08.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:08.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:08.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:08.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:08.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.173+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:08.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.173+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:08.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.228+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.241+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.311+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.323+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.409 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.395+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.407+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:08.168+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:08.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:08.168+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:08.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:08.171+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:08.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:08.171+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:08.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:08.171+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[93121]: osdmap e514: 8 total, 8 up, 8 in 2024-03-20T08:09:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:08.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[95692]: osdmap e514: 8 total, 8 up, 8 in 2024-03-20T08:09:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:08.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:08.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.432+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.446+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.470+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.482+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.518+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.530+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.541+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.603+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.616+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.627+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.661 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.659+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.695+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.720+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.745+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.757+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.769+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.793+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.917 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.805+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.855+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.893+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.904+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:08.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.916+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:08.707+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:08.707+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:08.707+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:08.707+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:08.708+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:08.708+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.953+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:08.990+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.001+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.013+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.037+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.074+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.087+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.126+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.148+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.159+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.172+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.180 INFO:tasks.rados.rados.0.smithi118.stdout:2230: done (1 left) 2024-03-20T08:09:09.180 INFO:tasks.rados.rados.0.smithi118.stdout:2231: rmattr oid 21 current snap is 230 2024-03-20T08:09:09.180 INFO:tasks.rados.rados.0.smithi118.stdout:2229: finishing rollback tid 0 to smithi11881747-31 2024-03-20T08:09:09.180 INFO:tasks.rados.rados.0.smithi118.stdout:2229: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:09:09.180 INFO:tasks.rados.rados.0.smithi118.stdout:2229: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:09:09.180 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 34000 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:09:09.180 INFO:tasks.rados.rados.0.smithi118.stdout:2229: done (1 left) 2024-03-20T08:09:09.181 INFO:tasks.rados.rados.0.smithi118.stdout:2232: snap_remove snap 212 2024-03-20T08:09:09.423 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:09.190+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:09.423 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:09.190+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: osdmap e515: 8 total, 8 up, 8 in 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: pgmap v452: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 700 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:09.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.189+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.192+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.240+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.253+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.266+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.276+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.301+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.326+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.605 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:09.182+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:09.606 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:09.183+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:09.606 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:09.183+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:09.606 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:09.178+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:09.606 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: osdmap e515: 8 total, 8 up, 8 in 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: pgmap v452: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 700 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:09.607 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: osdmap e515: 8 total, 8 up, 8 in 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: pgmap v452: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 700 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 10 op/s 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:09.608 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:09.678 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:09:09.607+0000 7fafa3585700 -1 osd.6 pg_epoch: 516 pg[5.9s1( v 512'1108 (0'0,512'1108] local-lis/les=505/506 n=22 ec=73/73 lis/c=505/505 les/c/f=506/506/0 sis=505) [7,6,1]p7(0) r=1 lpr=505 luod=0'0 crt=512'1108 mlcod 512'1108 active mbc={} ps=113] update_snap_map remove_oid 5:902426e2:::smithi11881747-46:ff failed with -2 2024-03-20T08:09:09.678 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:09:09.611+0000 7fafa758d700 -1 osd.6 pg_epoch: 516 pg[5.9s1( v 516'1112 (0'0,516'1112] local-lis/les=505/506 n=21 ec=73/73 lis/c=505/505 les/c/f=506/506/0 sis=505) [7,6,1]p7(0) r=1 lpr=505 luod=0'0 crt=516'1112 mlcod 516'1112 active mbc={} ps=113] update_snap_map remove_oid 5:95859c30:::smithi11881747-2:ff failed with -2 2024-03-20T08:09:09.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.447+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.459+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.483+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.521+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.533+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.583+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.595+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.680 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.680 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.636+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.680 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.650+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.680 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.680 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.678+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:09.604+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:09.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:09.604+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:09.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:09.604+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:09.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:09.605+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:09.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.722+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.748+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.784+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.844+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.856+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.881+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.919+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:09.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.931+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.185 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 7395 (ObjNum 545 snap 179 seq_num 545) dirty exists 2024-03-20T08:09:10.186 INFO:tasks.rados.rados.0.smithi118.stdout:2231: done (1 left) 2024-03-20T08:09:10.186 INFO:tasks.rados.rados.0.smithi118.stdout:2232: done (0 left) 2024-03-20T08:09:10.186 INFO:tasks.rados.rados.0.smithi118.stdout:2233: rollback oid 28 current snap is 230 2024-03-20T08:09:10.186 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 28 to 216 2024-03-20T08:09:10.186 INFO:tasks.rados.rados.0.smithi118.stdout:2234: read oid 11 snap -1 2024-03-20T08:09:10.186 INFO:tasks.rados.rados.0.smithi118.stdout:2234: expect deleted 2024-03-20T08:09:10.186 INFO:tasks.rados.rados.0.smithi118.stdout:2235: snap_create 2024-03-20T08:09:10.186 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: osdmap e516: 8 total, 8 up, 8 in 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:10.187 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.944+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.957+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.969+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:09.994+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.006+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.044+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.188 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.093+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.118+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.154+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.166+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.189 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.190 INFO:tasks.rados.rados.0.smithi118.stdout:2233: finishing rollback tid 0 to smithi11881747-28 2024-03-20T08:09:10.190 INFO:tasks.rados.rados.0.smithi118.stdout:2233: finishing rollback tid 1 to smithi11881747-28 2024-03-20T08:09:10.191 INFO:tasks.rados.rados.0.smithi118.stdout:2233: finishing rollback tid 2 to smithi11881747-28 2024-03-20T08:09:10.191 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 33926 (ObjNum 644 snap 211 seq_num 644) dirty exists 2024-03-20T08:09:10.441 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:10.190+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:10.441 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:10.190+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:10.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.184+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.184+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.195+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.207+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.333+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.442 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.382+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.395+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.407+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.430+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:10.186+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:10.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:10.186+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:10.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:10.187+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:10.183+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:10.183+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:10.183+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:10.183+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:10.183+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:10.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:10.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: osdmap e516: 8 total, 8 up, 8 in 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: osdmap e516: 8 total, 8 up, 8 in 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:10.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:10.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:10.206+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:10.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:10.188+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:10.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:10.188+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.454+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.489+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.513+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.537+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.621+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.633+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.645+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.669+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.693 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.865+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.876+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.901+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.912+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.924+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.935+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:10.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.948+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.190 INFO:tasks.rados.rados.0.smithi118.stdout:2233: done (2 left) 2024-03-20T08:09:11.190 INFO:tasks.rados.rados.0.smithi118.stdout:2234: done (1 left) 2024-03-20T08:09:11.190 INFO:tasks.rados.rados.0.smithi118.stdout:2235: done (0 left) 2024-03-20T08:09:11.190 INFO:tasks.rados.rados.0.smithi118.stdout:2236: copy_from oid 21 from oid 50 current snap is 231 2024-03-20T08:09:11.190 INFO:tasks.rados.rados.0.smithi118.stdout:2237: read oid 7 snap -1 2024-03-20T08:09:11.190 INFO:tasks.rados.rados.0.smithi118.stdout:2237: expect (ObjNum 687 snap 225 seq_num 687) 2024-03-20T08:09:11.191 INFO:tasks.rados.rados.0.smithi118.stdout:2238: read oid 17 snap -1 2024-03-20T08:09:11.191 INFO:tasks.rados.rados.0.smithi118.stdout:2238: expect (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:09:11.191 INFO:tasks.rados.rados.0.smithi118.stdout:2239: read oid 49 snap -1 2024-03-20T08:09:11.191 INFO:tasks.rados.rados.0.smithi118.stdout:2239: expect deleted 2024-03-20T08:09:11.191 INFO:tasks.rados.rados.0.smithi118.stdout:2240: delete oid 38 current snap is 231 2024-03-20T08:09:11.198 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:11.197+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:10.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.035+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.059+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.105+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.199 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.154+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.178+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.200 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.193+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:11.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: osdmap e517: 8 total, 8 up, 8 in 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:11.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:11.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:11.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-mon[87080]: pgmap v455: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 700 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 7 op/s 2024-03-20T08:09:11.208 INFO:tasks.rados.rados.0.smithi118.stdout:2239: done (4 left) 2024-03-20T08:09:11.208 INFO:tasks.rados.rados.0.smithi118.stdout:2240: done (3 left) 2024-03-20T08:09:11.208 INFO:tasks.rados.rados.0.smithi118.stdout:2241: snap_create 2024-03-20T08:09:11.215 INFO:tasks.rados.rados.0.smithi118.stdout:2237: expect (ObjNum 687 snap 225 seq_num 687) 2024-03-20T08:09:11.304 INFO:tasks.rados.rados.0.smithi118.stdout:2238: expect (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:09:11.449 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:11.197+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:11.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.203+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:11.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.268+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.293+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.340+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.352+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.474 INFO:tasks.rados.rados.0.smithi118.stdout:2236: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:09:11.474 INFO:tasks.rados.rados.0.smithi118.stdout:2236: finishing copy_from to smithi11881747-21 2024-03-20T08:09:11.474 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 7117 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:09:11.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:11.364+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:11.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:11.195+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:11.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:11.195+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:11.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:11.190+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:11.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:11.193+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:11.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:11.193+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:11.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:11.194+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:11.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:11.197+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: osdmap e517: 8 total, 8 up, 8 in 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[93121]: pgmap v455: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 700 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 7 op/s 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: osdmap e517: 8 total, 8 up, 8 in 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:11.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:11.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:11.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:11.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:11.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:11 smithi118 ceph-mon[95692]: pgmap v455: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 700 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 7 op/s 2024-03-20T08:09:11.710 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.485+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.497+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.519+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.532+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.543+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.566+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.711 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.601+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.637+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.674+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.686+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.712 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.710+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.723+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.761+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.796+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.831+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.855+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.963 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.866+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.938+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.950+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:11.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.962+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.195 INFO:tasks.rados.rados.0.smithi118.stdout:2236: done (3 left) 2024-03-20T08:09:12.195 INFO:tasks.rados.rados.0.smithi118.stdout:2237: done (2 left) 2024-03-20T08:09:12.195 INFO:tasks.rados.rados.0.smithi118.stdout:2238: done (1 left) 2024-03-20T08:09:12.196 INFO:tasks.rados.rados.0.smithi118.stdout:2241: done (0 left) 2024-03-20T08:09:12.196 INFO:tasks.rados.rados.0.smithi118.stdout:2242: copy_from oid 23 from oid 20 current snap is 232 2024-03-20T08:09:12.196 INFO:tasks.rados.rados.0.smithi118.stdout:2243: rmattr oid 4 current snap is 232 2024-03-20T08:09:12.196 INFO:tasks.rados.rados.0.smithi118.stdout:2243: done (1 left) 2024-03-20T08:09:12.196 INFO:tasks.rados.rados.0.smithi118.stdout:2244: snap_create 2024-03-20T08:09:12.208 INFO:tasks.rados.rados.0.smithi118.stdout:2242: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:09:12.210 INFO:tasks.rados.rados.0.smithi118.stdout:2242: finishing copy_from to smithi11881747-23 2024-03-20T08:09:12.210 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 7076 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:09:12.216 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:12.201+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:12.216 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:12.201+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:12.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: osdmap e518: 8 total, 8 up, 8 in 2024-03-20T08:09:12.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:12.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:12.216 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:12.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:12.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:12.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:12.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:12.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:12.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:12.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:12.217 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:12.217 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:11.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.033+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.044+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.056+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.079+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.102+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.115+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.128+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.140+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.176+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.203+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.203+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.204+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.470 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:09:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:09:12.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.254+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.267+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.301+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.313+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.360+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.386+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.434+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.446+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.472 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.471+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:12.201+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:12.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:12.196+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:12.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:12.196+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:12.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:12.197+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:12.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:12.195+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:12.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:12.195+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:12.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:12.198+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:12.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:12.198+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:12.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:12.198+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:12.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:12.198+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:12.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:12.198+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:12.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: osdmap e518: 8 total, 8 up, 8 in 2024-03-20T08:09:12.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:12.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:12.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:12.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:12.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: osdmap e518: 8 total, 8 up, 8 in 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:12.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:12.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:12.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:12.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:12.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:12.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:12.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:12.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:12.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.482+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.529+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.541+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.565+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.637+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.660+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.684+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.696+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.708+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.720+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.733 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.769+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.867+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.939+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.976+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:12.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:12.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:2242: done (1 left) 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:2244: done (0 left) 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:2245: rollback oid 37 current snap is 233 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 37 to 215 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:2246: read oid 25 snap -1 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:2246: expect (ObjNum 652 snap 213 seq_num 652) 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:2247: read oid 28 snap -1 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:2247: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:09:13.200 INFO:tasks.rados.rados.0.smithi118.stdout:2248: snap_remove snap 231 2024-03-20T08:09:13.243 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:13.204+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:13.243 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:13.205+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: osdmap e519: 8 total, 8 up, 8 in 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:13.244 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:13.245 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-mon[87080]: pgmap v458: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 704 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 5 op/s 2024-03-20T08:09:13.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.051+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.064+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.087+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.184+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.208+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.218+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.244+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.255+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.268+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.292+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.330+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.355+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.501 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.367+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.415+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.427+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.439+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.465+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.488+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:13.203+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:13.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:13.208+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:13.585 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:13.201+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:13.586 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:13.209+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:13.207+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:13.210+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:13.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:13.211+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:13.587 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:13.215+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: osdmap e519: 8 total, 8 up, 8 in 2024-03-20T08:09:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[93121]: pgmap v458: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 704 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 5 op/s 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: osdmap e519: 8 total, 8 up, 8 in 2024-03-20T08:09:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-mon[95692]: pgmap v458: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 704 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 5 op/s 2024-03-20T08:09:13.756 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.513+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.548+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.560+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.584+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.597+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.631+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.643+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.656+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.669+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.682+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.721+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.745+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:09:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:09:13] "GET /metrics HTTP/1.1" 200 36335 "" "Prometheus/2.43.0" 2024-03-20T08:09:14.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.769+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.816+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.838+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.861+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.872+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.908+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.919+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.964+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.988+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:13.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.204 INFO:tasks.rados.rados.0.smithi118.stdout:2248: done (3 left) 2024-03-20T08:09:14.204 INFO:tasks.rados.rados.0.smithi118.stdout:2249: delete oid 47 current snap is 233 2024-03-20T08:09:14.204 INFO:tasks.rados.rados.0.smithi118.stdout:2245: finishing rollback tid 1 to smithi11881747-37 2024-03-20T08:09:14.204 INFO:tasks.rados.rados.0.smithi118.stdout:2245: finishing rollback tid 2 to smithi11881747-37 2024-03-20T08:09:14.204 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 30067 (ObjNum 584 snap 188 seq_num 584) dirty exists 2024-03-20T08:09:14.205 INFO:tasks.rados.rados.0.smithi118.stdout:2247: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:09:14.266 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:14.211+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:14.266 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:14.211+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: osdmap e520: 8 total, 8 up, 8 in 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:14.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:14.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:14.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:14.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:14.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:14.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.024+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.047+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.083+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.096+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.120+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.131+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.143+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.271 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.168+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.271 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.179+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.271 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.191+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.271 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.204+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.271 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.211+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:14.272 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.211+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:14.272 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.272 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.231+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.266+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.373 INFO:tasks.rados.rados.0.smithi118.stdout:2246: expect (ObjNum 652 snap 213 seq_num 652) 2024-03-20T08:09:14.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.316+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.328+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.340+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.376+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.388+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.424+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.522 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.523 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.523 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.523 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.523 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.485+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.523 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.523 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.509+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.523 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.606 INFO:tasks.rados.rados.0.smithi118.stdout:2245: done (3 left) 2024-03-20T08:09:14.606 INFO:tasks.rados.rados.0.smithi118.stdout:2246: done (2 left) 2024-03-20T08:09:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:2247: done (1 left) 2024-03-20T08:09:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:2249: done (0 left) 2024-03-20T08:09:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 233 2024-03-20T08:09:14.607 INFO:tasks.rados.rados.0.smithi118.stdout:2250: seq_num 695 ranges {2916352=442368,3358720=540672,3899392=598016,4497408=466944} 2024-03-20T08:09:14.624 INFO:tasks.rados.rados.0.smithi118.stdout:2250: writing smithi11881747-23 from 2916352 to 3358720 tid 1 2024-03-20T08:09:14.628 INFO:tasks.rados.rados.0.smithi118.stdout:2250: writing smithi11881747-23 from 3358720 to 3899392 tid 2 2024-03-20T08:09:14.632 INFO:tasks.rados.rados.0.smithi118.stdout:2250: writing smithi11881747-23 from 3899392 to 4497408 tid 3 2024-03-20T08:09:14.635 INFO:tasks.rados.rados.0.smithi118.stdout:2250: writing smithi11881747-23 from 4497408 to 4964352 tid 4 2024-03-20T08:09:14.635 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 233 2024-03-20T08:09:14.635 INFO:tasks.rados.rados.0.smithi118.stdout:2250: finishing write tid 1 to smithi11881747-23 2024-03-20T08:09:14.635 INFO:tasks.rados.rados.0.smithi118.stdout:2250: finishing write tid 2 to smithi11881747-23 2024-03-20T08:09:14.635 INFO:tasks.rados.rados.0.smithi118.stdout:2251: seq_num 696 ranges {7110656=458752,7569408=417792,7987200=753664,8740864=434176} 2024-03-20T08:09:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:14.210+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:14.210+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:14.210+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:14.207+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:14.210+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:14.210+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:14.210+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:14.210+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: osdmap e520: 8 total, 8 up, 8 in 2024-03-20T08:09:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:14.674 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:14.204+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:14.675 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:14.204+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: osdmap e520: 8 total, 8 up, 8 in 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:14.677 INFO:tasks.rados.rados.0.smithi118.stdout:2251: writing smithi11881747-43 from 7110656 to 7569408 tid 1 2024-03-20T08:09:14.679 INFO:tasks.rados.rados.0.smithi118.stdout:2251: writing smithi11881747-43 from 7569408 to 7987200 tid 2 2024-03-20T08:09:14.683 INFO:tasks.rados.rados.0.smithi118.stdout:2251: writing smithi11881747-43 from 7987200 to 8740864 tid 3 2024-03-20T08:09:14.686 INFO:tasks.rados.rados.0.smithi118.stdout:2251: writing smithi11881747-43 from 8740864 to 9175040 tid 4 2024-03-20T08:09:14.686 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 233 2024-03-20T08:09:14.686 INFO:tasks.rados.rados.0.smithi118.stdout:2250: finishing write tid 3 to smithi11881747-23 2024-03-20T08:09:14.686 INFO:tasks.rados.rados.0.smithi118.stdout:2252: seq_num 697 ranges {2170880=475136,2646016=704512,3350528=557056,3907584=344064} 2024-03-20T08:09:14.701 INFO:tasks.rados.rados.0.smithi118.stdout:2252: writing smithi11881747-48 from 2170880 to 2646016 tid 1 2024-03-20T08:09:14.705 INFO:tasks.rados.rados.0.smithi118.stdout:2252: writing smithi11881747-48 from 2646016 to 3350528 tid 2 2024-03-20T08:09:14.708 INFO:tasks.rados.rados.0.smithi118.stdout:2252: writing smithi11881747-48 from 3350528 to 3907584 tid 3 2024-03-20T08:09:14.711 INFO:tasks.rados.rados.0.smithi118.stdout:2252: writing smithi11881747-48 from 3907584 to 4251648 tid 4 2024-03-20T08:09:14.711 INFO:tasks.rados.rados.0.smithi118.stdout:2253: read oid 10 snap -1 2024-03-20T08:09:14.712 INFO:tasks.rados.rados.0.smithi118.stdout:2253: expect (ObjNum 675 snap 223 seq_num 675) 2024-03-20T08:09:14.712 INFO:tasks.rados.rados.0.smithi118.stdout:2254: snap_remove snap 229 2024-03-20T08:09:14.778 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.778 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.778 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.558+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.778 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.571+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.778 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.583+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.778 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.631+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.644+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.655+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.743+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:14.779 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.803+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.863+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.958+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.970+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:14.993+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.029+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:14.865+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:15.211 INFO:tasks.rados.rados.0.smithi118.stdout:2254: done (4 left) 2024-03-20T08:09:15.211 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 233 2024-03-20T08:09:15.211 INFO:tasks.rados.rados.0.smithi118.stdout:2250: finishing write tid 4 to smithi11881747-23 2024-03-20T08:09:15.211 INFO:tasks.rados.rados.0.smithi118.stdout:2250: finishing write tid 5 to smithi11881747-23 2024-03-20T08:09:15.211 INFO:tasks.rados.rados.0.smithi118.stdout:2250: finishing write tid 6 to smithi11881747-23 2024-03-20T08:09:15.211 INFO:tasks.rados.rados.0.smithi118.stdout:2250: oid 23 updating version 0 to 33931 2024-03-20T08:09:15.211 INFO:tasks.rados.rados.0.smithi118.stdout:2250: oid 23 updating version 33931 to 33932 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2250: oid 23 version 33932 is already newer than 33928 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2250: oid 23 version 33932 is already newer than 33930 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2250: oid 23 version 33932 is already newer than 33929 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 33932 (ObjNum 695 snap 233 seq_num 695) dirty exists 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2250: left oid 23 (ObjNum 695 snap 233 seq_num 695) 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: finishing write tid 1 to smithi11881747-43 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: finishing write tid 2 to smithi11881747-43 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: finishing write tid 3 to smithi11881747-43 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: finishing write tid 4 to smithi11881747-43 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: finishing write tid 5 to smithi11881747-43 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: finishing write tid 6 to smithi11881747-43 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: oid 43 updating version 0 to 34002 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: oid 43 updating version 34002 to 34004 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: oid 43 updating version 34004 to 34005 2024-03-20T08:09:15.212 INFO:tasks.rados.rados.0.smithi118.stdout:2251: oid 43 version 34005 is already newer than 34001 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2251: oid 43 version 34005 is already newer than 34003 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 34005 (ObjNum 696 snap 233 seq_num 696) dirty exists 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2251: left oid 43 (ObjNum 696 snap 233 seq_num 696) 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: finishing write tid 1 to smithi11881747-48 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: finishing write tid 2 to smithi11881747-48 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: finishing write tid 3 to smithi11881747-48 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: finishing write tid 4 to smithi11881747-48 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: finishing write tid 5 to smithi11881747-48 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: finishing write tid 6 to smithi11881747-48 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: oid 48 updating version 0 to 9719 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: oid 48 updating version 9719 to 9721 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: oid 48 version 9721 is already newer than 9718 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: oid 48 version 9721 is already newer than 9717 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: oid 48 version 9721 is already newer than 9720 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 9721 (ObjNum 697 snap 233 seq_num 697) dirty exists 2024-03-20T08:09:15.213 INFO:tasks.rados.rados.0.smithi118.stdout:2252: left oid 48 (ObjNum 697 snap 233 seq_num 697) 2024-03-20T08:09:15.214 INFO:tasks.rados.rados.0.smithi118.stdout:2253: expect (ObjNum 675 snap 223 seq_num 675) 2024-03-20T08:09:15.278 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:15.213+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:15.279 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:15.213+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: osdmap e521: 8 total, 8 up, 8 in 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:15.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:15.280 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:15.280 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:15.280 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-mon[87080]: pgmap v461: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 703 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:09:15.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.040+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.052+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.064+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.112+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.125+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.137+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.188+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.201+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.216+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.218+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.266+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.292+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.351+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.376+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.498+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.536 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:15.210+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:15.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:15.211+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:15.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:15.308+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:15.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:15.308+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:15.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:15.308+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:15.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:15.308+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:15.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:15.209+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:15.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: osdmap e521: 8 total, 8 up, 8 in 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:15.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[93121]: pgmap v461: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 703 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:09:15.675 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:15.307+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: osdmap e521: 8 total, 8 up, 8 in 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:15.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:15.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:15 smithi118 ceph-mon[95692]: pgmap v461: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 703 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:09:15.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.535+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.559+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.572+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.585+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.633+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.656+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.740+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.751+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.762+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.774+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:15.868 INFO:tasks.rados.rados.0.smithi118.stdout:2255: seq_num 698 ranges {7356416=499712,7856128=589824,8445952=770048,9216000=245760} 2024-03-20T08:09:15.912 INFO:tasks.rados.rados.0.smithi118.stdout:2255: writing smithi11881747-16 from 7356416 to 7856128 tid 1 2024-03-20T08:09:15.916 INFO:tasks.rados.rados.0.smithi118.stdout:2255: writing smithi11881747-16 from 7856128 to 8445952 tid 2 2024-03-20T08:09:15.921 INFO:tasks.rados.rados.0.smithi118.stdout:2255: writing smithi11881747-16 from 8445952 to 9216000 tid 3 2024-03-20T08:09:15.923 INFO:tasks.rados.rados.0.smithi118.stdout:2255: writing smithi11881747-16 from 9216000 to 9461760 tid 4 2024-03-20T08:09:15.923 INFO:tasks.rados.rados.0.smithi118.stdout:2250: done (4 left) 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2251: done (3 left) 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2252: done (2 left) 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2253: done (1 left) 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2256: read oid 40 snap 211 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2256: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2255: finishing write tid 1 to smithi11881747-16 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2255: finishing write tid 2 to smithi11881747-16 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2257: read oid 43 snap -1 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2257: expect (ObjNum 696 snap 233 seq_num 696) 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2258: setattr oid 47 current snap is 233 2024-03-20T08:09:15.924 INFO:tasks.rados.rados.0.smithi118.stdout:2259: snap_create 2024-03-20T08:09:15.929 INFO:tasks.rados.rados.0.smithi118.stdout:2256: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:09:15.976 INFO:tasks.rados.rados.0.smithi118.stdout:2255: finishing write tid 3 to smithi11881747-16 2024-03-20T08:09:15.976 INFO:tasks.rados.rados.0.smithi118.stdout:2255: finishing write tid 4 to smithi11881747-16 2024-03-20T08:09:15.976 INFO:tasks.rados.rados.0.smithi118.stdout:2255: finishing write tid 5 to smithi11881747-16 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33938 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:2255: finishing write tid 6 to smithi11881747-16 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:2255: oid 16 updating version 0 to 33935 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:2255: oid 16 updating version 33935 to 33936 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:2255: oid 16 version 33936 is already newer than 33934 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:2255: oid 16 version 33936 is already newer than 33933 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:2255: oid 16 updating version 33936 to 33937 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 33937 (ObjNum 698 snap 233 seq_num 698) dirty exists 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:2255: left oid 16 (ObjNum 698 snap 233 seq_num 698) 2024-03-20T08:09:15.977 INFO:tasks.rados.rados.0.smithi118.stdout:2257: expect (ObjNum 696 snap 233 seq_num 696) 2024-03-20T08:09:16.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.821+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.844+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.884+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.897+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.923+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.945+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.974+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.986+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:15.999+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.011+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.022+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.060 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.047+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.060 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.312 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:16.218+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:16.312 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:16.218+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:16.312 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: osdmap e522: 8 total, 8 up, 8 in 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:16.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.082+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.105+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.154+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.190+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.221+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.221+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.230+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.266+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.278+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.289+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.316 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:09:16.569 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.360+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.371+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.396+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.448+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.473+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.498+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.509+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.557+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.632 INFO:tasks.rados.rados.0.smithi118.stdout:2255: done (4 left) 2024-03-20T08:09:16.632 INFO:tasks.rados.rados.0.smithi118.stdout:2256: done (3 left) 2024-03-20T08:09:16.632 INFO:tasks.rados.rados.0.smithi118.stdout:2257: done (2 left) 2024-03-20T08:09:16.632 INFO:tasks.rados.rados.0.smithi118.stdout:2258: done (1 left) 2024-03-20T08:09:16.632 INFO:tasks.rados.rados.0.smithi118.stdout:2259: done (0 left) 2024-03-20T08:09:16.632 INFO:tasks.rados.rados.0.smithi118.stdout:2260: delete oid 49 current snap is 234 2024-03-20T08:09:16.634 INFO:tasks.rados.rados.0.smithi118.stdout:2260: done (0 left) 2024-03-20T08:09:16.634 INFO:tasks.rados.rados.0.smithi118.stdout:2261: read oid 3 snap 232 2024-03-20T08:09:16.634 INFO:tasks.rados.rados.0.smithi118.stdout:2261: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:09:16.634 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 234 2024-03-20T08:09:16.634 INFO:tasks.rados.rados.0.smithi118.stdout:2262: seq_num 700 ranges {7757824=532480,8290304=409600,8699904=737280,9437184=458752} 2024-03-20T08:09:16.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:16.215+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:16.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:16.215+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:16.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:16.218+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:16.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:16.218+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:16.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:16.218+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:16.295+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:16.295+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:16.296+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:16.296+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:16.296+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: osdmap e522: 8 total, 8 up, 8 in 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: osdmap e522: 8 total, 8 up, 8 in 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset f6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:16.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:16.296+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:16.680 INFO:tasks.rados.rados.0.smithi118.stdout:2262: writing smithi11881747-10 from 7757824 to 8290304 tid 1 2024-03-20T08:09:16.683 INFO:tasks.rados.rados.0.smithi118.stdout:2262: writing smithi11881747-10 from 8290304 to 8699904 tid 2 2024-03-20T08:09:16.689 INFO:tasks.rados.rados.0.smithi118.stdout:2262: writing smithi11881747-10 from 8699904 to 9437184 tid 3 2024-03-20T08:09:16.692 INFO:tasks.rados.rados.0.smithi118.stdout:2262: writing smithi11881747-10 from 9437184 to 9895936 tid 4 2024-03-20T08:09:16.693 INFO:tasks.rados.rados.0.smithi118.stdout:2263: snap_create 2024-03-20T08:09:16.693 INFO:tasks.rados.rados.0.smithi118.stdout:2261: expect (ObjNum 410 snap 136 seq_num 410) 2024-03-20T08:09:16.693 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:09:16.693 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (8m) 2m ago 9m 27.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:09:16.693 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (9m) 18s ago 9m 90.1M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:09:16.693 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (10m) 2m ago 10m 15.4M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:09:16.693 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (10m) 18s ago 10m 21.6M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (5m) 18s ago 14m 452M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (6m) 2m ago 16m 508M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (5m) 2m ago 16m 56.2M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (4m) 18s ago 14m 45.9M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (5m) 2m ago 14m 51.1M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (9m) 2m ago 9m 15.5M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (9m) 18s ago 9m 15.5M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (3m) 2m ago 14m 334M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (3m) 2m ago 13m 299M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (2m) 2m ago 13m 251M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (2m) 2m ago 12m 12.7M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:09:16.694 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (88s) 18s ago 12m 284M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:09:16.695 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (63s) 18s ago 12m 313M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 72b569d10541 2024-03-20T08:09:16.695 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (24s) 18s ago 11m 14.9M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 dca658056148 2024-03-20T08:09:16.695 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (11m) 18s ago 11m 230M 2480M 18.2.2-904-g78dbc53a 3b5d1eaa4228 b48bb8ba0aee 2024-03-20T08:09:16.695 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (5m) 18s ago 10m 60.7M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:09:16.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.581+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.593+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.616+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.655+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.678+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.720+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.767+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.834 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.780+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.834 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.834 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.834 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.834 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.830+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: finishing write tid 1 to smithi11881747-10 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: finishing write tid 2 to smithi11881747-10 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: finishing write tid 3 to smithi11881747-10 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: finishing write tid 4 to smithi11881747-10 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: finishing write tid 5 to smithi11881747-10 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: finishing write tid 6 to smithi11881747-10 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: oid 10 updating version 0 to 34008 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: oid 10 updating version 34008 to 34009 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: oid 10 version 34009 is already newer than 34007 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: oid 10 version 34009 is already newer than 34006 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:2262: oid 10 updating version 34009 to 34010 2024-03-20T08:09:16.865 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 34010 (ObjNum 700 snap 234 seq_num 700) dirty exists 2024-03-20T08:09:16.866 INFO:tasks.rados.rados.0.smithi118.stdout:2262: left oid 10 (ObjNum 700 snap 234 seq_num 700) 2024-03-20T08:09:17.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.866+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.878+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.903+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.915+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.939+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.976+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:16.999+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.011+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.034+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.057+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.083 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.082+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.154 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:09:17.154 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:09:17.154 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:09:17.154 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:09:17.154 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:09:17.154 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 1, 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 7 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 3, 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 12 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:09:17.155 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:09:17.219 INFO:tasks.rados.rados.0.smithi118.stdout:2261: done (2 left) 2024-03-20T08:09:17.219 INFO:tasks.rados.rados.0.smithi118.stdout:2262: done (1 left) 2024-03-20T08:09:17.219 INFO:tasks.rados.rados.0.smithi118.stdout:2263: done (0 left) 2024-03-20T08:09:17.219 INFO:tasks.rados.rados.0.smithi118.stdout:2264: copy_from oid 45 from oid 2 current snap is 235 2024-03-20T08:09:17.220 INFO:tasks.rados.rados.0.smithi118.stdout:2265: read oid 19 snap 233 2024-03-20T08:09:17.220 INFO:tasks.rados.rados.0.smithi118.stdout:2265: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:09:17.220 INFO:tasks.rados.rados.0.smithi118.stdout:2266: delete oid 5 current snap is 235 2024-03-20T08:09:17.226 INFO:tasks.rados.rados.0.smithi118.stdout:2266: done (2 left) 2024-03-20T08:09:17.226 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 235 2024-03-20T08:09:17.226 INFO:tasks.rados.rados.0.smithi118.stdout:2267: seq_num 702 ranges {655360=565248,1220608=581632,1802240=745472,2547712=278528} 2024-03-20T08:09:17.233 INFO:tasks.rados.rados.0.smithi118.stdout:2267: writing smithi11881747-1 from 655360 to 1220608 tid 1 2024-03-20T08:09:17.237 INFO:tasks.rados.rados.0.smithi118.stdout:2267: writing smithi11881747-1 from 1220608 to 1802240 tid 2 2024-03-20T08:09:17.242 INFO:tasks.rados.rados.0.smithi118.stdout:2267: writing smithi11881747-1 from 1802240 to 2547712 tid 3 2024-03-20T08:09:17.243 INFO:tasks.rados.rados.0.smithi118.stdout:2267: writing smithi11881747-1 from 2547712 to 2826240 tid 4 2024-03-20T08:09:17.243 INFO:tasks.rados.rados.0.smithi118.stdout:2268: rollback oid 17 current snap is 235 2024-03-20T08:09:17.244 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 234 2024-03-20T08:09:17.244 INFO:tasks.rados.rados.0.smithi118.stdout:2265: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:09:17.285 INFO:tasks.rados.rados.0.smithi118.stdout:2265: done (3 left) 2024-03-20T08:09:17.285 INFO:tasks.rados.rados.0.smithi118.stdout:2269: setattr oid 4 current snap is 235 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2267: finishing write tid 1 to smithi11881747-1 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2267: finishing write tid 2 to smithi11881747-1 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2264: finishing copy_from racing read to smithi11881747-45 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2264: finishing copy_from to smithi11881747-45 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 23093 (ObjNum 685 snap 225 seq_num 685) dirty exists 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2268: finishing rollback tid 0 to smithi11881747-17 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2268: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2268: finishing rollback tid 2 to smithi11881747-17 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7056 (ObjNum 568 snap 185 seq_num 568) dirty exists 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2267: finishing write tid 3 to smithi11881747-1 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2267: finishing write tid 4 to smithi11881747-1 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2267: finishing write tid 5 to smithi11881747-1 2024-03-20T08:09:17.286 INFO:tasks.rados.rados.0.smithi118.stdout:2267: finishing write tid 6 to smithi11881747-1 2024-03-20T08:09:17.287 INFO:tasks.rados.rados.0.smithi118.stdout:2267: oid 1 updating version 0 to 34014 2024-03-20T08:09:17.287 INFO:tasks.rados.rados.0.smithi118.stdout:2267: oid 1 version 34014 is already newer than 34012 2024-03-20T08:09:17.287 INFO:tasks.rados.rados.0.smithi118.stdout:2267: oid 1 updating version 34014 to 34015 2024-03-20T08:09:17.287 INFO:tasks.rados.rados.0.smithi118.stdout:2267: oid 1 version 34015 is already newer than 34011 2024-03-20T08:09:17.287 INFO:tasks.rados.rados.0.smithi118.stdout:2267: oid 1 version 34015 is already newer than 34013 2024-03-20T08:09:17.287 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 34015 (ObjNum 702 snap 235 seq_num 702) dirty exists 2024-03-20T08:09:17.287 INFO:tasks.rados.rados.0.smithi118.stdout:2267: left oid 1 (ObjNum 702 snap 235 seq_num 702) 2024-03-20T08:09:17.288 INFO:tasks.rados.rados.0.smithi118.stdout:2264: done (3 left) 2024-03-20T08:09:17.288 INFO:tasks.rados.rados.0.smithi118.stdout:2267: done (2 left) 2024-03-20T08:09:17.288 INFO:tasks.rados.rados.0.smithi118.stdout:2268: done (1 left) 2024-03-20T08:09:17.288 INFO:tasks.rados.rados.0.smithi118.stdout:2270: read oid 5 snap -1 2024-03-20T08:09:17.288 INFO:tasks.rados.rados.0.smithi118.stdout:2270: expect deleted 2024-03-20T08:09:17.288 INFO:tasks.rados.rados.0.smithi118.stdout:2271: snap_remove snap 175 2024-03-20T08:09:17.335 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:17.225+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:17.335 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:17.231+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: osdmap e523: 8 total, 8 up, 8 in 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: pgmap v464: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 703 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 8.2 MiB/s rd, 712 KiB/s wr, 3 op/s 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:17.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:17.337 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/1697064471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.131+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.166+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.189+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.228+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.263+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.275+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.300+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:17.222+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:17.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:17.224+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:17.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:17.290+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:17.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:17.222+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:17.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:17.223+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:17.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:17.223+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:17.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:17.223+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:17.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: osdmap e523: 8 total, 8 up, 8 in 2024-03-20T08:09:17.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:17.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: pgmap v464: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 703 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 8.2 MiB/s rd, 712 KiB/s wr, 3 op/s 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:17.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/1697064471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:17.425 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:17.225+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: osdmap e523: 8 total, 8 up, 8 in 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: pgmap v464: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 703 MiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 8.2 MiB/s rd, 712 KiB/s wr, 3 op/s 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:17.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:17.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:17.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:17 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/1697064471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:17.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.361+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.385+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.397+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.421+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.445+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.493+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.505+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.530+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.590 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.552+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.591 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.849 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.654+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.665+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.701+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.751+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.764+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.776+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.787+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.849+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.111 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.932+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:17.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.028+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.039+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.112 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.064+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.113 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.076+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.113 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.113 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.113 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.111+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.224 INFO:tasks.rados.rados.0.smithi118.stdout:2271: done (2 left) 2024-03-20T08:09:18.224 INFO:tasks.rados.rados.0.smithi118.stdout:2272: read oid 44 snap -1 2024-03-20T08:09:18.224 INFO:tasks.rados.rados.0.smithi118.stdout:2272: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:09:18.224 INFO:tasks.rados.rados.0.smithi118.stdout:2273: rmattr oid 12 current snap is 235 2024-03-20T08:09:18.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 33917 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:09:18.224 INFO:tasks.rados.rados.0.smithi118.stdout:2269: done (3 left) 2024-03-20T08:09:18.225 INFO:tasks.rados.rados.0.smithi118.stdout:2270: done (2 left) 2024-03-20T08:09:18.225 INFO:tasks.rados.rados.0.smithi118.stdout:2274: snap_create 2024-03-20T08:09:18.236 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34002 (ObjNum 600 snap 196 seq_num 600) dirty exists 2024-03-20T08:09:18.247 INFO:tasks.rados.rados.0.smithi118.stdout:2272: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:09:18.368 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:18.236+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:18.368 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:18.236+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: osdmap e524: 8 total, 8 up, 8 in 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:18.369 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:18.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:18.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:18.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:18.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:18.370 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:18.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.135+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.147+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.185+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.231+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.231+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.248+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.260+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.284+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.332+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.344+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.355+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.381+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.419+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.431+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.443+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.455+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.479+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.560+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.571+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.582+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.605+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.617+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:18.230+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:18.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:18.230+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:18.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:18.230+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:18.227+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:18.228+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:18.230+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:18.230+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:18.233+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:18.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:18.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: osdmap e524: 8 total, 8 up, 8 in 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:18.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: osdmap e524: 8 total, 8 up, 8 in 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 113=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:18.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:18.227+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:18.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:18.446+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:18.677 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:18.446+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:18.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.717+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.764+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.775+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.788+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.823+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.846+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.858+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:18.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.905+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.916+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.954+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:18.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.016+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.050+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.074+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.086+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.098+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.109+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.122+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.230 INFO:tasks.rados.rados.0.smithi118.stdout:2272: done (2 left) 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:2273: done (1 left) 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:2274: done (0 left) 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:2275: read oid 39 snap 233 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:2275: expect (ObjNum 668 snap 217 seq_num 668) 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:2276: rollback oid 41 current snap is 236 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 41 to 227 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:2277: copy_from oid 40 from oid 12 current snap is 236 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 236 2024-03-20T08:09:19.231 INFO:tasks.rados.rados.0.smithi118.stdout:2278: seq_num 703 ranges {2826240=581632,3407872=458752,3866624=557056,4423680=417792,4841472=172032} 2024-03-20T08:09:19.250 INFO:tasks.rados.rados.0.smithi118.stdout:2278: writing smithi11881747-1 from 2826240 to 3407872 tid 1 2024-03-20T08:09:19.253 INFO:tasks.rados.rados.0.smithi118.stdout:2278: writing smithi11881747-1 from 3407872 to 3866624 tid 2 2024-03-20T08:09:19.256 INFO:tasks.rados.rados.0.smithi118.stdout:2278: writing smithi11881747-1 from 3866624 to 4423680 tid 3 2024-03-20T08:09:19.259 INFO:tasks.rados.rados.0.smithi118.stdout:2278: writing smithi11881747-1 from 4423680 to 4841472 tid 4 2024-03-20T08:09:19.260 INFO:tasks.rados.rados.0.smithi118.stdout:2278: writing smithi11881747-1 from 4841472 to 5013504 tid 5 2024-03-20T08:09:19.260 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 236 2024-03-20T08:09:19.261 INFO:tasks.rados.rados.0.smithi118.stdout:2276: finishing rollback tid 1 to smithi11881747-41 2024-03-20T08:09:19.261 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 33923 (ObjNum 177 snap 0 seq_num 0) dirty dne 2024-03-20T08:09:19.261 INFO:tasks.rados.rados.0.smithi118.stdout:2279: seq_num 704 ranges {0=598016,598016=745472,1343488=712704,2056192=147456} 2024-03-20T08:09:19.263 INFO:tasks.rados.rados.0.smithi118.stdout:2279: writing smithi11881747-11 from 0 to 598016 tid 1 2024-03-20T08:09:19.268 INFO:tasks.rados.rados.0.smithi118.stdout:2279: writing smithi11881747-11 from 598016 to 1343488 tid 2 2024-03-20T08:09:19.273 INFO:tasks.rados.rados.0.smithi118.stdout:2279: writing smithi11881747-11 from 1343488 to 2056192 tid 3 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2279: writing smithi11881747-11 from 2056192 to 2203648 tid 4 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2276: done (4 left) 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2280: read oid 35 snap -1 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2280: expect (ObjNum 677 snap 225 seq_num 677) 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2281: rmattr oid 6 current snap is 236 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2281: done (5 left) 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2282: snap_create 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2278: finishing write tid 1 to smithi11881747-1 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2278: finishing write tid 2 to smithi11881747-1 2024-03-20T08:09:19.275 INFO:tasks.rados.rados.0.smithi118.stdout:2275: expect (ObjNum 668 snap 217 seq_num 668) 2024-03-20T08:09:19.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:19.234+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:19.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:19.234+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:19.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: osdmap e525: 8 total, 8 up, 8 in 2024-03-20T08:09:19.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:19.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:19.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:19.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:19.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:19.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:19.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:19.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:19.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:19.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:19.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-mon[87080]: pgmap v467: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 716 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 9.4 MiB/s rd, 2.4 MiB/s wr, 14 op/s 2024-03-20T08:09:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.146+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.182+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.193+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.218+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.234+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.235+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.250+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.290+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.306+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.390+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: finishing write tid 3 to smithi11881747-1 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: finishing write tid 4 to smithi11881747-1 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: finishing write tid 5 to smithi11881747-1 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: finishing write tid 6 to smithi11881747-1 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: finishing write tid 7 to smithi11881747-1 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: oid 1 updating version 0 to 34017 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: oid 1 updating version 34017 to 34020 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: oid 1 version 34020 is already newer than 34016 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: oid 1 version 34020 is already newer than 34018 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: oid 1 version 34020 is already newer than 34019 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:2278: oid 1 updating version 34020 to 34021 2024-03-20T08:09:19.568 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 34021 (ObjNum 703 snap 236 seq_num 703) dirty exists 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2278: left oid 1 (ObjNum 703 snap 236 seq_num 703) 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: finishing write tid 1 to smithi11881747-11 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: finishing write tid 2 to smithi11881747-11 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: finishing write tid 3 to smithi11881747-11 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2277: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: finishing write tid 4 to smithi11881747-11 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2277: finishing copy_from to smithi11881747-40 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34002 (ObjNum 600 snap 196 seq_num 600) dirty exists 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: finishing write tid 5 to smithi11881747-11 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: finishing write tid 6 to smithi11881747-11 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: oid 11 updating version 0 to 23082 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: oid 11 updating version 23082 to 23085 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: oid 11 version 23085 is already newer than 23083 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: oid 11 version 23085 is already newer than 23084 2024-03-20T08:09:19.569 INFO:tasks.rados.rados.0.smithi118.stdout:2279: oid 11 updating version 23085 to 23086 2024-03-20T08:09:19.570 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 23086 (ObjNum 704 snap 236 seq_num 704) dirty exists 2024-03-20T08:09:19.570 INFO:tasks.rados.rados.0.smithi118.stdout:2279: left oid 11 (ObjNum 704 snap 236 seq_num 704) 2024-03-20T08:09:19.570 INFO:tasks.rados.rados.0.smithi118.stdout:2280: expect (ObjNum 677 snap 225 seq_num 677) 2024-03-20T08:09:19.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.431+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.444+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.457+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.470+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.537+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.647 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.647 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.647 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.647 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.590+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.647 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.603+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.647 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.617+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.648 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.631+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.648 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:19.232+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:19.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:19.232+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:19.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:19.234+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:19.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:19.234+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:19.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:19.234+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:19.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:19.235+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: osdmap e525: 8 total, 8 up, 8 in 2024-03-20T08:09:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:19.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:19.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:19.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:19.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:19.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:19.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:19.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:19.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[93121]: pgmap v467: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 716 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 9.4 MiB/s rd, 2.4 MiB/s wr, 14 op/s 2024-03-20T08:09:19.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:19.239+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:19.677 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:19.241+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: osdmap e525: 8 total, 8 up, 8 in 2024-03-20T08:09:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:19.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:19.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:19.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:19.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:19.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:19.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:19.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:19.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:19.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:19.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:19 smithi118 ceph-mon[95692]: pgmap v467: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 716 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 9.4 MiB/s rd, 2.4 MiB/s wr, 14 op/s 2024-03-20T08:09:19.906 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.906 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.671+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.906 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.685+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.906 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.698+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.794+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.819+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.863+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.876+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:19.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.165 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.922+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.165 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.165 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.165 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.962+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.165 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.973+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.986+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:19.997+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.021+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.032+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.043+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.055+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.079+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.092+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.116+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.140+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.165+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2275: done (5 left) 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2277: done (4 left) 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2278: done (3 left) 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2279: done (2 left) 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2280: done (1 left) 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2282: done (0 left) 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2283: read oid 44 snap 228 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2283: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2284: read oid 50 snap -1 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2284: expect (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2285: rmattr oid 6 current snap is 237 2024-03-20T08:09:20.236 INFO:tasks.rados.rados.0.smithi118.stdout:2285: done (2 left) 2024-03-20T08:09:20.237 INFO:tasks.rados.rados.0.smithi118.stdout:2286: snap_create 2024-03-20T08:09:20.255 INFO:tasks.rados.rados.0.smithi118.stdout:2283: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:09:20.351 INFO:tasks.rados.rados.0.smithi118.stdout:2284: expect (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:09:20.422 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:20.244+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:20.422 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:20.244+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: osdmap e526: 8 total, 8 up, 8 in 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:20.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:20.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:20.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:20.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:20.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:20.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-mon[87080]: Upgrade: osd.7 is safe to restart 2024-03-20T08:09:20.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.209+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.223+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.239+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.240+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:20.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.240+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:20.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.267+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.342+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.396+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.425 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.422+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:20.241+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:20.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:20.241+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:20.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:20.241+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: osdmap e526: 8 total, 8 up, 8 in 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[93121]: Upgrade: osd.7 is safe to restart 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: osdmap e526: 8 total, 8 up, 8 in 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:20.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["7"], "max": 16}]: dispatch 2024-03-20T08:09:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:20 smithi118 ceph-mon[95692]: Upgrade: osd.7 is safe to restart 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.451+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.465+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.492+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.548+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.560+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.570+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.583+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.595+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.630+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.642+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.738+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.751+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.763+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.774+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.811+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.834+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.871+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.895+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.919+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:20.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.932+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.944+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.978+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:20.990+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.051+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.148+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.160+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.184+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.241 INFO:tasks.rados.rados.0.smithi118.stdout:2283: done (2 left) 2024-03-20T08:09:21.241 INFO:tasks.rados.rados.0.smithi118.stdout:2284: done (1 left) 2024-03-20T08:09:21.241 INFO:tasks.rados.rados.0.smithi118.stdout:2286: done (0 left) 2024-03-20T08:09:21.241 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 238 2024-03-20T08:09:21.242 INFO:tasks.rados.rados.0.smithi118.stdout:2287: seq_num 705 ranges {0=614400,614400=630784,1245184=524288,1769472=450560} 2024-03-20T08:09:21.244 INFO:tasks.rados.rados.0.smithi118.stdout:2287: writing smithi11881747-41 from 0 to 614400 tid 1 2024-03-20T08:09:21.249 INFO:tasks.rados.rados.0.smithi118.stdout:2287: writing smithi11881747-41 from 614400 to 1245184 tid 2 2024-03-20T08:09:21.252 INFO:tasks.rados.rados.0.smithi118.stdout:2287: writing smithi11881747-41 from 1245184 to 1769472 tid 3 2024-03-20T08:09:21.255 INFO:tasks.rados.rados.0.smithi118.stdout:2287: writing smithi11881747-41 from 1769472 to 2220032 tid 4 2024-03-20T08:09:21.255 INFO:tasks.rados.rados.0.smithi118.stdout:2288: snap_remove snap 228 2024-03-20T08:09:21.444 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:21.245+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:21.445 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:21.245+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-mon[87080]: osdmap e527: 8 total, 8 up, 8 in 2024-03-20T08:09:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:21.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-mon[87080]: pgmap v470: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 721 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 25 MiB/s rd, 4.4 MiB/s wr, 30 op/s 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.247+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.263+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.322+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.359+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.396+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.408+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.421+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.445+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:21.245+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:21.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:21.245+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:21.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:21.241+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:21.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:21.241+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:21.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[93121]: osdmap e527: 8 total, 8 up, 8 in 2024-03-20T08:09:21.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:21.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:21.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:21.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[93121]: pgmap v470: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 721 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 25 MiB/s rd, 4.4 MiB/s wr, 30 op/s 2024-03-20T08:09:21.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[95692]: osdmap e527: 8 total, 8 up, 8 in 2024-03-20T08:09:21.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-mon[95692]: pgmap v470: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 721 MiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 25 MiB/s rd, 4.4 MiB/s wr, 30 op/s 2024-03-20T08:09:21.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:21.248+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:21.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:21.248+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:21.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:21.249+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:21.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:21.249+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:21.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:21.250+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:21.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.457+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.470+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.520+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.544+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.556+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.581+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.593+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.630+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.656+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:21.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.009 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.716+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.009 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.009 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.009 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.009 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.010 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.010 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.789+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.010 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.801+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.010 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.813+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.010 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.836+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.848+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.885+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:21.766+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:22.246 INFO:tasks.rados.rados.0.smithi118.stdout:2288: done (1 left) 2024-03-20T08:09:22.246 INFO:tasks.rados.rados.0.smithi118.stdout:2289: delete oid 10 current snap is 238 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: finishing write tid 1 to smithi11881747-41 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: finishing write tid 2 to smithi11881747-41 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: finishing write tid 3 to smithi11881747-41 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: finishing write tid 4 to smithi11881747-41 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: finishing write tid 5 to smithi11881747-41 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: finishing write tid 6 to smithi11881747-41 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: oid 41 updating version 0 to 33942 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: oid 41 version 33942 is already newer than 33940 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: oid 41 updating version 33942 to 33943 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: oid 41 version 33943 is already newer than 33941 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: oid 41 version 33943 is already newer than 33939 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 33943 (ObjNum 705 snap 238 seq_num 705) dirty exists 2024-03-20T08:09:22.247 INFO:tasks.rados.rados.0.smithi118.stdout:2287: left oid 41 (ObjNum 705 snap 238 seq_num 705) 2024-03-20T08:09:22.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: osdmap e528: 8 total, 8 up, 8 in 2024-03-20T08:09:22.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:22.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:22.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:22.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:22.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:22.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:22.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:22.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:22.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:22.259 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:22.260 INFO:tasks.rados.rados.0.smithi118.stdout:2287: done (1 left) 2024-03-20T08:09:22.260 INFO:tasks.rados.rados.0.smithi118.stdout:2289: done (0 left) 2024-03-20T08:09:22.260 INFO:tasks.rados.rados.0.smithi118.stdout:2290: setattr oid 9 current snap is 238 2024-03-20T08:09:22.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:21.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.011+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.145+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.261 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.209+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.256+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:22.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.256+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:22.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.263 INFO:tasks.rados.rados.0.smithi118.stdout:2291: rollback oid 23 current snap is 238 2024-03-20T08:09:22.263 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 23 to 218 2024-03-20T08:09:22.263 INFO:tasks.rados.rados.0.smithi118.stdout:2292: rollback oid 5 current snap is 238 2024-03-20T08:09:22.263 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 169 2024-03-20T08:09:22.263 INFO:tasks.rados.rados.0.smithi118.stdout:2293: rmattr oid 22 current snap is 238 2024-03-20T08:09:22.263 INFO:tasks.rados.rados.0.smithi118.stdout:2293: done (3 left) 2024-03-20T08:09:22.263 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 238 2024-03-20T08:09:22.263 INFO:tasks.rados.rados.0.smithi118.stdout:2294: seq_num 707 ranges {4751360=647168,5398528=802816,6201344=532480,6733824=270336} 2024-03-20T08:09:22.291 INFO:tasks.rados.rados.0.smithi118.stdout:2294: writing smithi11881747-27 from 4751360 to 5398528 tid 1 2024-03-20T08:09:22.296 INFO:tasks.rados.rados.0.smithi118.stdout:2294: writing smithi11881747-27 from 5398528 to 6201344 tid 2 2024-03-20T08:09:22.299 INFO:tasks.rados.rados.0.smithi118.stdout:2294: writing smithi11881747-27 from 6201344 to 6733824 tid 3 2024-03-20T08:09:22.301 INFO:tasks.rados.rados.0.smithi118.stdout:2294: writing smithi11881747-27 from 6733824 to 7004160 tid 4 2024-03-20T08:09:22.301 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 238 2024-03-20T08:09:22.302 INFO:tasks.rados.rados.0.smithi118.stdout:2291: finishing rollback tid 0 to smithi11881747-23 2024-03-20T08:09:22.302 INFO:tasks.rados.rados.0.smithi118.stdout:2295: seq_num 708 ranges {1122304=663552,1785856=737280,2523136=696320,3219456=172032} 2024-03-20T08:09:22.311 INFO:tasks.rados.rados.0.smithi118.stdout:2295: writing smithi11881747-26 from 1122304 to 1785856 tid 1 2024-03-20T08:09:22.315 INFO:tasks.rados.rados.0.smithi118.stdout:2295: writing smithi11881747-26 from 1785856 to 2523136 tid 2 2024-03-20T08:09:22.320 INFO:tasks.rados.rados.0.smithi118.stdout:2295: writing smithi11881747-26 from 2523136 to 3219456 tid 3 2024-03-20T08:09:22.321 INFO:tasks.rados.rados.0.smithi118.stdout:2295: writing smithi11881747-26 from 3219456 to 3391488 tid 4 2024-03-20T08:09:22.321 INFO:tasks.rados.rados.0.smithi118.stdout:2296: read oid 31 snap -1 2024-03-20T08:09:22.321 INFO:tasks.rados.rados.0.smithi118.stdout:2296: expect (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2297: snap_create 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2291: finishing rollback tid 1 to smithi11881747-23 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2291: finishing rollback tid 2 to smithi11881747-23 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 33946 (ObjNum 626 snap 203 seq_num 626) dirty exists 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 23087 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2292: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2292: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 23089 (ObjNum 495 snap 164 seq_num 495) dirty exists 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2294: finishing write tid 1 to smithi11881747-27 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2294: finishing write tid 2 to smithi11881747-27 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2294: finishing write tid 3 to smithi11881747-27 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2294: finishing write tid 4 to smithi11881747-27 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2294: finishing write tid 5 to smithi11881747-27 2024-03-20T08:09:22.322 INFO:tasks.rados.rados.0.smithi118.stdout:2294: finishing write tid 6 to smithi11881747-27 2024-03-20T08:09:22.323 INFO:tasks.rados.rados.0.smithi118.stdout:2294: oid 27 updating version 0 to 13619 2024-03-20T08:09:22.323 INFO:tasks.rados.rados.0.smithi118.stdout:2294: oid 27 updating version 13619 to 13621 2024-03-20T08:09:22.323 INFO:tasks.rados.rados.0.smithi118.stdout:2294: oid 27 version 13621 is already newer than 13620 2024-03-20T08:09:22.323 INFO:tasks.rados.rados.0.smithi118.stdout:2294: oid 27 version 13621 is already newer than 13617 2024-03-20T08:09:22.323 INFO:tasks.rados.rados.0.smithi118.stdout:2294: oid 27 version 13621 is already newer than 13618 2024-03-20T08:09:22.323 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 13621 (ObjNum 707 snap 238 seq_num 707) dirty exists 2024-03-20T08:09:22.323 INFO:tasks.rados.rados.0.smithi118.stdout:2294: left oid 27 (ObjNum 707 snap 238 seq_num 707) 2024-03-20T08:09:22.343 INFO:tasks.rados.rados.0.smithi118.stdout:2295: finishing write tid 1 to smithi11881747-26 2024-03-20T08:09:22.347 INFO:tasks.rados.rados.0.smithi118.stdout:2295: finishing write tid 2 to smithi11881747-26 2024-03-20T08:09:22.348 INFO:tasks.rados.rados.0.smithi118.stdout:2295: finishing write tid 3 to smithi11881747-26 2024-03-20T08:09:22.348 INFO:tasks.rados.rados.0.smithi118.stdout:2295: finishing write tid 4 to smithi11881747-26 2024-03-20T08:09:22.348 INFO:tasks.rados.rados.0.smithi118.stdout:2295: finishing write tid 5 to smithi11881747-26 2024-03-20T08:09:22.352 INFO:tasks.rados.rados.0.smithi118.stdout:2295: finishing write tid 6 to smithi11881747-26 2024-03-20T08:09:22.353 INFO:tasks.rados.rados.0.smithi118.stdout:2295: oid 26 updating version 0 to 34004 2024-03-20T08:09:22.353 INFO:tasks.rados.rados.0.smithi118.stdout:2295: oid 26 updating version 34004 to 34005 2024-03-20T08:09:22.353 INFO:tasks.rados.rados.0.smithi118.stdout:2295: oid 26 updating version 34005 to 34007 2024-03-20T08:09:22.353 INFO:tasks.rados.rados.0.smithi118.stdout:2295: oid 26 version 34007 is already newer than 34003 2024-03-20T08:09:22.353 INFO:tasks.rados.rados.0.smithi118.stdout:2295: oid 26 version 34007 is already newer than 34006 2024-03-20T08:09:22.353 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 34007 (ObjNum 708 snap 238 seq_num 708) dirty exists 2024-03-20T08:09:22.353 INFO:tasks.rados.rados.0.smithi118.stdout:2295: left oid 26 (ObjNum 708 snap 238 seq_num 708) 2024-03-20T08:09:22.358 INFO:tasks.rados.rados.0.smithi118.stdout:2296: expect (ObjNum 522 snap 169 seq_num 522) 2024-03-20T08:09:22.512 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:09:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:09:22.512 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:22.257+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:22.512 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:22.257+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:22.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.274+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.512 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.287+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.300+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.315+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.353+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.361+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.455+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:22.255+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:22.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:22.256+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:22.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:22.256+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:22.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: osdmap e528: 8 total, 8 up, 8 in 2024-03-20T08:09:22.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:22.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:22.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:22.254+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:22.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:22.254+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:22.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:22.254+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:22.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:22.254+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: osdmap e528: 8 total, 8 up, 8 in 2024-03-20T08:09:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:22.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:22.249+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:22.677 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:22.269+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:22.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.526+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.539+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.581+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.609+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.636+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.664+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.690+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.716+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.730+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.758+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:22.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.800+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.869+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.897+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.925+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.944+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:22.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.273 INFO:tasks.rados.rados.0.smithi118.stdout:2290: done (6 left) 2024-03-20T08:09:23.273 INFO:tasks.rados.rados.0.smithi118.stdout:2291: done (5 left) 2024-03-20T08:09:23.273 INFO:tasks.rados.rados.0.smithi118.stdout:2292: done (4 left) 2024-03-20T08:09:23.273 INFO:tasks.rados.rados.0.smithi118.stdout:2294: done (3 left) 2024-03-20T08:09:23.274 INFO:tasks.rados.rados.0.smithi118.stdout:2295: done (2 left) 2024-03-20T08:09:23.274 INFO:tasks.rados.rados.0.smithi118.stdout:2296: done (1 left) 2024-03-20T08:09:23.274 INFO:tasks.rados.rados.0.smithi118.stdout:2297: done (0 left) 2024-03-20T08:09:23.274 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 239 2024-03-20T08:09:23.274 INFO:tasks.rados.rados.0.smithi118.stdout:2298: seq_num 709 ranges {4792320=679936,5472256=614400,6086656=499712,6586368=491520} 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: osdmap e529: 8 total, 8 up, 8 in 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:23.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: pgmap v473: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 723 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 18 op/s 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: Upgrade: Updating osd.7 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-mon[87080]: Deploying daemon osd.7 on smithi196 2024-03-20T08:09:23.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.052+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.065+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.089+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.101+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.112+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.150+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.167+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.181+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.224+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.281 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:23.276+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:23.303 INFO:tasks.rados.rados.0.smithi118.stdout:2298: writing smithi11881747-6 from 4792320 to 5472256 tid 1 2024-03-20T08:09:23.307 INFO:tasks.rados.rados.0.smithi118.stdout:2298: writing smithi11881747-6 from 5472256 to 6086656 tid 2 2024-03-20T08:09:23.310 INFO:tasks.rados.rados.0.smithi118.stdout:2298: writing smithi11881747-6 from 6086656 to 6586368 tid 3 2024-03-20T08:09:23.313 INFO:tasks.rados.rados.0.smithi118.stdout:2298: writing smithi11881747-6 from 6586368 to 7077888 tid 4 2024-03-20T08:09:23.313 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 239 2024-03-20T08:09:23.313 INFO:tasks.rados.rados.0.smithi118.stdout:2299: seq_num 710 ranges {1581056=696320,2277376=499712,2777088=712704,3489792=393216} 2024-03-20T08:09:23.326 INFO:tasks.rados.rados.0.smithi118.stdout:2299: writing smithi11881747-36 from 1581056 to 2277376 tid 1 2024-03-20T08:09:23.329 INFO:tasks.rados.rados.0.smithi118.stdout:2299: writing smithi11881747-36 from 2277376 to 2777088 tid 2 2024-03-20T08:09:23.334 INFO:tasks.rados.rados.0.smithi118.stdout:2299: writing smithi11881747-36 from 2777088 to 3489792 tid 3 2024-03-20T08:09:23.337 INFO:tasks.rados.rados.0.smithi118.stdout:2299: writing smithi11881747-36 from 3489792 to 3883008 tid 4 2024-03-20T08:09:23.337 INFO:tasks.rados.rados.0.smithi118.stdout:2300: snap_create 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: finishing write tid 1 to smithi11881747-6 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: finishing write tid 2 to smithi11881747-6 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: finishing write tid 3 to smithi11881747-6 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: finishing write tid 4 to smithi11881747-6 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: finishing write tid 5 to smithi11881747-6 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: finishing write tid 6 to smithi11881747-6 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: oid 6 updating version 0 to 23098 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: oid 6 version 23098 is already newer than 23097 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: oid 6 version 23098 is already newer than 23094 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: oid 6 version 23098 is already newer than 23096 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: oid 6 version 23098 is already newer than 23095 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 23098 (ObjNum 709 snap 239 seq_num 709) dirty exists 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2298: left oid 6 (ObjNum 709 snap 239 seq_num 709) 2024-03-20T08:09:23.338 INFO:tasks.rados.rados.0.smithi118.stdout:2299: finishing write tid 1 to smithi11881747-36 2024-03-20T08:09:23.339 INFO:tasks.rados.rados.0.smithi118.stdout:2299: finishing write tid 2 to smithi11881747-36 2024-03-20T08:09:23.347 INFO:tasks.rados.rados.0.smithi118.stdout:2299: finishing write tid 3 to smithi11881747-36 2024-03-20T08:09:23.347 INFO:tasks.rados.rados.0.smithi118.stdout:2299: finishing write tid 4 to smithi11881747-36 2024-03-20T08:09:23.347 INFO:tasks.rados.rados.0.smithi118.stdout:2299: finishing write tid 5 to smithi11881747-36 2024-03-20T08:09:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:2299: finishing write tid 6 to smithi11881747-36 2024-03-20T08:09:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:2299: oid 36 updating version 0 to 33918 2024-03-20T08:09:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:2299: oid 36 version 33918 is already newer than 33915 2024-03-20T08:09:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:2299: oid 36 version 33918 is already newer than 33916 2024-03-20T08:09:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:2299: oid 36 version 33918 is already newer than 33917 2024-03-20T08:09:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:2299: oid 36 updating version 33918 to 33919 2024-03-20T08:09:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 33919 (ObjNum 710 snap 239 seq_num 710) dirty exists 2024-03-20T08:09:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:2299: left oid 36 (ObjNum 710 snap 239 seq_num 710) 2024-03-20T08:09:23.538 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:23.276+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:23.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.279+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.304+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.331+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.347+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.361+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.424+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.500+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.538+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:23.274+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:23.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:23.275+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:23.586 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:23.273+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:23.586 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:23.273+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:23.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:23.466+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:23.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:23.466+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:23.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:23.466+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:23.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:23.466+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:23.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:23.466+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:23.587 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:23.309+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:23.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:23.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: osdmap e529: 8 total, 8 up, 8 in 2024-03-20T08:09:23.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:23.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:23.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:23.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: pgmap v473: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 723 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 18 op/s 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: Upgrade: Updating osd.7 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:23.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[93121]: Deploying daemon osd.7 on smithi196 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: osdmap e529: 8 total, 8 up, 8 in 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:23.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:23.590 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:23.590 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: pgmap v473: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 723 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 19 MiB/s rd, 2.6 MiB/s wr, 18 op/s 2024-03-20T08:09:23.590 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: Upgrade: Updating osd.7 2024-03-20T08:09:23.590 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:23.590 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:23.590 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-20T08:09:23.590 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:23.590 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-mon[95692]: Deploying daemon osd.7 on smithi196 2024-03-20T08:09:23.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.600+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.635+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.646+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.659+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.725+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.737+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.749+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.760+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.783+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:09:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:09:23] "GET /metrics HTTP/1.1" 200 36335 "" "Prometheus/2.43.0" 2024-03-20T08:09:24.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.819+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.831+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.844+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.868+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.893+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.905+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.953+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:23.988+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.040+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2298: done (2 left) 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2299: done (1 left) 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2300: done (0 left) 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2301: rmattr oid 11 current snap is 240 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2301: done (0 left) 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2302: read oid 45 snap -1 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2302: expect (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2303: copy_from oid 30 from oid 9 current snap is 240 2024-03-20T08:09:24.280 INFO:tasks.rados.rados.0.smithi118.stdout:2304: snap_create 2024-03-20T08:09:24.286 INFO:tasks.rados.rados.0.smithi118.stdout:2303: finishing copy_from racing read to smithi11881747-30 2024-03-20T08:09:24.287 INFO:tasks.rados.rados.0.smithi118.stdout:2303: finishing copy_from to smithi11881747-30 2024-03-20T08:09:24.287 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 23087 (ObjNum 97 snap 0 seq_num 322634160) dirty exists 2024-03-20T08:09:24.293 INFO:tasks.rados.rados.0.smithi118.stdout:2302: expect (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:09:24.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:24.282+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:24.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:24.282+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: osdmap e530: 8 total, 8 up, 8 in 2024-03-20T08:09:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:24.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.065+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.091+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.102+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.125+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.163+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.198+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.222+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.235+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.258+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.284+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:24.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.284+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:24.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.286+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:24.277+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:24.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:24.280+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:24.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:24.280+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:24.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:24.281+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:24.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:24.282+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:24.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:24.283+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:24.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:24.283+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:24.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:24.288+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: osdmap e530: 8 total, 8 up, 8 in 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: osdmap e530: 8 total, 8 up, 8 in 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:24.713 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.311+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.713 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.322+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.713 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.713 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.406+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.431+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.455+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.479+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.515+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.528+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.541+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.714 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.602+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.715 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.724+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.748+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.760+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.782+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.820+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.846+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.872+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.908+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.932+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.947+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.961+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:24.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.975+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:24.909+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:25.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:24.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.016+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.030+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.059+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.074+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.087+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.112+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.124+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.136+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.148+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.239 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.172+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.185+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.198+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.210+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:2302: done (2 left) 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:2303: done (1 left) 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:2304: done (0 left) 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:2305: rollback oid 30 current snap is 241 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 239 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:2306: rollback oid 36 current snap is 241 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 36 to 232 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:2307: read oid 1 snap -1 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:2307: expect (ObjNum 703 snap 236 seq_num 703) 2024-03-20T08:09:25.284 INFO:tasks.rados.rados.0.smithi118.stdout:2308: setattr oid 21 current snap is 241 2024-03-20T08:09:25.286 INFO:tasks.rados.rados.0.smithi118.stdout:2309: snap_remove snap 204 2024-03-20T08:09:25.490 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:25.296+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:25.490 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:25.296+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: pgmap v475: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 739 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 7 op/s 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: osdmap e531: 8 total, 8 up, 8 in 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:25.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:25.492 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.250+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.261+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.273+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.293+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.325+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.337+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.365+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.492 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.379+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.405+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.417+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.454+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.466+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.493 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:25.286+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:25.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:25.296+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:25.309+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:25.309+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:25.310+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:25.310+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:25.310+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:25.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: pgmap v475: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 739 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 7 op/s 2024-03-20T08:09:25.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: osdmap e531: 8 total, 8 up, 8 in 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:25.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: pgmap v475: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 739 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 7 op/s 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: osdmap e531: 8 total, 8 up, 8 in 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:25.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:25.288+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:25.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:25.288+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:25.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:25.308+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:25.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.504+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.566+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.635+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.662+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.675+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.689+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.992 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.757+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.834+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.847+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.902+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.917+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.969+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:25.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:25.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.028+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.064+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.100+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.146+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.160+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.174+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.199+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.210+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.222+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.235+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.246+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.291 INFO:tasks.rados.rados.0.smithi118.stdout:2309: done (4 left) 2024-03-20T08:09:26.291 INFO:tasks.rados.rados.0.smithi118.stdout:2310: read oid 34 snap -1 2024-03-20T08:09:26.291 INFO:tasks.rados.rados.0.smithi118.stdout:2310: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:09:26.291 INFO:tasks.rados.rados.0.smithi118.stdout:2311: snap_remove snap 215 2024-03-20T08:09:26.503 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:26.294+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:26.503 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:26.294+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:26.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:26.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: osdmap e532: 8 total, 8 up, 8 in 2024-03-20T08:09:26.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:26.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.295+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.296+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.310+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.322+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.347+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.359+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.396+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.408+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.421+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.446+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.458+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.493+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:26.291+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:26.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:26.293+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:26.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:26.299+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:26.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:26.299+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:26.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:26.288+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:26.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:26.288+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:26.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:26.289+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:26.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:26.289+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: osdmap e532: 8 total, 8 up, 8 in 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:26.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: osdmap e532: 8 total, 8 up, 8 in 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:26.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.517+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.570+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.585+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.597+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.623+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.635+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.714+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:26.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.779+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.792+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.814+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.870+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.924+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.938+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.951+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.977+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:26.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.268 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:27 smithi196 systemd[1]: Stopping Ceph osd.7 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:09:27.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.268 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.063+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.076+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.129+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.269 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.198+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.212+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.270 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.290 INFO:tasks.rados.rados.0.smithi118.stdout:2311: done (5 left) 2024-03-20T08:09:27.290 INFO:tasks.rados.rados.0.smithi118.stdout:2312: rollback oid 18 current snap is 241 2024-03-20T08:09:27.290 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 18 to 235 2024-03-20T08:09:27.290 INFO:tasks.rados.rados.0.smithi118.stdout:2305: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:09:27.290 INFO:tasks.rados.rados.0.smithi118.stdout:2305: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:09:27.290 INFO:tasks.rados.rados.0.smithi118.stdout:2305: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:09:27.290 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 33920 (ObjNum 650 snap 213 seq_num 650) dirty exists 2024-03-20T08:09:27.291 INFO:tasks.rados.rados.0.smithi118.stdout:2306: finishing rollback tid 0 to smithi11881747-36 2024-03-20T08:09:27.291 INFO:tasks.rados.rados.0.smithi118.stdout:2306: finishing rollback tid 1 to smithi11881747-36 2024-03-20T08:09:27.291 INFO:tasks.rados.rados.0.smithi118.stdout:2306: finishing rollback tid 2 to smithi11881747-36 2024-03-20T08:09:27.291 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 33922 (ObjNum 667 snap 217 seq_num 667) dirty exists 2024-03-20T08:09:27.291 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 13857 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:09:27.291 INFO:tasks.rados.rados.0.smithi118.stdout:2307: expect (ObjNum 703 snap 236 seq_num 703) 2024-03-20T08:09:27.511 INFO:tasks.rados.rados.0.smithi118.stdout:2310: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:09:27.527 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:27.297+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:27.527 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:27.297+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:27.527 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[54676]: 2024-03-20T08:09:27.349+0000 7f102a44e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:09:27.527 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[54676]: 2024-03-20T08:09:27.349+0000 7f102a44e700 -1 osd.7 534 *** Got signal Terminated *** 2024-03-20T08:09:27.527 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[54676]: 2024-03-20T08:09:27.349+0000 7f102a44e700 -1 osd.7 534 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: pgmap v478: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 749 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 19 op/s 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: osdmap e533: 8 total, 8 up, 8 in 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:27.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:27.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:27.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.268+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.299+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:27.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.323+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.348+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.372+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.420+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.434+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.444+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.492+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:27.293+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:27.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:27.293+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:27.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:27.301+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:27.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:27.301+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:27.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:27.297+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:27.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:27.297+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:27.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:27.297+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:27.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:27.297+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:27.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:27.297+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:27.674 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:27.301+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: pgmap v478: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 749 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 19 op/s 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: osdmap e533: 8 total, 8 up, 8 in 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:27.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: pgmap v478: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 749 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 19 op/s 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: osdmap e533: 8 total, 8 up, 8 in 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:27.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:27.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:27.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:27.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:27.688 INFO:tasks.rados.rados.0.smithi118.stdout:2312: finishing rollback tid 1 to smithi11881747-18 2024-03-20T08:09:27.688 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 13613 (ObjNum 33 snap 0 seq_num 322988480) dirty dne 2024-03-20T08:09:27.688 INFO:tasks.rados.rados.0.smithi118.stdout:2305: done (5 left) 2024-03-20T08:09:27.688 INFO:tasks.rados.rados.0.smithi118.stdout:2306: done (4 left) 2024-03-20T08:09:27.688 INFO:tasks.rados.rados.0.smithi118.stdout:2307: done (3 left) 2024-03-20T08:09:27.688 INFO:tasks.rados.rados.0.smithi118.stdout:2308: done (2 left) 2024-03-20T08:09:27.688 INFO:tasks.rados.rados.0.smithi118.stdout:2310: done (1 left) 2024-03-20T08:09:27.689 INFO:tasks.rados.rados.0.smithi118.stdout:2312: done (0 left) 2024-03-20T08:09:27.689 INFO:tasks.rados.rados.0.smithi118.stdout:2313: delete oid 32 current snap is 241 2024-03-20T08:09:27.693 INFO:tasks.rados.rados.0.smithi118.stdout:2313: done (0 left) 2024-03-20T08:09:27.693 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 241 2024-03-20T08:09:27.693 INFO:tasks.rados.rados.0.smithi118.stdout:2314: seq_num 712 ranges {3006464=729088,3735552=671744,4407296=671744,5079040=262144} 2024-03-20T08:09:27.714 INFO:tasks.rados.rados.0.smithi118.stdout:2314: writing smithi11881747-34 from 3006464 to 3735552 tid 1 2024-03-20T08:09:27.718 INFO:tasks.rados.rados.0.smithi118.stdout:2314: writing smithi11881747-34 from 3735552 to 4407296 tid 2 2024-03-20T08:09:27.722 INFO:tasks.rados.rados.0.smithi118.stdout:2314: writing smithi11881747-34 from 4407296 to 5079040 tid 3 2024-03-20T08:09:27.724 INFO:tasks.rados.rados.0.smithi118.stdout:2314: writing smithi11881747-34 from 5079040 to 5341184 tid 4 2024-03-20T08:09:27.725 INFO:tasks.rados.rados.0.smithi118.stdout:2315: rmattr oid 44 current snap is 241 2024-03-20T08:09:27.725 INFO:tasks.rados.rados.0.smithi118.stdout:2314: finishing write tid 1 to smithi11881747-34 2024-03-20T08:09:27.725 INFO:tasks.rados.rados.0.smithi118.stdout:2316: rmattr oid 10 current snap is 241 2024-03-20T08:09:27.725 INFO:tasks.rados.rados.0.smithi118.stdout:2316: done (2 left) 2024-03-20T08:09:27.725 INFO:tasks.rados.rados.0.smithi118.stdout:2317: copy_from oid 23 from oid 37 current snap is 241 2024-03-20T08:09:27.725 INFO:tasks.rados.rados.0.smithi118.stdout:2318: copy_from oid 11 from oid 50 current snap is 241 2024-03-20T08:09:27.725 INFO:tasks.rados.rados.0.smithi118.stdout:2319: snap_remove snap 232 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.580+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.592+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.628+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.640+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.664+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.701+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.745+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:27.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.797+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.834+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.847+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.866+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.902+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.925+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.938+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.974+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:27.998+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.035+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.048+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.089+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.103+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.210+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.222+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.235+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.250+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2319: done (4 left) 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2320: rollback oid 1 current snap is 241 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 1 to 206 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2314: finishing write tid 2 to smithi11881747-34 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2314: finishing write tid 3 to smithi11881747-34 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2314: finishing write tid 4 to smithi11881747-34 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2314: finishing write tid 5 to smithi11881747-34 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2314: finishing write tid 6 to smithi11881747-34 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2314: oid 34 updating version 0 to 33923 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2314: oid 34 updating version 33923 to 33927 2024-03-20T08:09:28.306 INFO:tasks.rados.rados.0.smithi118.stdout:2314: oid 34 version 33927 is already newer than 33924 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2314: oid 34 version 33927 is already newer than 33926 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2314: oid 34 version 33927 is already newer than 33925 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 33927 (ObjNum 712 snap 241 seq_num 712) dirty exists 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2314: left oid 34 (ObjNum 712 snap 241 seq_num 712) 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 34008 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2317: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2317: finishing copy_from to smithi11881747-23 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 30067 (ObjNum 584 snap 188 seq_num 584) dirty exists 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2318: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2318: finishing copy_from to smithi11881747-11 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 7117 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2314: done (4 left) 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2315: done (3 left) 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2317: done (2 left) 2024-03-20T08:09:28.307 INFO:tasks.rados.rados.0.smithi118.stdout:2318: done (1 left) 2024-03-20T08:09:28.308 INFO:tasks.rados.rados.0.smithi118.stdout:2321: rollback oid 9 current snap is 241 2024-03-20T08:09:28.308 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 9 to 202 2024-03-20T08:09:28.308 INFO:tasks.rados.rados.0.smithi118.stdout:2322: read oid 48 snap -1 2024-03-20T08:09:28.308 INFO:tasks.rados.rados.0.smithi118.stdout:2322: expect (ObjNum 697 snap 233 seq_num 697) 2024-03-20T08:09:28.308 INFO:tasks.rados.rados.0.smithi118.stdout:2323: rollback oid 47 current snap is 241 2024-03-20T08:09:28.308 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 169 2024-03-20T08:09:28.308 INFO:tasks.rados.rados.0.smithi118.stdout:2324: snap_remove snap 230 2024-03-20T08:09:28.549 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:28.321+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:28.549 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:28.321+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: osdmap e534: 8 total, 8 up, 8 in 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:28.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:28.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: osd.7 marked itself down and dead 2024-03-20T08:09:28.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: pgmap v481: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 752 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:09:28.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:28.551 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.551 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.551 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.334+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.551 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.551 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.455+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.467+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.507+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:28.304+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:28.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:28.312+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:28.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:28.312+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:28.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:28.319+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:28.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:28.301+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:28.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:28.302+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:28.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:28.302+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:28.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:28.313+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: osdmap e534: 8 total, 8 up, 8 in 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: osd.7 marked itself down and dead 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: pgmap v481: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 752 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: osdmap e534: 8 total, 8 up, 8 in 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:28.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11e=[]:{ad=[a8],bf=[bf,be,b4,b3],cb=[cb,ca,c9,c8,c6,c5,c3,c2,c1],d6=[d6,d4,d3,d1,cf,cd],e2=[e2,e1],ea=[e9,e8,e5,e4,e3],fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],100=[100,ff,fe,fd]} for object 5:beabf7e5:::smithi11881747-7:a0 2024-03-20T08:09:28.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{e3=[e3,e0],f3=[f3,f2,f1,f0,eb,e8,e5],f4=[f4],fa=[fa,f8,f7,f5],100=[100],112=[112,111,110,10f,10e,10d,10c,10a,108,107,106,105,102,101],113=[113],126=[126,125,124,122,120,11f,11e,11d,11a,119,117,116,114]} for object 5:89304740:::smithi11881747-4:9c 2024-03-20T08:09:28.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:28.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:28.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: osd.7 marked itself down and dead 2024-03-20T08:09:28.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: pgmap v481: 121 pgs: 2 active+clean+snaptrim, 11 active+clean+snaptrim_error, 108 active+clean; 752 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 2.5 MiB/s wr, 20 op/s 2024-03-20T08:09:28.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:28.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.616+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.630+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.643+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.686+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.728+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.742+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.769+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.782+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:28.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.836+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.865+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.880+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:28.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.307 INFO:tasks.rados.rados.0.smithi118.stdout:2323: finishing rollback tid 0 to smithi11881747-47 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:2323: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33948 (ObjNum 161 snap 0 seq_num 0) dirty dne 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:2321: finishing rollback tid 0 to smithi11881747-9 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:2321: finishing rollback tid 1 to smithi11881747-9 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:2320: finishing rollback tid 0 to smithi11881747-1 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:2321: finishing rollback tid 2 to smithi11881747-9 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 23092 (ObjNum 622 snap 202 seq_num 622) dirty exists 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:2320: finishing rollback tid 1 to smithi11881747-1 2024-03-20T08:09:29.308 INFO:tasks.rados.rados.0.smithi118.stdout:2320: finishing rollback tid 2 to smithi11881747-1 2024-03-20T08:09:29.309 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 34025 (ObjNum 637 snap 205 seq_num 637) dirty exists 2024-03-20T08:09:29.309 INFO:tasks.rados.rados.0.smithi118.stdout:2322: expect (ObjNum 697 snap 233 seq_num 697) 2024-03-20T08:09:29.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:09:29.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-20T08:09:29.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:29.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:29.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:29.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: osdmap e535: 8 total, 7 up, 8 in 2024-03-20T08:09:29.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:29.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:29.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:29.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:29.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:29.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:29.336 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:29.311+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:29.336 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:29.311+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:29 smithi196 podman[107371]: 2024-03-20 08:09:29.137481018 +0000 UTC m=+1.917644587 container died b48bb8ba0aee6666fe3779f51b1a7b34b60dc18d7ba96fdefad0449870b3f35c (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=reef-78dbc53, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.068+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.084+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.100+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.145+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.163+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.201+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.338 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.268+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.295+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.469 INFO:tasks.rados.rados.0.smithi118.stdout:2320: done (4 left) 2024-03-20T08:09:29.470 INFO:tasks.rados.rados.0.smithi118.stdout:2321: done (3 left) 2024-03-20T08:09:29.470 INFO:tasks.rados.rados.0.smithi118.stdout:2322: done (2 left) 2024-03-20T08:09:29.470 INFO:tasks.rados.rados.0.smithi118.stdout:2323: done (1 left) 2024-03-20T08:09:29.470 INFO:tasks.rados.rados.0.smithi118.stdout:2324: done (0 left) 2024-03-20T08:09:29.470 INFO:tasks.rados.rados.0.smithi118.stdout:2325: rollback oid 10 current snap is 241 2024-03-20T08:09:29.470 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 219 2024-03-20T08:09:29.470 INFO:tasks.rados.rados.0.smithi118.stdout:2326: delete oid 37 current snap is 241 2024-03-20T08:09:29.472 INFO:tasks.rados.rados.0.smithi118.stdout:2325: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:09:29.474 INFO:tasks.rados.rados.0.smithi118.stdout:2325: finishing rollback tid 2 to smithi11881747-10 2024-03-20T08:09:29.474 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 34027 (ObjNum 472 snap 156 seq_num 472) dirty exists 2024-03-20T08:09:29.475 INFO:tasks.rados.rados.0.smithi118.stdout:2325: done (1 left) 2024-03-20T08:09:29.475 INFO:tasks.rados.rados.0.smithi118.stdout:2326: done (0 left) 2024-03-20T08:09:29.475 INFO:tasks.rados.rados.0.smithi118.stdout:2327: rollback oid 11 current snap is 241 2024-03-20T08:09:29.475 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 184 2024-03-20T08:09:29.475 INFO:tasks.rados.rados.0.smithi118.stdout:2328: snap_create 2024-03-20T08:09:29.478 INFO:tasks.rados.rados.0.smithi118.stdout:2327: finishing rollback tid 0 to smithi11881747-11 2024-03-20T08:09:29.478 INFO:tasks.rados.rados.0.smithi118.stdout:2327: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:09:29.478 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 23094 (ObjNum 129 snap 0 seq_num 322910480) dirty dne 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.441+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.454+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.468+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.546+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.559+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.571+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:29.316+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:29.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:29.316+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:29.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: osdmap e535: 8 total, 7 up, 8 in 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:29.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:29.305+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:29.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:29.308+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:29.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:29.308+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:29.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:29.312+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:29.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:09:29.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-20T08:09:29.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:29.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:29.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: osdmap e535: 8 total, 7 up, 8 in 2024-03-20T08:09:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:29.675 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:29.304+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:29.675 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:29.306+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:29.675 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:29.306+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:29.839 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.599+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.839 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.839 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.839 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.839 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.650+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.728+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.814+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:29.840 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:29 smithi196 podman[107371]: 2024-03-20 08:09:29.64789781 +0000 UTC m=+2.428061378 container cleanup b48bb8ba0aee6666fe3779f51b1a7b34b60dc18d7ba96fdefad0449870b3f35c (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=reef-78dbc53, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:09:29.841 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:29 smithi196 bash[107371]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7 2024-03-20T08:09:30.101 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:29 smithi196 podman[107403]: 2024-03-20 08:09:29.933142288 +0000 UTC m=+0.792766986 container remove b48bb8ba0aee6666fe3779f51b1a7b34b60dc18d7ba96fdefad0449870b3f35c (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=reef-78dbc53, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:09:30.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.866+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.888+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.900+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.924+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.936+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.951+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.965+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.980+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:29.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.008+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.022+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.048+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.104 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.317 INFO:tasks.rados.rados.0.smithi118.stdout:2327: done (1 left) 2024-03-20T08:09:30.317 INFO:tasks.rados.rados.0.smithi118.stdout:2328: done (0 left) 2024-03-20T08:09:30.317 INFO:tasks.rados.rados.0.smithi118.stdout:2329: snap_create 2024-03-20T08:09:30.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: osdmap e536: 8 total, 7 up, 8 in 2024-03-20T08:09:30.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:30.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[93121]: pgmap v484: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 2 active+undersized+degraded, 21 active+undersized, 11 stale+active+clean, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 74 active+clean; 760 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 6.2 MiB/s rd, 912 KiB/s wr, 10 op/s; 18/1455 objects degraded (1.237%) 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: osdmap e536: 8 total, 7 up, 8 in 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:30.318 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:30.319 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:30.319 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:30.319 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:30.319 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:30.319 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:30.319 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:30.319 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-mon[95692]: pgmap v484: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 2 active+undersized+degraded, 21 active+undersized, 11 stale+active+clean, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 74 active+clean; 760 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 6.2 MiB/s rd, 912 KiB/s wr, 10 op/s; 18/1455 objects degraded (1.237%) 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: osdmap e536: 8 total, 7 up, 8 in 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:30.363 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-mon[87080]: pgmap v484: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 2 active+undersized+degraded, 21 active+undersized, 11 stale+active+clean, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 74 active+clean; 760 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 6.2 MiB/s rd, 912 KiB/s wr, 10 op/s; 18/1455 objects degraded (1.237%) 2024-03-20T08:09:30.364 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:30.320+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:30.364 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:30.321+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:30.364 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:30 smithi196 podman[107487]: 2024-03-20 08:09:30.238856416 +0000 UTC m=+0.022541701 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:09:30.364 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:30 smithi196 podman[107487]: 2024-03-20 08:09:30.342621812 +0000 UTC m=+0.126307101 container create bc51f4d2c56a62a099bd91a24337bdcfb2877997f8ebaff26e8256556267d750 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-20T08:09:30.364 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.364 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.145+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.160+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.175+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.189+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.204+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.365 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.322+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.366 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.610 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:30.317+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:30.611 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:30.320+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:30.611 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:30.320+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:30.611 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:30.321+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:30.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:30.319+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:30.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:30.319+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:30.611 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:30.319+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:30.618 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:30 smithi196 podman[107487]: 2024-03-20 08:09:30.612678701 +0000 UTC m=+0.396364000 container init bc51f4d2c56a62a099bd91a24337bdcfb2877997f8ebaff26e8256556267d750 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:09:30.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.375+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.445+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.471+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.538+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.573+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.597+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.877 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:30 smithi196 podman[107487]: 2024-03-20 08:09:30.620323172 +0000 UTC m=+0.404008464 container start bc51f4d2c56a62a099bd91a24337bdcfb2877997f8ebaff26e8256556267d750 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-20T08:09:30.878 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:30 smithi196 podman[107487]: 2024-03-20 08:09:30.721378789 +0000 UTC m=+0.505064085 container attach bc51f4d2c56a62a099bd91a24337bdcfb2877997f8ebaff26e8256556267d750 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-20T08:09:30.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.878 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.678+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.691+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.718+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.757+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.784+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.796+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.821+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.848+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.863+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:30.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:30.610+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:31.138 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:31 smithi196 podman[107487]: 2024-03-20 08:09:31.021691082 +0000 UTC m=+0.805376368 container died bc51f4d2c56a62a099bd91a24337bdcfb2877997f8ebaff26e8256556267d750 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-20T08:09:31.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.915+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.958+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.971+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:30.984+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.029+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.127+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.139+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.320 INFO:tasks.rados.rados.0.smithi118.stdout:2329: done (0 left) 2024-03-20T08:09:31.321 INFO:tasks.rados.rados.0.smithi118.stdout:2330: snap_create 2024-03-20T08:09:31.396 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:31.326+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:31.396 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:31.326+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:31.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: Health check failed: Degraded data redundancy: 18/1455 objects degraded (1.237%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:31.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: osdmap e537: 8 total, 7 up, 8 in 2024-03-20T08:09:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:31.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:31.397 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.153+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.179+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.191+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.204+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.268+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.281+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.398 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.399 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.399 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.399 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.360+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.399 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.372+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.399 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.399 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.396+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.421+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.444+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.457+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.493+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.505+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.518+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.530+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.567+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.604+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.616+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.639+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.674 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:31.320+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:31.675 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:31.321+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: Health check failed: Degraded data redundancy: 18/1455 objects degraded (1.237%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: osdmap e537: 8 total, 7 up, 8 in 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: Health check failed: Degraded data redundancy: 18/1455 objects degraded (1.237%), 3 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: osdmap e537: 8 total, 7 up, 8 in 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:31.910 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:31 smithi196 podman[107532]: 2024-03-20 08:09:31.760673185 +0000 UTC m=+0.732691772 container remove bc51f4d2c56a62a099bd91a24337bdcfb2877997f8ebaff26e8256556267d750 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:09:31.910 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:31 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.7.service: Deactivated successfully. 2024-03-20T08:09:31.910 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:31 smithi196 systemd[1]: Stopped Ceph osd.7 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:09:31.910 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:31 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.7.service: Consumed 1min 39.462s CPU time. 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.666+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.679+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.691+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.752+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.819+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.833+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.869+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:31.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.970+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.983+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:31.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.071+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.104+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.137+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.171 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:32 smithi196 systemd[1]: Starting Ceph osd.7 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:09:32.327 INFO:tasks.rados.rados.0.smithi118.stdout:2330: done (0 left) 2024-03-20T08:09:32.327 INFO:tasks.rados.rados.0.smithi118.stdout:2331: read oid 21 snap -1 2024-03-20T08:09:32.327 INFO:tasks.rados.rados.0.smithi118.stdout:2331: expect (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:09:32.327 INFO:tasks.rados.rados.0.smithi118.stdout:2332: read oid 49 snap 213 2024-03-20T08:09:32.327 INFO:tasks.rados.rados.0.smithi118.stdout:2332: expect (ObjNum 628 snap 204 seq_num 628) 2024-03-20T08:09:32.328 INFO:tasks.rados.rados.0.smithi118.stdout:2333: read oid 6 snap -1 2024-03-20T08:09:32.328 INFO:tasks.rados.rados.0.smithi118.stdout:2333: expect (ObjNum 709 snap 239 seq_num 709) 2024-03-20T08:09:32.328 INFO:tasks.rados.rados.0.smithi118.stdout:2334: read oid 12 snap -1 2024-03-20T08:09:32.328 INFO:tasks.rados.rados.0.smithi118.stdout:2334: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:09:32.328 INFO:tasks.rados.rados.0.smithi118.stdout:2335: snap_create 2024-03-20T08:09:32.337 INFO:tasks.rados.rados.0.smithi118.stdout:2332: expect (ObjNum 628 snap 204 seq_num 628) 2024-03-20T08:09:32.375 INFO:tasks.rados.rados.0.smithi118.stdout:2331: expect (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:09:32.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:32.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:32.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-mon[87080]: osdmap e538: 8 total, 7 up, 8 in 2024-03-20T08:09:32.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:32.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:32.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:32.421 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-mon[87080]: pgmap v487: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 7 active+undersized+degraded, 36 active+undersized, 3 stale+active+clean, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 62 active+clean; 763 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 MiB/s wr, 5 op/s; 44/1467 objects degraded (2.999%) 2024-03-20T08:09:32.422 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:32 smithi196 podman[107628]: 2024-03-20 08:09:32.231767986 +0000 UTC m=+0.028321290 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:09:32.422 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:09:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:09:32.422 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:32.332+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:32.332+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.169+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.214+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.241+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.281+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.292+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.370+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.421+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.536 INFO:tasks.rados.rados.0.smithi118.stdout:2334: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:09:32.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:32.337+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:32.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:32.337+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:32.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:32.327+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:32.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:32.327+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:32.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:32.327+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[93121]: osdmap e538: 8 total, 7 up, 8 in 2024-03-20T08:09:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:32.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[93121]: pgmap v487: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 7 active+undersized+degraded, 36 active+undersized, 3 stale+active+clean, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 62 active+clean; 763 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 MiB/s wr, 5 op/s; 44/1467 objects degraded (2.999%) 2024-03-20T08:09:32.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:32.330+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:32.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:32.331+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:32.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:32.332+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:32.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:32.332+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:32.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:32.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:32.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[95692]: osdmap e538: 8 total, 7 up, 8 in 2024-03-20T08:09:32.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:32.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:32.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:32.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-mon[95692]: pgmap v487: 121 pgs: 1 active+undersized+degraded+snaptrim_error, 7 active+undersized+degraded, 36 active+undersized, 3 stale+active+clean, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 62 active+clean; 763 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 MiB/s wr, 5 op/s; 44/1467 objects degraded (2.999%) 2024-03-20T08:09:32.675 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:32.335+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:32.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.435+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.446+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.678 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.460+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.472+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.507+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.519+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.544+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.616+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.654+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.679 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.678+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.729+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.765+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.934 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.790+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.802+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.838+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.861+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.884+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:32.935 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.191 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:33 smithi196 podman[107628]: 2024-03-20 08:09:33.17107127 +0000 UTC m=+0.967624569 container create ed2b273624b97550c503ed2814482c4bc0047d1e27aaf8aaf3d453472f8327ef (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:09:33.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.946+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.958+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.970+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.983+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:32.995+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.019+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.043+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.065+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.077+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.102+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.128+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.154+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.191+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.311 INFO:tasks.rados.rados.0.smithi118.stdout:2333: expect (ObjNum 709 snap 239 seq_num 709) 2024-03-20T08:09:33.483 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:33.330+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:33.483 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:33.330+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.240+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.295+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.307+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.360+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.396+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.421+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.446+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.469+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: osdmap e539: 8 total, 7 up, 8 in 2024-03-20T08:09:33.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:33.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:33.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:33.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:33.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:33.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:33.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:33.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:33.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:33.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:33.328+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:33.586 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:33.328+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:33.586 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:33.326+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:33.586 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: osdmap e539: 8 total, 7 up, 8 in 2024-03-20T08:09:33.586 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:33.586 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:33.586 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: osdmap e539: 8 total, 7 up, 8 in 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:33.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:33.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:33.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:33.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:33.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2331: done (4 left) 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2332: done (3 left) 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2333: done (2 left) 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2334: done (1 left) 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2335: done (0 left) 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2336: read oid 42 snap -1 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2336: expect deleted 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2337: copy_from oid 40 from oid 4 current snap is 245 2024-03-20T08:09:33.693 INFO:tasks.rados.rados.0.smithi118.stdout:2338: read oid 43 snap -1 2024-03-20T08:09:33.694 INFO:tasks.rados.rados.0.smithi118.stdout:2338: expect (ObjNum 696 snap 233 seq_num 696) 2024-03-20T08:09:33.694 INFO:tasks.rados.rados.0.smithi118.stdout:2339: rollback oid 39 current snap is 245 2024-03-20T08:09:33.694 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 213 2024-03-20T08:09:33.694 INFO:tasks.rados.rados.0.smithi118.stdout:2340: copy_from oid 3 from oid 17 current snap is 245 2024-03-20T08:09:33.694 INFO:tasks.rados.rados.0.smithi118.stdout:2341: snap_remove snap 222 2024-03-20T08:09:33.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.490+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.515+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.538+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.753 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.597+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.620+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.632+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.754 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:09:33] "GET /metrics HTTP/1.1" 200 36332 "" "Prometheus/2.43.0" 2024-03-20T08:09:33.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:33.699+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:33.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:33.696+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:33.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:33.696+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:33.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:33.697+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:34.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.767+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.791+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.803+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.943+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.968+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.005 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:33.993+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.004+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.016+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.029+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.040+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.064+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.140+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.195+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.208+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.244+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.339 INFO:tasks.rados.rados.0.smithi118.stdout:2339: finishing rollback tid 0 to smithi11881747-39 2024-03-20T08:09:34.339 INFO:tasks.rados.rados.0.smithi118.stdout:2339: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:09:34.339 INFO:tasks.rados.rados.0.smithi118.stdout:2339: finishing rollback tid 2 to smithi11881747-39 2024-03-20T08:09:34.340 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 13624 (ObjNum 649 snap 213 seq_num 649) dirty exists 2024-03-20T08:09:34.340 INFO:tasks.rados.rados.0.smithi118.stdout:2337: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:09:34.340 INFO:tasks.rados.rados.0.smithi118.stdout:2337: finishing copy_from to smithi11881747-40 2024-03-20T08:09:34.340 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 33917 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:09:34.340 INFO:tasks.rados.rados.0.smithi118.stdout:2340: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:09:34.340 INFO:tasks.rados.rados.0.smithi118.stdout:2340: finishing copy_from to smithi11881747-3 2024-03-20T08:09:34.340 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 7056 (ObjNum 568 snap 185 seq_num 568) dirty exists 2024-03-20T08:09:34.340 INFO:tasks.rados.rados.0.smithi118.stdout:2338: expect (ObjNum 696 snap 233 seq_num 696) 2024-03-20T08:09:34.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.292+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.316+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.360+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.371+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.438+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.492+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.522 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:34.342+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:34.522 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:34.342+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:34.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: osdmap e540: 8 total, 7 up, 8 in 2024-03-20T08:09:34.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:34.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:34.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:34.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:34.522 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:34.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:34.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:34.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:34.523 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-mon[87080]: pgmap v490: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 763 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 MiB/s wr, 3 op/s; 70/1473 objects degraded (4.752%) 2024-03-20T08:09:34.523 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 podman[107628]: 2024-03-20 08:09:34.384131349 +0000 UTC m=+2.180684654 container init ed2b273624b97550c503ed2814482c4bc0047d1e27aaf8aaf3d453472f8327ef (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-20T08:09:34.523 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 podman[107628]: 2024-03-20 08:09:34.391753772 +0000 UTC m=+2.188307066 container start ed2b273624b97550c503ed2814482c4bc0047d1e27aaf8aaf3d453472f8327ef (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-20T08:09:34.523 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 podman[107628]: 2024-03-20 08:09:34.492291268 +0000 UTC m=+2.288844564 container attach ed2b273624b97550c503ed2814482c4bc0047d1e27aaf8aaf3d453472f8327ef (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:09:34.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:34.342+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:34.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:34.343+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:34.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:34.343+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:34.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:34.338+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:34.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:34.567+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:34.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:34.567+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:34.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:34.568+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:34.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:34.568+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: osdmap e540: 8 total, 7 up, 8 in 2024-03-20T08:09:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:34.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:34.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[93121]: pgmap v490: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 763 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 MiB/s wr, 3 op/s; 70/1473 objects degraded (4.752%) 2024-03-20T08:09:34.675 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:34.341+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:34.675 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:34.341+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:34.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: osdmap e540: 8 total, 7 up, 8 in 2024-03-20T08:09:34.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:34.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:34.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:34.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:34.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:34.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:34.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:34.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:34.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:34 smithi118 ceph-mon[95692]: pgmap v490: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 763 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 MiB/s wr, 3 op/s; 70/1473 objects degraded (4.752%) 2024-03-20T08:09:34.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.534+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.546+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.572+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.584+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.664+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.679+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.693+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.705+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.718+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.733+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.748+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.761+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.775+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2336: done (5 left) 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2337: done (4 left) 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2338: done (3 left) 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2339: done (2 left) 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2340: done (1 left) 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2341: done (0 left) 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2342: rmattr oid 42 current snap is 245 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2342: done (0 left) 2024-03-20T08:09:34.952 INFO:tasks.rados.rados.0.smithi118.stdout:2343: snap_create 2024-03-20T08:09:35.027 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate[107641]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-20T08:09:35.027 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate[107641]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-20T08:09:35.027 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 bash[107628]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-20T08:09:35.027 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 bash[107628]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-7 --no-mon-config --dev /dev/mapper/vg_nvme-lv_1 2024-03-20T08:09:35.027 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 bash[107628]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 bash[107628]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 bash[107628]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 bash[107628]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 bash[107628]: --> ceph-volume raw activate successful for osd ID: 7 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate[107641]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_1 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate[107641]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate[107641]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-7/block 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate[107641]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate[107641]: --> ceph-volume raw activate successful for osd ID: 7 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:35 smithi196 podman[107628]: 2024-03-20 08:09:35.023953581 +0000 UTC m=+2.820506875 container died ed2b273624b97550c503ed2814482c4bc0047d1e27aaf8aaf3d453472f8327ef (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.803+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.831+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.844+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.858+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.872+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.900+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.928+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.968+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:34.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.011+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.055+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.079+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.092+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.190+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.203+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.216+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.241+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.252+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.343 INFO:tasks.rados.rados.0.smithi118.stdout:2343: done (0 left) 2024-03-20T08:09:35.344 INFO:tasks.rados.rados.0.smithi118.stdout:2344: delete oid 15 current snap is 246 2024-03-20T08:09:35.350 INFO:tasks.rados.rados.0.smithi118.stdout:2344: done (0 left) 2024-03-20T08:09:35.350 INFO:tasks.rados.rados.0.smithi118.stdout:2345: snap_remove snap 227 2024-03-20T08:09:35.545 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:35.350+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:35.545 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:35.350+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:35.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:35.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:35.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: osdmap e541: 8 total, 7 up, 8 in 2024-03-20T08:09:35.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:35.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:35.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:35.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:35.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:35.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:35.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:35.547 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:35.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.302+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.326+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.339+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.358+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.370+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.382+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.406+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.430+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.465+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.488+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.546+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.596 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:35.344+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:35.597 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:35.344+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:35.597 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:35.349+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:35.597 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:35.597 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:35.597 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: osdmap e541: 8 total, 7 up, 8 in 2024-03-20T08:09:35.597 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: osdmap e541: 8 total, 7 up, 8 in 2024-03-20T08:09:35.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:35.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:35.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:35.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:35.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:35.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:35.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:35.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:35.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.558+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.619+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.631+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.643+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.656+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.717+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.728+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.740+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.752+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.764+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.803+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:35.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:35.597+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:35.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:35.596+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:35.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:35.596+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:35.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:35.597+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:36.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.852+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.876+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.900+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.913+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.927+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.965+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:35.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.018+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.316 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.316 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.079+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.316 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.316 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.316 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.316 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.126+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.169+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.181+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.193+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.241+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.252+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.265+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.317 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.316+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.348 INFO:tasks.rados.rados.0.smithi118.stdout:2345: done (0 left) 2024-03-20T08:09:36.349 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 246 2024-03-20T08:09:36.349 INFO:tasks.rados.rados.0.smithi118.stdout:2346: seq_num 715 ranges {3604480=778240,4382720=770048,5152768=499712,5652480=335872} 2024-03-20T08:09:36.372 INFO:tasks.rados.rados.0.smithi118.stdout:2346: writing smithi11881747-21 from 3604480 to 4382720 tid 1 2024-03-20T08:09:36.378 INFO:tasks.rados.rados.0.smithi118.stdout:2346: writing smithi11881747-21 from 4382720 to 5152768 tid 2 2024-03-20T08:09:36.380 INFO:tasks.rados.rados.0.smithi118.stdout:2346: writing smithi11881747-21 from 5152768 to 5652480 tid 3 2024-03-20T08:09:36.383 INFO:tasks.rados.rados.0.smithi118.stdout:2346: writing smithi11881747-21 from 5652480 to 5988352 tid 4 2024-03-20T08:09:36.383 INFO:tasks.rados.rados.0.smithi118.stdout:2347: rmattr oid 18 current snap is 246 2024-03-20T08:09:36.383 INFO:tasks.rados.rados.0.smithi118.stdout:2347: done (1 left) 2024-03-20T08:09:36.383 INFO:tasks.rados.rados.0.smithi118.stdout:2348: copy_from oid 49 from oid 22 current snap is 246 2024-03-20T08:09:36.383 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 246 2024-03-20T08:09:36.383 INFO:tasks.rados.rados.0.smithi118.stdout:2349: seq_num 716 ranges {1294336=794624,2088960=655360,2744320=704512,3448832=245760} 2024-03-20T08:09:36.394 INFO:tasks.rados.rados.0.smithi118.stdout:2349: writing smithi11881747-30 from 1294336 to 2088960 tid 1 2024-03-20T08:09:36.398 INFO:tasks.rados.rados.0.smithi118.stdout:2349: writing smithi11881747-30 from 2088960 to 2744320 tid 2 2024-03-20T08:09:36.402 INFO:tasks.rados.rados.0.smithi118.stdout:2349: writing smithi11881747-30 from 2744320 to 3448832 tid 3 2024-03-20T08:09:36.403 INFO:tasks.rados.rados.0.smithi118.stdout:2349: writing smithi11881747-30 from 3448832 to 3694592 tid 4 2024-03-20T08:09:36.403 INFO:tasks.rados.rados.0.smithi118.stdout:2350: snap_remove snap 234 2024-03-20T08:09:36.575 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:36 smithi196 podman[107786]: 2024-03-20 08:09:36.324102193 +0000 UTC m=+1.293144015 container remove ed2b273624b97550c503ed2814482c4bc0047d1e27aaf8aaf3d453472f8327ef (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:09:36.576 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:36.355+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:36.576 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:36.356+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:36.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: osdmap e542: 8 total, 7 up, 8 in 2024-03-20T08:09:36.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:36.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:36.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:36.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:36.576 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:36.577 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:36.577 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:36.577 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:36.577 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-mon[87080]: pgmap v493: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 768 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 14 MiB/s rd, 0 B/s wr, 4 op/s; 70/1476 objects degraded (4.743%) 2024-03-20T08:09:36.577 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.577 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.344+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.577 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.577 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.577 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.439+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:36.386+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:36.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:36.387+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:36.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:36.348+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:36.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:36.348+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:36.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:36.349+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: osdmap e542: 8 total, 7 up, 8 in 2024-03-20T08:09:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:36.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[93121]: pgmap v493: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 768 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 14 MiB/s rd, 0 B/s wr, 4 op/s; 70/1476 objects degraded (4.743%) 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: osdmap e542: 8 total, 7 up, 8 in 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:36.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:36.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:36.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:36.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:36.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:36.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:36.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:36 smithi118 ceph-mon[95692]: pgmap v493: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 768 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 14 MiB/s rd, 0 B/s wr, 4 op/s; 70/1476 objects degraded (4.743%) 2024-03-20T08:09:36.827 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:36 smithi196 podman[107843]: 2024-03-20 08:09:36.484293301 +0000 UTC m=+0.025820115 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:09:36.827 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:36 smithi196 podman[107843]: 2024-03-20 08:09:36.595612084 +0000 UTC m=+0.137138881 container create 52392403da44e54428ee449b4f9d69c2bf26bbb5c63f470720d5a8b566f1c480 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.587+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.600+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.638+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.686+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.742+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.754+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.790+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.802+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:36.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.077 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:36 smithi196 podman[107843]: 2024-03-20 08:09:36.836281855 +0000 UTC m=+0.377808660 container init 52392403da44e54428ee449b4f9d69c2bf26bbb5c63f470720d5a8b566f1c480 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-20T08:09:37.078 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:36 smithi196 podman[107843]: 2024-03-20 08:09:36.844652715 +0000 UTC m=+0.386179519 container start 52392403da44e54428ee449b4f9d69c2bf26bbb5c63f470720d5a8b566f1c480 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-20T08:09:37.078 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-osd[107880]: -- 172.21.15.196:0/3607796767 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb29431680 con 0x55cb293fac00 2024-03-20T08:09:37.078 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:36 smithi196 bash[107843]: 52392403da44e54428ee449b4f9d69c2bf26bbb5c63f470720d5a8b566f1c480 2024-03-20T08:09:37.078 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:36 smithi196 systemd[1]: Started Ceph osd.7 for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:09:37.078 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.078 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.078 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.895+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.908+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.932+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.945+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.958+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:36.987+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.000+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.040+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.051+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.077+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.091+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.339 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.159+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.186+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.200+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.228+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.270+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.284+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.325+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.340 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.353 INFO:tasks.rados.rados.0.smithi118.stdout:2350: done (3 left) 2024-03-20T08:09:37.353 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 246 2024-03-20T08:09:37.353 INFO:tasks.rados.rados.0.smithi118.stdout:2346: finishing write tid 1 to smithi11881747-21 2024-03-20T08:09:37.353 INFO:tasks.rados.rados.0.smithi118.stdout:2349: finishing write tid 1 to smithi11881747-30 2024-03-20T08:09:37.353 INFO:tasks.rados.rados.0.smithi118.stdout:2349: finishing write tid 2 to smithi11881747-30 2024-03-20T08:09:37.353 INFO:tasks.rados.rados.0.smithi118.stdout:2346: finishing write tid 2 to smithi11881747-21 2024-03-20T08:09:37.353 INFO:tasks.rados.rados.0.smithi118.stdout:2349: finishing write tid 3 to smithi11881747-30 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: finishing write tid 4 to smithi11881747-30 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: finishing write tid 5 to smithi11881747-30 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2348: finishing copy_from to smithi11881747-49 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2348: got expected ENOENT (src dne) 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2348: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: finishing write tid 6 to smithi11881747-30 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: oid 30 updating version 0 to 33923 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: oid 30 updating version 33923 to 33925 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: oid 30 version 33925 is already newer than 33921 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: oid 30 version 33925 is already newer than 33922 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: oid 30 version 33925 is already newer than 33924 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 33925 (ObjNum 716 snap 246 seq_num 716) dirty exists 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2349: left oid 30 (ObjNum 716 snap 246 seq_num 716) 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2346: finishing write tid 3 to smithi11881747-21 2024-03-20T08:09:37.354 INFO:tasks.rados.rados.0.smithi118.stdout:2346: finishing write tid 4 to smithi11881747-21 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2346: finishing write tid 5 to smithi11881747-21 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2346: finishing write tid 6 to smithi11881747-21 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2346: oid 21 updating version 0 to 17961 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2346: oid 21 updating version 17961 to 17965 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2346: oid 21 version 17965 is already newer than 17962 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2346: oid 21 version 17965 is already newer than 17963 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2346: oid 21 version 17965 is already newer than 17964 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 17965 (ObjNum 715 snap 246 seq_num 715) dirty exists 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2346: left oid 21 (ObjNum 715 snap 246 seq_num 715) 2024-03-20T08:09:37.355 INFO:tasks.rados.rados.0.smithi118.stdout:2351: seq_num 717 ranges {4546560=16384} 2024-03-20T08:09:37.380 INFO:tasks.rados.rados.0.smithi118.stdout:2351: writing smithi11881747-25 from 4546560 to 4562944 tid 1 2024-03-20T08:09:37.380 INFO:tasks.rados.rados.0.smithi118.stdout:2346: done (3 left) 2024-03-20T08:09:37.380 INFO:tasks.rados.rados.0.smithi118.stdout:2348: done (2 left) 2024-03-20T08:09:37.380 INFO:tasks.rados.rados.0.smithi118.stdout:2349: done (1 left) 2024-03-20T08:09:37.380 INFO:tasks.rados.rados.0.smithi118.stdout:2352: rollback oid 5 current snap is 246 2024-03-20T08:09:37.380 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 219 2024-03-20T08:09:37.380 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 246 2024-03-20T08:09:37.381 INFO:tasks.rados.rados.0.smithi118.stdout:2353: seq_num 718 ranges {0=32768} 2024-03-20T08:09:37.381 INFO:tasks.rados.rados.0.smithi118.stdout:2353: writing smithi11881747-37 from 0 to 32768 tid 1 2024-03-20T08:09:37.381 INFO:tasks.rados.rados.0.smithi118.stdout:2354: snap_create 2024-03-20T08:09:37.385 INFO:tasks.rados.rados.0.smithi118.stdout:2351: finishing write tid 1 to smithi11881747-25 2024-03-20T08:09:37.386 INFO:tasks.rados.rados.0.smithi118.stdout:2351: finishing write tid 2 to smithi11881747-25 2024-03-20T08:09:37.386 INFO:tasks.rados.rados.0.smithi118.stdout:2352: finishing rollback tid 0 to smithi11881747-5 2024-03-20T08:09:37.387 INFO:tasks.rados.rados.0.smithi118.stdout:2352: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:09:37.387 INFO:tasks.rados.rados.0.smithi118.stdout:2351: finishing write tid 3 to smithi11881747-25 2024-03-20T08:09:37.387 INFO:tasks.rados.rados.0.smithi118.stdout:2351: oid 25 updating version 0 to 23095 2024-03-20T08:09:37.387 INFO:tasks.rados.rados.0.smithi118.stdout:2351: oid 25 updating version 23095 to 23096 2024-03-20T08:09:37.387 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 23096 (ObjNum 717 snap 246 seq_num 717) dirty exists 2024-03-20T08:09:37.387 INFO:tasks.rados.rados.0.smithi118.stdout:2351: left oid 25 (ObjNum 717 snap 246 seq_num 717) 2024-03-20T08:09:37.388 INFO:tasks.rados.rados.0.smithi118.stdout:2352: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:09:37.388 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 23099 (ObjNum 628 snap 204 seq_num 628) dirty exists 2024-03-20T08:09:37.392 INFO:tasks.rados.rados.0.smithi118.stdout:2353: finishing write tid 1 to smithi11881747-37 2024-03-20T08:09:37.392 INFO:tasks.rados.rados.0.smithi118.stdout:2353: finishing write tid 2 to smithi11881747-37 2024-03-20T08:09:37.398 INFO:tasks.rados.rados.0.smithi118.stdout:2353: finishing write tid 3 to smithi11881747-37 2024-03-20T08:09:37.398 INFO:tasks.rados.rados.0.smithi118.stdout:2353: oid 37 updating version 0 to 34004 2024-03-20T08:09:37.398 INFO:tasks.rados.rados.0.smithi118.stdout:2353: oid 37 updating version 34004 to 34005 2024-03-20T08:09:37.398 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 34005 (ObjNum 718 snap 246 seq_num 718) dirty exists 2024-03-20T08:09:37.398 INFO:tasks.rados.rados.0.smithi118.stdout:2353: left oid 37 (ObjNum 718 snap 246 seq_num 718) 2024-03-20T08:09:37.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:37.147+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:37.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:37.353+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:37.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:37.355+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:37.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:37.355+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 70/1476 objects degraded (4.743%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: osdmap e543: 8 total, 7 up, 8 in 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 70/1476 objects degraded (4.743%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: osdmap e543: 8 total, 7 up, 8 in 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:37.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:37.296+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:37.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:37.296+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:37.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:37.297+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:37.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:37.297+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:37.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:37.382+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:37.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:37.383+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:37.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:37.391+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:37.602 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:37.395+0000 7ffb672ed7c0 -1 Falling back to public interface 2024-03-20T08:09:37.602 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:37.356+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:37.602 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:37.356+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 70/1476 objects degraded (4.743%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: osdmap e543: 8 total, 7 up, 8 in 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:37.603 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:37.604 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.372+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.433+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.477+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.489+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.604 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.504+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.605 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.516+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.605 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.531+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.605 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.545+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.605 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.605 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.605 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.605 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.642+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.752+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.858 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.858 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.858 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.858 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.858 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.858 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:37.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:37.467+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.869+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.885+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.934+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.961+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.972+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:37.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.071+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.100+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.359 INFO:tasks.rados.rados.0.smithi118.stdout:2351: done (3 left) 2024-03-20T08:09:38.359 INFO:tasks.rados.rados.0.smithi118.stdout:2352: done (2 left) 2024-03-20T08:09:38.359 INFO:tasks.rados.rados.0.smithi118.stdout:2353: done (1 left) 2024-03-20T08:09:38.359 INFO:tasks.rados.rados.0.smithi118.stdout:2354: done (0 left) 2024-03-20T08:09:38.359 INFO:tasks.rados.rados.0.smithi118.stdout:2355: snap_remove snap 233 2024-03-20T08:09:38.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:38.362+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:38.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:38.362+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: osdmap e544: 8 total, 7 up, 8 in 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:38.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:38.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:38.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-mon[87080]: pgmap v496: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 775 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 7 op/s; 70/1485 objects degraded (4.714%) 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.186+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.215+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.367 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.245+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.258+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.285+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.323+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.365+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.438+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.490+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.504+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.516+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.529+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.554+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.627+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:38.358+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:38.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:38.358+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:38.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:38.358+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: osdmap e544: 8 total, 7 up, 8 in 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[95692]: pgmap v496: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 775 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 7 op/s; 70/1485 objects degraded (4.714%) 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: osdmap e544: 8 total, 7 up, 8 in 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:38.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:38.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:38.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:38.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:38.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:38.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:38.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-mon[93121]: pgmap v496: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 775 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 7 op/s; 70/1485 objects degraded (4.714%) 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.655+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.684+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.729+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.787+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.886 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.828+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.856+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.871+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:38.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.900+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.945+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:38.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.050+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.063+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.112+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.124+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.355 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:38.998+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:39.355 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:38.998+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:39.355 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:38.998+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:39.355 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:38.998+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:39.356 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:38.999+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:39.356 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:39.000+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:39.356 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:39.000+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:39.363 INFO:tasks.rados.rados.0.smithi118.stdout:2355: done (0 left) 2024-03-20T08:09:39.363 INFO:tasks.rados.rados.0.smithi118.stdout:2356: read oid 16 snap -1 2024-03-20T08:09:39.364 INFO:tasks.rados.rados.0.smithi118.stdout:2356: expect (ObjNum 698 snap 233 seq_num 698) 2024-03-20T08:09:39.364 INFO:tasks.rados.rados.0.smithi118.stdout:2357: read oid 15 snap -1 2024-03-20T08:09:39.364 INFO:tasks.rados.rados.0.smithi118.stdout:2357: expect deleted 2024-03-20T08:09:39.364 INFO:tasks.rados.rados.0.smithi118.stdout:2358: snap_create 2024-03-20T08:09:39.388 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:39.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:39.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:39.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-mon[87080]: osdmap e545: 8 total, 7 up, 8 in 2024-03-20T08:09:39.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:39.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=sqlstore.transactions t=2024-03-20T08:09:39.246000827Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-20T08:09:39.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[62730]: logger=cleanup t=2024-03-20T08:09:39.248369018Z level=info msg="Completed cleanup jobs" duration=12.719627ms 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:39.372+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:39.372+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.150+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.163+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.175+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.201+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.286+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.327+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.398 INFO:tasks.rados.rados.0.smithi118.stdout:2356: expect (ObjNum 698 snap 233 seq_num 698) 2024-03-20T08:09:39.662 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.400+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.438+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.463+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.475+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.535+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.549+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.663 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.589+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.601+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.649+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.664 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:39.368+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:39.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:39.368+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:39.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:39.364+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:39.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:39.365+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:39.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:39.365+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:39.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:39.364+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[93121]: osdmap e545: 8 total, 7 up, 8 in 2024-03-20T08:09:39.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:39.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[95692]: osdmap e545: 8 total, 7 up, 8 in 2024-03-20T08:09:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:39.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:39.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.698+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.762+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.796+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.833+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.865+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.881+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.909+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:39.925 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.923+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.936+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.976+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:39.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.003+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.016+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.071+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.103+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.135+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.181+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.365 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:40.010+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:40.369 INFO:tasks.rados.rados.0.smithi118.stdout:2356: done (2 left) 2024-03-20T08:09:40.369 INFO:tasks.rados.rados.0.smithi118.stdout:2357: done (1 left) 2024-03-20T08:09:40.369 INFO:tasks.rados.rados.0.smithi118.stdout:2358: done (0 left) 2024-03-20T08:09:40.369 INFO:tasks.rados.rados.0.smithi118.stdout:2359: setattr oid 33 current snap is 248 2024-03-20T08:09:40.371 INFO:tasks.rados.rados.0.smithi118.stdout:2360: delete oid 35 current snap is 248 2024-03-20T08:09:40.375 INFO:tasks.rados.rados.0.smithi118.stdout:2360: done (1 left) 2024-03-20T08:09:40.375 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 248 2024-03-20T08:09:40.375 INFO:tasks.rados.rados.0.smithi118.stdout:2361: seq_num 720 ranges {0=65536} 2024-03-20T08:09:40.375 INFO:tasks.rados.rados.0.smithi118.stdout:2361: writing smithi11881747-49 from 0 to 65536 tid 1 2024-03-20T08:09:40.375 INFO:tasks.rados.rados.0.smithi118.stdout:2362: snap_create 2024-03-20T08:09:40.380 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 13620 (ObjNum 691 snap 227 seq_num 691) dirty exists 2024-03-20T08:09:40.381 INFO:tasks.rados.rados.0.smithi118.stdout:2361: finishing write tid 1 to smithi11881747-49 2024-03-20T08:09:40.381 INFO:tasks.rados.rados.0.smithi118.stdout:2361: finishing write tid 2 to smithi11881747-49 2024-03-20T08:09:40.383 INFO:tasks.rados.rados.0.smithi118.stdout:2361: finishing write tid 3 to smithi11881747-49 2024-03-20T08:09:40.383 INFO:tasks.rados.rados.0.smithi118.stdout:2361: oid 49 updating version 0 to 13626 2024-03-20T08:09:40.383 INFO:tasks.rados.rados.0.smithi118.stdout:2361: oid 49 version 13626 is already newer than 13625 2024-03-20T08:09:40.383 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 13626 (ObjNum 720 snap 248 seq_num 720) dirty exists 2024-03-20T08:09:40.384 INFO:tasks.rados.rados.0.smithi118.stdout:2361: left oid 49 (ObjNum 720 snap 248 seq_num 720) 2024-03-20T08:09:40.441 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:40.374+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:40.441 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:40.375+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: osdmap e546: 8 total, 7 up, 8 in 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:40.442 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:40.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-mon[87080]: pgmap v499: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 777 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 7.7 MiB/s rd, 2.3 MiB/s wr, 7 op/s; 71/1491 objects degraded (4.762%) 2024-03-20T08:09:40.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.195+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.209+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.222+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.443 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.251+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.265+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.278+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.296+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.310+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.353+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.368+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.444 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.411+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.441+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:40.376+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:40.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:40.371+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:40.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:40.371+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:40.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:40.371+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:40.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:40.382+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:40.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:40.383+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:40.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:40.368+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:40.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:40.368+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:40.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:40.368+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:40.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:40.369+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:40.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:40.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:40.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:40.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:40.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:40.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: osdmap e546: 8 total, 7 up, 8 in 2024-03-20T08:09:40.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:40.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:40.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:40.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:40.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:40.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:40.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:40.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:40.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[93121]: pgmap v499: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 777 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 7.7 MiB/s rd, 2.3 MiB/s wr, 7 op/s; 71/1491 objects degraded (4.762%) 2024-03-20T08:09:40.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:40.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: osdmap e546: 8 total, 7 up, 8 in 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:40.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:40 smithi118 ceph-mon[95692]: pgmap v499: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 777 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 7.7 MiB/s rd, 2.3 MiB/s wr, 7 op/s; 71/1491 objects degraded (4.762%) 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.455+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.494+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.507+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.548+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.588+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.601+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.614+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.627+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.964 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.722+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.738+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.754+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.826+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.868+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.883+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.965 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.925+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:40.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.978+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:40.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.007+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.020+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.102+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.229 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.213+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.374 INFO:tasks.rados.rados.0.smithi118.stdout:2359: done (2 left) 2024-03-20T08:09:41.374 INFO:tasks.rados.rados.0.smithi118.stdout:2361: done (1 left) 2024-03-20T08:09:41.374 INFO:tasks.rados.rados.0.smithi118.stdout:2362: done (0 left) 2024-03-20T08:09:41.374 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 249 2024-03-20T08:09:41.374 INFO:tasks.rados.rados.0.smithi118.stdout:2363: seq_num 721 ranges {0=81920} 2024-03-20T08:09:41.374 INFO:tasks.rados.rados.0.smithi118.stdout:2363: writing smithi11881747-32 from 0 to 81920 tid 1 2024-03-20T08:09:41.374 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 249 2024-03-20T08:09:41.374 INFO:tasks.rados.rados.0.smithi118.stdout:2364: seq_num 722 ranges {9256960=98304} 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2364: writing smithi11881747-12 from 9256960 to 9355264 tid 1 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2365: rmattr oid 2 current snap is 249 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2365: done (2 left) 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2366: rollback oid 46 current snap is 249 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 221 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2363: finishing write tid 1 to smithi11881747-32 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2363: finishing write tid 2 to smithi11881747-32 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2363: finishing write tid 3 to smithi11881747-32 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2363: oid 32 updating version 0 to 34003 2024-03-20T08:09:41.427 INFO:tasks.rados.rados.0.smithi118.stdout:2363: oid 32 version 34003 is already newer than 34002 2024-03-20T08:09:41.428 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34003 (ObjNum 721 snap 249 seq_num 721) dirty exists 2024-03-20T08:09:41.428 INFO:tasks.rados.rados.0.smithi118.stdout:2363: left oid 32 (ObjNum 721 snap 249 seq_num 721) 2024-03-20T08:09:41.428 INFO:tasks.rados.rados.0.smithi118.stdout:2363: done (2 left) 2024-03-20T08:09:41.428 INFO:tasks.rados.rados.0.smithi118.stdout:2367: delete oid 14 current snap is 249 2024-03-20T08:09:41.431 INFO:tasks.rados.rados.0.smithi118.stdout:2366: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:09:41.431 INFO:tasks.rados.rados.0.smithi118.stdout:2366: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:09:41.431 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 23101 (ObjNum 145 snap 0 seq_num 323008512) dirty dne 2024-03-20T08:09:41.432 INFO:tasks.rados.rados.0.smithi118.stdout:2366: done (2 left) 2024-03-20T08:09:41.432 INFO:tasks.rados.rados.0.smithi118.stdout:2367: done (1 left) 2024-03-20T08:09:41.432 INFO:tasks.rados.rados.0.smithi118.stdout:2368: setattr oid 47 current snap is 249 2024-03-20T08:09:41.436 INFO:tasks.rados.rados.0.smithi118.stdout:2369: rmattr oid 48 current snap is 249 2024-03-20T08:09:41.436 INFO:tasks.rados.rados.0.smithi118.stdout:2369: done (2 left) 2024-03-20T08:09:41.436 INFO:tasks.rados.rados.0.smithi118.stdout:2370: read oid 14 snap -1 2024-03-20T08:09:41.437 INFO:tasks.rados.rados.0.smithi118.stdout:2370: expect deleted 2024-03-20T08:09:41.437 INFO:tasks.rados.rados.0.smithi118.stdout:2371: copy_from oid 21 from oid 37 current snap is 249 2024-03-20T08:09:41.437 INFO:tasks.rados.rados.0.smithi118.stdout:2372: setattr oid 8 current snap is 249 2024-03-20T08:09:41.439 INFO:tasks.rados.rados.0.smithi118.stdout:2364: finishing write tid 1 to smithi11881747-12 2024-03-20T08:09:41.439 INFO:tasks.rados.rados.0.smithi118.stdout:2364: finishing write tid 2 to smithi11881747-12 2024-03-20T08:09:41.439 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33949 (ObjNum 369 snap 0 seq_num 1890249808) dirty exists 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:2368: done (4 left) 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:2370: done (3 left) 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:2373: rollback oid 33 current snap is 249 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 241 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:2374: rollback oid 27 current snap is 249 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 218 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:2375: copy_from oid 24 from oid 31 current snap is 249 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:2376: read oid 26 snap 210 2024-03-20T08:09:41.441 INFO:tasks.rados.rados.0.smithi118.stdout:2376: expect (ObjNum 620 snap 202 seq_num 620) 2024-03-20T08:09:41.442 INFO:tasks.rados.rados.0.smithi118.stdout:2377: delete oid 38 current snap is 249 2024-03-20T08:09:41.447 INFO:tasks.rados.rados.0.smithi118.stdout:2364: finishing write tid 3 to smithi11881747-12 2024-03-20T08:09:41.447 INFO:tasks.rados.rados.0.smithi118.stdout:2364: oid 12 updating version 0 to 34003 2024-03-20T08:09:41.447 INFO:tasks.rados.rados.0.smithi118.stdout:2364: oid 12 updating version 34003 to 34004 2024-03-20T08:09:41.447 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34004 (ObjNum 722 snap 249 seq_num 722) dirty exists 2024-03-20T08:09:41.447 INFO:tasks.rados.rados.0.smithi118.stdout:2364: left oid 12 (ObjNum 722 snap 249 seq_num 722) 2024-03-20T08:09:41.447 INFO:tasks.rados.rados.0.smithi118.stdout:2374: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:09:41.447 INFO:tasks.rados.rados.0.smithi118.stdout:2374: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:09:41.448 INFO:tasks.rados.rados.0.smithi118.stdout:2374: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:09:41.448 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 13629 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:09:41.450 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34007 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:09:41.455 INFO:tasks.rados.rados.0.smithi118.stdout:2373: finishing rollback tid 0 to smithi11881747-33 2024-03-20T08:09:41.455 INFO:tasks.rados.rados.0.smithi118.stdout:2373: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:09:41.457 INFO:tasks.rados.rados.0.smithi118.stdout:2373: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:09:41.457 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 13624 (ObjNum 691 snap 227 seq_num 691) dirty exists 2024-03-20T08:09:41.460 INFO:tasks.rados.rados.0.smithi118.stdout:2376: expect (ObjNum 620 snap 202 seq_num 620) 2024-03-20T08:09:41.480 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:41.379+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:41.480 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:41.379+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:41.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:41.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:41.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:41.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:41.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: osdmap e547: 8 total, 7 up, 8 in 2024-03-20T08:09:41.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:41.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:41.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:41.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:41.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:41.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:41.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.226+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.240+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.302+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.366+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:2371: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:2375: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:2371: finishing copy_from to smithi11881747-21 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34005 (ObjNum 718 snap 246 seq_num 718) dirty exists 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:2375: finishing copy_from to smithi11881747-24 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 34000 (ObjNum 522 snap 169 seq_num 522) dirty exists 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:2364: done (7 left) 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:2371: done (6 left) 2024-03-20T08:09:41.503 INFO:tasks.rados.rados.0.smithi118.stdout:2372: done (5 left) 2024-03-20T08:09:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2373: done (4 left) 2024-03-20T08:09:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2374: done (3 left) 2024-03-20T08:09:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2375: done (2 left) 2024-03-20T08:09:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2376: done (1 left) 2024-03-20T08:09:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2377: done (0 left) 2024-03-20T08:09:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2378: rmattr oid 22 current snap is 249 2024-03-20T08:09:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2378: done (0 left) 2024-03-20T08:09:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2379: delete oid 19 current snap is 249 2024-03-20T08:09:41.509 INFO:tasks.rados.rados.0.smithi118.stdout:2379: done (0 left) 2024-03-20T08:09:41.509 INFO:tasks.rados.rados.0.smithi118.stdout:2380: copy_from oid 11 from oid 18 current snap is 249 2024-03-20T08:09:41.509 INFO:tasks.rados.rados.0.smithi118.stdout:2381: read oid 33 snap 206 2024-03-20T08:09:41.509 INFO:tasks.rados.rados.0.smithi118.stdout:2381: expect (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:09:41.509 INFO:tasks.rados.rados.0.smithi118.stdout:2382: read oid 43 snap -1 2024-03-20T08:09:41.509 INFO:tasks.rados.rados.0.smithi118.stdout:2382: expect (ObjNum 696 snap 233 seq_num 696) 2024-03-20T08:09:41.509 INFO:tasks.rados.rados.0.smithi118.stdout:2383: delete oid 46 current snap is 249 2024-03-20T08:09:41.510 INFO:tasks.rados.rados.0.smithi118.stdout:2380: finishing copy_from to smithi11881747-11 2024-03-20T08:09:41.510 INFO:tasks.rados.rados.0.smithi118.stdout:2380: got expected ENOENT (src dne) 2024-03-20T08:09:41.510 INFO:tasks.rados.rados.0.smithi118.stdout:2380: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:09:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:2380: done (3 left) 2024-03-20T08:09:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:2383: done (2 left) 2024-03-20T08:09:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:2384: read oid 1 snap -1 2024-03-20T08:09:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:2384: expect (ObjNum 637 snap 205 seq_num 637) 2024-03-20T08:09:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:2385: read oid 38 snap -1 2024-03-20T08:09:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:2385: expect deleted 2024-03-20T08:09:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 249 2024-03-20T08:09:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:2386: seq_num 727 ranges {4251648=180224} 2024-03-20T08:09:41.539 INFO:tasks.rados.rados.0.smithi118.stdout:2386: writing smithi11881747-48 from 4251648 to 4431872 tid 1 2024-03-20T08:09:41.540 INFO:tasks.rados.rados.0.smithi118.stdout:2387: delete oid 16 current snap is 249 2024-03-20T08:09:41.543 INFO:tasks.rados.rados.0.smithi118.stdout:2381: expect (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:09:41.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:41.378+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:41.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:41.378+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:41.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:41.374+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:41.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:41.429+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:41.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:41.429+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:41.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:41.431+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:41.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:41.374+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: osdmap e547: 8 total, 7 up, 8 in 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: osdmap e547: 8 total, 7 up, 8 in 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:41.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.493+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.510+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.568+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.580+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.639+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.693+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.721+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:41.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.751+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.764+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.792+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.855+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.905+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.939+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.954+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:41.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.001+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.262 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.068+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.083+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.124+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.139+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.153+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.248+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.275 INFO:tasks.rados.rados.0.smithi118.stdout:2386: finishing write tid 1 to smithi11881747-48 2024-03-20T08:09:42.275 INFO:tasks.rados.rados.0.smithi118.stdout:2386: finishing write tid 2 to smithi11881747-48 2024-03-20T08:09:42.275 INFO:tasks.rados.rados.0.smithi118.stdout:2382: expect (ObjNum 696 snap 233 seq_num 696) 2024-03-20T08:09:42.515 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:09:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:09:42.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: osdmap e548: 8 total, 7 up, 8 in 2024-03-20T08:09:42.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:42.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:42.767 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:42.768 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:42.768 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:42.768 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:42.768 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:42.768 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:42.768 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:42.768 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-mon[87080]: pgmap v502: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 778 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 662 KiB/s wr, 11 op/s; 73/1515 objects degraded (4.818%) 2024-03-20T08:09:42.768 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.276+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.332+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.359+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.388+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.430+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.458+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: osdmap e548: 8 total, 7 up, 8 in 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:42.771 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[95692]: pgmap v502: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 778 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 662 KiB/s wr, 11 op/s; 73/1515 objects degraded (4.818%) 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.530+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.555+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.567+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.582+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.596+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.638+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.666+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.697+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:42.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: osdmap e548: 8 total, 7 up, 8 in 2024-03-20T08:09:42.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 118=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset fd=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 124=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:42.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:42 smithi118 ceph-mon[93121]: pgmap v502: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 778 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 662 KiB/s wr, 11 op/s; 73/1515 objects degraded (4.818%) 2024-03-20T08:09:42.900 INFO:tasks.rados.rados.0.smithi118.stdout:2386: finishing write tid 3 to smithi11881747-48 2024-03-20T08:09:42.900 INFO:tasks.rados.rados.0.smithi118.stdout:2386: oid 48 updating version 0 to 34007 2024-03-20T08:09:42.900 INFO:tasks.rados.rados.0.smithi118.stdout:2386: oid 48 version 34007 is already newer than 34006 2024-03-20T08:09:42.900 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 34007 (ObjNum 727 snap 249 seq_num 727) dirty exists 2024-03-20T08:09:42.901 INFO:tasks.rados.rados.0.smithi118.stdout:2386: left oid 48 (ObjNum 727 snap 249 seq_num 727) 2024-03-20T08:09:42.901 INFO:tasks.rados.rados.0.smithi118.stdout:2384: expect (ObjNum 637 snap 205 seq_num 637) 2024-03-20T08:09:42.937 INFO:tasks.rados.rados.0.smithi118.stdout:2381: done (5 left) 2024-03-20T08:09:42.937 INFO:tasks.rados.rados.0.smithi118.stdout:2382: done (4 left) 2024-03-20T08:09:42.937 INFO:tasks.rados.rados.0.smithi118.stdout:2384: done (3 left) 2024-03-20T08:09:42.938 INFO:tasks.rados.rados.0.smithi118.stdout:2385: done (2 left) 2024-03-20T08:09:42.938 INFO:tasks.rados.rados.0.smithi118.stdout:2386: done (1 left) 2024-03-20T08:09:42.938 INFO:tasks.rados.rados.0.smithi118.stdout:2387: done (0 left) 2024-03-20T08:09:42.938 INFO:tasks.rados.rados.0.smithi118.stdout:2388: delete oid 47 current snap is 249 2024-03-20T08:09:42.939 INFO:tasks.rados.rados.0.smithi118.stdout:2388: done (0 left) 2024-03-20T08:09:42.939 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 249 2024-03-20T08:09:42.939 INFO:tasks.rados.rados.0.smithi118.stdout:2389: seq_num 730 ranges {1228800=229376} 2024-03-20T08:09:42.947 INFO:tasks.rados.rados.0.smithi118.stdout:2389: writing smithi11881747-8 from 1228800 to 1458176 tid 1 2024-03-20T08:09:42.947 INFO:tasks.rados.rados.0.smithi118.stdout:2390: setattr oid 49 current snap is 249 2024-03-20T08:09:42.949 INFO:tasks.rados.rados.0.smithi118.stdout:2391: snap_remove snap 237 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.801+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.833+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.030 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.945+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.031 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.031 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.980+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.031 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:42.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.031 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.030+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.092+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.108+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.124+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.139+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.226+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.241+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.256+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.545 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 73/1515 objects degraded (4.818%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:43.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.546 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:43.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.347+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.378+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.456+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.470+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.485+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.498+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.528+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:09:43] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:09:43.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 73/1515 objects degraded (4.818%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:43.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:43.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 73/1515 objects degraded (4.818%), 11 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:43.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:43.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:43.747 INFO:tasks.rados.rados.0.smithi118.stdout:2391: done (2 left) 2024-03-20T08:09:43.747 INFO:tasks.rados.rados.0.smithi118.stdout:2392: delete oid 41 current snap is 249 2024-03-20T08:09:43.748 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 13630 (ObjNum 720 snap 248 seq_num 720) dirty exists 2024-03-20T08:09:43.748 INFO:tasks.rados.rados.0.smithi118.stdout:2389: finishing write tid 1 to smithi11881747-8 2024-03-20T08:09:43.748 INFO:tasks.rados.rados.0.smithi118.stdout:2389: finishing write tid 2 to smithi11881747-8 2024-03-20T08:09:43.748 INFO:tasks.rados.rados.0.smithi118.stdout:2389: finishing write tid 3 to smithi11881747-8 2024-03-20T08:09:43.748 INFO:tasks.rados.rados.0.smithi118.stdout:2389: oid 8 updating version 0 to 34008 2024-03-20T08:09:43.748 INFO:tasks.rados.rados.0.smithi118.stdout:2389: oid 8 updating version 34008 to 34009 2024-03-20T08:09:43.748 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34009 (ObjNum 730 snap 249 seq_num 730) dirty exists 2024-03-20T08:09:43.748 INFO:tasks.rados.rados.0.smithi118.stdout:2389: left oid 8 (ObjNum 730 snap 249 seq_num 730) 2024-03-20T08:09:43.755 INFO:tasks.rados.rados.0.smithi118.stdout:2389: done (2 left) 2024-03-20T08:09:43.755 INFO:tasks.rados.rados.0.smithi118.stdout:2390: done (1 left) 2024-03-20T08:09:43.755 INFO:tasks.rados.rados.0.smithi118.stdout:2392: done (0 left) 2024-03-20T08:09:43.755 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 249 2024-03-20T08:09:43.755 INFO:tasks.rados.rados.0.smithi118.stdout:2393: seq_num 732 ranges {32768=262144} 2024-03-20T08:09:43.757 INFO:tasks.rados.rados.0.smithi118.stdout:2393: writing smithi11881747-21 from 32768 to 294912 tid 1 2024-03-20T08:09:43.757 INFO:tasks.rados.rados.0.smithi118.stdout:2394: rollback oid 24 current snap is 249 2024-03-20T08:09:43.757 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 236 2024-03-20T08:09:43.757 INFO:tasks.rados.rados.0.smithi118.stdout:2395: copy_from oid 44 from oid 15 current snap is 249 2024-03-20T08:09:43.757 INFO:tasks.rados.rados.0.smithi118.stdout:2396: rmattr oid 11 current snap is 249 2024-03-20T08:09:43.757 INFO:tasks.rados.rados.0.smithi118.stdout:2396: done (3 left) 2024-03-20T08:09:43.757 INFO:tasks.rados.rados.0.smithi118.stdout:2397: rmattr oid 47 current snap is 249 2024-03-20T08:09:43.757 INFO:tasks.rados.rados.0.smithi118.stdout:2397: done (3 left) 2024-03-20T08:09:43.758 INFO:tasks.rados.rados.0.smithi118.stdout:2398: delete oid 39 current snap is 249 2024-03-20T08:09:43.760 INFO:tasks.rados.rados.0.smithi118.stdout:2398: done (3 left) 2024-03-20T08:09:43.760 INFO:tasks.rados.rados.0.smithi118.stdout:2399: read oid 37 snap -1 2024-03-20T08:09:43.760 INFO:tasks.rados.rados.0.smithi118.stdout:2399: expect (ObjNum 718 snap 246 seq_num 718) 2024-03-20T08:09:43.760 INFO:tasks.rados.rados.0.smithi118.stdout:2400: read oid 33 snap -1 2024-03-20T08:09:43.760 INFO:tasks.rados.rados.0.smithi118.stdout:2400: expect (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:09:43.760 INFO:tasks.rados.rados.0.smithi118.stdout:2401: snap_create 2024-03-20T08:09:43.761 INFO:tasks.rados.rados.0.smithi118.stdout:2399: expect (ObjNum 718 snap 246 seq_num 718) 2024-03-20T08:09:43.774 INFO:tasks.rados.rados.0.smithi118.stdout:2393: finishing write tid 1 to smithi11881747-21 2024-03-20T08:09:43.774 INFO:tasks.rados.rados.0.smithi118.stdout:2393: finishing write tid 2 to smithi11881747-21 2024-03-20T08:09:43.775 INFO:tasks.rados.rados.0.smithi118.stdout:2393: finishing write tid 3 to smithi11881747-21 2024-03-20T08:09:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:2393: oid 21 updating version 0 to 34009 2024-03-20T08:09:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:2393: oid 21 version 34009 is already newer than 34008 2024-03-20T08:09:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34009 (ObjNum 732 snap 249 seq_num 732) dirty exists 2024-03-20T08:09:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:2393: left oid 21 (ObjNum 732 snap 249 seq_num 732) 2024-03-20T08:09:43.780 INFO:tasks.rados.rados.0.smithi118.stdout:2395: finishing copy_from to smithi11881747-44 2024-03-20T08:09:43.780 INFO:tasks.rados.rados.0.smithi118.stdout:2395: got expected ENOENT (src dne) 2024-03-20T08:09:43.780 INFO:tasks.rados.rados.0.smithi118.stdout:2395: finishing copy_from racing read to smithi11881747-44 2024-03-20T08:09:43.781 INFO:tasks.rados.rados.0.smithi118.stdout:2394: finishing rollback tid 0 to smithi11881747-24 2024-03-20T08:09:43.783 INFO:tasks.rados.rados.0.smithi118.stdout:2394: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:09:43.783 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 34010 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-03-20T08:09:43.793 INFO:tasks.rados.rados.0.smithi118.stdout:2400: expect (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:09:43.804 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:43.754+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:43.804 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:43.754+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.562+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.673+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.718+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.754+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.805 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.788+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:43.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.056 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:43.866+0000 7ffb672ed7c0 -1 osd.7 534 log_to_monitors true 2024-03-20T08:09:44.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.822+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.855+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.871+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.900+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.931+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.946+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.973+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:43.986+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.057 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.043+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.058 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:43.748+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:44.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:43.750+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:44.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:43.750+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:44.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:43.751+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:44.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:43.748+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:44.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:43.748+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:44.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:43.750+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:44.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:43.753+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:44.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:43.753+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:44.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:43.753+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:44.318 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.071+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.318 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.318 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.111+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.138+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.178+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.206+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.248+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.261+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.319 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.320 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.332+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.360+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.386+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.400+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.415+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.442+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.470+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.485+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.529+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.573 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.544+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.574 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.559+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.753 INFO:tasks.rados.rados.0.smithi118.stdout:2393: done (5 left) 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:2394: done (4 left) 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:2395: done (3 left) 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:2399: done (2 left) 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:2400: done (1 left) 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:2401: done (0 left) 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:2402: rmattr oid 38 current snap is 250 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:2402: done (0 left) 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 250 2024-03-20T08:09:44.754 INFO:tasks.rados.rados.0.smithi118.stdout:2403: seq_num 734 ranges {0=294912} 2024-03-20T08:09:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:2403: writing smithi11881747-22 from 0 to 294912 tid 1 2024-03-20T08:09:44.755 INFO:tasks.rados.rados.0.smithi118.stdout:2404: snap_create 2024-03-20T08:09:44.789 INFO:tasks.rados.rados.0.smithi118.stdout:2403: finishing write tid 1 to smithi11881747-22 2024-03-20T08:09:44.789 INFO:tasks.rados.rados.0.smithi118.stdout:2403: finishing write tid 2 to smithi11881747-22 2024-03-20T08:09:44.791 INFO:tasks.rados.rados.0.smithi118.stdout:2403: finishing write tid 3 to smithi11881747-22 2024-03-20T08:09:44.792 INFO:tasks.rados.rados.0.smithi118.stdout:2403: oid 22 updating version 0 to 34011 2024-03-20T08:09:44.792 INFO:tasks.rados.rados.0.smithi118.stdout:2403: oid 22 version 34011 is already newer than 34010 2024-03-20T08:09:44.792 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 34011 (ObjNum 734 snap 250 seq_num 734) dirty exists 2024-03-20T08:09:44.792 INFO:tasks.rados.rados.0.smithi118.stdout:2403: left oid 22 (ObjNum 734 snap 250 seq_num 734) 2024-03-20T08:09:44.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:44.764+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:44.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:44.764+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: osdmap e549: 8 total, 7 up, 8 in 2024-03-20T08:09:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:44.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: from='osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: pgmap v504: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 778 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 9.9 MiB/s rd, 560 KiB/s wr, 20 op/s; 74/1527 objects degraded (4.846%) 2024-03-20T08:09:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.571+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.584+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.679+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.717+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.730+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:44.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.789+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.162 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:44.906+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.162 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.162 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.118+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.162 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.134+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.162 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.148+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.162 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.162+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:44.752+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:45.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:44.752+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:45.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: osdmap e549: 8 total, 7 up, 8 in 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:45.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: from='osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: pgmap v504: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 778 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 9.9 MiB/s rd, 560 KiB/s wr, 20 op/s; 74/1527 objects degraded (4.846%) 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: osdmap e549: 8 total, 7 up, 8 in 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:45.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: from='osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: pgmap v504: 121 pgs: 2 active+undersized+degraded+snaptrim_error, 9 active+undersized+degraded, 39 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 778 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 9.9 MiB/s rd, 560 KiB/s wr, 20 op/s; 74/1527 objects degraded (4.846%) 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:45.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:45.419 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:45.223+0000 7ffb5473d700 -1 osd.7 534 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-20T08:09:45.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.191+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.206+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.219+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.235+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.268+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.420 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.282+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.385+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.396+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.408+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.421 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.420+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.432+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.445+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.456+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.483+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.671 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.560+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.585+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.646+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.672 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.658+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.673 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.671+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.764 INFO:tasks.rados.rados.0.smithi118.stdout:2403: done (1 left) 2024-03-20T08:09:45.764 INFO:tasks.rados.rados.0.smithi118.stdout:2404: done (0 left) 2024-03-20T08:09:45.765 INFO:tasks.rados.rados.0.smithi118.stdout:2405: read oid 13 snap 249 2024-03-20T08:09:45.765 INFO:tasks.rados.rados.0.smithi118.stdout:2405: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:09:45.765 INFO:tasks.rados.rados.0.smithi118.stdout:2406: copy_from oid 5 from oid 50 current snap is 251 2024-03-20T08:09:45.765 INFO:tasks.rados.rados.0.smithi118.stdout:2407: setattr oid 11 current snap is 251 2024-03-20T08:09:45.765 INFO:tasks.rados.rados.0.smithi118.stdout:2408: rollback oid 40 current snap is 251 2024-03-20T08:09:45.766 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 242 2024-03-20T08:09:45.766 INFO:tasks.rados.rados.0.smithi118.stdout:2409: read oid 49 snap -1 2024-03-20T08:09:45.766 INFO:tasks.rados.rados.0.smithi118.stdout:2409: expect (ObjNum 720 snap 248 seq_num 720) 2024-03-20T08:09:45.766 INFO:tasks.rados.rados.0.smithi118.stdout:2410: snap_remove snap 245 2024-03-20T08:09:45.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:45.767+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:45.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:45.771+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: osdmap e550: 8 total, 7 up, 8 in 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: from='osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:45.925 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-mon[87080]: from='osd.7 ' entity='osd.7' 2024-03-20T08:09:45.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.684+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.695+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.719+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.731+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.770+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.824+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.836+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.864+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.877+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.888+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.900+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.928 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.912+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:45.928 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.925+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:45.757+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:46.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:45.756+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:46.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:45.756+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:46.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:45.756+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:46.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:45.761+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:46.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:45.761+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:46.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:45.769+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:46.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:45.765+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:46.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-20T08:09:46.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: osdmap e550: 8 total, 7 up, 8 in 2024-03-20T08:09:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: from='osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:09:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:09:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[93121]: from='osd.7 ' entity='osd.7' 2024-03-20T08:09:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-20T08:09:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: osdmap e550: 8 total, 7 up, 8 in 2024-03-20T08:09:46.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: from='osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:09:46.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-20T08:09:46.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:46.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:46.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:45 smithi118 ceph-mon[95692]: from='osd.7 ' entity='osd.7' 2024-03-20T08:09:46.183 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.183 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.183 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.961+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.973+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.985+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:45.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.184 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.035+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.059+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.084+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.096+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.185 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.122+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.159+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.186 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.434 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.198+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.434 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.209+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.434 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.222+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.434 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.234+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.434 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.434 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.259+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.295+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.331+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.380+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.405+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.434+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.689 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.492+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.519+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.531+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.556+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.568+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.580+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.592+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.617+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.677+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.689+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.755 INFO:tasks.rados.rados.0.smithi118.stdout:2410: done (5 left) 2024-03-20T08:09:46.755 INFO:tasks.rados.rados.0.smithi118.stdout:2411: rollback oid 21 current snap is 251 2024-03-20T08:09:46.755 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 206 2024-03-20T08:09:46.755 INFO:tasks.rados.rados.0.smithi118.stdout:2409: expect (ObjNum 720 snap 248 seq_num 720) 2024-03-20T08:09:46.761 INFO:tasks.rados.rados.0.smithi118.stdout:2405: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:09:46.947 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 23100 (ObjNum 33 snap 0 seq_num 323408048) dirty exists 2024-03-20T08:09:46.947 INFO:tasks.rados.rados.0.smithi118.stdout:2408: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:09:46.947 INFO:tasks.rados.rados.0.smithi118.stdout:2405: done (5 left) 2024-03-20T08:09:46.947 INFO:tasks.rados.rados.0.smithi118.stdout:2407: done (4 left) 2024-03-20T08:09:46.947 INFO:tasks.rados.rados.0.smithi118.stdout:2409: done (3 left) 2024-03-20T08:09:46.947 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 251 2024-03-20T08:09:46.947 INFO:tasks.rados.rados.0.smithi118.stdout:2408: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:09:46.947 INFO:tasks.rados.rados.0.smithi118.stdout:2412: seq_num 735 ranges {65536=311296} 2024-03-20T08:09:46.948 INFO:tasks.rados.rados.0.smithi118.stdout:2412: writing smithi11881747-49 from 65536 to 376832 tid 1 2024-03-20T08:09:46.948 INFO:tasks.rados.rados.0.smithi118.stdout:2413: read oid 16 snap 169 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:2413: expect (ObjNum 381 snap 129 seq_num 381) 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:2414: delete oid 24 current snap is 251 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:2408: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34008 (ObjNum 600 snap 196 seq_num 600) dirty exists 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:2406: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:2406: finishing copy_from to smithi11881747-5 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 7117 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:2411: finishing rollback tid 0 to smithi11881747-21 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:2411: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:2411: finishing rollback tid 2 to smithi11881747-21 2024-03-20T08:09:46.949 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34014 (ObjNum 545 snap 179 seq_num 545) dirty exists 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391] boot 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: osdmap e551: 8 total, 8 up, 8 in 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:46.952 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:46.953 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:46.953 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:46.953 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:46.953 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: pgmap v507: 121 pgs: 5 peering, 2 active+undersized+degraded+snaptrim_error, 8 active+undersized+degraded, 35 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 782 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 18 MiB/s rd, 2.0 MiB/s wr, 32 op/s; 72/1542 objects degraded (4.669%) 2024-03-20T08:09:46.953 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-mon[87080]: osdmap e552: 8 total, 8 up, 8 in 2024-03-20T08:09:46.953 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.702+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.729+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.805+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.819+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.819+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.954 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.924+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:46.955 INFO:tasks.rados.rados.0.smithi118.stdout:2412: finishing write tid 1 to smithi11881747-49 2024-03-20T08:09:46.955 INFO:tasks.rados.rados.0.smithi118.stdout:2412: finishing write tid 2 to smithi11881747-49 2024-03-20T08:09:46.955 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:46.763+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:46.955 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:46.764+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:46.955 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:46.843+0000 7ffb41f18700 -1 osd.7 pg_epoch: 552 pg[5.9s0( v 548'1146 (0'0,548'1146] local-lis/les=551/552 n=21 ec=73/73 lis/c=551/551 les/c/f=552/552/0 sis=551) [7,6,1]p7(0) r=0 lpr=551 crt=548'1146 mlcod 0'0 active+clean+snaptrim trimq=[11a~1,11f~1,122~1,124~3,129~1,131~1] ps=121] update_snap_map remove_oid 5:902426e2:::smithi11881747-46:11c failed with -2 2024-03-20T08:09:46.956 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:46.846+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:46.956 INFO:tasks.rados.rados.0.smithi118.stdout:2412: finishing write tid 3 to smithi11881747-49 2024-03-20T08:09:46.956 INFO:tasks.rados.rados.0.smithi118.stdout:2412: oid 49 updating version 0 to 13632 2024-03-20T08:09:46.956 INFO:tasks.rados.rados.0.smithi118.stdout:2412: oid 49 updating version 13632 to 13633 2024-03-20T08:09:46.956 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 13633 (ObjNum 735 snap 251 seq_num 735) dirty exists 2024-03-20T08:09:46.956 INFO:tasks.rados.rados.0.smithi118.stdout:2412: left oid 49 (ObjNum 735 snap 251 seq_num 735) 2024-03-20T08:09:46.965 INFO:tasks.rados.rados.0.smithi118.stdout:2413: expect (ObjNum 381 snap 129 seq_num 381) 2024-03-20T08:09:47.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:46.764+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:47.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:46.764+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:46.754+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:46.754+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:46.756+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:46.756+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:46.913+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:47.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:46.758+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:47.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:46.759+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:47.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:46.759+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:09:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391] boot 2024-03-20T08:09:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: osdmap e551: 8 total, 8 up, 8 in 2024-03-20T08:09:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:09:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: pgmap v507: 121 pgs: 5 peering, 2 active+undersized+degraded+snaptrim_error, 8 active+undersized+degraded, 35 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 782 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 18 MiB/s rd, 2.0 MiB/s wr, 32 op/s; 72/1542 objects degraded (4.669%) 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[93121]: osdmap e552: 8 total, 8 up, 8 in 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-20T08:09:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: osd.7 [v2:172.21.15.196:6824/4230059391,v1:172.21.15.196:6825/4230059391] boot 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: osdmap e551: 8 total, 8 up, 8 in 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: pgmap v507: 121 pgs: 5 peering, 2 active+undersized+degraded+snaptrim_error, 8 active+undersized+degraded, 35 active+undersized, 2 active+clean+snaptrim, 9 active+clean+snaptrim_error, 60 active+clean; 782 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 18 MiB/s rd, 2.0 MiB/s wr, 32 op/s; 72/1542 objects degraded (4.669%) 2024-03-20T08:09:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:46 smithi118 ceph-mon[95692]: osdmap e552: 8 total, 8 up, 8 in 2024-03-20T08:09:47.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.973+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.986+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.207 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:46.999+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.068+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.095+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.109+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.122+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.135+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.193+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.208 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.207+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.209 INFO:tasks.rados.rados.0.smithi118.stdout:2406: done (5 left) 2024-03-20T08:09:47.209 INFO:tasks.rados.rados.0.smithi118.stdout:2408: done (4 left) 2024-03-20T08:09:47.209 INFO:tasks.rados.rados.0.smithi118.stdout:2411: done (3 left) 2024-03-20T08:09:47.209 INFO:tasks.rados.rados.0.smithi118.stdout:2412: done (2 left) 2024-03-20T08:09:47.209 INFO:tasks.rados.rados.0.smithi118.stdout:2413: done (1 left) 2024-03-20T08:09:47.209 INFO:tasks.rados.rados.0.smithi118.stdout:2414: done (0 left) 2024-03-20T08:09:47.209 INFO:tasks.rados.rados.0.smithi118.stdout:2415: delete oid 23 current snap is 251 2024-03-20T08:09:47.211 INFO:tasks.rados.rados.0.smithi118.stdout:2415: done (0 left) 2024-03-20T08:09:47.211 INFO:tasks.rados.rados.0.smithi118.stdout:2416: setattr oid 9 current snap is 251 2024-03-20T08:09:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:2417: snap_remove snap 239 2024-03-20T08:09:47.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.222+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.234+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.248+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.289+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.316+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.328+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.385+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.428+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.454+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.468 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.543 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.605+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.633+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.647+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.675+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.785 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 23101 (ObjNum 622 snap 202 seq_num 622) dirty exists 2024-03-20T08:09:47.785 INFO:tasks.rados.rados.0.smithi118.stdout:2416: done (1 left) 2024-03-20T08:09:47.785 INFO:tasks.rados.rados.0.smithi118.stdout:2417: done (0 left) 2024-03-20T08:09:47.785 INFO:tasks.rados.rados.0.smithi118.stdout:2418: snap_create 2024-03-20T08:09:47.920 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:47.920 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:47.920 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:47.920 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:47.920 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:47.920 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:47.921 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:47.921 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:47.921 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:47.921 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:47.921 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:47.921 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:47.788+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:47.921 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:47.788+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:47.921 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:47.789+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:47.784+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:47.784+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:47.785+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:47.785+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:47.922 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:09:47.922 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (9m) 2m ago 10m 27.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:09:47.922 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (10m) 5s ago 10m 86.8M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (11m) 2m ago 11m 15.4M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 2909c37f9371 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (11m) 5s ago 11m 22.0M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (6m) 5s ago 15m 452M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (6m) 2m ago 16m 508M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (5m) 2m ago 17m 56.2M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (5m) 5s ago 15m 51.5M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (5m) 2m ago 15m 51.1M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (9m) 2m ago 9m 15.5M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (9m) 5s ago 9m 15.6M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (4m) 2m ago 14m 334M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (3m) 2m ago 14m 299M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (3m) 2m ago 13m 251M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (2m) 2m ago 13m 12.7M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (119s) 5s ago 13m 339M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (94s) 5s ago 12m 272M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 72b569d10541 2024-03-20T08:09:47.923 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (55s) 5s ago 12m 273M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 dca658056148 2024-03-20T08:09:47.924 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (11s) 5s ago 11m 15.4M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 52392403da44 2024-03-20T08:09:47.924 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (6m) 5s ago 10m 60.7M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:47.925 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:47.786+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:47.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:47.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:47.984 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:47.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:47.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:47.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:47.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:47.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:47.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:47.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:47.985 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:47.985 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:47.790+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:47.985 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:47.790+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:47.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.741+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.783+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.790+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:47.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.800+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.858+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.908+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.933+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.947+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:47.988 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:47.933+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:48.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:47.934+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:48.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:47.998+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.021+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.033+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.082+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.095+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.110+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.124+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.150+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.162+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.212+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.224+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.238 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.237+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.362 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:09:48.362 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:09:48.362 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:09:48.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:09:48.362 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:09:48.362 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:09:48.362 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:09:48.362 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 8 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 18.2.2-904-g78dbc53a (78dbc53aa74481debe12189646c683a43626f88b) reef (stable)": 2, 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 13 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:09:48.363 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:09:48.494 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.249+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.494 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.494 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.309+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.321+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.345+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.381+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.407+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.495 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.444+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.468+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.496 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.531+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.556+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.757 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.665+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.678+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.705+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.717+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.758 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.741+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:48.790 INFO:tasks.rados.rados.0.smithi118.stdout:2418: done (0 left) 2024-03-20T08:09:48.791 INFO:tasks.rados.rados.0.smithi118.stdout:2419: read oid 42 snap -1 2024-03-20T08:09:48.791 INFO:tasks.rados.rados.0.smithi118.stdout:2419: expect deleted 2024-03-20T08:09:48.791 INFO:tasks.rados.rados.0.smithi118.stdout:2420: snap_remove snap 184 2024-03-20T08:09:49.009 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:48.787+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:49.009 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:48.796+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:49.009 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:48.796+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: osdmap e553: 8 total, 8 up, 8 in 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:49.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: pgmap v510: 121 pgs: 5 peering, 1 active+undersized+degraded+snaptrim_error, 6 active+undersized+degraded, 28 active+undersized, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 69 active+clean; 798 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 25 op/s; 49/1554 objects degraded (3.153%); 4.4 MiB/s, 0 objects/s recovering 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/3392002000' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:49.011 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.757+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.791+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.791+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.802+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.894+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.906+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.012 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.952+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.975+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.986+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:48.998+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.013 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.013 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:09:49.013 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.013 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:49.013 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.013 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-20T08:09:49.014 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-20T08:09:49.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-03-20T08:09:49.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-03-20T08:09:49.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-03-20T08:09:49.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:48 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-03-20T08:09:49.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:48.789+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:49.045 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:48.789+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:49.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:49.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:49.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: osdmap e553: 8 total, 8 up, 8 in 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:49.046 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: pgmap v510: 121 pgs: 5 peering, 1 active+undersized+degraded+snaptrim_error, 6 active+undersized+degraded, 28 active+undersized, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 69 active+clean; 798 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 25 op/s; 49/1554 objects degraded (3.153%); 4.4 MiB/s, 0 objects/s recovering 2024-03-20T08:09:49.047 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3392002000' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.047 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.047 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.047 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:49.047 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:49.047 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:49.047 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: osdmap e553: 8 total, 8 up, 8 in 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:49.048 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: pgmap v510: 121 pgs: 5 peering, 1 active+undersized+degraded+snaptrim_error, 6 active+undersized+degraded, 28 active+undersized, 2 active+clean+snaptrim, 10 active+clean+snaptrim_error, 69 active+clean; 798 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 25 op/s; 49/1554 objects degraded (3.153%); 4.4 MiB/s, 0 objects/s recovering 2024-03-20T08:09:49.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/3392002000' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.049 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:49.049 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:48.849+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:49.049 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:48.788+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:49.049 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:48.788+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-20T08:09:49.050 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-03-20T08:09:49.051 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-20T08:09:49.052 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]: dispatch 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.6"}]': finished 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]: dispatch 2024-03-20T08:09:49.053 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:48 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.7"}]': finished 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.046+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.058+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.082+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.095+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.155+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.168+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.180+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.192+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.217+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.241+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.044+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:49.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.044+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:49.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.045+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:49.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.045+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:49.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.045+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:49.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.276+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.288+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.311+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.322+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.334+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.358+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.395+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.407+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.421+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.469+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.493+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.505+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.518+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.774 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.529+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.774 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.590+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.602+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.665+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.689+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.775 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.725+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.738+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.763+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.776 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.774+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:49.789 INFO:tasks.rados.rados.0.smithi118.stdout:2420: done (1 left) 2024-03-20T08:09:49.789 INFO:tasks.rados.rados.0.smithi118.stdout:2421: copy_from oid 21 from oid 12 current snap is 252 2024-03-20T08:09:49.789 INFO:tasks.rados.rados.0.smithi118.stdout:2419: done (1 left) 2024-03-20T08:09:49.790 INFO:tasks.rados.rados.0.smithi118.stdout:2422: copy_from oid 26 from oid 15 current snap is 252 2024-03-20T08:09:49.790 INFO:tasks.rados.rados.0.smithi118.stdout:2423: delete oid 37 current snap is 252 2024-03-20T08:09:49.803 INFO:tasks.rados.rados.0.smithi118.stdout:2423: done (2 left) 2024-03-20T08:09:49.803 INFO:tasks.rados.rados.0.smithi118.stdout:2424: read oid 32 snap -1 2024-03-20T08:09:49.803 INFO:tasks.rados.rados.0.smithi118.stdout:2424: expect (ObjNum 721 snap 249 seq_num 721) 2024-03-20T08:09:49.803 INFO:tasks.rados.rados.0.smithi118.stdout:2425: snap_remove snap 251 2024-03-20T08:09:50.030 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:09:49.815+0000 7fafa758d700 -1 osd.6 pg_epoch: 555 pg[5.9s1( v 552'1150 (0'0,552'1150] local-lis/les=551/552 n=20 ec=73/73 lis/c=551/551 les/c/f=552/552/0 sis=551) [7,6,1]p7(0) r=1 lpr=551 luod=0'0 crt=552'1150 mlcod 552'1150 active mbc={} ps=121] update_snap_map remove_oid 5:902426e2:::smithi11881747-46:f6 failed with -2 2024-03-20T08:09:50.030 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:49.804+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:50.030 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:49.804+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:50.030 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:49.813+0000 7ffb40715700 -1 osd.7 pg_epoch: 555 pg[5.4s0( v 552'277 (0'0,552'277] local-lis/les=551/552 n=4 ec=73/73 lis/c=551/551 les/c/f=552/552/0 sis=551) [7,2,5]p7(0) r=0 lpr=551 crt=552'277 lcod 534'275 mlcod 534'275 active+clean+snaptrim trimq=[f4~1] ps=130] update_snap_map remove_oid 5:27618338:::smithi11881747-17:f4 failed with -2 2024-03-20T08:09:50.031 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:49.814+0000 7ffb3df10700 -1 osd.7 pg_epoch: 555 pg[5.9s0( v 552'1150 (0'0,552'1150] local-lis/les=551/552 n=20 ec=73/73 lis/c=551/551 les/c/f=552/552/0 sis=551) [7,6,1]p7(0) r=0 lpr=551 crt=552'1150 lcod 552'1148 mlcod 552'1148 active+clean+snaptrim trimq=[f4~1,11a~1,11f~1,122~1,124~3,129~1,12b~1,131~1] ps=121] update_snap_map remove_oid 5:902426e2:::smithi11881747-46:f6 failed with -2 2024-03-20T08:09:50.031 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:49.819+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:50.031 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:49.821+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:09:50.031 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:49.821+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:50.031 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:50.031 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all osd 2024-03-20T08:09:50.031 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: Health check update: Degraded data redundancy: 49/1554 objects degraded (3.153%), 7 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:50.031 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: Upgrade: Setting require_osd_release to 19 squid 2024-03-20T08:09:50.031 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-20T08:09:50.031 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:50.032 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:50.032 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:50.032 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:50.032 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: osdmap e554: 8 total, 8 up, 8 in 2024-03-20T08:09:50.032 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:50.032 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:50.032 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:50.032 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:50.032 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.786+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.797+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.809+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.867+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.879+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.916+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.929+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:49.993+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.006+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.018+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.030+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:49.795+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:50.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:49.795+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:50.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:49.797+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:50.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.789+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:50.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.789+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:50.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.790+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:50.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:49.790+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all osd 2024-03-20T08:09:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: Health check update: Degraded data redundancy: 49/1554 objects degraded (3.153%), 7 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: Upgrade: Setting require_osd_release to 19 squid 2024-03-20T08:09:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: osdmap e554: 8 total, 8 up, 8 in 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:50.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:49.804+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:50.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:49.910+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all osd 2024-03-20T08:09:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: Health check update: Degraded data redundancy: 49/1554 objects degraded (3.153%), 7 pgs degraded (PG_DEGRADED) 2024-03-20T08:09:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: Upgrade: Setting require_osd_release to 19 squid 2024-03-20T08:09:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-20T08:09:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],f6=[f4],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:11c 2024-03-20T08:09:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:50.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:50.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: osdmap e554: 8 total, 8 up, 8 in 2024-03-20T08:09:50.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:50.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:50.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:50.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:50.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.078+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.091+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.104+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.141+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.201+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.237+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.250+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.288+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.545 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.337+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.374+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.424+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.546 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.460+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.472+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.485+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.496+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.509+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.533+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.547 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.545+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.559+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.571+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.584+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.618+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.629+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.702+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:50.803 INFO:tasks.rados.rados.0.smithi118.stdout:2425: done (3 left) 2024-03-20T08:09:50.803 INFO:tasks.rados.rados.0.smithi118.stdout:2426: copy_from oid 37 from oid 44 current snap is 252 2024-03-20T08:09:50.803 INFO:tasks.rados.rados.0.smithi118.stdout:2422: finishing copy_from to smithi11881747-26 2024-03-20T08:09:50.804 INFO:tasks.rados.rados.0.smithi118.stdout:2422: got expected ENOENT (src dne) 2024-03-20T08:09:50.804 INFO:tasks.rados.rados.0.smithi118.stdout:2422: finishing copy_from racing read to smithi11881747-26 2024-03-20T08:09:50.804 INFO:tasks.rados.rados.0.smithi118.stdout:2424: expect (ObjNum 721 snap 249 seq_num 721) 2024-03-20T08:09:50.806 INFO:tasks.rados.rados.0.smithi118.stdout:2421: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:09:50.806 INFO:tasks.rados.rados.0.smithi118.stdout:2421: finishing copy_from to smithi11881747-21 2024-03-20T08:09:50.806 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34004 (ObjNum 722 snap 249 seq_num 722) dirty exists 2024-03-20T08:09:50.806 INFO:tasks.rados.rados.0.smithi118.stdout:2421: done (3 left) 2024-03-20T08:09:50.806 INFO:tasks.rados.rados.0.smithi118.stdout:2422: done (2 left) 2024-03-20T08:09:50.806 INFO:tasks.rados.rados.0.smithi118.stdout:2424: done (1 left) 2024-03-20T08:09:50.806 INFO:tasks.rados.rados.0.smithi118.stdout:2427: delete oid 20 current snap is 252 2024-03-20T08:09:50.814 INFO:tasks.rados.rados.0.smithi118.stdout:2427: done (1 left) 2024-03-20T08:09:50.815 INFO:tasks.rados.rados.0.smithi118.stdout:2428: read oid 3 snap -1 2024-03-20T08:09:50.815 INFO:tasks.rados.rados.0.smithi118.stdout:2428: expect (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:09:50.815 INFO:tasks.rados.rados.0.smithi118.stdout:2429: snap_create 2024-03-20T08:09:50.837 INFO:tasks.rados.rados.0.smithi118.stdout:2428: expect (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:09:50.998 INFO:tasks.rados.rados.0.smithi118.stdout:2426: finishing copy_from racing read to smithi11881747-37 2024-03-20T08:09:50.998 INFO:tasks.rados.rados.0.smithi118.stdout:2426: finishing copy_from to smithi11881747-37 2024-03-20T08:09:50.999 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 34008 (ObjNum 503 snap 166 seq_num 503) dirty exists 2024-03-20T08:09:51.051 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:50.807+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:51.051 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:50.807+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: osdmap e555: 8 total, 8 up, 8 in 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:51.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: Upgrade: It appears safe to stop mds.a 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:51.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-mon[87080]: pgmap v513: 121 pgs: 5 peering, 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 101 active+clean; 796 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 10 MiB/s rd, 268 KiB/s wr, 8 op/s; 7.9 MiB/s, 1 objects/s recovering 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.805+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.805+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.831+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.870+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.882+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.893+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.918+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.954+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.966+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:50.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.016+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.028+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.039+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:50.816+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:51.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:50.802+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:51.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:50.803+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:51.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:50.815+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:51.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:50.815+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:51.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:50.806+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:51.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:50.806+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:51.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:50.807+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:51.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:50.807+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:51.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:50.807+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:51.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:50.792+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 13 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:09:51.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:51.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:51.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:51.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:51.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-20T08:09:51.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: osdmap e555: 8 total, 8 up, 8 in 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: Upgrade: It appears safe to stop mds.a 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:51.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[93121]: pgmap v513: 121 pgs: 5 peering, 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 101 active+clean; 796 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 10 MiB/s rd, 268 KiB/s wr, 8 op/s; 7.9 MiB/s, 1 objects/s recovering 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-20T08:09:51.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: osdmap e555: 8 total, 8 up, 8 in 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["a"]}]: dispatch 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: Upgrade: It appears safe to stop mds.a 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:51.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:51.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:51.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:51.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:51.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:50 smithi118 ceph-mon[95692]: pgmap v513: 121 pgs: 5 peering, 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 101 active+clean; 796 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 10 MiB/s rd, 268 KiB/s wr, 8 op/s; 7.9 MiB/s, 1 objects/s recovering 2024-03-20T08:09:51.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.064+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.111+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.124+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.136+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.149+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.173+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.208+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.267+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.292+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.305 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.563 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:51.371+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:51.563 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:51.371+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.317+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.366+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.379+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.392+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.405+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.454+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.490+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.806 INFO:tasks.rados.rados.0.smithi118.stdout:2426: done (2 left) 2024-03-20T08:09:51.807 INFO:tasks.rados.rados.0.smithi118.stdout:2428: done (1 left) 2024-03-20T08:09:51.807 INFO:tasks.rados.rados.0.smithi118.stdout:2429: done (0 left) 2024-03-20T08:09:51.807 INFO:tasks.rados.rados.0.smithi118.stdout:2430: read oid 47 snap -1 2024-03-20T08:09:51.807 INFO:tasks.rados.rados.0.smithi118.stdout:2430: expect deleted 2024-03-20T08:09:51.807 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 253 2024-03-20T08:09:51.807 INFO:tasks.rados.rados.0.smithi118.stdout:2431: seq_num 740 ranges {9355264=401408} 2024-03-20T08:09:51.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 13 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:09:51.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/1554 objects degraded (3.153%), 7 pgs degraded) 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: osdmap e556: 8 total, 8 up, 8 in 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:51.814 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:51.815 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:51.813+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:51.815 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:51.813+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:51.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.601+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.614+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.628+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.815 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.651+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.676+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:51.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.811+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:51.860 INFO:tasks.rados.rados.0.smithi118.stdout:2431: writing smithi11881747-12 from 9355264 to 9756672 tid 1 2024-03-20T08:09:51.860 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 253 2024-03-20T08:09:51.860 INFO:tasks.rados.rados.0.smithi118.stdout:2432: seq_num 741 ranges {1458176=417792} 2024-03-20T08:09:51.870 INFO:tasks.rados.rados.0.smithi118.stdout:2432: writing smithi11881747-8 from 1458176 to 1875968 tid 1 2024-03-20T08:09:51.871 INFO:tasks.rados.rados.0.smithi118.stdout:2433: delete oid 31 current snap is 253 2024-03-20T08:09:51.871 INFO:tasks.rados.rados.0.smithi118.stdout:2431: finishing write tid 1 to smithi11881747-12 2024-03-20T08:09:51.871 INFO:tasks.rados.rados.0.smithi118.stdout:2431: finishing write tid 2 to smithi11881747-12 2024-03-20T08:09:51.874 INFO:tasks.rados.rados.0.smithi118.stdout:2430: done (3 left) 2024-03-20T08:09:51.874 INFO:tasks.rados.rados.0.smithi118.stdout:2433: done (2 left) 2024-03-20T08:09:51.874 INFO:tasks.rados.rados.0.smithi118.stdout:2434: delete oid 10 current snap is 253 2024-03-20T08:09:51.875 INFO:tasks.rados.rados.0.smithi118.stdout:2431: finishing write tid 3 to smithi11881747-12 2024-03-20T08:09:51.875 INFO:tasks.rados.rados.0.smithi118.stdout:2431: oid 12 updating version 0 to 34006 2024-03-20T08:09:51.875 INFO:tasks.rados.rados.0.smithi118.stdout:2431: oid 12 version 34006 is already newer than 34005 2024-03-20T08:09:51.875 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34006 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:09:51.875 INFO:tasks.rados.rados.0.smithi118.stdout:2431: left oid 12 (ObjNum 740 snap 253 seq_num 740) 2024-03-20T08:09:51.879 INFO:tasks.rados.rados.0.smithi118.stdout:2431: done (2 left) 2024-03-20T08:09:51.879 INFO:tasks.rados.rados.0.smithi118.stdout:2434: done (1 left) 2024-03-20T08:09:51.879 INFO:tasks.rados.rados.0.smithi118.stdout:2435: read oid 46 snap -1 2024-03-20T08:09:51.879 INFO:tasks.rados.rados.0.smithi118.stdout:2435: expect deleted 2024-03-20T08:09:51.879 INFO:tasks.rados.rados.0.smithi118.stdout:2436: read oid 44 snap -1 2024-03-20T08:09:51.879 INFO:tasks.rados.rados.0.smithi118.stdout:2436: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:09:51.880 INFO:tasks.rados.rados.0.smithi118.stdout:2437: snap_remove snap 243 2024-03-20T08:09:52.066 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:51.865+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:52.067 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:51.865+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:09:52.067 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:51.865+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:52.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.848+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.862+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.923+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.947+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.970+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:51.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.008+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.020+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.055+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:51.806+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:52.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:51.806+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:52.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:51.806+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:52.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:51.809+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:52.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:51.810+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:52.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:51.810+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:52.070 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:51.810+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:52.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 13 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:09:52.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/1554 objects degraded (3.153%), 7 pgs degraded) 2024-03-20T08:09:52.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: osdmap e556: 8 total, 8 up, 8 in 2024-03-20T08:09:52.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:52.071 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 13 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 49/1554 objects degraded (3.153%), 7 pgs degraded) 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: osdmap e556: 8 total, 8 up, 8 in 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:52.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:52.073 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:51.882+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:52.328 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:09:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:09:52.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.081+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.105+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.153+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.165+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.217+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.329 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.330 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.328+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:52.069+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.354+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.367+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.380+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.392+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.404+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.427+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.439+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.450+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.462+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.475+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.489+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.501+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.513+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.526+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.549+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.574+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.814 INFO:tasks.rados.rados.0.smithi118.stdout:2437: done (3 left) 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:2438: delete oid 7 current snap is 253 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:2432: finishing write tid 1 to smithi11881747-8 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:2432: finishing write tid 2 to smithi11881747-8 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:2432: finishing write tid 3 to smithi11881747-8 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:2432: oid 8 updating version 0 to 34012 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:2432: oid 8 version 34012 is already newer than 34011 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34012 (ObjNum 741 snap 253 seq_num 741) dirty exists 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:2432: left oid 8 (ObjNum 741 snap 253 seq_num 741) 2024-03-20T08:09:52.815 INFO:tasks.rados.rados.0.smithi118.stdout:2436: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:09:52.840 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:52.823+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:52.840 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:52.824+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:52.840 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:52.819+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:52.840 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:52.819+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: osdmap e557: 8 total, 8 up, 8 in 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:52.841 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:52.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:52.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:52.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:52.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:52.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: pgmap v516: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 806 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 8 op/s; 3.5 MiB/s, 1 objects/s recovering 2024-03-20T08:09:52.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:52.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:09:52.842 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:52.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.597+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.620+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.632+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.645+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.670+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.682+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.695+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.720+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.745+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.757+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.843 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.820+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:52.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.821+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:52.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.844 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:52.904 INFO:tasks.rados.rados.0.smithi118.stdout:2432: done (3 left) 2024-03-20T08:09:52.904 INFO:tasks.rados.rados.0.smithi118.stdout:2435: done (2 left) 2024-03-20T08:09:52.905 INFO:tasks.rados.rados.0.smithi118.stdout:2436: done (1 left) 2024-03-20T08:09:52.905 INFO:tasks.rados.rados.0.smithi118.stdout:2438: done (0 left) 2024-03-20T08:09:52.905 INFO:tasks.rados.rados.0.smithi118.stdout:2439: snap_create 2024-03-20T08:09:53.082 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:52.818+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:53.082 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:52.818+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:53.082 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:52.812+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:53.083 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:52.818+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:53.083 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:52.820+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:53.083 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:52.824+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:53.083 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:52.824+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:53.083 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:52.824+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:53.083 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:53.083 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:53.083 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: osdmap e557: 8 total, 8 up, 8 in 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: pgmap v516: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 806 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 8 op/s; 3.5 MiB/s, 1 objects/s recovering 2024-03-20T08:09:53.084 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:53.085 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:09:53.085 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:53.085 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:52.815+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:53.085 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:52.815+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:53.085 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:53.085 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:53.085 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:53.085 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: osdmap e557: 8 total, 8 up, 8 in 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:53.086 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: pgmap v516: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 806 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 123 KiB/s rd, 2.4 MiB/s wr, 8 op/s; 3.5 MiB/s, 1 objects/s recovering 2024-03-20T08:09:53.087 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:53.087 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:09:53.087 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:52 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:09:53.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.888+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.900+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.986+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:52.998+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.011+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.024+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.035+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.085+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.109+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.121+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.134+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.146+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.168+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.180+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.193+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.204+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.240+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.265+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.288+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.300+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.325+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.337+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.386+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.397+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.422+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.447+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.485+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.498+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.559+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.611 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.597+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.612 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:09:53] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:09:53.819 INFO:tasks.rados.rados.0.smithi118.stdout:2439: done (0 left) 2024-03-20T08:09:53.819 INFO:tasks.rados.rados.0.smithi118.stdout:2440: read oid 50 snap -1 2024-03-20T08:09:53.819 INFO:tasks.rados.rados.0.smithi118.stdout:2440: expect (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:09:53.819 INFO:tasks.rados.rados.0.smithi118.stdout:2441: copy_from oid 3 from oid 12 current snap is 254 2024-03-20T08:09:53.819 INFO:tasks.rados.rados.0.smithi118.stdout:2442: copy_from oid 42 from oid 16 current snap is 254 2024-03-20T08:09:53.819 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 254 2024-03-20T08:09:53.819 INFO:tasks.rados.rados.0.smithi118.stdout:2443: seq_num 745 ranges {0=483328} 2024-03-20T08:09:53.822 INFO:tasks.rados.rados.0.smithi118.stdout:2443: writing smithi11881747-38 from 0 to 483328 tid 1 2024-03-20T08:09:53.822 INFO:tasks.rados.rados.0.smithi118.stdout:2444: snap_create 2024-03-20T08:09:53.822 INFO:tasks.rados.rados.0.smithi118.stdout:2442: finishing copy_from to smithi11881747-42 2024-03-20T08:09:53.822 INFO:tasks.rados.rados.0.smithi118.stdout:2442: got expected ENOENT (src dne) 2024-03-20T08:09:53.822 INFO:tasks.rados.rados.0.smithi118.stdout:2442: finishing copy_from racing read to smithi11881747-42 2024-03-20T08:09:53.833 INFO:tasks.rados.rados.0.smithi118.stdout:2443: finishing write tid 1 to smithi11881747-38 2024-03-20T08:09:53.834 INFO:tasks.rados.rados.0.smithi118.stdout:2443: finishing write tid 2 to smithi11881747-38 2024-03-20T08:09:53.839 INFO:tasks.rados.rados.0.smithi118.stdout:2443: finishing write tid 3 to smithi11881747-38 2024-03-20T08:09:53.839 INFO:tasks.rados.rados.0.smithi118.stdout:2443: oid 38 updating version 0 to 34015 2024-03-20T08:09:53.839 INFO:tasks.rados.rados.0.smithi118.stdout:2443: oid 38 updating version 34015 to 34016 2024-03-20T08:09:53.839 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 34016 (ObjNum 745 snap 254 seq_num 745) dirty exists 2024-03-20T08:09:53.839 INFO:tasks.rados.rados.0.smithi118.stdout:2443: left oid 38 (ObjNum 745 snap 254 seq_num 745) 2024-03-20T08:09:53.844 INFO:tasks.rados.rados.0.smithi118.stdout:2440: expect (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:09:53.867 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:53.821+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:53.868 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:53.821+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:53.868 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:53.826+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:53.868 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:53.826+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:09:53.868 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:53.826+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:53.868 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: Upgrade: Updating mds.a 2024-03-20T08:09:53.868 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: Deploying daemon mds.a on smithi118 2024-03-20T08:09:53.868 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: osdmap e558: 8 total, 8 up, 8 in 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:53.869 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.623+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.635+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.647+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.660+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.674+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.686+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.737+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.870 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.761+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.798+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.820+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.835+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.871 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:53.992 INFO:tasks.rados.rados.0.smithi118.stdout:2441: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:09:53.992 INFO:tasks.rados.rados.0.smithi118.stdout:2441: finishing copy_from to smithi11881747-3 2024-03-20T08:09:53.992 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34006 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.905+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:53.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.001+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.062+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:53.816+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:53.816+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:53.816+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:54.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:53.852+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:53.821+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:53.821+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:53.821+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:53.821+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:54.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:53.815+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: Upgrade: Updating mds.a 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: Deploying daemon mds.a on smithi118 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: osdmap e558: 8 total, 8 up, 8 in 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:54.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: Upgrade: Updating mds.a 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: Deploying daemon mds.a on smithi118 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: osdmap e558: 8 total, 8 up, 8 in 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:54.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:54.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.148+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.172+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.210+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.222+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.235+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.247+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.259+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.326+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.339+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.462+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.475+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.537+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.587+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.600+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.764 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:54 smithi118 systemd[1]: Stopping Ceph mds.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:09:54.819 INFO:tasks.rados.rados.0.smithi118.stdout:2440: done (4 left) 2024-03-20T08:09:54.819 INFO:tasks.rados.rados.0.smithi118.stdout:2441: done (3 left) 2024-03-20T08:09:54.819 INFO:tasks.rados.rados.0.smithi118.stdout:2442: done (2 left) 2024-03-20T08:09:54.819 INFO:tasks.rados.rados.0.smithi118.stdout:2443: done (1 left) 2024-03-20T08:09:54.820 INFO:tasks.rados.rados.0.smithi118.stdout:2444: done (0 left) 2024-03-20T08:09:54.820 INFO:tasks.rados.rados.0.smithi118.stdout:2445: read oid 8 snap -1 2024-03-20T08:09:54.820 INFO:tasks.rados.rados.0.smithi118.stdout:2445: expect (ObjNum 741 snap 253 seq_num 741) 2024-03-20T08:09:54.820 INFO:tasks.rados.rados.0.smithi118.stdout:2446: read oid 28 snap -1 2024-03-20T08:09:54.820 INFO:tasks.rados.rados.0.smithi118.stdout:2446: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:09:54.820 INFO:tasks.rados.rados.0.smithi118.stdout:2447: rmattr oid 14 current snap is 255 2024-03-20T08:09:54.820 INFO:tasks.rados.rados.0.smithi118.stdout:2447: done (2 left) 2024-03-20T08:09:54.820 INFO:tasks.rados.rados.0.smithi118.stdout:2448: delete oid 18 current snap is 255 2024-03-20T08:09:54.823 INFO:tasks.rados.rados.0.smithi118.stdout:2448: done (2 left) 2024-03-20T08:09:54.823 INFO:tasks.rados.rados.0.smithi118.stdout:2449: snap_remove snap 219 2024-03-20T08:09:54.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:54.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: osdmap e559: 8 total, 8 up, 8 in 2024-03-20T08:09:54.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:54.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:54.880 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:54.881 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-mon[87080]: pgmap v519: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 808 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 9 op/s 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.637+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.685+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.697+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.709+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.734+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.786+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.822+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:54.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.822+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:54.883 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:54.821+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:54.883 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:54.821+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:54.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.828+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.855+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.868+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:54.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.893+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.906+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.918+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.931+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:54.994+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.019+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.031+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.043+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.055+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.132 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.133 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.081+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.133 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.093+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.133 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.133 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.118+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.133 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.131+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:54.826+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:55.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:54.827+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:55.172 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[67464]: 2024-03-20T08:09:54.763+0000 7ff7006de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:09:55.173 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[67464]: 2024-03-20T08:09:54.763+0000 7ff7006de700 -1 mds.a *** got signal Terminated *** 2024-03-20T08:09:55.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:54.820+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:55.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:54.822+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:55.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:54.822+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:55.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:54.823+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:55.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:54.823+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:55.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: osdmap e559: 8 total, 8 up, 8 in 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[93121]: pgmap v519: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 808 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 9 op/s 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: osdmap e559: 8 total, 8 up, 8 in 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:55.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-mon[95692]: pgmap v519: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 808 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 9 op/s 2024-03-20T08:09:55.176 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:54.817+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:55.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:54.822+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:55.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:54.822+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:55.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:55.250+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:55.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:55.251+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.180+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.193+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.206+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.218+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.230+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.270+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.367+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.379+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.416+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.488+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.650 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.500+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.549+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.599+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.637+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.649+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.835 INFO:tasks.rados.rados.0.smithi118.stdout:2449: done (2 left) 2024-03-20T08:09:55.835 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 255 2024-03-20T08:09:55.835 INFO:tasks.rados.rados.0.smithi118.stdout:2450: seq_num 747 ranges {9355264=516096} 2024-03-20T08:09:55.889 INFO:tasks.rados.rados.0.smithi118.stdout:2450: writing smithi11881747-21 from 9355264 to 9871360 tid 1 2024-03-20T08:09:55.889 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 255 2024-03-20T08:09:55.889 INFO:tasks.rados.rados.0.smithi118.stdout:2451: seq_num 748 ranges {0=532480} 2024-03-20T08:09:55.891 INFO:tasks.rados.rados.0.smithi118.stdout:2451: writing smithi11881747-39 from 0 to 532480 tid 1 2024-03-20T08:09:55.892 INFO:tasks.rados.rados.0.smithi118.stdout:2452: snap_remove snap 241 2024-03-20T08:09:55.911 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:55.839+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:55.911 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:55.840+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:55.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:55.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: osdmap e560: 8 total, 8 up, 8 in 2024-03-20T08:09:55.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:55.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:55.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:55.911 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:55.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:55.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:55.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:55.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:55.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:55.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:55.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-mon[87080]: fsmap cephfs:1 {0=b=up:active} 2024-03-20T08:09:55.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.683+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.720+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.758+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.784+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.833+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.913 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.840+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:55.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.862+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.875+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.887+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.899+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:55.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.172 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:56.072+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:56.172 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:56.072+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:09:56.172 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:56.072+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:56.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:55.833+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:56.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:55.834+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:56.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.924+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.962+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:55.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.048+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.124+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.148+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:55.837+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:55.838+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:56.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:55.838+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: osdmap e560: 8 total, 8 up, 8 in 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[93121]: fsmap cephfs:1 {0=b=up:active} 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: osdmap e560: 8 total, 8 up, 8 in 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:56.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:55 smithi118 ceph-mon[95692]: fsmap cephfs:1 {0=b=up:active} 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.431 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.282+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.295+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.308+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.321+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.358+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.382+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.432 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.430+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.458+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.470+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.531+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.544+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.556+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.568+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.580+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.606+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.630+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.643+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.655+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.679+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.841 INFO:tasks.rados.rados.0.smithi118.stdout:2452: done (4 left) 2024-03-20T08:09:56.841 INFO:tasks.rados.rados.0.smithi118.stdout:2453: setattr oid 10 current snap is 255 2024-03-20T08:09:56.841 INFO:tasks.rados.rados.0.smithi118.stdout:2445: expect (ObjNum 741 snap 253 seq_num 741) 2024-03-20T08:09:56.945 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:56.913+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:56.945 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:56.914+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:56.946 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:56.845+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:56.946 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:56.845+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:56.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:56.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:56.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: osdmap e561: 8 total, 8 up, 8 in 2024-03-20T08:09:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:56.947 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-mon[87080]: pgmap v522: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 808 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 7.4 MiB/s rd, 642 KiB/s wr, 6 op/s 2024-03-20T08:09:56.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.703+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.728+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.803+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.843+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.847+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.847+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.857+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.869+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.880+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.893+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.949 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:56.954 INFO:tasks.rados.rados.0.smithi118.stdout:2446: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:09:57.132 INFO:tasks.rados.rados.0.smithi118.stdout:2451: finishing write tid 1 to smithi11881747-39 2024-03-20T08:09:57.132 INFO:tasks.rados.rados.0.smithi118.stdout:2451: finishing write tid 2 to smithi11881747-39 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2451: finishing write tid 3 to smithi11881747-39 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2451: oid 39 updating version 0 to 13635 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2451: oid 39 updating version 13635 to 13636 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 13636 (ObjNum 748 snap 255 seq_num 748) dirty exists 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2451: left oid 39 (ObjNum 748 snap 255 seq_num 748) 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2450: finishing write tid 1 to smithi11881747-21 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2450: finishing write tid 2 to smithi11881747-21 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2450: finishing write tid 3 to smithi11881747-21 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2450: oid 21 updating version 0 to 34018 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2450: oid 21 version 34018 is already newer than 34017 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34018 (ObjNum 747 snap 255 seq_num 747) dirty exists 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2450: left oid 21 (ObjNum 747 snap 255 seq_num 747) 2024-03-20T08:09:57.133 INFO:tasks.rados.rados.0.smithi118.stdout:2445: done (4 left) 2024-03-20T08:09:57.134 INFO:tasks.rados.rados.0.smithi118.stdout:2446: done (3 left) 2024-03-20T08:09:57.134 INFO:tasks.rados.rados.0.smithi118.stdout:2450: done (2 left) 2024-03-20T08:09:57.134 INFO:tasks.rados.rados.0.smithi118.stdout:2451: done (1 left) 2024-03-20T08:09:57.134 INFO:tasks.rados.rados.0.smithi118.stdout:2454: rmattr oid 39 current snap is 255 2024-03-20T08:09:57.134 INFO:tasks.rados.rados.0.smithi118.stdout:2454: done (1 left) 2024-03-20T08:09:57.134 INFO:tasks.rados.rados.0.smithi118.stdout:2455: rollback oid 23 current snap is 255 2024-03-20T08:09:57.134 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 23 to 213 2024-03-20T08:09:57.134 INFO:tasks.rados.rados.0.smithi118.stdout:2456: snap_remove snap 202 2024-03-20T08:09:57.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:56.843+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:57.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:56.843+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:57.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:57.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: osdmap e561: 8 total, 8 up, 8 in 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[93121]: pgmap v522: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 808 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 7.4 MiB/s rd, 642 KiB/s wr, 6 op/s 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.809+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.809+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.810+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.810+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.837+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.838+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.838+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.838+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:57.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:56.838+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:57.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:56.911+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:57.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:56.838+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:57.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:56.838+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: osdmap e561: 8 total, 8 up, 8 in 2024-03-20T08:09:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:56 smithi118 ceph-mon[95692]: pgmap v522: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 808 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 7.4 MiB/s rd, 642 KiB/s wr, 6 op/s 2024-03-20T08:09:57.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.984+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:56.997+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.033+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.057+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.081+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.143+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.167+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.205 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.179+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.191+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.206 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.458 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.458 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.231+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.458 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.244+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.458 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.260+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.458 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.273+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.458 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.458 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.301+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.315+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.328+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.340+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.354+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.367+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.380+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.406+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.419+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.431+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.459 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.444+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.485+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.511+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.564+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.591+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.620+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.633+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.648+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.661+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.690+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.721+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.844 INFO:tasks.rados.rados.0.smithi118.stdout:2456: done (2 left) 2024-03-20T08:09:57.844 INFO:tasks.rados.rados.0.smithi118.stdout:2457: delete oid 19 current snap is 255 2024-03-20T08:09:57.844 INFO:tasks.rados.rados.0.smithi118.stdout:2455: finishing rollback tid 1 to smithi11881747-23 2024-03-20T08:09:57.844 INFO:tasks.rados.rados.0.smithi118.stdout:2455: finishing rollback tid 2 to smithi11881747-23 2024-03-20T08:09:57.844 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 33955 (ObjNum 626 snap 203 seq_num 626) dirty exists 2024-03-20T08:09:57.844 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 34030 (ObjNum 65 snap 0 seq_num 0) dirty exists 2024-03-20T08:09:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:2453: done (2 left) 2024-03-20T08:09:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:2455: done (1 left) 2024-03-20T08:09:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:2457: done (0 left) 2024-03-20T08:09:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:2458: read oid 50 snap -1 2024-03-20T08:09:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:2458: expect (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:09:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:2459: snap_remove snap 254 2024-03-20T08:09:57.975 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:57.853+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:57.975 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:57.853+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:57.975 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: osdmap e562: 8 total, 8 up, 8 in 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:57.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:57.977 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:57.977 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:57.977 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:57.977 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:57.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.767+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.822+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.852+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.893+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.935+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.978 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:57.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.976+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:57.843+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:58.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:57.846+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:58.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:57.848+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:58.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:57.848+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:58.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:57.848+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: osdmap e562: 8 total, 8 up, 8 in 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: osdmap e562: 8 total, 8 up, 8 in 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:58.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:57.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.002+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.015+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.043+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.072+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.085+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.098+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.126+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.140+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.154+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.181+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.194+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.236+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.263+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.274+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.302+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.360+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.391+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.434+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.446+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.459+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.501+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.581+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.607+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.620+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.632+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.674+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.702+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.729+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.742+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.851 INFO:tasks.rados.rados.0.smithi118.stdout:2459: done (1 left) 2024-03-20T08:09:58.851 INFO:tasks.rados.rados.0.smithi118.stdout:2460: read oid 33 snap -1 2024-03-20T08:09:58.851 INFO:tasks.rados.rados.0.smithi118.stdout:2460: expect (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:09:58.851 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 255 2024-03-20T08:09:58.851 INFO:tasks.rados.rados.0.smithi118.stdout:2461: seq_num 750 ranges {532480=565248} 2024-03-20T08:09:58.857 INFO:tasks.rados.rados.0.smithi118.stdout:2461: writing smithi11881747-39 from 532480 to 1097728 tid 1 2024-03-20T08:09:58.857 INFO:tasks.rados.rados.0.smithi118.stdout:2462: snap_create 2024-03-20T08:09:58.857 INFO:tasks.rados.rados.0.smithi118.stdout:2458: expect (ObjNum 663 snap 216 seq_num 663) 2024-03-20T08:09:58.995 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:58.853+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:58.995 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:58.856+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:58.996 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:58.812+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:58.996 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:58.813+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:09:58.996 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:58.813+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:58.996 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:58.996 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: osdmap e563: 8 total, 8 up, 8 in 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: pgmap v525: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 821 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 7.8 MiB/s rd, 2.6 MiB/s wr, 7 op/s 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:58.997 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.770+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.784+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.855+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.855+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.998 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.882+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.922+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.947+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.971+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:58.999 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:58.996+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.008 INFO:tasks.rados.rados.0.smithi118.stdout:2461: finishing write tid 1 to smithi11881747-39 2024-03-20T08:09:59.008 INFO:tasks.rados.rados.0.smithi118.stdout:2461: finishing write tid 2 to smithi11881747-39 2024-03-20T08:09:59.008 INFO:tasks.rados.rados.0.smithi118.stdout:2461: finishing write tid 3 to smithi11881747-39 2024-03-20T08:09:59.008 INFO:tasks.rados.rados.0.smithi118.stdout:2461: oid 39 updating version 0 to 13637 2024-03-20T08:09:59.008 INFO:tasks.rados.rados.0.smithi118.stdout:2461: oid 39 updating version 13637 to 13638 2024-03-20T08:09:59.008 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 13638 (ObjNum 750 snap 255 seq_num 750) dirty exists 2024-03-20T08:09:59.008 INFO:tasks.rados.rados.0.smithi118.stdout:2461: left oid 39 (ObjNum 750 snap 255 seq_num 750) 2024-03-20T08:09:59.008 INFO:tasks.rados.rados.0.smithi118.stdout:2460: expect (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:09:59.129 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:58.850+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:59.129 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:58.851+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:59.129 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:58 smithi118 podman[121276]: 2024-03-20 08:09:58.770330182 +0000 UTC m=+4.134849637 container died 2909c37f937131c8a703b63a3b882aebeae9e8f8a7d66634ac4808d442b21253 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=reef-78dbc53, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-20T08:09:59.130 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:58.862+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:59.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.811+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:59.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.812+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:59.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.812+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:59.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.812+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:59.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.854+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:59.130 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.854+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:59.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.855+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:59.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.858+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:09:59.131 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:58.858+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:59.131 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:59.131 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:59.131 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:59.131 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: osdmap e563: 8 total, 8 up, 8 in 2024-03-20T08:09:59.131 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:59.131 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:59.131 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: pgmap v525: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 821 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 7.8 MiB/s rd, 2.6 MiB/s wr, 7 op/s 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:59.132 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: osdmap e563: 8 total, 8 up, 8 in 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: pgmap v525: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 821 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 7.8 MiB/s rd, 2.6 MiB/s wr, 7 op/s 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:09:59.133 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:09:59.134 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:58.851+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:09:59.134 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:58.851+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:59.255 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.255 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.255 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.255 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.048+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.255 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.060+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.255 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.122+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.134+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.183+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.195+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.218+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.231+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.256 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.422 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:59 smithi118 podman[121276]: 2024-03-20 08:09:59.321177299 +0000 UTC m=+4.685696750 container cleanup 2909c37f937131c8a703b63a3b882aebeae9e8f8a7d66634ac4808d442b21253 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, RELEASE=reef-78dbc53, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-20T08:09:59.422 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:59 smithi118 bash[121276]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a 2024-03-20T08:09:59.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.255+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.268+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.281+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.513 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.354+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.367+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.380+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.392+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.405+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.430+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.489+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.514 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.771 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:59.572+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:59.771 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:09:59.573+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:59.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.526+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.539+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.602+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.627+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.772 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.711+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.723+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.773 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:09:59.856 INFO:tasks.rados.rados.0.smithi118.stdout:2458: done (3 left) 2024-03-20T08:09:59.856 INFO:tasks.rados.rados.0.smithi118.stdout:2460: done (2 left) 2024-03-20T08:09:59.857 INFO:tasks.rados.rados.0.smithi118.stdout:2461: done (1 left) 2024-03-20T08:09:59.857 INFO:tasks.rados.rados.0.smithi118.stdout:2462: done (0 left) 2024-03-20T08:09:59.857 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 256 2024-03-20T08:09:59.857 INFO:tasks.rados.rados.0.smithi118.stdout:2463: seq_num 751 ranges {827392=581632} 2024-03-20T08:09:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:2463: writing smithi11881747-9 from 827392 to 1409024 tid 1 2024-03-20T08:09:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:2464: read oid 18 snap -1 2024-03-20T08:09:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:2464: expect deleted 2024-03-20T08:09:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 256 2024-03-20T08:09:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:2465: seq_num 752 ranges {9871360=598016} 2024-03-20T08:09:59.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:59.860+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:09:59.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:59.860+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:59.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:09:59.862+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:59.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:59.857+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:59.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:59.857+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:59.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:59.858+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:59.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:09:59.858+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:59.923 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:59 smithi118 podman[121295]: 2024-03-20 08:09:59.640145466 +0000 UTC m=+0.865949325 container remove 2909c37f937131c8a703b63a3b882aebeae9e8f8a7d66634ac4808d442b21253 (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-78dbc53, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-20T08:09:59.924 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:59 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.a.service: Deactivated successfully. 2024-03-20T08:09:59.924 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:59 smithi118 systemd[1]: Stopped Ceph mds.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:09:59.924 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:09:59 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.a.service: Consumed 1.218s CPU time. 2024-03-20T08:09:59.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:59.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:59.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: osdmap e564: 8 total, 8 up, 8 in 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:59.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:09:59.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: osdmap e564: 8 total, 8 up, 8 in 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:09:59.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:59.928 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:09:59.857+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:09:59.928 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:09:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:09:59.872+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:2465: writing smithi11881747-21 from 9871360 to 10469376 tid 1 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:2466: snap_create 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:2463: finishing write tid 1 to smithi11881747-9 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:2463: finishing write tid 2 to smithi11881747-9 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:2463: finishing write tid 3 to smithi11881747-9 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:2463: oid 9 updating version 0 to 23103 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:2463: oid 9 version 23103 is already newer than 23102 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 23103 (ObjNum 751 snap 256 seq_num 751) dirty exists 2024-03-20T08:09:59.929 INFO:tasks.rados.rados.0.smithi118.stdout:2463: left oid 9 (ObjNum 751 snap 256 seq_num 751) 2024-03-20T08:09:59.948 INFO:tasks.rados.rados.0.smithi118.stdout:2465: finishing write tid 1 to smithi11881747-21 2024-03-20T08:09:59.948 INFO:tasks.rados.rados.0.smithi118.stdout:2465: finishing write tid 2 to smithi11881747-21 2024-03-20T08:09:59.951 INFO:tasks.rados.rados.0.smithi118.stdout:2465: finishing write tid 3 to smithi11881747-21 2024-03-20T08:09:59.951 INFO:tasks.rados.rados.0.smithi118.stdout:2465: oid 21 updating version 0 to 34020 2024-03-20T08:09:59.952 INFO:tasks.rados.rados.0.smithi118.stdout:2465: oid 21 version 34020 is already newer than 34019 2024-03-20T08:09:59.952 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34020 (ObjNum 752 snap 256 seq_num 752) dirty exists 2024-03-20T08:09:59.952 INFO:tasks.rados.rados.0.smithi118.stdout:2465: left oid 21 (ObjNum 752 snap 256 seq_num 752) 2024-03-20T08:10:00.025 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:59.860+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:00.025 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:09:59.861+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:00.025 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:00.025 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: osdmap e564: 8 total, 8 up, 8 in 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:00.026 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:00.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:00.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:00.027 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:00.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.784+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.796+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.027 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.846+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.863+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.876+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.923+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.936+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.961+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:09:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:09:59.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.000+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.272 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:00 smithi118 systemd[1]: Starting Ceph mds.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR Possible data damage: 13 pgs snaptrim_error 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : [ERR] PG_DAMAGED: Possible data damage: 13 pgs snaptrim_error 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.0 is active+clean+snaptrim_error, acting [3,1,4] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.1 is active+clean+snaptrim_error, acting [4,3,7] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.3 is active+clean+snaptrim_error, acting [0,3,6] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.4 is active+clean+snaptrim_error, acting [7,2,5] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.5 is active+clean+snaptrim_error, acting [0,1,4] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.6 is active+clean+snaptrim_error, acting [2,5,1] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.7 is active+clean+snaptrim_error, acting [5,1,0] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.8 is active+clean+snaptrim_error, acting [2,0,1] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.9 is active+clean+snaptrim_error, acting [7,6,1] 2024-03-20T08:10:00.273 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.a is active+clean+snaptrim_error, acting [2,4,3] 2024-03-20T08:10:00.274 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.c is active+clean+snaptrim_error, acting [1,4,0] 2024-03-20T08:10:00.274 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.d is active+clean+snaptrim_error, acting [2,7,5] 2024-03-20T08:10:00.274 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:09:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.f is active+clean+snaptrim_error, acting [5,4,6] 2024-03-20T08:10:00.275 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:00.099+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:00.275 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:00.100+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:00.275 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:00.100+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.087+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.140+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.152+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.191+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.239+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.263+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.276+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.537 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.537 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.537 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.537 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.537 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.537 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.435+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.447+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.460+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.511+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.538 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.539 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.610 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:00 smithi118 podman[121424]: 2024-03-20 08:10:00.177541677 +0000 UTC m=+0.019000252 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:10:00.610 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:00 smithi118 podman[121424]: 2024-03-20 08:10:00.305492219 +0000 UTC m=+0.146950789 container create b9e9c53270c72ac57f1e92041ba3493d746b3db2af3c4940aca1423e9508908a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:10:00.610 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:00 smithi118 podman[121424]: 2024-03-20 08:10:00.611777349 +0000 UTC m=+0.453235919 container init b9e9c53270c72ac57f1e92041ba3493d746b3db2af3c4940aca1423e9508908a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:10:00.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.797 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.562+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.588+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.600+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.636+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.661+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.674+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.697+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.709+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.734+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.797+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:00.860 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: osdmap e565: 8 total, 8 up, 8 in 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:00.861 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:00.862 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:00 smithi118 podman[121424]: 2024-03-20 08:10:00.617316855 +0000 UTC m=+0.458775425 container start b9e9c53270c72ac57f1e92041ba3493d746b3db2af3c4940aca1423e9508908a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:10:00.862 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[121438]: starting mds.a at 2024-03-20T08:10:00.862 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:00 smithi118 bash[121424]: b9e9c53270c72ac57f1e92041ba3493d746b3db2af3c4940aca1423e9508908a 2024-03-20T08:10:00.862 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:00 smithi118 systemd[1]: Started Ceph mds.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: osdmap e565: 8 total, 8 up, 8 in 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:00.863 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: Health detail: HEALTH_ERR Possible data damage: 13 pgs snaptrim_error 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: [ERR] PG_DAMAGED: Possible data damage: 13 pgs snaptrim_error 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.0 is active+clean+snaptrim_error, acting [3,1,4] 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.1 is active+clean+snaptrim_error, acting [4,3,7] 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.3 is active+clean+snaptrim_error, acting [0,3,6] 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.4 is active+clean+snaptrim_error, acting [7,2,5] 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.5 is active+clean+snaptrim_error, acting [0,1,4] 2024-03-20T08:10:00.864 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.6 is active+clean+snaptrim_error, acting [2,5,1] 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.7 is active+clean+snaptrim_error, acting [5,1,0] 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.8 is active+clean+snaptrim_error, acting [2,0,1] 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.9 is active+clean+snaptrim_error, acting [7,6,1] 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.a is active+clean+snaptrim_error, acting [2,4,3] 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.c is active+clean+snaptrim_error, acting [1,4,0] 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.d is active+clean+snaptrim_error, acting [2,7,5] 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pg 5.f is active+clean+snaptrim_error, acting [5,4,6] 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: pgmap v528: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 822 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 11 op/s 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:00.865 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2463: done (3 left) 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2464: done (2 left) 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2465: done (1 left) 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2466: done (0 left) 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2467: rmattr oid 11 current snap is 257 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2468: read oid 40 snap -1 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2468: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2469: copy_from oid 44 from oid 5 current snap is 257 2024-03-20T08:10:00.866 INFO:tasks.rados.rados.0.smithi118.stdout:2470: snap_remove snap 221 2024-03-20T08:10:01.047 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:00.871+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:01.047 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:00.872+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:01.048 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:00.863+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:01.048 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:00.864+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:01.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.835+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.048 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.866+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.866+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.887+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.922+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.935+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.947+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.973+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:00.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.049 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.011+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: osdmap e565: 8 total, 8 up, 8 in 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:01.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: Health detail: HEALTH_ERR Possible data damage: 13 pgs snaptrim_error 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: [ERR] PG_DAMAGED: Possible data damage: 13 pgs snaptrim_error 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.0 is active+clean+snaptrim_error, acting [3,1,4] 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.1 is active+clean+snaptrim_error, acting [4,3,7] 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.3 is active+clean+snaptrim_error, acting [0,3,6] 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.4 is active+clean+snaptrim_error, acting [7,2,5] 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.5 is active+clean+snaptrim_error, acting [0,1,4] 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.6 is active+clean+snaptrim_error, acting [2,5,1] 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.7 is active+clean+snaptrim_error, acting [5,1,0] 2024-03-20T08:10:01.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.8 is active+clean+snaptrim_error, acting [2,0,1] 2024-03-20T08:10:01.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.022+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.035+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.048+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.9 is active+clean+snaptrim_error, acting [7,6,1] 2024-03-20T08:10:01.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.a is active+clean+snaptrim_error, acting [2,4,3] 2024-03-20T08:10:01.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.c is active+clean+snaptrim_error, acting [1,4,0] 2024-03-20T08:10:01.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.d is active+clean+snaptrim_error, acting [2,7,5] 2024-03-20T08:10:01.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pg 5.f is active+clean+snaptrim_error, acting [5,4,6] 2024-03-20T08:10:01.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: pgmap v528: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 822 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 11 op/s 2024-03-20T08:10:01.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:01.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:01.053 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:00 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:01.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:00.867+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:01.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:00.877+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:01.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:00.863+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:01.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:00.875+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:01.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:00.875+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:00.870+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:00.870+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:00.871+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:00.872+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:01.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:00.875+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: Health detail: HEALTH_ERR Possible data damage: 13 pgs snaptrim_error 2024-03-20T08:10:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: [ERR] PG_DAMAGED: Possible data damage: 13 pgs snaptrim_error 2024-03-20T08:10:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.0 is active+clean+snaptrim_error, acting [3,1,4] 2024-03-20T08:10:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.1 is active+clean+snaptrim_error, acting [4,3,7] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.3 is active+clean+snaptrim_error, acting [0,3,6] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.4 is active+clean+snaptrim_error, acting [7,2,5] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.5 is active+clean+snaptrim_error, acting [0,1,4] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.6 is active+clean+snaptrim_error, acting [2,5,1] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.7 is active+clean+snaptrim_error, acting [5,1,0] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.8 is active+clean+snaptrim_error, acting [2,0,1] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.9 is active+clean+snaptrim_error, acting [7,6,1] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.a is active+clean+snaptrim_error, acting [2,4,3] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.c is active+clean+snaptrim_error, acting [1,4,0] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.d is active+clean+snaptrim_error, acting [2,7,5] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pg 5.f is active+clean+snaptrim_error, acting [5,4,6] 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: pgmap v528: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 822 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 6.2 MiB/s rd, 2.8 MiB/s wr, 11 op/s 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:01.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:00 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:01.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.062+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.074+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.098+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.124+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.148+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.159+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.171+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.331+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.368+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.392+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.404+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.416+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.428+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.440+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.466+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.825 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.825 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.587+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.825 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.600+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.825 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.825 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.825 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.639+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.664+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.702+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.715+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.764+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.776+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.788+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:01.862 INFO:tasks.rados.rados.0.smithi118.stdout:2470: done (3 left) 2024-03-20T08:10:01.862 INFO:tasks.rados.rados.0.smithi118.stdout:2471: copy_from oid 50 from oid 10 current snap is 257 2024-03-20T08:10:01.862 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 23104 (ObjNum 289 snap 0 seq_num 1) dirty exists 2024-03-20T08:10:01.862 INFO:tasks.rados.rados.0.smithi118.stdout:2469: finishing copy_from racing read to smithi11881747-44 2024-03-20T08:10:01.862 INFO:tasks.rados.rados.0.smithi118.stdout:2469: finishing copy_from to smithi11881747-44 2024-03-20T08:10:01.862 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 7117 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:10:01.862 INFO:tasks.rados.rados.0.smithi118.stdout:2468: expect (ObjNum 600 snap 196 seq_num 600) 2024-03-20T08:10:02.013 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:01.867+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:02.013 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:01.867+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:02.013 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:01.867+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:02.013 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:02.013 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:02.013 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:boot 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: osdmap e566: 8 total, 8 up, 8 in 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:02.014 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:02.015 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:02.015 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:01.861+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:02.015 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:02.015 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:02.015 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:02.015 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:boot 2024-03-20T08:10:02.015 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:10:02.015 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: osdmap e566: 8 total, 8 up, 8 in 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:02.016 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:02.075 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:02.075 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:02.075 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:02.075 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:boot 2024-03-20T08:10:02.075 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: fsmap cephfs:1 {0=b=up:active} 1 up:standby 2024-03-20T08:10:02.075 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:02.075 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: osdmap e566: 8 total, 8 up, 8 in 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:02.076 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:02.076 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.837+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.076 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.849+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.863+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.866+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.883+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.947+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.973+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:01.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.077 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.078 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.036+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.078 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:01.863+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:02.078 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:01.863+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:02.078 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:02.012+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:02.078 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:02.013+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:02.079 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:02.013+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:02.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.049+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.079 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.330 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:10:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.100+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.124+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.162+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.176+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.213+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.331 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.239+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.266+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.292+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:02.012+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:02.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.012+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:02.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.012+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:02.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.013+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:02.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.013+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:02.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.381+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.407+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.432+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.446+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.472+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.485+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.497+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.532+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.544+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.557+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.570+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.583+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.654 INFO:tasks.rados.rados.0.smithi118.stdout:2467: done (3 left) 2024-03-20T08:10:02.655 INFO:tasks.rados.rados.0.smithi118.stdout:2468: done (2 left) 2024-03-20T08:10:02.655 INFO:tasks.rados.rados.0.smithi118.stdout:2469: done (1 left) 2024-03-20T08:10:02.656 INFO:tasks.rados.rados.0.smithi118.stdout:2472: read oid 37 snap -1 2024-03-20T08:10:02.656 INFO:tasks.rados.rados.0.smithi118.stdout:2472: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:10:02.656 INFO:tasks.rados.rados.0.smithi118.stdout:2473: read oid 7 snap -1 2024-03-20T08:10:02.656 INFO:tasks.rados.rados.0.smithi118.stdout:2473: expect deleted 2024-03-20T08:10:02.656 INFO:tasks.rados.rados.0.smithi118.stdout:2474: snap_create 2024-03-20T08:10:02.656 INFO:tasks.rados.rados.0.smithi118.stdout:2471: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:10:02.656 INFO:tasks.rados.rados.0.smithi118.stdout:2471: finishing copy_from to smithi11881747-50 2024-03-20T08:10:02.656 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 34030 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:10:02.663 INFO:tasks.rados.rados.0.smithi118.stdout:2472: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:10:02.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.595+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.621+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.633+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.646+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.658+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.672+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.685+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.710+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.748+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.837 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.838 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.824+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.838 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.836+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:02.872 INFO:tasks.rados.rados.0.smithi118.stdout:2471: done (3 left) 2024-03-20T08:10:02.873 INFO:tasks.rados.rados.0.smithi118.stdout:2472: done (2 left) 2024-03-20T08:10:02.873 INFO:tasks.rados.rados.0.smithi118.stdout:2473: done (1 left) 2024-03-20T08:10:02.873 INFO:tasks.rados.rados.0.smithi118.stdout:2474: done (0 left) 2024-03-20T08:10:02.873 INFO:tasks.rados.rados.0.smithi118.stdout:2475: setattr oid 42 current snap is 258 2024-03-20T08:10:02.873 INFO:tasks.rados.rados.0.smithi118.stdout:2476: rollback oid 29 current snap is 258 2024-03-20T08:10:02.873 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 209 2024-03-20T08:10:02.873 INFO:tasks.rados.rados.0.smithi118.stdout:2477: snap_create 2024-03-20T08:10:02.881 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 23105 (ObjNum 65 snap 0 seq_num 1565743970) dirty exists 2024-03-20T08:10:02.882 INFO:tasks.rados.rados.0.smithi118.stdout:2476: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:10:02.887 INFO:tasks.rados.rados.0.smithi118.stdout:2476: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:10:02.887 INFO:tasks.rados.rados.0.smithi118.stdout:2476: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:10:02.887 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33958 (ObjNum 638 snap 206 seq_num 638) dirty exists 2024-03-20T08:10:03.094 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:02.936+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:03.094 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:02.937+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:03.094 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:02.879+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:03.094 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:02.879+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: osdmap e567: 8 total, 8 up, 8 in 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:03.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:03.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-mon[87080]: pgmap v531: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 823 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 410 KiB/s wr, 9 op/s 2024-03-20T08:10:03.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.847+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.859+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.874+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.877+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:03.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.877+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:03.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.096 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.903+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.917+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.943+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.954+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:02.995+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.008+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.056+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.097 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.082+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.098 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:02.875+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:03.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:02.878+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:03.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.870+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:03.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.870+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:03.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.872+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:03.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.872+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:03.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:02.872+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: osdmap e567: 8 total, 8 up, 8 in 2024-03-20T08:10:03.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[95692]: pgmap v531: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 823 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 410 KiB/s wr, 9 op/s 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:03.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: osdmap e567: 8 total, 8 up, 8 in 2024-03-20T08:10:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-mon[93121]: pgmap v531: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 823 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.6 MiB/s rd, 410 KiB/s wr, 9 op/s 2024-03-20T08:10:03.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:02.882+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:03.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:02.877+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:03.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:02.877+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:03.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.122+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.181+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.206+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.218+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.268+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.293+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.348 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.358+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.382+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.395+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.407+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.434+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.447+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.460+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.472+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.484+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.509+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.523+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.548+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.560+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.584+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.596+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.858 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.620+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.632+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.658+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.671+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.697+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.710+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.723+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.734+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.747+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.758+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.782+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.806+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.818+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.841+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:03.866 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:10:03] "GET /metrics HTTP/1.1" 200 36172 "" "Prometheus/2.43.0" 2024-03-20T08:10:03.874 INFO:tasks.rados.rados.0.smithi118.stdout:2475: done (2 left) 2024-03-20T08:10:03.874 INFO:tasks.rados.rados.0.smithi118.stdout:2476: done (1 left) 2024-03-20T08:10:03.874 INFO:tasks.rados.rados.0.smithi118.stdout:2477: done (0 left) 2024-03-20T08:10:03.874 INFO:tasks.rados.rados.0.smithi118.stdout:2478: copy_from oid 16 from oid 7 current snap is 259 2024-03-20T08:10:03.874 INFO:tasks.rados.rados.0.smithi118.stdout:2479: read oid 14 snap -1 2024-03-20T08:10:03.874 INFO:tasks.rados.rados.0.smithi118.stdout:2479: expect deleted 2024-03-20T08:10:03.874 INFO:tasks.rados.rados.0.smithi118.stdout:2480: delete oid 40 current snap is 259 2024-03-20T08:10:03.875 INFO:tasks.rados.rados.0.smithi118.stdout:2478: finishing copy_from to smithi11881747-16 2024-03-20T08:10:03.875 INFO:tasks.rados.rados.0.smithi118.stdout:2478: got expected ENOENT (src dne) 2024-03-20T08:10:03.875 INFO:tasks.rados.rados.0.smithi118.stdout:2478: finishing copy_from racing read to smithi11881747-16 2024-03-20T08:10:03.883 INFO:tasks.rados.rados.0.smithi118.stdout:2478: done (2 left) 2024-03-20T08:10:03.883 INFO:tasks.rados.rados.0.smithi118.stdout:2479: done (1 left) 2024-03-20T08:10:03.883 INFO:tasks.rados.rados.0.smithi118.stdout:2480: done (0 left) 2024-03-20T08:10:03.883 INFO:tasks.rados.rados.0.smithi118.stdout:2481: snap_remove snap 240 2024-03-20T08:10:04.116 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:03.872+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:04.116 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:03.873+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:04.116 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:03.873+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:04.116 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:03.889+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:04.116 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:03.890+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: osdmap e568: 8 total, 8 up, 8 in 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:04.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:04.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:04.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:04.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:04.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:04.118 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.118 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.890+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:04.118 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.894+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.118 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.118 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.918+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.944+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.969+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:03.993+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.017+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.029+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.092+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.103+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.119 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.116+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:03.885+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:04.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:03.885+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:04.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:03.886+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:04.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:03.874+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:03.871+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:03.872+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:03.872+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:03.872+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:04.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:03.882+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: osdmap e568: 8 total, 8 up, 8 in 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: osdmap e568: 8 total, 8 up, 8 in 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.152+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.178+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.191+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.203+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.215+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.227+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.239+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.251+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.274+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.285+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.309+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.321+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.344+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.370+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.384+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.623 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.421+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.434+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.448+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.473+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.485+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.497+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.510+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.535+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.549+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.635+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.686+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.711+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.748+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.881 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.784+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.809+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.824+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:04.885 INFO:tasks.rados.rados.0.smithi118.stdout:2481: done (0 left) 2024-03-20T08:10:04.885 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 259 2024-03-20T08:10:04.886 INFO:tasks.rados.rados.0.smithi118.stdout:2482: seq_num 754 ranges {4562944=630784} 2024-03-20T08:10:04.915 INFO:tasks.rados.rados.0.smithi118.stdout:2482: writing smithi11881747-25 from 4562944 to 5193728 tid 1 2024-03-20T08:10:04.915 INFO:tasks.rados.rados.0.smithi118.stdout:2483: snap_remove snap 211 2024-03-20T08:10:05.133 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:04.887+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:05.134 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:04.887+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:05.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:05.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:05.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:05.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:05.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:05.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:05.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: osdmap e569: 8 total, 8 up, 8 in 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: pgmap v534: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 823 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 288 KiB/s wr, 5 op/s 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:05.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.889+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.889+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.923+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.947+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.960+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.985+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:04.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.051+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.086+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.098+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.122+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:04.888+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:05.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:04.888+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:05.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:04.884+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:05.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:04.884+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:05.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:04.885+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:05.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:04.885+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:05.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:04.885+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: osdmap e569: 8 total, 8 up, 8 in 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: pgmap v534: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 823 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 288 KiB/s wr, 5 op/s 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: osdmap e569: 8 total, 8 up, 8 in 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:05.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:05.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:05.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:05.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:05.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:05.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: pgmap v534: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 823 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 288 KiB/s wr, 5 op/s 2024-03-20T08:10:05.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:05.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:05.176 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:04.920+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:05.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:04.884+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:05.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:04.884+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:05.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:05.273+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:05.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:05.274+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:05.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.172+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.197+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.210+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.224+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.260+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.308+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.368+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.381+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.406+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.419+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.431+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.457+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.470+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.494+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.517+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.645 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.529+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.566+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.604+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.617+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.630+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.646 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.893 INFO:tasks.rados.rados.0.smithi118.stdout:2483: done (1 left) 2024-03-20T08:10:05.894 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 259 2024-03-20T08:10:05.894 INFO:tasks.rados.rados.0.smithi118.stdout:2484: seq_num 755 ranges {0=647168} 2024-03-20T08:10:05.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:05.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:05.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: osdmap e570: 8 total, 8 up, 8 in 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:05.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:05.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:05.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.669+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.695+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.731+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.896 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.780+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.804+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.816+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.829+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.841+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.878+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.897 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.893+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:05.898 INFO:tasks.rados.rados.0.smithi118.stdout:2484: writing smithi11881747-31 from 0 to 647168 tid 1 2024-03-20T08:10:05.898 INFO:tasks.rados.rados.0.smithi118.stdout:2485: copy_from oid 1 from oid 7 current snap is 259 2024-03-20T08:10:05.898 INFO:tasks.rados.rados.0.smithi118.stdout:2482: finishing write tid 1 to smithi11881747-25 2024-03-20T08:10:05.898 INFO:tasks.rados.rados.0.smithi118.stdout:2482: finishing write tid 2 to smithi11881747-25 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:2482: finishing write tid 3 to smithi11881747-25 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:2482: oid 25 updating version 0 to 23107 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:2482: oid 25 version 23107 is already newer than 23106 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 23107 (ObjNum 754 snap 259 seq_num 754) dirty exists 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:2482: left oid 25 (ObjNum 754 snap 259 seq_num 754) 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:2482: done (2 left) 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:2486: rollback oid 47 current snap is 259 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 209 2024-03-20T08:10:05.899 INFO:tasks.rados.rados.0.smithi118.stdout:2487: delete oid 23 current snap is 259 2024-03-20T08:10:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:2486: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:10:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:2486: finishing rollback tid 2 to smithi11881747-47 2024-03-20T08:10:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33960 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:10:05.903 INFO:tasks.rados.rados.0.smithi118.stdout:2486: done (3 left) 2024-03-20T08:10:05.903 INFO:tasks.rados.rados.0.smithi118.stdout:2487: done (2 left) 2024-03-20T08:10:05.903 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 259 2024-03-20T08:10:05.903 INFO:tasks.rados.rados.0.smithi118.stdout:2488: seq_num 757 ranges {0=688128} 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2488: writing smithi11881747-18 from 0 to 688128 tid 1 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2489: copy_from oid 42 from oid 11 current snap is 259 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2484: finishing write tid 1 to smithi11881747-31 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2484: finishing write tid 2 to smithi11881747-31 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2485: finishing copy_from to smithi11881747-1 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2485: got expected ENOENT (src dne) 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2485: finishing copy_from racing read to smithi11881747-1 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2484: finishing write tid 3 to smithi11881747-31 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2484: oid 31 updating version 0 to 34031 2024-03-20T08:10:05.908 INFO:tasks.rados.rados.0.smithi118.stdout:2484: oid 31 updating version 34031 to 34032 2024-03-20T08:10:05.909 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 34032 (ObjNum 755 snap 259 seq_num 755) dirty exists 2024-03-20T08:10:05.909 INFO:tasks.rados.rados.0.smithi118.stdout:2484: left oid 31 (ObjNum 755 snap 259 seq_num 755) 2024-03-20T08:10:05.909 INFO:tasks.rados.rados.0.smithi118.stdout:2484: done (3 left) 2024-03-20T08:10:05.909 INFO:tasks.rados.rados.0.smithi118.stdout:2485: done (2 left) 2024-03-20T08:10:05.909 INFO:tasks.rados.rados.0.smithi118.stdout:2490: snap_remove snap 253 2024-03-20T08:10:06.151 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:05.902+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:06.151 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:05.902+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:06.151 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:05.902+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:06.151 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:05.898+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:06.151 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:05.905+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:06.152 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.894+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:06.152 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.152 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.927+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.152 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.939+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.152 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.152 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.152 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:05.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.000+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.012+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.098+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.111+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.153 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:05.890+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:06.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:05.890+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:06.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:05.891+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:06.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:05.901+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:06.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:05.902+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:06.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:05.902+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:06.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:05.903+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:06.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:05.894+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:06.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:05.902+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:06.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:06.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:06.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:06.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: osdmap e570: 8 total, 8 up, 8 in 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:06.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:06.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:06.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:06.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: osdmap e570: 8 total, 8 up, 8 in 2024-03-20T08:10:06.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:06.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:05 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:06.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.164+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.202+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.410 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.240+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.252+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.264+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.301+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.314+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.325+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.339+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.374+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.398+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.411 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.422+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.445+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.458+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.483+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.533+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.546+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.669 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.558+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.581+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.604+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.618+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.632+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.645+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.656+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.670 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2490: done (2 left) 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2491: read oid 33 snap -1 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2491: expect (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2492: read oid 6 snap -1 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2492: expect (ObjNum 709 snap 239 seq_num 709) 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2493: copy_from oid 20 from oid 31 current snap is 259 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2494: read oid 37 snap -1 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2494: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2495: rollback oid 49 current snap is 259 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 49 to 236 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2496: read oid 28 snap -1 2024-03-20T08:10:06.901 INFO:tasks.rados.rados.0.smithi118.stdout:2496: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2497: read oid 47 snap -1 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2497: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2498: snap_create 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2489: finishing copy_from racing read to smithi11881747-42 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2489: finishing copy_from to smithi11881747-42 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 23104 (ObjNum 129 snap 0 seq_num 321811152) dirty exists 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2488: finishing write tid 1 to smithi11881747-18 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2488: finishing write tid 2 to smithi11881747-18 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2488: finishing write tid 3 to smithi11881747-18 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2488: oid 18 updating version 0 to 13640 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2488: oid 18 version 13640 is already newer than 13639 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 13640 (ObjNum 757 snap 259 seq_num 757) dirty exists 2024-03-20T08:10:06.902 INFO:tasks.rados.rados.0.smithi118.stdout:2488: left oid 18 (ObjNum 757 snap 259 seq_num 757) 2024-03-20T08:10:06.908 INFO:tasks.rados.rados.0.smithi118.stdout:2495: finishing rollback tid 0 to smithi11881747-49 2024-03-20T08:10:06.909 INFO:tasks.rados.rados.0.smithi118.stdout:2495: finishing rollback tid 1 to smithi11881747-49 2024-03-20T08:10:06.909 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 13642 (ObjNum 33 snap 0 seq_num 1616999248) dirty dne 2024-03-20T08:10:06.913 INFO:tasks.rados.rados.0.smithi118.stdout:2493: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:10:06.916 INFO:tasks.rados.rados.0.smithi118.stdout:2494: expect (ObjNum 503 snap 166 seq_num 503) 2024-03-20T08:10:06.928 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:06.908+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:06.929 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:06.898+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:06.929 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:06.899+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:06.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: osdmap e571: 8 total, 8 up, 8 in 2024-03-20T08:10:06.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:06.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:06.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:06.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:06.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:06.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:06.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:06.930 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:06.930 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:06.930 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:06.930 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:06.930 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:06.930 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:06.930 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-mon[87080]: pgmap v537: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 827 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-03-20T08:10:06.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.930 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.694+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.718+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.730+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.754+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.789+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.801+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.814+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.851+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.931 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.865+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:06.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.901+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:06.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.901+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:06.932 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.013 INFO:tasks.rados.rados.0.smithi118.stdout:2493: finishing copy_from to smithi11881747-20 2024-03-20T08:10:07.013 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 34032 (ObjNum 755 snap 259 seq_num 755) dirty exists 2024-03-20T08:10:07.013 INFO:tasks.rados.rados.0.smithi118.stdout:2496: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:10:07.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:06.901+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:07.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:06.902+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:07.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:06.894+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:07.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:06.895+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:07.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:06.895+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:07.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:06.896+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:07.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:06.896+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: osdmap e571: 8 total, 8 up, 8 in 2024-03-20T08:10:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:07.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:07.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:07.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:07.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:07.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:07.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:07.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:07.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:07.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[93121]: pgmap v537: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 827 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-03-20T08:10:07.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:06.907+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:07.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:06.898+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:07.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:06.898+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:07.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: osdmap e571: 8 total, 8 up, 8 in 2024-03-20T08:10:07.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:07.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:07.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 11d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:07.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:07.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:07.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:07.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:07.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:07.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:07.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:07.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:07.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:07.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:07.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:06 smithi118 ceph-mon[95692]: pgmap v537: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 827 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 15 MiB/s rd, 1.2 MiB/s wr, 10 op/s 2024-03-20T08:10:07.190 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:06.929+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:07.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.932+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.976+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:06.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.002+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.016+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.191 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.040+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.066+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.078+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.091+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.154+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.166+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.192 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.178+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.191+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.193 INFO:tasks.rados.rados.0.smithi118.stdout:2497: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:10:07.220 INFO:tasks.rados.rados.0.smithi118.stdout:2491: expect (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:10:07.448 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.217+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.231+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.266+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.290+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.302+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.326+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.352+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.388+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.701 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.701 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.475+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.701 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.488+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.701 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.701 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.701 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.701 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.600+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.612+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.638+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.872 INFO:tasks.rados.rados.0.smithi118.stdout:2492: expect (ObjNum 709 snap 239 seq_num 709) 2024-03-20T08:10:07.953 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:07.905+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:07.953 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:07.905+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:07.953 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: osdmap e572: 8 total, 8 up, 8 in 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:07.954 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.724+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.737+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.773+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.786+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.811+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.837+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.862+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.874+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.900+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.907+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.916+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.928+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.939+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:07.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.953+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.075 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:07.905+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:08.075 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:07.905+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:08.075 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:07.905+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:08.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:07.901+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:08.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:07.901+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:08.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:07.901+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:08.075 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:07.902+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:08.076 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:07.903+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:08.076 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:08.076 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:08.076 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:08.076 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:08.076 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: osdmap e572: 8 total, 8 up, 8 in 2024-03-20T08:10:08.076 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:08.076 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: osdmap e572: 8 total, 8 up, 8 in 2024-03-20T08:10:08.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:08.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:08.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:08.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:08.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:08.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:08.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:08.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:08.078 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:08.210 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:08.200+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:08.211 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:08.201+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:08.211 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:08.201+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:08.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.966+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.978+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.211 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:07.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.038+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.051+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.063+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.113+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.125+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.140+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.155+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.169+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.212 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.184+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.213 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2488: done (9 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2489: done (8 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2491: done (7 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2492: done (6 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2493: done (5 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2494: done (4 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2495: done (3 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2496: done (2 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2497: done (1 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2498: done (0 left) 2024-03-20T08:10:08.276 INFO:tasks.rados.rados.0.smithi118.stdout:2499: delete oid 23 current snap is 260 2024-03-20T08:10:08.277 INFO:tasks.rados.rados.0.smithi118.stdout:2499: done (0 left) 2024-03-20T08:10:08.278 INFO:tasks.rados.rados.0.smithi118.stdout:2500: delete oid 41 current snap is 260 2024-03-20T08:10:08.279 INFO:tasks.rados.rados.0.smithi118.stdout:2500: done (0 left) 2024-03-20T08:10:08.279 INFO:tasks.rados.rados.0.smithi118.stdout:2501: read oid 40 snap -1 2024-03-20T08:10:08.279 INFO:tasks.rados.rados.0.smithi118.stdout:2501: expect deleted 2024-03-20T08:10:08.279 INFO:tasks.rados.rados.0.smithi118.stdout:2502: snap_create 2024-03-20T08:10:08.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:08.200+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.223+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.235+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.247+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.259+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.465 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.331+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.367+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.390+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.415+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.427+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.440+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.490+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.526+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.538+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.723 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.573+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.599+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.611+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.636+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.661+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.674+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.686+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.711+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.907 INFO:tasks.rados.rados.0.smithi118.stdout:2501: done (1 left) 2024-03-20T08:10:08.907 INFO:tasks.rados.rados.0.smithi118.stdout:2502: done (0 left) 2024-03-20T08:10:08.907 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 261 2024-03-20T08:10:08.907 INFO:tasks.rados.rados.0.smithi118.stdout:2503: seq_num 760 ranges {0=737280} 2024-03-20T08:10:08.911 INFO:tasks.rados.rados.0.smithi118.stdout:2503: writing smithi11881747-35 from 0 to 737280 tid 1 2024-03-20T08:10:08.911 INFO:tasks.rados.rados.0.smithi118.stdout:2504: snap_create 2024-03-20T08:10:08.919 INFO:tasks.rados.rados.0.smithi118.stdout:2503: finishing write tid 1 to smithi11881747-35 2024-03-20T08:10:08.919 INFO:tasks.rados.rados.0.smithi118.stdout:2503: finishing write tid 2 to smithi11881747-35 2024-03-20T08:10:08.921 INFO:tasks.rados.rados.0.smithi118.stdout:2503: finishing write tid 3 to smithi11881747-35 2024-03-20T08:10:08.921 INFO:tasks.rados.rados.0.smithi118.stdout:2503: oid 35 updating version 0 to 23103 2024-03-20T08:10:08.922 INFO:tasks.rados.rados.0.smithi118.stdout:2503: oid 35 version 23103 is already newer than 23102 2024-03-20T08:10:08.922 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 23103 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:10:08.922 INFO:tasks.rados.rados.0.smithi118.stdout:2503: left oid 35 (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:10:08.973 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:08.917+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:08.973 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:08.920+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:08.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:08.911+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:08.973 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:08.912+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: osdmap e573: 8 total, 8 up, 8 in 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:08.974 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:08.975 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:08.975 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:08.975 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:08.975 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-mon[87080]: pgmap v540: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 828 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 11 MiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-03-20T08:10:08.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.747+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.821+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.846+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.872+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.885+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.898+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.910+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.910+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.934+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.950+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:08.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:08.905+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:09.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:08.905+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:09.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:08.920+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: osdmap e573: 8 total, 8 up, 8 in 2024-03-20T08:10:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:09.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[93121]: pgmap v540: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 828 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 11 MiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: osdmap e573: 8 total, 8 up, 8 in 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:09.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:08 smithi118 ceph-mon[95692]: pgmap v540: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 828 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 11 MiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-03-20T08:10:09.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:08.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.017+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.053+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.077+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.089+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.101+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.115+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.126+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.150+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.163+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.175+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.199+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.225+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.237+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.250+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.288+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.325+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.338+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.350+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.376+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.440+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.464+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.480 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.488+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.514+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.600+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.638+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.650+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.724+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.907 INFO:tasks.rados.rados.0.smithi118.stdout:2503: done (1 left) 2024-03-20T08:10:09.907 INFO:tasks.rados.rados.0.smithi118.stdout:2504: done (0 left) 2024-03-20T08:10:09.907 INFO:tasks.rados.rados.0.smithi118.stdout:2505: read oid 2 snap -1 2024-03-20T08:10:09.907 INFO:tasks.rados.rados.0.smithi118.stdout:2505: expect (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:10:09.908 INFO:tasks.rados.rados.0.smithi118.stdout:2506: snap_create 2024-03-20T08:10:09.937 INFO:tasks.rados.rados.0.smithi118.stdout:2505: expect (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:10:09.993 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:09.993 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: osdmap e574: 8 total, 8 up, 8 in 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:09.994 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:09.995 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:09.995 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:09.911+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:09.995 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:09.911+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:09.995 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:09.912+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:09.995 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:09.912+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:09.995 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:09.917+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.803+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.815+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.869+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.894+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.906+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.915+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.966+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:09.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:09.909+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:10.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:09.910+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:10.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:09.911+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:10.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:09.774+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:10.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:09.775+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:10.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:09.775+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:10.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:09.776+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:10.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:09.776+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: osdmap e574: 8 total, 8 up, 8 in 2024-03-20T08:10:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:10.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:10.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:09.906+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: osdmap e574: 8 total, 8 up, 8 in 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:10:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:10.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:09.775+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:10.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:09.775+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:10.176 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:09.907+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:09.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.022+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.036+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.257 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.174+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.188+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.201+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.258 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.271+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.299+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.326+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.355+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.368+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.381+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.395+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.490+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.521 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.505+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.780 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.519+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.546+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.559+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.589+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.615+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.627+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.677+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.689+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.729+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.754+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.780+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:10.918 INFO:tasks.rados.rados.0.smithi118.stdout:2505: done (1 left) 2024-03-20T08:10:10.918 INFO:tasks.rados.rados.0.smithi118.stdout:2506: done (0 left) 2024-03-20T08:10:10.918 INFO:tasks.rados.rados.0.smithi118.stdout:2507: snap_create 2024-03-20T08:10:11.042 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:10.922+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:11.042 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:10.922+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:11.042 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: osdmap e575: 8 total, 8 up, 8 in 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:11.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: Upgrade: It appears safe to stop mds.b 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: pgmap v543: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 831 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 16 MiB/s rd, 664 KiB/s wr, 9 op/s 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:11.044 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.797+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.838+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.864+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.919+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.922+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.922+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.934+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.958+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.972+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:10.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.000+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.028+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:10.916+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:11.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:10.917+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:11.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:10.804+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:11.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:10.804+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:11.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:10.804+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:11.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:10.804+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:11.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:11.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: osdmap e575: 8 total, 8 up, 8 in 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: Upgrade: It appears safe to stop mds.b 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: pgmap v543: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 831 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 16 MiB/s rd, 664 KiB/s wr, 9 op/s 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:11.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: osdmap e575: 8 total, 8 up, 8 in 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds ok-to-stop", "ids": ["b"]}]: dispatch 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: Upgrade: It appears safe to stop mds.b 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: pgmap v543: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 831 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 16 MiB/s rd, 664 KiB/s wr, 9 op/s 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:11.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:11.302 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.057+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.302 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.071+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.302 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.084+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.129+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.169+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.181+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.193+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.204+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.217+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.254+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.266+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.303 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.278+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.304 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.302+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.563 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:11.317+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:11.564 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:11.317+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:11.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.315+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.326+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.564 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.389+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.426+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.452+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.526+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.539+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.552+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:11.314+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:11.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:11.314+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:11.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.590+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.636+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.645+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.662+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.674+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.700+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.725+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.738+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.763+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.776+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.789+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.802+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.830 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.814+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2507: done (0 left) 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2508: copy_from oid 41 from oid 34 current snap is 264 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2509: read oid 14 snap 213 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2509: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2510: read oid 13 snap -1 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2510: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2511: read oid 1 snap -1 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2511: expect (ObjNum 637 snap 205 seq_num 637) 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 264 2024-03-20T08:10:11.923 INFO:tasks.rados.rados.0.smithi118.stdout:2512: seq_num 761 ranges {0=753664} 2024-03-20T08:10:11.928 INFO:tasks.rados.rados.0.smithi118.stdout:2512: writing smithi11881747-50 from 0 to 753664 tid 1 2024-03-20T08:10:11.928 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 264 2024-03-20T08:10:11.928 INFO:tasks.rados.rados.0.smithi118.stdout:2513: seq_num 762 ranges {0=770048} 2024-03-20T08:10:11.933 INFO:tasks.rados.rados.0.smithi118.stdout:2513: writing smithi11881747-11 from 0 to 770048 tid 1 2024-03-20T08:10:11.933 INFO:tasks.rados.rados.0.smithi118.stdout:2514: snap_create 2024-03-20T08:10:11.933 INFO:tasks.rados.rados.0.smithi118.stdout:2509: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:10:11.963 INFO:tasks.rados.rados.0.smithi118.stdout:2511: expect (ObjNum 637 snap 205 seq_num 637) 2024-03-20T08:10:11.993 INFO:tasks.rados.rados.0.smithi118.stdout:2510: expect (ObjNum 447 snap 148 seq_num 447) 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: osdmap e576: 8 total, 8 up, 8 in 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:12.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:12.083 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:12.083 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:12.083 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:11.924+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:12.083 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:11.924+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:12.083 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:12.039+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:12.083 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:12.039+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:12.083 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:12.039+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.866+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.891+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.916+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.928+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:12.084 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:11.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.021+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.058+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.085 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.082+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:11.925+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:12.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:11.925+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:12.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:11.927+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: osdmap e576: 8 total, 8 up, 8 in 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:12.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.786+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.786+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.786+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.786+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.786+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.926+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.927+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.928+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:12.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:11.929+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:12.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:11.783+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:12.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:11.921+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:12.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:11.923+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: osdmap e576: 8 total, 8 up, 8 in 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:12.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:12.178 INFO:tasks.rados.rados.0.smithi118.stdout:2512: finishing write tid 1 to smithi11881747-50 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2513: finishing write tid 1 to smithi11881747-11 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2513: finishing write tid 2 to smithi11881747-11 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2513: finishing write tid 3 to smithi11881747-11 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2513: oid 11 updating version 0 to 23109 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2513: oid 11 version 23109 is already newer than 23108 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 23109 (ObjNum 762 snap 264 seq_num 762) dirty exists 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2513: left oid 11 (ObjNum 762 snap 264 seq_num 762) 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2512: finishing write tid 2 to smithi11881747-50 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2508: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2512: finishing write tid 3 to smithi11881747-50 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2512: oid 50 updating version 0 to 41296 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2512: oid 50 updating version 41296 to 41297 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 41297 (ObjNum 761 snap 264 seq_num 761) dirty exists 2024-03-20T08:10:12.179 INFO:tasks.rados.rados.0.smithi118.stdout:2512: left oid 50 (ObjNum 761 snap 264 seq_num 761) 2024-03-20T08:10:12.180 INFO:tasks.rados.rados.0.smithi118.stdout:2508: finishing copy_from to smithi11881747-41 2024-03-20T08:10:12.180 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 33927 (ObjNum 712 snap 241 seq_num 712) dirty exists 2024-03-20T08:10:12.343 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:10:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.095+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.106+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.145+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.182+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.193+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.344 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.266+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.331+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.368+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.405+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.416+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.454+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.490+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.516+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.528+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.540+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.559+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.647+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.887 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.659+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.696+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.757+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.780+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.792+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:12.928 INFO:tasks.rados.rados.0.smithi118.stdout:2508: done (6 left) 2024-03-20T08:10:12.928 INFO:tasks.rados.rados.0.smithi118.stdout:2509: done (5 left) 2024-03-20T08:10:12.928 INFO:tasks.rados.rados.0.smithi118.stdout:2510: done (4 left) 2024-03-20T08:10:12.928 INFO:tasks.rados.rados.0.smithi118.stdout:2511: done (3 left) 2024-03-20T08:10:12.928 INFO:tasks.rados.rados.0.smithi118.stdout:2512: done (2 left) 2024-03-20T08:10:12.929 INFO:tasks.rados.rados.0.smithi118.stdout:2513: done (1 left) 2024-03-20T08:10:12.929 INFO:tasks.rados.rados.0.smithi118.stdout:2514: done (0 left) 2024-03-20T08:10:12.929 INFO:tasks.rados.rados.0.smithi118.stdout:2515: read oid 29 snap -1 2024-03-20T08:10:12.929 INFO:tasks.rados.rados.0.smithi118.stdout:2515: expect (ObjNum 638 snap 206 seq_num 638) 2024-03-20T08:10:12.929 INFO:tasks.rados.rados.0.smithi118.stdout:2516: snap_create 2024-03-20T08:10:12.954 INFO:tasks.rados.rados.0.smithi118.stdout:2515: expect (ObjNum 638 snap 206 seq_num 638) 2024-03-20T08:10:13.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:12.931+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:13.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:12.931+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:13.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:12.937+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:13.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:12.937+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: osdmap e577: 8 total, 8 up, 8 in 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-mon[87080]: pgmap v546: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 831 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 5.5 MiB/s rd, 664 KiB/s wr, 5 op/s 2024-03-20T08:10:13.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.909+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.927+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:13.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.927+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:13.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.952+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:12.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.029+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.403 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.043+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.210+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.223+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.236+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.407 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.271+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.311+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.350+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.375+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.408 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:12.932+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:13.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:12.932+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:13.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:12.927+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:13.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:12.935+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:13.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:12.936+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:13.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:12.931+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:13.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:12.932+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:13.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:12.932+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:13.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:12.932+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:13.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:12.934+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:13.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:13.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: osdmap e577: 8 total, 8 up, 8 in 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:13.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[95692]: pgmap v546: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 831 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 5.5 MiB/s rd, 664 KiB/s wr, 5 op/s 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: osdmap e577: 8 total, 8 up, 8 in 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:13.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:13.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:12 smithi118 ceph-mon[93121]: pgmap v546: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 831 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 5.5 MiB/s rd, 664 KiB/s wr, 5 op/s 2024-03-20T08:10:13.750 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.750 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.438+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.468+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.509+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.533+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.646+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.751 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:10:13] "GET /metrics HTTP/1.1" 200 36400 "" "Prometheus/2.43.0" 2024-03-20T08:10:13.928 INFO:tasks.rados.rados.0.smithi118.stdout:2515: done (1 left) 2024-03-20T08:10:13.928 INFO:tasks.rados.rados.0.smithi118.stdout:2516: done (0 left) 2024-03-20T08:10:13.928 INFO:tasks.rados.rados.0.smithi118.stdout:2517: rollback oid 30 current snap is 266 2024-03-20T08:10:13.929 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 226 2024-03-20T08:10:13.929 INFO:tasks.rados.rados.0.smithi118.stdout:2518: delete oid 21 current snap is 266 2024-03-20T08:10:13.939 INFO:tasks.rados.rados.0.smithi118.stdout:2517: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:10:13.939 INFO:tasks.rados.rados.0.smithi118.stdout:2517: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:10:13.939 INFO:tasks.rados.rados.0.smithi118.stdout:2517: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:10:13.939 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 33928 (ObjNum 650 snap 213 seq_num 650) dirty exists 2024-03-20T08:10:13.950 INFO:tasks.rados.rados.0.smithi118.stdout:2517: done (1 left) 2024-03-20T08:10:13.950 INFO:tasks.rados.rados.0.smithi118.stdout:2518: done (0 left) 2024-03-20T08:10:13.950 INFO:tasks.rados.rados.0.smithi118.stdout:2519: snap_create 2024-03-20T08:10:14.003 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:13.934+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:14.004 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:13.934+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:14.004 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:13.939+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:14.004 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:13.940+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:14.004 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:13.940+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:14.004 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:14.004 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: osdmap e578: 8 total, 8 up, 8 in 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:14.006 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:14.006 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:14.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.762+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.776+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.788+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.006 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.800+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.812+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.824+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.849+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.885+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.908+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.938+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.938+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.007 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.954+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:13.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.008 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.017+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.065+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.078+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.103+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.128+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.165+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.177+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.190+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.202+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.214+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.225+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.312 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:13.930+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:14.313 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:13.930+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:14.313 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:13.931+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:14.313 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:13.927+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:14.313 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:13.927+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:14.313 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:13.928+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:14.313 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:13.929+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:14.313 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: osdmap e578: 8 total, 8 up, 8 in 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:14.314 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:14.315 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:14.315 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:13.940+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:14.315 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:14.315 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:14.315 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: osdmap e578: 8 total, 8 up, 8 in 2024-03-20T08:10:14.315 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:14.315 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:14.315 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:14.316 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:14.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.276+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.287+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.525 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.300+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.311+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.402+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.439+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.526 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.454+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:14.312+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.661+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.675+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.702+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.729+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.742+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:14.941 INFO:tasks.rados.rados.0.smithi118.stdout:2519: done (0 left) 2024-03-20T08:10:14.941 INFO:tasks.rados.rados.0.smithi118.stdout:2520: snap_create 2024-03-20T08:10:15.050 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:14.950+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:15.050 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:14.950+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:15.050 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: osdmap e579: 8 total, 8 up, 8 in 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:15.051 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: pgmap v549: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 836 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 1 op/s 2024-03-20T08:10:15.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:15.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:15.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:10:15.052 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:15.052 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:14.972+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:15.052 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:14.973+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:15.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.830+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.843+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.858+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.886+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.900+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.929+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.949+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.949+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:14.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.019+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.054 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.050+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.065+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.080+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.096+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.110+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.125+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.137+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.312 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.198+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.210+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.224+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.250+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.274+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.299+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.313 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:14.943+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:15.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:14.943+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:14.940+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:14.942+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:14.942+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:14.942+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:14.942+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:15.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:14.971+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: osdmap e579: 8 total, 8 up, 8 in 2024-03-20T08:10:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:15.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: pgmap v549: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 836 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 1 op/s 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:10:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: osdmap e579: 8 total, 8 up, 8 in 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: pgmap v549: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 836 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 1 op/s 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-20T08:10:15.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:15.427 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:14.940+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:15.428 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:14.940+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:15.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.323+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.350+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.565 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.378+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.391+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.421+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.446+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.470+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.523+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.538+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.552+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.565+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.591+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.603+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.616+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.654+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.705+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.718+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.730+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.780+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.816+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:15.941 INFO:tasks.rados.rados.0.smithi118.stdout:2520: done (0 left) 2024-03-20T08:10:15.941 INFO:tasks.rados.rados.0.smithi118.stdout:2521: delete oid 36 current snap is 268 2024-03-20T08:10:15.949 INFO:tasks.rados.rados.0.smithi118.stdout:2521: done (0 left) 2024-03-20T08:10:15.949 INFO:tasks.rados.rados.0.smithi118.stdout:2522: delete oid 1 current snap is 268 2024-03-20T08:10:15.953 INFO:tasks.rados.rados.0.smithi118.stdout:2522: done (0 left) 2024-03-20T08:10:15.953 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 268 2024-03-20T08:10:15.953 INFO:tasks.rados.rados.0.smithi118.stdout:2523: seq_num 766 ranges {0=434176,434176=401408} 2024-03-20T08:10:15.954 INFO:tasks.rados.rados.0.smithi118.stdout:2523: writing smithi11881747-21 from 0 to 434176 tid 1 2024-03-20T08:10:15.958 INFO:tasks.rados.rados.0.smithi118.stdout:2523: writing smithi11881747-21 from 434176 to 835584 tid 2 2024-03-20T08:10:15.958 INFO:tasks.rados.rados.0.smithi118.stdout:2524: copy_from oid 19 from oid 49 current snap is 268 2024-03-20T08:10:15.958 INFO:tasks.rados.rados.0.smithi118.stdout:2525: read oid 9 snap -1 2024-03-20T08:10:15.958 INFO:tasks.rados.rados.0.smithi118.stdout:2525: expect (ObjNum 751 snap 256 seq_num 751) 2024-03-20T08:10:15.958 INFO:tasks.rados.rados.0.smithi118.stdout:2526: rmattr oid 25 current snap is 268 2024-03-20T08:10:15.959 INFO:tasks.rados.rados.0.smithi118.stdout:2526: done (3 left) 2024-03-20T08:10:15.959 INFO:tasks.rados.rados.0.smithi118.stdout:2527: read oid 35 snap -1 2024-03-20T08:10:15.959 INFO:tasks.rados.rados.0.smithi118.stdout:2527: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:10:15.959 INFO:tasks.rados.rados.0.smithi118.stdout:2528: delete oid 20 current snap is 268 2024-03-20T08:10:15.960 INFO:tasks.rados.rados.0.smithi118.stdout:2524: finishing copy_from to smithi11881747-19 2024-03-20T08:10:15.960 INFO:tasks.rados.rados.0.smithi118.stdout:2524: got expected ENOENT (src dne) 2024-03-20T08:10:15.960 INFO:tasks.rados.rados.0.smithi118.stdout:2524: finishing copy_from racing read to smithi11881747-19 2024-03-20T08:10:15.961 INFO:tasks.rados.rados.0.smithi118.stdout:2523: finishing write tid 1 to smithi11881747-21 2024-03-20T08:10:15.962 INFO:tasks.rados.rados.0.smithi118.stdout:2524: done (4 left) 2024-03-20T08:10:15.962 INFO:tasks.rados.rados.0.smithi118.stdout:2528: done (3 left) 2024-03-20T08:10:15.962 INFO:tasks.rados.rados.0.smithi118.stdout:2529: snap_remove snap 218 2024-03-20T08:10:16.077 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:15.962+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:16.077 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:15.962+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:16.078 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:15.966+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: Upgrade: Updating mds.b 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: Deploying daemon mds.b on smithi196 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: osdmap e580: 8 total, 8 up, 8 in 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:16.078 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:16.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:16.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:16.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:16.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:16.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:16.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:16.079 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:16.079 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:15.943+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:16.079 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:15.944+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.829+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.855+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.874+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.903+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.916+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.928+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.950+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:16.080 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.957+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.971+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:15.997+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.021+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.035+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.065+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.078+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.103+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.116+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.154+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.333 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.165+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.333 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.178+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.333 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.190+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.333 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.202+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.333 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.216+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.228+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.240+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.251+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.276+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.289+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.301+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:15.941+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:16.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:15.945+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:16.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:15.949+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:16.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:15.949+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:16.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:15.940+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:16.071+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:16.071+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:16.071+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:16.072+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: Upgrade: Updating mds.b 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: Deploying daemon mds.b on smithi196 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: osdmap e580: 8 total, 8 up, 8 in 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: Upgrade: Updating mds.b 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: Deploying daemon mds.b on smithi196 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: osdmap e580: 8 total, 8 up, 8 in 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:16.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.376+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.390+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.415+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.426+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.454+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.492+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.534+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.547+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.587+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.845 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:16 smithi196 systemd[1]: Stopping Ceph mds.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:10:16.845 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.600+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.845 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.845 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.845 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.695+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.711+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.724+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.846 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.830+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.847 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.845+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:16.951 INFO:tasks.rados.rados.0.smithi118.stdout:2529: done (3 left) 2024-03-20T08:10:16.952 INFO:tasks.rados.rados.0.smithi118.stdout:2530: rmattr oid 24 current snap is 268 2024-03-20T08:10:16.952 INFO:tasks.rados.rados.0.smithi118.stdout:2530: done (3 left) 2024-03-20T08:10:16.952 INFO:tasks.rados.rados.0.smithi118.stdout:2531: read oid 42 snap -1 2024-03-20T08:10:16.952 INFO:tasks.rados.rados.0.smithi118.stdout:2531: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:10:16.952 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 268 2024-03-20T08:10:16.952 INFO:tasks.rados.rados.0.smithi118.stdout:2525: expect (ObjNum 751 snap 256 seq_num 751) 2024-03-20T08:10:17.010 INFO:tasks.rados.rados.0.smithi118.stdout:2532: seq_num 768 ranges {0=466944,466944=401408} 2024-03-20T08:10:17.012 INFO:tasks.rados.rados.0.smithi118.stdout:2532: writing smithi11881747-7 from 0 to 466944 tid 1 2024-03-20T08:10:17.015 INFO:tasks.rados.rados.0.smithi118.stdout:2532: writing smithi11881747-7 from 466944 to 868352 tid 2 2024-03-20T08:10:17.015 INFO:tasks.rados.rados.0.smithi118.stdout:2525: done (4 left) 2024-03-20T08:10:17.015 INFO:tasks.rados.rados.0.smithi118.stdout:2533: delete oid 31 current snap is 268 2024-03-20T08:10:17.015 INFO:tasks.rados.rados.0.smithi118.stdout:2527: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:10:17.044 INFO:tasks.rados.rados.0.smithi118.stdout:2523: finishing write tid 2 to smithi11881747-21 2024-03-20T08:10:17.044 INFO:tasks.rados.rados.0.smithi118.stdout:2523: finishing write tid 3 to smithi11881747-21 2024-03-20T08:10:17.044 INFO:tasks.rados.rados.0.smithi118.stdout:2523: finishing write tid 4 to smithi11881747-21 2024-03-20T08:10:17.045 INFO:tasks.rados.rados.0.smithi118.stdout:2523: oid 21 updating version 0 to 34023 2024-03-20T08:10:17.045 INFO:tasks.rados.rados.0.smithi118.stdout:2523: oid 21 version 34023 is already newer than 34022 2024-03-20T08:10:17.045 INFO:tasks.rados.rados.0.smithi118.stdout:2523: oid 21 updating version 34023 to 34024 2024-03-20T08:10:17.045 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34024 (ObjNum 766 snap 268 seq_num 766) dirty exists 2024-03-20T08:10:17.045 INFO:tasks.rados.rados.0.smithi118.stdout:2523: left oid 21 (ObjNum 766 snap 268 seq_num 766) 2024-03-20T08:10:17.045 INFO:tasks.rados.rados.0.smithi118.stdout:2531: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:10:17.047 INFO:tasks.rados.rados.0.smithi118.stdout:2532: finishing write tid 1 to smithi11881747-7 2024-03-20T08:10:17.047 INFO:tasks.rados.rados.0.smithi118.stdout:2532: finishing write tid 2 to smithi11881747-7 2024-03-20T08:10:17.047 INFO:tasks.rados.rados.0.smithi118.stdout:2523: done (4 left) 2024-03-20T08:10:17.047 INFO:tasks.rados.rados.0.smithi118.stdout:2527: done (3 left) 2024-03-20T08:10:17.048 INFO:tasks.rados.rados.0.smithi118.stdout:2531: done (2 left) 2024-03-20T08:10:17.048 INFO:tasks.rados.rados.0.smithi118.stdout:2533: done (1 left) 2024-03-20T08:10:17.048 INFO:tasks.rados.rados.0.smithi118.stdout:2534: snap_create 2024-03-20T08:10:17.048 INFO:tasks.rados.rados.0.smithi118.stdout:2532: finishing write tid 3 to smithi11881747-7 2024-03-20T08:10:17.048 INFO:tasks.rados.rados.0.smithi118.stdout:2532: finishing write tid 4 to smithi11881747-7 2024-03-20T08:10:17.048 INFO:tasks.rados.rados.0.smithi118.stdout:2532: oid 7 updating version 0 to 34010 2024-03-20T08:10:17.048 INFO:tasks.rados.rados.0.smithi118.stdout:2532: oid 7 version 34010 is already newer than 34008 2024-03-20T08:10:17.049 INFO:tasks.rados.rados.0.smithi118.stdout:2532: oid 7 version 34010 is already newer than 34009 2024-03-20T08:10:17.049 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 34010 (ObjNum 768 snap 268 seq_num 768) dirty exists 2024-03-20T08:10:17.049 INFO:tasks.rados.rados.0.smithi118.stdout:2532: left oid 7 (ObjNum 768 snap 268 seq_num 768) 2024-03-20T08:10:17.096 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b[58688]: 2024-03-20T08:10:17.015+0000 7f3592cad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:10:17.096 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b[58688]: 2024-03-20T08:10:17.015+0000 7f3592cad700 -1 mds.b *** got signal Terminated *** 2024-03-20T08:10:17.096 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:17.022+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:17.096 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:17.027+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:17.097 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:17.029+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:17.097 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:17.029+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:17.097 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:17.029+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:17.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:17.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: osdmap e581: 8 total, 8 up, 8 in 2024-03-20T08:10:17.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:17.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:17.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:17.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:17.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:17.097 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:17.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:17.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:17.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:17.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:17.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:17.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:17.098 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-mon[87080]: pgmap v552: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 838 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:10:17.098 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:16.956+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:17.098 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:16.956+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:17.099 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:17.025+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:17.099 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:17.027+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:17.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.875+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.890+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.903+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.918+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.099 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.954+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.954+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.960+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.971+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.983+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:16.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.008+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.034+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.046+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.059+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.071+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.096+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:16.951+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:17.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:16.951+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:17.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:17.039+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:17.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:16.949+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:17.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:16.951+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:17.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:17.026+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:17.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:17.026+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:17.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:17.026+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:17.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:17.035+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:17.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:17.015+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:17.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:17.022+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:17.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:17.022+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:17.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:17.022+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:17.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:17.022+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:17.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:17.031+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:17.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:17.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: osdmap e581: 8 total, 8 up, 8 in 2024-03-20T08:10:17.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[93121]: pgmap v552: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 838 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:10:17.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:10:17.016+0000 7f75111b8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: osdmap e581: 8 total, 8 up, 8 in 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:17.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:17.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:16 smithi118 ceph-mon[95692]: pgmap v552: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 838 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 5 op/s 2024-03-20T08:10:17.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.136+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.161+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.172+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.358 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.184+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.197+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.209+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.222+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.235+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.259+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.283+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.296+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.308+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.321+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.334+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.370+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.406+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.419+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.431+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.444+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.458+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.535+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.584+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.597+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.859 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.646+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.670+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.860 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.819+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:17.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.037 INFO:tasks.rados.rados.0.smithi118.stdout:2532: done (1 left) 2024-03-20T08:10:18.037 INFO:tasks.rados.rados.0.smithi118.stdout:2534: done (0 left) 2024-03-20T08:10:18.037 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 269 2024-03-20T08:10:18.037 INFO:tasks.rados.rados.0.smithi118.stdout:2535: seq_num 770 ranges {4431872=499712,4931584=401408} 2024-03-20T08:10:18.064 INFO:tasks.rados.rados.0.smithi118.stdout:2535: writing smithi11881747-48 from 4431872 to 4931584 tid 1 2024-03-20T08:10:18.066 INFO:tasks.rados.rados.0.smithi118.stdout:2535: writing smithi11881747-48 from 4931584 to 5332992 tid 2 2024-03-20T08:10:18.066 INFO:tasks.rados.rados.0.smithi118.stdout:2536: delete oid 9 current snap is 269 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2536: done (1 left) 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2537: read oid 25 snap -1 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2537: expect (ObjNum 754 snap 259 seq_num 754) 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2538: read oid 18 snap -1 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2538: expect (ObjNum 757 snap 259 seq_num 757) 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2539: copy_from oid 45 from oid 1 current snap is 269 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2540: read oid 20 snap -1 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2540: expect deleted 2024-03-20T08:10:18.076 INFO:tasks.rados.rados.0.smithi118.stdout:2541: rollback oid 21 current snap is 269 2024-03-20T08:10:18.077 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 252 2024-03-20T08:10:18.077 INFO:tasks.rados.rados.0.smithi118.stdout:2542: snap_create 2024-03-20T08:10:18.080 INFO:tasks.rados.rados.0.smithi118.stdout:2539: finishing copy_from to smithi11881747-45 2024-03-20T08:10:18.080 INFO:tasks.rados.rados.0.smithi118.stdout:2539: got expected ENOENT (src dne) 2024-03-20T08:10:18.081 INFO:tasks.rados.rados.0.smithi118.stdout:2539: finishing copy_from racing read to smithi11881747-45 2024-03-20T08:10:18.081 INFO:tasks.rados.rados.0.smithi118.stdout:2538: expect (ObjNum 757 snap 259 seq_num 757) 2024-03-20T08:10:18.104 INFO:tasks.rados.rados.0.smithi118.stdout:2535: finishing write tid 1 to smithi11881747-48 2024-03-20T08:10:18.104 INFO:tasks.rados.rados.0.smithi118.stdout:2537: expect (ObjNum 754 snap 259 seq_num 754) 2024-03-20T08:10:18.120 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:18.120 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:18.120 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:18.120 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: osdmap e582: 8 total, 8 up, 8 in 2024-03-20T08:10:18.120 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:18.120 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: osdmap e583: 8 total, 8 up, 8 in 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: Standby daemon mds.a assigned to filesystem cephfs as rank 0 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: fsmap cephfs:0/1 1 up:standby, 1 failed 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:18.121 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: fsmap cephfs:1/1 {0=a=up:replay} 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:18.122 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:18.122 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.871+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.909+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.922+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.933+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.958+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.971+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.983+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:17.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.008+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.021+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.035+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.050+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.123 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.072+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.072+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.096+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.108+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:18.076+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:18.076+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:18.036+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:18.124 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:18.037+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:18.372 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.120+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.373 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.133+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.373 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.373 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.158+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.373 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.373 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.373 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.208+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.244+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.295+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.307+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.332+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.358+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:18.069+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:18.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:18.074+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:18.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:18.080+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:18.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:18.080+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:18.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:18.080+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:18.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:18.081+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:18.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:18.037+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:18.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:18.037+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: osdmap e582: 8 total, 8 up, 8 in 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: osdmap e583: 8 total, 8 up, 8 in 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: Standby daemon mds.a assigned to filesystem cephfs as rank 0 2024-03-20T08:10:18.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: fsmap cephfs:0/1 1 up:standby, 1 failed 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: fsmap cephfs:1/1 {0=a=up:replay} 2024-03-20T08:10:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:18.426 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:18.036+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:18.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:18.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: osdmap e582: 8 total, 8 up, 8 in 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: osdmap e583: 8 total, 8 up, 8 in 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: Standby daemon mds.a assigned to filesystem cephfs as rank 0 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: fsmap cephfs:0/1 1 up:standby, 1 failed 2024-03-20T08:10:18.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: fsmap cephfs:1/1 {0=a=up:replay} 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:18.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:18.501 INFO:tasks.rados.rados.0.smithi118.stdout:2535: finishing write tid 2 to smithi11881747-48 2024-03-20T08:10:18.501 INFO:tasks.rados.rados.0.smithi118.stdout:2535: finishing write tid 3 to smithi11881747-48 2024-03-20T08:10:18.501 INFO:tasks.rados.rados.0.smithi118.stdout:2541: finishing rollback tid 0 to smithi11881747-21 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:2535: finishing write tid 4 to smithi11881747-48 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:2535: oid 48 updating version 0 to 34027 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:2535: oid 48 version 34027 is already newer than 34026 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:2535: oid 48 version 34027 is already newer than 34025 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 34027 (ObjNum 770 snap 269 seq_num 770) dirty exists 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:2535: left oid 48 (ObjNum 770 snap 269 seq_num 770) 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:2541: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:2541: finishing rollback tid 2 to smithi11881747-21 2024-03-20T08:10:18.502 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34030 (ObjNum 722 snap 249 seq_num 722) dirty exists 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.384+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.599+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.776 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:10:18.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.636+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.648+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.879 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.698+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.710+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.737+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.748+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.775+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.803+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.815+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.852+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.866+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:18.880 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.879+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.042 INFO:tasks.rados.rados.0.smithi118.stdout:2535: done (6 left) 2024-03-20T08:10:19.042 INFO:tasks.rados.rados.0.smithi118.stdout:2537: done (5 left) 2024-03-20T08:10:19.042 INFO:tasks.rados.rados.0.smithi118.stdout:2538: done (4 left) 2024-03-20T08:10:19.043 INFO:tasks.rados.rados.0.smithi118.stdout:2539: done (3 left) 2024-03-20T08:10:19.043 INFO:tasks.rados.rados.0.smithi118.stdout:2540: done (2 left) 2024-03-20T08:10:19.043 INFO:tasks.rados.rados.0.smithi118.stdout:2541: done (1 left) 2024-03-20T08:10:19.043 INFO:tasks.rados.rados.0.smithi118.stdout:2542: done (0 left) 2024-03-20T08:10:19.043 INFO:tasks.rados.rados.0.smithi118.stdout:2543: snap_remove snap 246 2024-03-20T08:10:19.136 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:reconnect 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: fsmap cephfs:1/1 {0=a=up:reconnect} 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: osdmap e584: 8 total, 8 up, 8 in 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:19.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:19.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:19.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:19.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-mon[87080]: pgmap v556: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 840 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.1 MiB/s rd, 725 KiB/s wr, 8 op/s 2024-03-20T08:10:19.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.905+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.918+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:18.991+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.004+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.016+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.046+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.048+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.074+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.087+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:19.044+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:19.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:19.044+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:19.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.112+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.125+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.137+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.157 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:10:19.157 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (10m) 14s ago 10m 27.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:10:19.157 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (10m) 36s ago 10m 86.8M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:10:19.157 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (18s) 14s ago 11m 12.5M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 b9e9c53270c7 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (11m) 36s ago 11m 22.0M - 18.2.2-904-g78dbc53a 3b5d1eaa4228 e5e6b027dc58 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (6m) 36s ago 15m 452M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (7m) 14s ago 17m 514M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (6m) 14s ago 17m 67.3M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (5m) 36s ago 15m 51.5M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (6m) 14s ago 15m 61.0M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (10m) 14s ago 10m 15.6M - 1.5.0 0da6a335fe13 e22b97acea5c 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (10m) 36s ago 10m 15.6M - 1.5.0 0da6a335fe13 fd3b2992729c 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (4m) 14s ago 15m 329M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (4m) 14s ago 14m 513M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (3m) 14s ago 14m 388M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (3m) 14s ago 13m 141M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (2m) 36s ago 13m 339M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (2m) 36s ago 13m 272M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 72b569d10541 2024-03-20T08:10:19.158 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (86s) 36s ago 12m 273M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 dca658056148 2024-03-20T08:10:19.159 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (42s) 36s ago 12m 15.4M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 52392403da44 2024-03-20T08:10:19.159 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (6m) 36s ago 11m 60.7M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:10:19.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.149+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.163+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.176+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.202+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.214+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.275+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.287+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.300+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.313+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.337+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.361+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:19.043+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:19.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:19.043+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:19.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:19.043+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:19.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:reconnect 2024-03-20T08:10:19.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: fsmap cephfs:1/1 {0=a=up:reconnect} 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: osdmap e584: 8 total, 8 up, 8 in 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[93121]: pgmap v556: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 840 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.1 MiB/s rd, 725 KiB/s wr, 8 op/s 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:reconnect 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: fsmap cephfs:1/1 {0=a=up:reconnect} 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: osdmap e584: 8 total, 8 up, 8 in 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-mon[95692]: pgmap v556: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 840 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 7.1 MiB/s rd, 725 KiB/s wr, 8 op/s 2024-03-20T08:10:19.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:19.043+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:19.611 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:10:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:10:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:10:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:10:19.611 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 8 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 1 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 14 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:10:19.612 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.423+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.473+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.538+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.552+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.591+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.914 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:19.676+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:19.914 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:19.676+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:19.914 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:19.676+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.679+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.691+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.740+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.764+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.776+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.788+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.800+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.850+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.862+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.875+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.900+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.913+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:19.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:19.746+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:19.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:19.704+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:19.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:19.704+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:19.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:19.705+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:19.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:19.705+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:19.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:19.705+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:20.049 INFO:tasks.rados.rados.0.smithi118.stdout:2543: done (0 left) 2024-03-20T08:10:20.049 INFO:tasks.rados.rados.0.smithi118.stdout:2544: delete oid 47 current snap is 270 2024-03-20T08:10:20.052 INFO:tasks.rados.rados.0.smithi118.stdout:2544: done (0 left) 2024-03-20T08:10:20.052 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 270 2024-03-20T08:10:20.052 INFO:tasks.rados.rados.0.smithi118.stdout:2545: seq_num 773 ranges {4751360=548864,5300224=401408} 2024-03-20T08:10:20.083 INFO:tasks.rados.rados.0.smithi118.stdout:2545: writing smithi11881747-27 from 4751360 to 5300224 tid 1 2024-03-20T08:10:20.086 INFO:tasks.rados.rados.0.smithi118.stdout:2545: writing smithi11881747-27 from 5300224 to 5701632 tid 2 2024-03-20T08:10:20.086 INFO:tasks.rados.rados.0.smithi118.stdout:2546: read oid 18 snap -1 2024-03-20T08:10:20.086 INFO:tasks.rados.rados.0.smithi118.stdout:2546: expect (ObjNum 757 snap 259 seq_num 757) 2024-03-20T08:10:20.086 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 270 2024-03-20T08:10:20.086 INFO:tasks.rados.rados.0.smithi118.stdout:2547: seq_num 774 ranges {9175040=565248,9740288=401408} 2024-03-20T08:10:20.142 INFO:tasks.rados.rados.0.smithi118.stdout:2547: writing smithi11881747-43 from 9175040 to 9740288 tid 1 2024-03-20T08:10:20.145 INFO:tasks.rados.rados.0.smithi118.stdout:2547: writing smithi11881747-43 from 9740288 to 10141696 tid 2 2024-03-20T08:10:20.145 INFO:tasks.rados.rados.0.smithi118.stdout:2548: read oid 8 snap 263 2024-03-20T08:10:20.145 INFO:tasks.rados.rados.0.smithi118.stdout:2548: expect (ObjNum 741 snap 253 seq_num 741) 2024-03-20T08:10:20.145 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 270 2024-03-20T08:10:20.145 INFO:tasks.rados.rados.0.smithi118.stdout:2545: finishing write tid 1 to smithi11881747-27 2024-03-20T08:10:20.145 INFO:tasks.rados.rados.0.smithi118.stdout:2549: seq_num 775 ranges {5341184=589824,5931008=401408} 2024-03-20T08:10:20.165 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:20.054+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:20.166 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:20.055+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:20.166 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:20.049+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:20.166 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:20.049+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:20.166 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:20.166 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:rejoin 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: fsmap cephfs:1/1 {0=a=up:rejoin} 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: osdmap e585: 8 total, 8 up, 8 in 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/2046292814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:20.167 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:20.168 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:20.168 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:20.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.939+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.951+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.978+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:19.990+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.051+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.052+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.060+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.086+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.113+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.152+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.165+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.178 INFO:tasks.rados.rados.0.smithi118.stdout:2549: writing smithi11881747-34 from 5341184 to 5931008 tid 1 2024-03-20T08:10:20.180 INFO:tasks.rados.rados.0.smithi118.stdout:2549: writing smithi11881747-34 from 5931008 to 6332416 tid 2 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2550: delete oid 17 current snap is 270 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2545: finishing write tid 2 to smithi11881747-27 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2545: finishing write tid 3 to smithi11881747-27 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2545: finishing write tid 4 to smithi11881747-27 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2545: oid 27 updating version 0 to 34035 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2545: oid 27 updating version 34035 to 34036 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2545: oid 27 version 34036 is already newer than 34034 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 34036 (ObjNum 773 snap 270 seq_num 773) dirty exists 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2545: left oid 27 (ObjNum 773 snap 270 seq_num 773) 2024-03-20T08:10:20.181 INFO:tasks.rados.rados.0.smithi118.stdout:2546: expect (ObjNum 757 snap 259 seq_num 757) 2024-03-20T08:10:20.206 INFO:tasks.rados.rados.0.smithi118.stdout:2545: done (5 left) 2024-03-20T08:10:20.206 INFO:tasks.rados.rados.0.smithi118.stdout:2546: done (4 left) 2024-03-20T08:10:20.206 INFO:tasks.rados.rados.0.smithi118.stdout:2550: done (3 left) 2024-03-20T08:10:20.206 INFO:tasks.rados.rados.0.smithi118.stdout:2551: setattr oid 24 current snap is 270 2024-03-20T08:10:20.207 INFO:tasks.rados.rados.0.smithi118.stdout:2547: finishing write tid 1 to smithi11881747-43 2024-03-20T08:10:20.207 INFO:tasks.rados.rados.0.smithi118.stdout:2547: finishing write tid 2 to smithi11881747-43 2024-03-20T08:10:20.207 INFO:tasks.rados.rados.0.smithi118.stdout:2547: finishing write tid 3 to smithi11881747-43 2024-03-20T08:10:20.207 INFO:tasks.rados.rados.0.smithi118.stdout:2548: expect (ObjNum 741 snap 253 seq_num 741) 2024-03-20T08:10:20.321 INFO:tasks.rados.rados.0.smithi118.stdout:2547: finishing write tid 4 to smithi11881747-43 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2547: oid 43 updating version 0 to 34037 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2547: oid 43 version 34037 is already newer than 34036 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2547: oid 43 version 34037 is already newer than 34035 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 34037 (ObjNum 774 snap 270 seq_num 774) dirty exists 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2547: left oid 43 (ObjNum 774 snap 270 seq_num 774) 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2549: finishing write tid 1 to smithi11881747-34 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2549: finishing write tid 2 to smithi11881747-34 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2549: finishing write tid 3 to smithi11881747-34 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2549: finishing write tid 4 to smithi11881747-34 2024-03-20T08:10:20.322 INFO:tasks.rados.rados.0.smithi118.stdout:2549: oid 34 updating version 0 to 33929 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2549: oid 34 updating version 33929 to 33931 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2549: oid 34 version 33931 is already newer than 33930 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 33931 (ObjNum 775 snap 270 seq_num 775) dirty exists 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2549: left oid 34 (ObjNum 775 snap 270 seq_num 775) 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2547: done (3 left) 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2548: done (2 left) 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2549: done (1 left) 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2552: read oid 10 snap -1 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2552: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2553: delete oid 26 current snap is 270 2024-03-20T08:10:20.323 INFO:tasks.rados.rados.0.smithi118.stdout:2552: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:10:20.329 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 43812 (ObjNum 80145 snap 0 seq_num 321898128) dirty exists 2024-03-20T08:10:20.329 INFO:tasks.rados.rados.0.smithi118.stdout:2551: done (2 left) 2024-03-20T08:10:20.329 INFO:tasks.rados.rados.0.smithi118.stdout:2552: done (1 left) 2024-03-20T08:10:20.329 INFO:tasks.rados.rados.0.smithi118.stdout:2553: done (0 left) 2024-03-20T08:10:20.329 INFO:tasks.rados.rados.0.smithi118.stdout:2554: delete oid 48 current snap is 270 2024-03-20T08:10:20.348 INFO:tasks.rados.rados.0.smithi118.stdout:2554: done (0 left) 2024-03-20T08:10:20.348 INFO:tasks.rados.rados.0.smithi118.stdout:2555: snap_create 2024-03-20T08:10:20.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:20.056+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:20.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:20.056+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:20.048+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:20.048+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:20.049+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:20.050+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:rejoin 2024-03-20T08:10:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: fsmap cephfs:1/1 {0=a=up:rejoin} 2024-03-20T08:10:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-03-20T08:10:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: osdmap e585: 8 total, 8 up, 8 in 2024-03-20T08:10:20.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/2046292814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:20.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:rejoin 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: fsmap cephfs:1/1 {0=a=up:rejoin} 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: daemon mds.a is now active in filesystem cephfs as rank 0 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: osdmap e585: 8 total, 8 up, 8 in 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/2046292814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:20.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:20.052+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:20.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:20.052+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:20.427 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:20.088+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:20.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.194+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.268+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.293+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.338+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.352+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.377+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.686 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.537+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.548+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.573+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.687 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.635+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.648+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.688 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.685+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.710+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.746+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.783+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.843+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.855+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.904+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:20.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.054 INFO:tasks.rados.rados.0.smithi118.stdout:2555: done (0 left) 2024-03-20T08:10:21.054 INFO:tasks.rados.rados.0.smithi118.stdout:2556: snap_remove snap 247 2024-03-20T08:10:21.136 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:21.056+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:21.136 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:21.056+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:21.136 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:21.056+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:21.136 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:10:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[121438]: 2024-03-20T08:10:20.869+0000 7febac2d9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-20T08:10:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:21.053+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:21.053+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:21.053+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:21.054+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:21.137 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:21.054+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:21.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:21.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-20T08:10:21.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:active 2024-03-20T08:10:21.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: fsmap cephfs:1 {0=a=up:active} 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: osdmap e586: 8 total, 8 up, 8 in 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:21.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[93121]: pgmap v559: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 840 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 5.9 MiB/s rd, 501 KiB/s wr, 11 op/s 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:active 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: fsmap cephfs:1 {0=a=up:active} 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: osdmap e586: 8 total, 8 up, 8 in 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:21.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:21.140 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:21.140 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:21.140 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:21.140 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:21.140 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:21.140 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:21.140 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:21.140 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-mon[95692]: pgmap v559: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 840 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 5.9 MiB/s rd, 501 KiB/s wr, 11 op/s 2024-03-20T08:10:21.200 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:21.057+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:21.200 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:21.057+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:21.200 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:21.136+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:21.200 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:21.136+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:21.200 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:21.136+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: mds.? [v2:172.21.15.118:6834/2279894254,v1:172.21.15.118:6835/2279894254] up:active 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: fsmap cephfs:1 {0=a=up:active} 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: osdmap e586: 8 total, 8 up, 8 in 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:21.201 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:21.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:21.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:21.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:21.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:21.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:21.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:21.202 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-mon[87080]: pgmap v559: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 840 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 5.9 MiB/s rd, 501 KiB/s wr, 11 op/s 2024-03-20T08:10:21.202 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.969+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:20.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.007+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.053+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.059+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.111+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.124+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.150+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.162+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.175+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.199+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:21.138+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:21.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:21.136+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.225+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.250+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.275+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.287+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.299+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.461 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.324+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.350+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.374+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.423+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.462 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.460+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.724 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.485+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.560+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.725 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.649+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.674+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.710+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.726 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.975 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.748+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.773+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.798+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.836+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.848+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.861+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.976 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.874+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.936+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.962+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:21.977 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.061 INFO:tasks.rados.rados.0.smithi118.stdout:2556: done (0 left) 2024-03-20T08:10:22.061 INFO:tasks.rados.rados.0.smithi118.stdout:2557: rmattr oid 26 current snap is 271 2024-03-20T08:10:22.061 INFO:tasks.rados.rados.0.smithi118.stdout:2557: done (0 left) 2024-03-20T08:10:22.061 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 271 2024-03-20T08:10:22.062 INFO:tasks.rados.rados.0.smithi118.stdout:2558: seq_num 779 ranges {0=655360,655360=401408} 2024-03-20T08:10:22.065 INFO:tasks.rados.rados.0.smithi118.stdout:2558: writing smithi11881747-1 from 0 to 655360 tid 1 2024-03-20T08:10:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:2558: writing smithi11881747-1 from 655360 to 1056768 tid 2 2024-03-20T08:10:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:2559: copy_from oid 15 from oid 9 current snap is 271 2024-03-20T08:10:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 271 2024-03-20T08:10:22.069 INFO:tasks.rados.rados.0.smithi118.stdout:2560: seq_num 780 ranges {5341184=671744,6012928=401408} 2024-03-20T08:10:22.103 INFO:tasks.rados.rados.0.smithi118.stdout:2560: writing smithi11881747-41 from 5341184 to 6012928 tid 1 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2560: writing smithi11881747-41 from 6012928 to 6414336 tid 2 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2561: delete oid 38 current snap is 271 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2558: finishing write tid 1 to smithi11881747-1 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2558: finishing write tid 2 to smithi11881747-1 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2558: finishing write tid 3 to smithi11881747-1 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2559: finishing copy_from to smithi11881747-15 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2559: got expected ENOENT (src dne) 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2559: finishing copy_from racing read to smithi11881747-15 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2558: finishing write tid 4 to smithi11881747-1 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2558: oid 1 updating version 0 to 34038 2024-03-20T08:10:22.106 INFO:tasks.rados.rados.0.smithi118.stdout:2558: oid 1 updating version 34038 to 34039 2024-03-20T08:10:22.107 INFO:tasks.rados.rados.0.smithi118.stdout:2558: oid 1 updating version 34039 to 34040 2024-03-20T08:10:22.107 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 34040 (ObjNum 779 snap 271 seq_num 779) dirty exists 2024-03-20T08:10:22.107 INFO:tasks.rados.rados.0.smithi118.stdout:2558: left oid 1 (ObjNum 779 snap 271 seq_num 779) 2024-03-20T08:10:22.113 INFO:tasks.rados.rados.0.smithi118.stdout:2560: finishing write tid 1 to smithi11881747-41 2024-03-20T08:10:22.113 INFO:tasks.rados.rados.0.smithi118.stdout:2560: finishing write tid 2 to smithi11881747-41 2024-03-20T08:10:22.113 INFO:tasks.rados.rados.0.smithi118.stdout:2560: finishing write tid 3 to smithi11881747-41 2024-03-20T08:10:22.115 INFO:tasks.rados.rados.0.smithi118.stdout:2560: finishing write tid 4 to smithi11881747-41 2024-03-20T08:10:22.115 INFO:tasks.rados.rados.0.smithi118.stdout:2560: oid 41 updating version 0 to 33963 2024-03-20T08:10:22.115 INFO:tasks.rados.rados.0.smithi118.stdout:2560: oid 41 updating version 33963 to 33965 2024-03-20T08:10:22.115 INFO:tasks.rados.rados.0.smithi118.stdout:2560: oid 41 version 33965 is already newer than 33964 2024-03-20T08:10:22.115 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 33965 (ObjNum 780 snap 271 seq_num 780) dirty exists 2024-03-20T08:10:22.115 INFO:tasks.rados.rados.0.smithi118.stdout:2560: left oid 41 (ObjNum 780 snap 271 seq_num 780) 2024-03-20T08:10:22.119 INFO:tasks.rados.rados.0.smithi118.stdout:2558: done (3 left) 2024-03-20T08:10:22.119 INFO:tasks.rados.rados.0.smithi118.stdout:2559: done (2 left) 2024-03-20T08:10:22.119 INFO:tasks.rados.rados.0.smithi118.stdout:2560: done (1 left) 2024-03-20T08:10:22.119 INFO:tasks.rados.rados.0.smithi118.stdout:2561: done (0 left) 2024-03-20T08:10:22.120 INFO:tasks.rados.rados.0.smithi118.stdout:2562: rollback oid 2 current snap is 271 2024-03-20T08:10:22.120 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 242 2024-03-20T08:10:22.120 INFO:tasks.rados.rados.0.smithi118.stdout:2563: rollback oid 12 current snap is 271 2024-03-20T08:10:22.120 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 12 to 268 2024-03-20T08:10:22.120 INFO:tasks.rados.rados.0.smithi118.stdout:2564: rollback oid 50 current snap is 271 2024-03-20T08:10:22.120 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 50 to 235 2024-03-20T08:10:22.120 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 271 2024-03-20T08:10:22.120 INFO:tasks.rados.rados.0.smithi118.stdout:2565: seq_num 782 ranges {0=704512,704512=401408} 2024-03-20T08:10:22.124 INFO:tasks.rados.rados.0.smithi118.stdout:2565: writing smithi11881747-46 from 0 to 704512 tid 1 2024-03-20T08:10:22.127 INFO:tasks.rados.rados.0.smithi118.stdout:2565: writing smithi11881747-46 from 704512 to 1105920 tid 2 2024-03-20T08:10:22.127 INFO:tasks.rados.rados.0.smithi118.stdout:2566: snap_create 2024-03-20T08:10:22.129 INFO:tasks.rados.rados.0.smithi118.stdout:2562: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:10:22.129 INFO:tasks.rados.rados.0.smithi118.stdout:2562: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:10:22.133 INFO:tasks.rados.rados.0.smithi118.stdout:2563: finishing rollback tid 0 to smithi11881747-12 2024-03-20T08:10:22.133 INFO:tasks.rados.rados.0.smithi118.stdout:2562: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:10:22.133 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 23106 (ObjNum 685 snap 225 seq_num 685) dirty exists 2024-03-20T08:10:22.134 INFO:tasks.rados.rados.0.smithi118.stdout:2565: finishing write tid 1 to smithi11881747-46 2024-03-20T08:10:22.134 INFO:tasks.rados.rados.0.smithi118.stdout:2563: finishing rollback tid 1 to smithi11881747-12 2024-03-20T08:10:22.134 INFO:tasks.rados.rados.0.smithi118.stdout:2563: finishing rollback tid 2 to smithi11881747-12 2024-03-20T08:10:22.134 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34013 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:10:22.135 INFO:tasks.rados.rados.0.smithi118.stdout:2565: finishing write tid 2 to smithi11881747-46 2024-03-20T08:10:22.135 INFO:tasks.rados.rados.0.smithi118.stdout:2565: finishing write tid 3 to smithi11881747-46 2024-03-20T08:10:22.138 INFO:tasks.rados.rados.0.smithi118.stdout:2565: finishing write tid 4 to smithi11881747-46 2024-03-20T08:10:22.138 INFO:tasks.rados.rados.0.smithi118.stdout:2565: oid 46 updating version 0 to 23109 2024-03-20T08:10:22.138 INFO:tasks.rados.rados.0.smithi118.stdout:2565: oid 46 version 23109 is already newer than 23108 2024-03-20T08:10:22.138 INFO:tasks.rados.rados.0.smithi118.stdout:2565: oid 46 version 23109 is already newer than 23107 2024-03-20T08:10:22.138 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 23109 (ObjNum 782 snap 271 seq_num 782) dirty exists 2024-03-20T08:10:22.138 INFO:tasks.rados.rados.0.smithi118.stdout:2565: left oid 46 (ObjNum 782 snap 271 seq_num 782) 2024-03-20T08:10:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:2564: finishing rollback tid 0 to smithi11881747-50 2024-03-20T08:10:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:2564: finishing rollback tid 1 to smithi11881747-50 2024-03-20T08:10:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:2564: finishing rollback tid 2 to smithi11881747-50 2024-03-20T08:10:22.140 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 44388 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:10:22.233 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:22.065+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:22.233 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:22.065+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: osdmap e587: 8 total, 8 up, 8 in 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:22.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:22.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:22.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:22.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:22.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:22.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:22.235 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:22.123+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:22.235 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:22.137+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:22.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:21.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.051+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.068+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:22.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.068+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.069+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.083+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.095+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.108+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.143+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.169+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.194+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.233+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:22.066+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:22.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:22.066+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:22.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:22.110+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:22.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:22.110+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:22.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:22.106+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:22.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:22.107+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:22.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:22.108+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:22.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:22.110+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:22.424 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:22.061+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: osdmap e587: 8 total, 8 up, 8 in 2024-03-20T08:10:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: osdmap e587: 8 total, 8 up, 8 in 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:22.487 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:22 smithi196 podman[111984]: 2024-03-20 08:10:22.293836258 +0000 UTC m=+5.406329902 container died e5e6b027dc5893db1e3f2204d656aab142a8b00d9d63c471a645e6097fc36bbf (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-20T08:10:22.487 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:10:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:10:22.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.258+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.270+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.313+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.362+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.386+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.424+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.462+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.489 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.740 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:22 smithi196 podman[111984]: 2024-03-20 08:10:22.679373585 +0000 UTC m=+5.791867228 container cleanup e5e6b027dc5893db1e3f2204d656aab142a8b00d9d63c471a645e6097fc36bbf (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=reef-78dbc53, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-03-20T08:10:22.740 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:22 smithi196 bash[111984]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b 2024-03-20T08:10:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.548+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.560+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.572+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.658+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.684+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.726+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:22.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.001 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:22 smithi196 podman[112000]: 2024-03-20 08:10:22.897983168 +0000 UTC m=+0.606249139 container remove e5e6b027dc5893db1e3f2204d656aab142a8b00d9d63c471a645e6097fc36bbf (image=quay.ceph.io/ceph-ci/ceph@sha256:564807290f99cbc6bf1c015df7a27cb3be1130ef836fbbe3e338d182b77d711b, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-78dbc53, GIT_CLEAN=True) 2024-03-20T08:10:23.001 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.765+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.792+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.805+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.831+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.855+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.893+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.944+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:22.985+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:2562: done (4 left) 2024-03-20T08:10:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:2563: done (3 left) 2024-03-20T08:10:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:2564: done (2 left) 2024-03-20T08:10:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:2565: done (1 left) 2024-03-20T08:10:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:2566: done (0 left) 2024-03-20T08:10:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:2567: read oid 19 snap -1 2024-03-20T08:10:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:2567: expect deleted 2024-03-20T08:10:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:2568: snap_remove snap 270 2024-03-20T08:10:23.264 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:23.068+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:23.264 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:23.068+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:23.265 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:23 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.b.service: Deactivated successfully. 2024-03-20T08:10:23.265 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:23 smithi196 systemd[1]: Stopped Ceph mds.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:10:23.265 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:23 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.b.service: Consumed 1.825s CPU time. 2024-03-20T08:10:23.265 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.030+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.042+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.055+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.069+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.071+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.101+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.266 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.210+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.267 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:23.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: osdmap e588: 8 total, 8 up, 8 in 2024-03-20T08:10:23.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:23.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:23.267 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:23.268 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-mon[87080]: pgmap v562: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 841 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 7.2 MiB/s rd, 242 KiB/s wr, 12 op/s 2024-03-20T08:10:23.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:23.067+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:23.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:23.067+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:23.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:23.067+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:23.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:23.153+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: osdmap e588: 8 total, 8 up, 8 in 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:23.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[95692]: pgmap v562: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 841 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 7.2 MiB/s rd, 242 KiB/s wr, 12 op/s 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: osdmap e588: 8 total, 8 up, 8 in 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-mon[93121]: pgmap v562: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 841 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 7.2 MiB/s rd, 242 KiB/s wr, 12 op/s 2024-03-20T08:10:23.528 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:23 smithi196 systemd[1]: Starting Ceph mds.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.304+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.326+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.338+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.364+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.378+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.431+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.529 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.444+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.469+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.483+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.530 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.782 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:23 smithi196 podman[112126]: 2024-03-20 08:10:23.662401683 +0000 UTC m=+0.024870802 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:10:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.543+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.560+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.621+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.635+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.649+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.665+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.678+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.691+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.717+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.729+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.742+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:10:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:10:23] "GET /metrics HTTP/1.1" 200 36400 "" "Prometheus/2.43.0" 2024-03-20T08:10:24.040 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:23 smithi196 podman[112126]: 2024-03-20 08:10:23.822214919 +0000 UTC m=+0.184684034 container create 237cfe0be47629abdd4e3b2f9e916702944b5bb23203447f2c05075543a70535 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-20T08:10:24.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.794+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.818+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.831+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.869+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.894+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.919+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.931+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.945+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:23.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.002+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.026+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.066 INFO:tasks.rados.rados.0.smithi118.stdout:2568: done (1 left) 2024-03-20T08:10:24.067 INFO:tasks.rados.rados.0.smithi118.stdout:2569: snap_create 2024-03-20T08:10:24.295 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-mon[87080]: osdmap e589: 8 total, 8 up, 8 in 2024-03-20T08:10:24.295 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:24.295 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:24.295 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:24.295 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:24.296 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:24.296 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:24.296 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:24.272+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:24.296 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:24.273+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:24.296 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:24.273+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:24.296 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.066+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.066+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.084+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.111+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.129+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.156+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.168+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.181+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.207+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.251+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.298 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:24 smithi196 podman[112126]: 2024-03-20 08:10:24.102747288 +0000 UTC m=+0.465216410 container init 237cfe0be47629abdd4e3b2f9e916702944b5bb23203447f2c05075543a70535 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-20T08:10:24.298 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:24 smithi196 podman[112126]: 2024-03-20 08:10:24.109552431 +0000 UTC m=+0.472021537 container start 237cfe0be47629abdd4e3b2f9e916702944b5bb23203447f2c05075543a70535 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2) 2024-03-20T08:10:24.298 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b[112139]: starting mds.b at 2024-03-20T08:10:24.298 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:24 smithi196 bash[112126]: 237cfe0be47629abdd4e3b2f9e916702944b5bb23203447f2c05075543a70535 2024-03-20T08:10:24.298 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:10:24 smithi196 systemd[1]: Started Ceph mds.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:10:24.299 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.266+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.299 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.299 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.299 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:24.070+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:24.299 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:24.070+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:24.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:24.072+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:24.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:24.072+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:24.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:24.274+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[93121]: osdmap e589: 8 total, 8 up, 8 in 2024-03-20T08:10:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:24.066+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:24.066+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:24.067+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:24.067+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:24.067+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:24.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[95692]: osdmap e589: 8 total, 8 up, 8 in 2024-03-20T08:10:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:24.425 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:24.275+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.307+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.331+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.345+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.360+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.430+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.558 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.445+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.559 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.459+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.559 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.559 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.488+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.559 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.559 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.515+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.559 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.529+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.559 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.542+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.559 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.556+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.820 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.820 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.582+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.820 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.820 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.695+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.708+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.736+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.749+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.763+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.778+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.792+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.821 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.805+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:24.822 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.819+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.891+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.905+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.936+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.951+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.982+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:24.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.024+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.041+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.071 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.079 INFO:tasks.rados.rados.0.smithi118.stdout:2567: done (1 left) 2024-03-20T08:10:25.079 INFO:tasks.rados.rados.0.smithi118.stdout:2569: done (0 left) 2024-03-20T08:10:25.079 INFO:tasks.rados.rados.0.smithi118.stdout:2570: copy_from oid 50 from oid 43 current snap is 273 2024-03-20T08:10:25.079 INFO:tasks.rados.rados.0.smithi118.stdout:2571: snap_create 2024-03-20T08:10:25.138 INFO:tasks.rados.rados.0.smithi118.stdout:2570: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:10:25.143 INFO:tasks.rados.rados.0.smithi118.stdout:2570: finishing copy_from to smithi11881747-50 2024-03-20T08:10:25.143 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 34037 (ObjNum 774 snap 270 seq_num 774) dirty exists 2024-03-20T08:10:25.321 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:25.083+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:25.322 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:25.084+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:25.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:25.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:25.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: osdmap e590: 8 total, 8 up, 8 in 2024-03-20T08:10:25.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:25.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:25.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:25.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:25.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: pgmap v565: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 852 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 6.2 MiB/s rd, 724 KiB/s wr, 24 op/s 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:25.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:25.323 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.071+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.085+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.127+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.146+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.158+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.197+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.209+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.235+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.247+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.259+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.284+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.309+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.322+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:25.315+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:25.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:25.315+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:25.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:25.082+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:25.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:25.082+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:25.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:25.082+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:25.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:25.079+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:25.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:25.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: osdmap e590: 8 total, 8 up, 8 in 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: pgmap v565: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 852 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 6.2 MiB/s rd, 724 KiB/s wr, 24 op/s 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:25.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: osdmap e590: 8 total, 8 up, 8 in 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:25.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: pgmap v565: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 852 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 6.2 MiB/s rd, 724 KiB/s wr, 24 op/s 2024-03-20T08:10:25.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:25.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:25.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:25.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:25.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:25.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:25.575 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.575 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.575 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.575 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.378+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.575 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.394+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.408+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.420+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.434+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.490+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.521+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.535+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.549+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.576 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.562+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.827 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:25.578+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:25.827 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:25.579+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:25.827 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.601+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.627+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.665+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.677+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.701+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.714+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.763+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.776+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.789+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.801+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:25.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:2570: done (1 left) 2024-03-20T08:10:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:2571: done (0 left) 2024-03-20T08:10:26.080 INFO:tasks.rados.rados.0.smithi118.stdout:2572: rollback oid 44 current snap is 274 2024-03-20T08:10:26.081 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 44 to 265 2024-03-20T08:10:26.081 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 274 2024-03-20T08:10:26.081 INFO:tasks.rados.rados.0.smithi118.stdout:2573: seq_num 783 ranges {0=720896,720896=401408} 2024-03-20T08:10:26.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.852+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.081 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.876+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.890+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.936+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:25.994+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.082 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.085 INFO:tasks.rados.rados.0.smithi118.stdout:2573: writing smithi11881747-42 from 0 to 720896 tid 1 2024-03-20T08:10:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:2573: writing smithi11881747-42 from 720896 to 1122304 tid 2 2024-03-20T08:10:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:2574: rollback oid 5 current snap is 274 2024-03-20T08:10:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 5 to 210 2024-03-20T08:10:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 274 2024-03-20T08:10:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:2575: seq_num 784 ranges {10141696=737280,10878976=401408} 2024-03-20T08:10:26.151 INFO:tasks.rados.rados.0.smithi118.stdout:2575: writing smithi11881747-50 from 10141696 to 10878976 tid 1 2024-03-20T08:10:26.158 INFO:tasks.rados.rados.0.smithi118.stdout:2575: writing smithi11881747-50 from 10878976 to 11280384 tid 2 2024-03-20T08:10:26.158 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 274 2024-03-20T08:10:26.158 INFO:tasks.rados.rados.0.smithi118.stdout:2573: finishing write tid 1 to smithi11881747-42 2024-03-20T08:10:26.158 INFO:tasks.rados.rados.0.smithi118.stdout:2576: seq_num 785 ranges {81920=753664,835584=401408} 2024-03-20T08:10:26.161 INFO:tasks.rados.rados.0.smithi118.stdout:2576: writing smithi11881747-32 from 81920 to 835584 tid 1 2024-03-20T08:10:26.164 INFO:tasks.rados.rados.0.smithi118.stdout:2576: writing smithi11881747-32 from 835584 to 1236992 tid 2 2024-03-20T08:10:26.164 INFO:tasks.rados.rados.0.smithi118.stdout:2577: read oid 41 snap -1 2024-03-20T08:10:26.164 INFO:tasks.rados.rados.0.smithi118.stdout:2577: expect (ObjNum 780 snap 271 seq_num 780) 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2573: finishing write tid 2 to smithi11881747-42 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2573: finishing write tid 3 to smithi11881747-42 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2574: finishing rollback tid 0 to smithi11881747-5 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2573: finishing write tid 4 to smithi11881747-42 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2573: oid 42 updating version 0 to 23112 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2573: oid 42 updating version 23112 to 23113 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2573: oid 42 version 23113 is already newer than 23111 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 23113 (ObjNum 783 snap 274 seq_num 783) dirty exists 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2573: left oid 42 (ObjNum 783 snap 274 seq_num 783) 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2574: finishing rollback tid 1 to smithi11881747-5 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2573: done (5 left) 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2578: delete oid 42 current snap is 274 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2574: finishing rollback tid 2 to smithi11881747-5 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 23116 (ObjNum 642 snap 209 seq_num 642) dirty exists 2024-03-20T08:10:26.165 INFO:tasks.rados.rados.0.smithi118.stdout:2572: finishing rollback tid 0 to smithi11881747-44 2024-03-20T08:10:26.166 INFO:tasks.rados.rados.0.smithi118.stdout:2572: finishing rollback tid 1 to smithi11881747-44 2024-03-20T08:10:26.166 INFO:tasks.rados.rados.0.smithi118.stdout:2572: finishing rollback tid 2 to smithi11881747-44 2024-03-20T08:10:26.166 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 45552 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:10:26.167 INFO:tasks.rados.rados.0.smithi118.stdout:2575: finishing write tid 1 to smithi11881747-50 2024-03-20T08:10:26.168 INFO:tasks.rados.rados.0.smithi118.stdout:2572: done (5 left) 2024-03-20T08:10:26.169 INFO:tasks.rados.rados.0.smithi118.stdout:2574: done (4 left) 2024-03-20T08:10:26.169 INFO:tasks.rados.rados.0.smithi118.stdout:2578: done (3 left) 2024-03-20T08:10:26.169 INFO:tasks.rados.rados.0.smithi118.stdout:2579: delete oid 18 current snap is 274 2024-03-20T08:10:26.171 INFO:tasks.rados.rados.0.smithi118.stdout:2575: finishing write tid 2 to smithi11881747-50 2024-03-20T08:10:26.173 INFO:tasks.rados.rados.0.smithi118.stdout:2576: finishing write tid 1 to smithi11881747-32 2024-03-20T08:10:26.175 INFO:tasks.rados.rados.0.smithi118.stdout:2575: finishing write tid 3 to smithi11881747-50 2024-03-20T08:10:26.175 INFO:tasks.rados.rados.0.smithi118.stdout:2579: done (3 left) 2024-03-20T08:10:26.175 INFO:tasks.rados.rados.0.smithi118.stdout:2580: rollback oid 24 current snap is 274 2024-03-20T08:10:26.175 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 272 2024-03-20T08:10:26.176 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 274 2024-03-20T08:10:26.176 INFO:tasks.rados.rados.0.smithi118.stdout:2581: seq_num 788 ranges {6332416=802816,7135232=401408} 2024-03-20T08:10:26.216 INFO:tasks.rados.rados.0.smithi118.stdout:2581: writing smithi11881747-34 from 6332416 to 7135232 tid 1 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2581: writing smithi11881747-34 from 7135232 to 7536640 tid 2 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2582: read oid 9 snap -1 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2582: expect deleted 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2583: read oid 26 snap -1 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2583: expect deleted 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2584: read oid 46 snap 216 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2584: expect deleted 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2585: rollback oid 6 current snap is 274 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 255 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2576: finishing write tid 2 to smithi11881747-32 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2576: finishing write tid 3 to smithi11881747-32 2024-03-20T08:10:26.220 INFO:tasks.rados.rados.0.smithi118.stdout:2576: finishing write tid 4 to smithi11881747-32 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2576: oid 32 updating version 0 to 34016 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2576: oid 32 version 34016 is already newer than 34015 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2576: oid 32 version 34016 is already newer than 34014 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34016 (ObjNum 785 snap 274 seq_num 785) dirty exists 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2576: left oid 32 (ObjNum 785 snap 274 seq_num 785) 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2575: finishing write tid 4 to smithi11881747-50 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2575: oid 50 updating version 0 to 45570 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2575: oid 50 version 45570 is already newer than 45569 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2575: oid 50 version 45570 is already newer than 45568 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 45570 (ObjNum 784 snap 274 seq_num 784) dirty exists 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2575: left oid 50 (ObjNum 784 snap 274 seq_num 784) 2024-03-20T08:10:26.221 INFO:tasks.rados.rados.0.smithi118.stdout:2577: expect (ObjNum 780 snap 271 seq_num 780) 2024-03-20T08:10:26.334 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:26.087+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:26.334 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:26.088+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:26.335 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:26.224+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:26.335 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:26.224+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:26.335 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:26.231+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:26.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:26.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: osdmap e591: 8 total, 8 up, 8 in 2024-03-20T08:10:26.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: mds.? [v2:172.21.15.196:6832/320115093,v1:172.21.15.196:6833/320115093] up:boot 2024-03-20T08:10:26.335 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: fsmap cephfs:1 {0=a=up:active} 1 up:standby 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:26.336 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:26.336 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.087+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:26.336 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.087+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:26.336 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.169+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.215+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.249+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.292+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.306+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.337 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:26.086+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:26.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:26.087+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:26.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:26.080+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:26.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:26.175+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:26.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:26.168+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:26.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:26.168+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:26.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:26.169+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:26.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:26.173+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: osdmap e591: 8 total, 8 up, 8 in 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: mds.? [v2:172.21.15.196:6832/320115093,v1:172.21.15.196:6833/320115093] up:boot 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: fsmap cephfs:1 {0=a=up:active} 1 up:standby 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: osdmap e591: 8 total, 8 up, 8 in 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: mds.? [v2:172.21.15.196:6832/320115093,v1:172.21.15.196:6833/320115093] up:boot 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: fsmap cephfs:1 {0=a=up:active} 1 up:standby 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 13e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:26.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:26.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.351+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.366+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.382+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.395+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.409+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.421+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.445+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.459+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.471+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.509+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.534+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.586 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.558+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.675 INFO:tasks.rados.rados.0.smithi118.stdout:2580: finishing rollback tid 0 to smithi11881747-24 2024-03-20T08:10:26.675 INFO:tasks.rados.rados.0.smithi118.stdout:2580: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2580: finishing rollback tid 2 to smithi11881747-24 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 45576 (ObjNum 33 snap 0 seq_num 0) dirty exists 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2581: finishing write tid 1 to smithi11881747-34 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2581: finishing write tid 2 to smithi11881747-34 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2581: finishing write tid 3 to smithi11881747-34 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2585: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2585: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2585: finishing rollback tid 2 to smithi11881747-6 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 23112 (ObjNum 709 snap 239 seq_num 709) dirty exists 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2575: done (8 left) 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2576: done (7 left) 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2577: done (6 left) 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2580: done (5 left) 2024-03-20T08:10:26.676 INFO:tasks.rados.rados.0.smithi118.stdout:2582: done (4 left) 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2583: done (3 left) 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2584: done (2 left) 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2585: done (1 left) 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 274 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2581: finishing write tid 4 to smithi11881747-34 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2581: oid 34 updating version 0 to 33933 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2581: oid 34 updating version 33933 to 33934 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2581: oid 34 version 33934 is already newer than 33932 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 33934 (ObjNum 788 snap 274 seq_num 788) dirty exists 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2581: left oid 34 (ObjNum 788 snap 274 seq_num 788) 2024-03-20T08:10:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:2586: seq_num 789 ranges {11280384=417792,11698176=663552,12361728=139264} 2024-03-20T08:10:26.741 INFO:tasks.rados.rados.0.smithi118.stdout:2586: writing smithi11881747-50 from 11280384 to 11698176 tid 1 2024-03-20T08:10:26.746 INFO:tasks.rados.rados.0.smithi118.stdout:2586: writing smithi11881747-50 from 11698176 to 12361728 tid 2 2024-03-20T08:10:26.747 INFO:tasks.rados.rados.0.smithi118.stdout:2586: writing smithi11881747-50 from 12361728 to 12500992 tid 3 2024-03-20T08:10:26.747 INFO:tasks.rados.rados.0.smithi118.stdout:2581: done (1 left) 2024-03-20T08:10:26.747 INFO:tasks.rados.rados.0.smithi118.stdout:2587: delete oid 39 current snap is 274 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:2587: done (1 left) 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:2588: rollback oid 46 current snap is 274 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 226 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:2589: rollback oid 21 current snap is 274 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 266 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:2590: copy_from oid 5 from oid 43 current snap is 274 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:2591: read oid 19 snap 268 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:2591: expect deleted 2024-03-20T08:10:26.750 INFO:tasks.rados.rados.0.smithi118.stdout:2592: snap_remove snap 269 2024-03-20T08:10:26.841 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.600+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.841 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.614+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.841 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.841 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.643+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.841 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.841 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.672+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.841 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.687+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.711+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.784+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.802+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:26.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.841+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.092 INFO:tasks.rados.rados.0.smithi118.stdout:2592: done (5 left) 2024-03-20T08:10:27.092 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 274 2024-03-20T08:10:27.092 INFO:tasks.rados.rados.0.smithi118.stdout:2593: seq_num 791 ranges {0=450560,450560=475136,925696=327680} 2024-03-20T08:10:27.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.866+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.891+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.905+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.918+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.956+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:26.993+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.018+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.043+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.056+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.094 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: osdmap e592: 8 total, 8 up, 8 in 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:27.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:27.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:27.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:27.096 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-mon[87080]: pgmap v568: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 859 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 543 KiB/s rd, 1.4 MiB/s wr, 30 op/s 2024-03-20T08:10:27.096 INFO:tasks.rados.rados.0.smithi118.stdout:2593: writing smithi11881747-9 from 0 to 450560 tid 1 2024-03-20T08:10:27.098 INFO:tasks.rados.rados.0.smithi118.stdout:2593: writing smithi11881747-9 from 450560 to 925696 tid 2 2024-03-20T08:10:27.101 INFO:tasks.rados.rados.0.smithi118.stdout:2593: writing smithi11881747-9 from 925696 to 1253376 tid 3 2024-03-20T08:10:27.101 INFO:tasks.rados.rados.0.smithi118.stdout:2594: read oid 24 snap 226 2024-03-20T08:10:27.101 INFO:tasks.rados.rados.0.smithi118.stdout:2594: expect (ObjNum 687 snap 225 seq_num 687) 2024-03-20T08:10:27.101 INFO:tasks.rados.rados.0.smithi118.stdout:2595: snap_remove snap 260 2024-03-20T08:10:27.349 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:27.092+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:27.349 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:27.092+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:27.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.096+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:27.349 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.115+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.231+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.247+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.350 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.315+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.327+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.351 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:27.095+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:27.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:27.095+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:27.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:27.095+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: osdmap e592: 8 total, 8 up, 8 in 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[93121]: pgmap v568: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 859 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 543 KiB/s rd, 1.4 MiB/s wr, 30 op/s 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: osdmap e592: 8 total, 8 up, 8 in 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14a=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-mon[95692]: pgmap v568: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 859 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 543 KiB/s rd, 1.4 MiB/s wr, 30 op/s 2024-03-20T08:10:27.426 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:27.089+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:27.426 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:27.090+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:27.426 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:27.091+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:27.426 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:27.091+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:27.426 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:27.092+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:27.426 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:27.101+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:27.427 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:27.093+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:27.427 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:27.093+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:27.607 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:27.403+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:27.607 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:27.404+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:27.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.399+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.438+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.475+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.488+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.511+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.523+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.534+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.560+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.866 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.622+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.866 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.866 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.646+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.866 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.866 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.866 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.684+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.866 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.866 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.712+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.788+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.802+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.832+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:27.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.091 INFO:tasks.rados.rados.0.smithi118.stdout:2595: done (7 left) 2024-03-20T08:10:28.091 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 274 2024-03-20T08:10:28.092 INFO:tasks.rados.rados.0.smithi118.stdout:2596: seq_num 792 ranges {1056768=466944,1523712=761856,2285568=40960} 2024-03-20T08:10:28.100 INFO:tasks.rados.rados.0.smithi118.stdout:2596: writing smithi11881747-1 from 1056768 to 1523712 tid 1 2024-03-20T08:10:28.105 INFO:tasks.rados.rados.0.smithi118.stdout:2596: writing smithi11881747-1 from 1523712 to 2285568 tid 2 2024-03-20T08:10:28.105 INFO:tasks.rados.rados.0.smithi118.stdout:2596: writing smithi11881747-1 from 2285568 to 2326528 tid 3 2024-03-20T08:10:28.105 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 274 2024-03-20T08:10:28.105 INFO:tasks.rados.rados.0.smithi118.stdout:2597: seq_num 793 ranges {0=483328,483328=647168,1130496=155648} 2024-03-20T08:10:28.108 INFO:tasks.rados.rados.0.smithi118.stdout:2597: writing smithi11881747-17 from 0 to 483328 tid 1 2024-03-20T08:10:28.111 INFO:tasks.rados.rados.0.smithi118.stdout:2597: writing smithi11881747-17 from 483328 to 1130496 tid 2 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:2597: writing smithi11881747-17 from 1130496 to 1286144 tid 3 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:2598: snap_create 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:2588: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:2588: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:2588: finishing rollback tid 2 to smithi11881747-46 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 23115 (ObjNum 681 snap 225 seq_num 681) dirty exists 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:2586: finishing write tid 1 to smithi11881747-50 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:2586: finishing write tid 2 to smithi11881747-50 2024-03-20T08:10:28.113 INFO:tasks.rados.rados.0.smithi118.stdout:2586: finishing write tid 3 to smithi11881747-50 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2586: finishing write tid 4 to smithi11881747-50 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2589: finishing rollback tid 0 to smithi11881747-21 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2589: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 37043 (ObjNum 177 snap 0 seq_num 0) dirty dne 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2586: finishing write tid 5 to smithi11881747-50 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2586: oid 50 updating version 0 to 45742 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2586: oid 50 version 45742 is already newer than 45741 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2586: oid 50 version 45742 is already newer than 45739 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2586: oid 50 version 45742 is already newer than 45740 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 45742 (ObjNum 789 snap 274 seq_num 789) dirty exists 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2586: left oid 50 (ObjNum 789 snap 274 seq_num 789) 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2590: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2590: finishing copy_from to smithi11881747-5 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 34037 (ObjNum 774 snap 270 seq_num 774) dirty exists 2024-03-20T08:10:28.114 INFO:tasks.rados.rados.0.smithi118.stdout:2593: finishing write tid 1 to smithi11881747-9 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: finishing write tid 2 to smithi11881747-9 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: finishing write tid 3 to smithi11881747-9 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: finishing write tid 4 to smithi11881747-9 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: finishing write tid 5 to smithi11881747-9 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: oid 9 updating version 0 to 34040 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: oid 9 version 34040 is already newer than 34038 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: oid 9 version 34040 is already newer than 34039 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: oid 9 updating version 34040 to 34041 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 34041 (ObjNum 791 snap 274 seq_num 791) dirty exists 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2593: left oid 9 (ObjNum 791 snap 274 seq_num 791) 2024-03-20T08:10:28.115 INFO:tasks.rados.rados.0.smithi118.stdout:2594: expect (ObjNum 687 snap 225 seq_num 687) 2024-03-20T08:10:28.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:28.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:28.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: osdmap e593: 8 total, 8 up, 8 in 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:28.119 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:28.120 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.861+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.880+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.894+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.938+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.953+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.120 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:27.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.022+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.046+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.083+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.096+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.097+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.118+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.121 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:28.094+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:28.122 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:28.094+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:28.122 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:28.116+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:28.215 INFO:tasks.rados.rados.0.smithi118.stdout:2596: finishing write tid 1 to smithi11881747-1 2024-03-20T08:10:28.215 INFO:tasks.rados.rados.0.smithi118.stdout:2596: finishing write tid 2 to smithi11881747-1 2024-03-20T08:10:28.215 INFO:tasks.rados.rados.0.smithi118.stdout:2596: finishing write tid 3 to smithi11881747-1 2024-03-20T08:10:28.215 INFO:tasks.rados.rados.0.smithi118.stdout:2596: finishing write tid 4 to smithi11881747-1 2024-03-20T08:10:28.215 INFO:tasks.rados.rados.0.smithi118.stdout:2597: finishing write tid 1 to smithi11881747-17 2024-03-20T08:10:28.215 INFO:tasks.rados.rados.0.smithi118.stdout:2597: finishing write tid 2 to smithi11881747-17 2024-03-20T08:10:28.215 INFO:tasks.rados.rados.0.smithi118.stdout:2597: finishing write tid 3 to smithi11881747-17 2024-03-20T08:10:28.215 INFO:tasks.rados.rados.0.smithi118.stdout:2596: finishing write tid 5 to smithi11881747-1 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2596: oid 1 updating version 0 to 34043 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2596: oid 1 updating version 34043 to 34044 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2596: oid 1 version 34044 is already newer than 34041 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2596: oid 1 version 34044 is already newer than 34042 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 34044 (ObjNum 792 snap 274 seq_num 792) dirty exists 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2596: left oid 1 (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2597: finishing write tid 4 to smithi11881747-17 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2597: finishing write tid 5 to smithi11881747-17 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2597: oid 17 updating version 0 to 7059 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2597: oid 17 updating version 7059 to 7061 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2597: oid 17 version 7061 is already newer than 7058 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2597: oid 17 version 7061 is already newer than 7060 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7061 (ObjNum 793 snap 274 seq_num 793) dirty exists 2024-03-20T08:10:28.216 INFO:tasks.rados.rados.0.smithi118.stdout:2597: left oid 17 (ObjNum 793 snap 274 seq_num 793) 2024-03-20T08:10:28.369 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:28.120+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:28.369 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:28.120+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.137+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.150+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.191+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.219+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.246+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.275+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.306+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.322+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.354+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:28.093+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:28.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:28.097+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:28.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:28.098+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:28.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:28.090+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:28.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:28.091+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:28.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:28.091+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:28.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:28.092+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:28.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:28.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:28.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: osdmap e593: 8 total, 8 up, 8 in 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:28.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: osdmap e593: 8 total, 8 up, 8 in 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:28.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 149=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:28.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.416+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.433+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.447+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.479+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.493+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.636+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.718+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.763+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.778+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.792+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.822+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.862+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.875+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.889+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:28.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:28.453+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:29.103 INFO:tasks.rados.rados.0.smithi118.stdout:2586: done (9 left) 2024-03-20T08:10:29.103 INFO:tasks.rados.rados.0.smithi118.stdout:2588: done (8 left) 2024-03-20T08:10:29.103 INFO:tasks.rados.rados.0.smithi118.stdout:2589: done (7 left) 2024-03-20T08:10:29.103 INFO:tasks.rados.rados.0.smithi118.stdout:2590: done (6 left) 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:2591: done (5 left) 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:2593: done (4 left) 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:2594: done (3 left) 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:2596: done (2 left) 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:2597: done (1 left) 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:2598: done (0 left) 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:2599: copy_from oid 10 from oid 15 current snap is 275 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 275 2024-03-20T08:10:29.104 INFO:tasks.rados.rados.0.smithi118.stdout:2600: seq_num 794 ranges {2228224=499712,2727936=532480,3260416=270336} 2024-03-20T08:10:29.118 INFO:tasks.rados.rados.0.smithi118.stdout:2600: writing smithi11881747-37 from 2228224 to 2727936 tid 1 2024-03-20T08:10:29.121 INFO:tasks.rados.rados.0.smithi118.stdout:2600: writing smithi11881747-37 from 2727936 to 3260416 tid 2 2024-03-20T08:10:29.124 INFO:tasks.rados.rados.0.smithi118.stdout:2600: writing smithi11881747-37 from 3260416 to 3530752 tid 3 2024-03-20T08:10:29.124 INFO:tasks.rados.rados.0.smithi118.stdout:2601: copy_from oid 6 from oid 9 current snap is 275 2024-03-20T08:10:29.124 INFO:tasks.rados.rados.0.smithi118.stdout:2599: finishing copy_from to smithi11881747-10 2024-03-20T08:10:29.124 INFO:tasks.rados.rados.0.smithi118.stdout:2599: got expected ENOENT (src dne) 2024-03-20T08:10:29.124 INFO:tasks.rados.rados.0.smithi118.stdout:2599: finishing copy_from racing read to smithi11881747-10 2024-03-20T08:10:29.124 INFO:tasks.rados.rados.0.smithi118.stdout:2599: done (2 left) 2024-03-20T08:10:29.124 INFO:tasks.rados.rados.0.smithi118.stdout:2602: snap_remove snap 210 2024-03-20T08:10:29.138 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:29.101+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:29.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:29.102+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:29.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:29.109+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:29.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:29.110+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:29.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: osdmap e594: 8 total, 8 up, 8 in 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: pgmap v571: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 860 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-03-20T08:10:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.901+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.982+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:28.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.028+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.042+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.056+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.071+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.111+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:29.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.118+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:29.117+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:29.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:29.114+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:29.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:29.114+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:29.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:29.115+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:29.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:29.102+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:29.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:29.102+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:29.100+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:29.100+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:29.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:29.102+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:29.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:29.102+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:29.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:29.102+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: osdmap e594: 8 total, 8 up, 8 in 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: pgmap v571: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 860 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: osdmap e594: 8 total, 8 up, 8 in 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:29.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: pgmap v571: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 860 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-03-20T08:10:29.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.178+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.193+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.234+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.261+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.275+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.289+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.302+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.315+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.429 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.384+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.493+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.507+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.579+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.636+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.717+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.979 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.730+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.769+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.811+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.849+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.861+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.874+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.900+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:29.980 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2601: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2601: finishing copy_from to smithi11881747-6 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 34041 (ObjNum 791 snap 274 seq_num 791) dirty exists 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2600: finishing write tid 1 to smithi11881747-37 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2600: finishing write tid 2 to smithi11881747-37 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2600: finishing write tid 3 to smithi11881747-37 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2600: finishing write tid 4 to smithi11881747-37 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2600: finishing write tid 5 to smithi11881747-37 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2600: oid 37 updating version 0 to 34012 2024-03-20T08:10:30.108 INFO:tasks.rados.rados.0.smithi118.stdout:2600: oid 37 updating version 34012 to 34014 2024-03-20T08:10:30.109 INFO:tasks.rados.rados.0.smithi118.stdout:2600: oid 37 version 34014 is already newer than 34013 2024-03-20T08:10:30.109 INFO:tasks.rados.rados.0.smithi118.stdout:2600: oid 37 version 34014 is already newer than 34011 2024-03-20T08:10:30.109 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 34014 (ObjNum 794 snap 275 seq_num 794) dirty exists 2024-03-20T08:10:30.109 INFO:tasks.rados.rados.0.smithi118.stdout:2600: left oid 37 (ObjNum 794 snap 275 seq_num 794) 2024-03-20T08:10:30.109 INFO:tasks.rados.rados.0.smithi118.stdout:2600: done (2 left) 2024-03-20T08:10:30.109 INFO:tasks.rados.rados.0.smithi118.stdout:2601: done (1 left) 2024-03-20T08:10:30.109 INFO:tasks.rados.rados.0.smithi118.stdout:2602: done (0 left) 2024-03-20T08:10:30.109 INFO:tasks.rados.rados.0.smithi118.stdout:2603: delete oid 49 current snap is 275 2024-03-20T08:10:30.111 INFO:tasks.rados.rados.0.smithi118.stdout:2603: done (0 left) 2024-03-20T08:10:30.111 INFO:tasks.rados.rados.0.smithi118.stdout:2604: rmattr oid 13 current snap is 275 2024-03-20T08:10:30.111 INFO:tasks.rados.rados.0.smithi118.stdout:2605: setattr oid 5 current snap is 275 2024-03-20T08:10:30.115 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 34042 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:10:30.115 INFO:tasks.rados.rados.0.smithi118.stdout:2604: done (1 left) 2024-03-20T08:10:30.115 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 275 2024-03-20T08:10:30.115 INFO:tasks.rados.rados.0.smithi118.stdout:2606: seq_num 796 ranges {0=540672,540672=704512,1245184=98304} 2024-03-20T08:10:30.119 INFO:tasks.rados.rados.0.smithi118.stdout:2606: writing smithi11881747-42 from 0 to 540672 tid 1 2024-03-20T08:10:30.124 INFO:tasks.rados.rados.0.smithi118.stdout:2606: writing smithi11881747-42 from 540672 to 1245184 tid 2 2024-03-20T08:10:30.125 INFO:tasks.rados.rados.0.smithi118.stdout:2606: writing smithi11881747-42 from 1245184 to 1343488 tid 3 2024-03-20T08:10:30.125 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 275 2024-03-20T08:10:30.125 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 34043 (ObjNum 774 snap 270 seq_num 774) dirty exists 2024-03-20T08:10:30.125 INFO:tasks.rados.rados.0.smithi118.stdout:2606: finishing write tid 1 to smithi11881747-42 2024-03-20T08:10:30.125 INFO:tasks.rados.rados.0.smithi118.stdout:2607: seq_num 797 ranges {3514368=557056,4071424=589824,4661248=212992} 2024-03-20T08:10:30.148 INFO:tasks.rados.rados.0.smithi118.stdout:2607: writing smithi11881747-46 from 3514368 to 4071424 tid 1 2024-03-20T08:10:30.153 INFO:tasks.rados.rados.0.smithi118.stdout:2607: writing smithi11881747-46 from 4071424 to 4661248 tid 2 2024-03-20T08:10:30.154 INFO:tasks.rados.rados.0.smithi118.stdout:2607: writing smithi11881747-46 from 4661248 to 4874240 tid 3 2024-03-20T08:10:30.155 INFO:tasks.rados.rados.0.smithi118.stdout:2605: done (2 left) 2024-03-20T08:10:30.155 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 275 2024-03-20T08:10:30.155 INFO:tasks.rados.rados.0.smithi118.stdout:2606: finishing write tid 2 to smithi11881747-42 2024-03-20T08:10:30.155 INFO:tasks.rados.rados.0.smithi118.stdout:2608: seq_num 798 ranges {5193728=573440,5767168=466944,6234112=335872} 2024-03-20T08:10:30.186 INFO:tasks.rados.rados.0.smithi118.stdout:2608: writing smithi11881747-25 from 5193728 to 5767168 tid 1 2024-03-20T08:10:30.189 INFO:tasks.rados.rados.0.smithi118.stdout:2608: writing smithi11881747-25 from 5767168 to 6234112 tid 2 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:2608: writing smithi11881747-25 from 6234112 to 6569984 tid 3 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:2609: rollback oid 50 current snap is 275 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 50 to 225 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:2606: finishing write tid 3 to smithi11881747-42 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:2606: finishing write tid 4 to smithi11881747-42 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:2606: finishing write tid 5 to smithi11881747-42 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:2606: oid 42 updating version 0 to 34046 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:2606: oid 42 version 34046 is already newer than 34045 2024-03-20T08:10:30.192 INFO:tasks.rados.rados.0.smithi118.stdout:2606: oid 42 version 34046 is already newer than 34044 2024-03-20T08:10:30.193 INFO:tasks.rados.rados.0.smithi118.stdout:2606: oid 42 updating version 34046 to 34047 2024-03-20T08:10:30.193 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 34047 (ObjNum 796 snap 275 seq_num 796) dirty exists 2024-03-20T08:10:30.193 INFO:tasks.rados.rados.0.smithi118.stdout:2606: left oid 42 (ObjNum 796 snap 275 seq_num 796) 2024-03-20T08:10:30.193 INFO:tasks.rados.rados.0.smithi118.stdout:2606: done (3 left) 2024-03-20T08:10:30.193 INFO:tasks.rados.rados.0.smithi118.stdout:2610: delete oid 43 current snap is 275 2024-03-20T08:10:30.205 INFO:tasks.rados.rados.0.smithi118.stdout:2610: done (3 left) 2024-03-20T08:10:30.205 INFO:tasks.rados.rados.0.smithi118.stdout:2611: snap_create 2024-03-20T08:10:30.205 INFO:tasks.rados.rados.0.smithi118.stdout:2608: finishing write tid 1 to smithi11881747-25 2024-03-20T08:10:30.206 INFO:tasks.rados.rados.0.smithi118.stdout:2608: finishing write tid 2 to smithi11881747-25 2024-03-20T08:10:30.206 INFO:tasks.rados.rados.0.smithi118.stdout:2608: finishing write tid 3 to smithi11881747-25 2024-03-20T08:10:30.206 INFO:tasks.rados.rados.0.smithi118.stdout:2608: finishing write tid 4 to smithi11881747-25 2024-03-20T08:10:30.208 INFO:tasks.rados.rados.0.smithi118.stdout:2608: finishing write tid 5 to smithi11881747-25 2024-03-20T08:10:30.208 INFO:tasks.rados.rados.0.smithi118.stdout:2608: oid 25 updating version 0 to 34051 2024-03-20T08:10:30.208 INFO:tasks.rados.rados.0.smithi118.stdout:2608: oid 25 version 34051 is already newer than 34050 2024-03-20T08:10:30.208 INFO:tasks.rados.rados.0.smithi118.stdout:2608: oid 25 version 34051 is already newer than 34048 2024-03-20T08:10:30.208 INFO:tasks.rados.rados.0.smithi118.stdout:2608: oid 25 version 34051 is already newer than 34049 2024-03-20T08:10:30.208 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 34051 (ObjNum 798 snap 275 seq_num 798) dirty exists 2024-03-20T08:10:30.208 INFO:tasks.rados.rados.0.smithi118.stdout:2608: left oid 25 (ObjNum 798 snap 275 seq_num 798) 2024-03-20T08:10:30.210 INFO:tasks.rados.rados.0.smithi118.stdout:2607: finishing write tid 1 to smithi11881747-46 2024-03-20T08:10:30.211 INFO:tasks.rados.rados.0.smithi118.stdout:2607: finishing write tid 2 to smithi11881747-46 2024-03-20T08:10:30.212 INFO:tasks.rados.rados.0.smithi118.stdout:2607: finishing write tid 3 to smithi11881747-46 2024-03-20T08:10:30.213 INFO:tasks.rados.rados.0.smithi118.stdout:2607: finishing write tid 4 to smithi11881747-46 2024-03-20T08:10:30.216 INFO:tasks.rados.rados.0.smithi118.stdout:2607: finishing write tid 5 to smithi11881747-46 2024-03-20T08:10:30.216 INFO:tasks.rados.rados.0.smithi118.stdout:2607: oid 46 updating version 0 to 34042 2024-03-20T08:10:30.216 INFO:tasks.rados.rados.0.smithi118.stdout:2607: oid 46 updating version 34042 to 34043 2024-03-20T08:10:30.216 INFO:tasks.rados.rados.0.smithi118.stdout:2607: oid 46 updating version 34043 to 34044 2024-03-20T08:10:30.216 INFO:tasks.rados.rados.0.smithi118.stdout:2607: oid 46 updating version 34044 to 34045 2024-03-20T08:10:30.216 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 34045 (ObjNum 797 snap 275 seq_num 797) dirty exists 2024-03-20T08:10:30.216 INFO:tasks.rados.rados.0.smithi118.stdout:2607: left oid 46 (ObjNum 797 snap 275 seq_num 797) 2024-03-20T08:10:30.223 INFO:tasks.rados.rados.0.smithi118.stdout:2609: finishing rollback tid 0 to smithi11881747-50 2024-03-20T08:10:30.223 INFO:tasks.rados.rados.0.smithi118.stdout:2609: finishing rollback tid 1 to smithi11881747-50 2024-03-20T08:10:30.224 INFO:tasks.rados.rados.0.smithi118.stdout:2609: finishing rollback tid 2 to smithi11881747-50 2024-03-20T08:10:30.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 46718 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:10:30.233 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:30.110+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:30.234 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:30.110+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:30.234 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:30.204+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:30.234 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:30.204+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:30.234 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:30.207+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:30.234 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: osdmap e595: 8 total, 8 up, 8 in 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:30.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.939+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.951+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:29.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.016+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.055+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.081+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.236 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.113+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:30.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.113+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:30.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.124+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.138+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.163+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.237 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.234+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:30.113+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:30.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:30.113+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:30.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:30.111+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:30.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:30.106+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:30.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:30.107+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:30.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:30.107+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:30.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:30.108+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:30.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:30.113+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: osdmap e595: 8 total, 8 up, 8 in 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: osdmap e595: 8 total, 8 up, 8 in 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.247+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.261+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.286+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.309+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.321+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.345+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.383+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.407+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.445+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.459+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.486 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.484+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.742 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.539+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.632+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.701+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.730+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:30.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.757+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.813+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.826+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.852+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.909+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.922+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.936+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.976+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.004 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:30.989+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.114 INFO:tasks.rados.rados.0.smithi118.stdout:2607: done (3 left) 2024-03-20T08:10:31.114 INFO:tasks.rados.rados.0.smithi118.stdout:2608: done (2 left) 2024-03-20T08:10:31.114 INFO:tasks.rados.rados.0.smithi118.stdout:2609: done (1 left) 2024-03-20T08:10:31.114 INFO:tasks.rados.rados.0.smithi118.stdout:2611: done (0 left) 2024-03-20T08:10:31.114 INFO:tasks.rados.rados.0.smithi118.stdout:2612: copy_from oid 8 from oid 3 current snap is 276 2024-03-20T08:10:31.114 INFO:tasks.rados.rados.0.smithi118.stdout:2613: setattr oid 13 current snap is 276 2024-03-20T08:10:31.115 INFO:tasks.rados.rados.0.smithi118.stdout:2614: delete oid 23 current snap is 276 2024-03-20T08:10:31.118 INFO:tasks.rados.rados.0.smithi118.stdout:2614: done (2 left) 2024-03-20T08:10:31.118 INFO:tasks.rados.rados.0.smithi118.stdout:2615: copy_from oid 2 from oid 24 current snap is 276 2024-03-20T08:10:31.118 INFO:tasks.rados.rados.0.smithi118.stdout:2616: snap_create 2024-03-20T08:10:31.125 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 34052 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:10:31.130 INFO:tasks.rados.rados.0.smithi118.stdout:2615: finishing copy_from racing read to smithi11881747-2 2024-03-20T08:10:31.133 INFO:tasks.rados.rados.0.smithi118.stdout:2615: finishing copy_from to smithi11881747-2 2024-03-20T08:10:31.133 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45576 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:10:31.165 INFO:tasks.rados.rados.0.smithi118.stdout:2612: finishing copy_from racing read to smithi11881747-8 2024-03-20T08:10:31.165 INFO:tasks.rados.rados.0.smithi118.stdout:2612: finishing copy_from to smithi11881747-8 2024-03-20T08:10:31.165 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34006 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:10:31.257 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:31.257 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:31.257 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:31.257 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:31.257 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:31.257 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: osdmap e596: 8 total, 8 up, 8 in 2024-03-20T08:10:31.257 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: pgmap v574: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 879 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 6.5 MiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:31.258 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.018+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.059+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.120+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.259 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.158+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.203+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:31.117+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:31.260 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:31.118+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:31.261 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:31.125+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:31.261 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:31.126+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:31.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:31.121+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:31.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:31.121+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:31.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:31.123+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:31.111+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:31.112+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:31.113+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:31.113+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:31.114+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:31.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: osdmap e596: 8 total, 8 up, 8 in 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:31.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: pgmap v574: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 879 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 6.5 MiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: osdmap e596: 8 total, 8 up, 8 in 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:31.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: pgmap v574: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 879 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 6.5 MiB/s rd, 4.2 MiB/s wr, 12 op/s 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:31.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:31.426 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:31.119+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:31.427 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:31.119+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:31.427 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:31.119+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:31.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.287+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.358+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.414+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.428+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.442+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.470+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.511 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.538+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.557+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.585+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.599+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.708+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.721+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.750+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:31.765 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.764+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.779+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.834+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.874+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.905+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.016 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:31.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.120 INFO:tasks.rados.rados.0.smithi118.stdout:2612: done (3 left) 2024-03-20T08:10:32.121 INFO:tasks.rados.rados.0.smithi118.stdout:2613: done (2 left) 2024-03-20T08:10:32.121 INFO:tasks.rados.rados.0.smithi118.stdout:2615: done (1 left) 2024-03-20T08:10:32.121 INFO:tasks.rados.rados.0.smithi118.stdout:2616: done (0 left) 2024-03-20T08:10:32.121 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 277 2024-03-20T08:10:32.121 INFO:tasks.rados.rados.0.smithi118.stdout:2617: seq_num 801 ranges {9756672=622592,10379264=573440,10952704=229376} 2024-03-20T08:10:32.178 INFO:tasks.rados.rados.0.smithi118.stdout:2617: writing smithi11881747-12 from 9756672 to 10379264 tid 1 2024-03-20T08:10:32.181 INFO:tasks.rados.rados.0.smithi118.stdout:2617: writing smithi11881747-12 from 10379264 to 10952704 tid 2 2024-03-20T08:10:32.183 INFO:tasks.rados.rados.0.smithi118.stdout:2617: writing smithi11881747-12 from 10952704 to 11182080 tid 3 2024-03-20T08:10:32.183 INFO:tasks.rados.rados.0.smithi118.stdout:2618: read oid 33 snap -1 2024-03-20T08:10:32.183 INFO:tasks.rados.rados.0.smithi118.stdout:2618: expect (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:10:32.183 INFO:tasks.rados.rados.0.smithi118.stdout:2619: rmattr oid 31 current snap is 277 2024-03-20T08:10:32.183 INFO:tasks.rados.rados.0.smithi118.stdout:2619: done (2 left) 2024-03-20T08:10:32.183 INFO:tasks.rados.rados.0.smithi118.stdout:2620: rollback oid 27 current snap is 277 2024-03-20T08:10:32.184 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 250 2024-03-20T08:10:32.184 INFO:tasks.rados.rados.0.smithi118.stdout:2621: delete oid 21 current snap is 277 2024-03-20T08:10:32.188 INFO:tasks.rados.rados.0.smithi118.stdout:2620: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:10:32.189 INFO:tasks.rados.rados.0.smithi118.stdout:2620: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:10:32.192 INFO:tasks.rados.rados.0.smithi118.stdout:2620: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:10:32.192 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 34041 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:10:32.207 INFO:tasks.rados.rados.0.smithi118.stdout:2620: done (3 left) 2024-03-20T08:10:32.208 INFO:tasks.rados.rados.0.smithi118.stdout:2621: done (2 left) 2024-03-20T08:10:32.208 INFO:tasks.rados.rados.0.smithi118.stdout:2622: snap_remove snap 273 2024-03-20T08:10:32.276 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:32.125+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:32.276 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:32.125+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: osdmap e597: 8 total, 8 up, 8 in 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:32.277 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:32.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:32.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:32.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:32.278 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:32.278 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:32.187+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:32.278 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:32.187+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:32.278 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:32.188+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.069+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.083+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.111+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.125+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.125+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.145+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.206+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.221+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.236+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.250+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.280 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.276+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:32.121+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:32.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:32.120+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:32.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:32.121+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:32.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:32.188+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:32.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:32.181+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:32.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:32.182+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:32.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:32.183+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:32.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:32.189+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: osdmap e597: 8 total, 8 up, 8 in 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:32.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: osdmap e597: 8 total, 8 up, 8 in 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:32.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:32.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:32.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:32.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:32.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:32.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:32.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:32.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:32 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:32.526 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:10:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:10:32.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.334+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.362+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.375+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.527 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.431+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.444+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.472+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.528 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.783 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.554+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.583+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.627+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.644+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.691+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.720+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.734+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.758+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.784 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:32.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.783+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.036 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.832+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.858+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.871+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.885+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.897+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.925+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.037 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.965+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.038 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.038 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:32.995+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.038 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.038 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.024+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.038 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.036+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:2622: done (2 left) 2024-03-20T08:10:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:2623: read oid 23 snap 276 2024-03-20T08:10:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:2623: expect deleted 2024-03-20T08:10:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 277 2024-03-20T08:10:33.131 INFO:tasks.rados.rados.0.smithi118.stdout:2624: seq_num 803 ranges {7536640=655360,8192000=745472,8937472=57344} 2024-03-20T08:10:33.178 INFO:tasks.rados.rados.0.smithi118.stdout:2624: writing smithi11881747-34 from 7536640 to 8192000 tid 1 2024-03-20T08:10:33.183 INFO:tasks.rados.rados.0.smithi118.stdout:2624: writing smithi11881747-34 from 8192000 to 8937472 tid 2 2024-03-20T08:10:33.184 INFO:tasks.rados.rados.0.smithi118.stdout:2624: writing smithi11881747-34 from 8937472 to 8994816 tid 3 2024-03-20T08:10:33.184 INFO:tasks.rados.rados.0.smithi118.stdout:2625: delete oid 5 current snap is 277 2024-03-20T08:10:33.184 INFO:tasks.rados.rados.0.smithi118.stdout:2617: finishing write tid 1 to smithi11881747-12 2024-03-20T08:10:33.184 INFO:tasks.rados.rados.0.smithi118.stdout:2617: finishing write tid 2 to smithi11881747-12 2024-03-20T08:10:33.184 INFO:tasks.rados.rados.0.smithi118.stdout:2617: finishing write tid 3 to smithi11881747-12 2024-03-20T08:10:33.184 INFO:tasks.rados.rados.0.smithi118.stdout:2617: finishing write tid 4 to smithi11881747-12 2024-03-20T08:10:33.185 INFO:tasks.rados.rados.0.smithi118.stdout:2617: finishing write tid 5 to smithi11881747-12 2024-03-20T08:10:33.185 INFO:tasks.rados.rados.0.smithi118.stdout:2617: oid 12 updating version 0 to 34016 2024-03-20T08:10:33.185 INFO:tasks.rados.rados.0.smithi118.stdout:2617: oid 12 version 34016 is already newer than 34015 2024-03-20T08:10:33.185 INFO:tasks.rados.rados.0.smithi118.stdout:2617: oid 12 version 34016 is already newer than 34014 2024-03-20T08:10:33.185 INFO:tasks.rados.rados.0.smithi118.stdout:2617: oid 12 updating version 34016 to 34017 2024-03-20T08:10:33.185 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34017 (ObjNum 801 snap 277 seq_num 801) dirty exists 2024-03-20T08:10:33.185 INFO:tasks.rados.rados.0.smithi118.stdout:2617: left oid 12 (ObjNum 801 snap 277 seq_num 801) 2024-03-20T08:10:33.185 INFO:tasks.rados.rados.0.smithi118.stdout:2618: expect (ObjNum 691 snap 227 seq_num 691) 2024-03-20T08:10:33.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:33.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: osdmap e598: 8 total, 8 up, 8 in 2024-03-20T08:10:33.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:33.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:33.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-mon[87080]: pgmap v577: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 903 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 24 op/s 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.049+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.074+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.098+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.122+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.135+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:33.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.156+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.180+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.197+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.210+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.223+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.235+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.247+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.274+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:33.131+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:33.289 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:33.131+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:33.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:33.132+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:33.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:33.132+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:33.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:33.132+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:33.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:33.126+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:33.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:33.126+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:33.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:33.128+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:33.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:33.128+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:33.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:33.128+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:33.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:33.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: osdmap e598: 8 total, 8 up, 8 in 2024-03-20T08:10:33.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:33.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[93121]: pgmap v577: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 903 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 24 op/s 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: osdmap e598: 8 total, 8 up, 8 in 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:33.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-mon[95692]: pgmap v577: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 903 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 24 op/s 2024-03-20T08:10:33.426 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:33.190+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:33.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:33.128+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:33.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:33.128+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:33.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.286+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.313+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.326+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.338+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.362+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.389+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.438+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.463+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.543 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.544 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.539+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.603+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.628+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.654+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.798 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.694+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.710+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.737+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.799 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:33.844 INFO:tasks.rados.rados.0.smithi118.stdout:2617: done (4 left) 2024-03-20T08:10:33.844 INFO:tasks.rados.rados.0.smithi118.stdout:2618: done (3 left) 2024-03-20T08:10:33.845 INFO:tasks.rados.rados.0.smithi118.stdout:2623: done (2 left) 2024-03-20T08:10:33.845 INFO:tasks.rados.rados.0.smithi118.stdout:2625: done (1 left) 2024-03-20T08:10:33.845 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 277 2024-03-20T08:10:33.845 INFO:tasks.rados.rados.0.smithi118.stdout:2624: finishing write tid 1 to smithi11881747-34 2024-03-20T08:10:33.845 INFO:tasks.rados.rados.0.smithi118.stdout:2624: finishing write tid 2 to smithi11881747-34 2024-03-20T08:10:33.845 INFO:tasks.rados.rados.0.smithi118.stdout:2626: seq_num 805 ranges {9756672=688128,10444800=507904,10952704=294912} 2024-03-20T08:10:33.903 INFO:tasks.rados.rados.0.smithi118.stdout:2626: writing smithi11881747-3 from 9756672 to 10444800 tid 1 2024-03-20T08:10:33.906 INFO:tasks.rados.rados.0.smithi118.stdout:2626: writing smithi11881747-3 from 10444800 to 10952704 tid 2 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:2626: writing smithi11881747-3 from 10952704 to 11247616 tid 3 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 277 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:2624: finishing write tid 3 to smithi11881747-34 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:2624: finishing write tid 4 to smithi11881747-34 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:2624: finishing write tid 5 to smithi11881747-34 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:2624: oid 34 updating version 0 to 33935 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:2624: oid 34 updating version 33935 to 33937 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:2624: oid 34 updating version 33937 to 33938 2024-03-20T08:10:33.909 INFO:tasks.rados.rados.0.smithi118.stdout:2624: oid 34 version 33938 is already newer than 33936 2024-03-20T08:10:33.910 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 33938 (ObjNum 803 snap 277 seq_num 803) dirty exists 2024-03-20T08:10:33.910 INFO:tasks.rados.rados.0.smithi118.stdout:2624: left oid 34 (ObjNum 803 snap 277 seq_num 803) 2024-03-20T08:10:33.910 INFO:tasks.rados.rados.0.smithi118.stdout:2627: seq_num 806 ranges {0=704512,704512=442368,1146880=360448} 2024-03-20T08:10:33.913 INFO:tasks.rados.rados.0.smithi118.stdout:2627: writing smithi11881747-23 from 0 to 704512 tid 1 2024-03-20T08:10:33.916 INFO:tasks.rados.rados.0.smithi118.stdout:2627: writing smithi11881747-23 from 704512 to 1146880 tid 2 2024-03-20T08:10:33.919 INFO:tasks.rados.rados.0.smithi118.stdout:2627: writing smithi11881747-23 from 1146880 to 1507328 tid 3 2024-03-20T08:10:33.919 INFO:tasks.rados.rados.0.smithi118.stdout:2624: done (2 left) 2024-03-20T08:10:33.919 INFO:tasks.rados.rados.0.smithi118.stdout:2628: snap_create 2024-03-20T08:10:33.919 INFO:tasks.rados.rados.0.smithi118.stdout:2627: finishing write tid 1 to smithi11881747-23 2024-03-20T08:10:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:10:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:10:33] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:10:33.923 INFO:tasks.rados.rados.0.smithi118.stdout:2627: finishing write tid 2 to smithi11881747-23 2024-03-20T08:10:33.923 INFO:tasks.rados.rados.0.smithi118.stdout:2627: finishing write tid 3 to smithi11881747-23 2024-03-20T08:10:33.924 INFO:tasks.rados.rados.0.smithi118.stdout:2627: finishing write tid 4 to smithi11881747-23 2024-03-20T08:10:33.925 INFO:tasks.rados.rados.0.smithi118.stdout:2626: finishing write tid 1 to smithi11881747-3 2024-03-20T08:10:33.925 INFO:tasks.rados.rados.0.smithi118.stdout:2626: finishing write tid 2 to smithi11881747-3 2024-03-20T08:10:33.925 INFO:tasks.rados.rados.0.smithi118.stdout:2626: finishing write tid 3 to smithi11881747-3 2024-03-20T08:10:33.925 INFO:tasks.rados.rados.0.smithi118.stdout:2626: finishing write tid 4 to smithi11881747-3 2024-03-20T08:10:33.926 INFO:tasks.rados.rados.0.smithi118.stdout:2627: finishing write tid 5 to smithi11881747-23 2024-03-20T08:10:33.926 INFO:tasks.rados.rados.0.smithi118.stdout:2627: oid 23 updating version 0 to 33967 2024-03-20T08:10:33.926 INFO:tasks.rados.rados.0.smithi118.stdout:2627: oid 23 updating version 33967 to 33969 2024-03-20T08:10:33.926 INFO:tasks.rados.rados.0.smithi118.stdout:2627: oid 23 version 33969 is already newer than 33966 2024-03-20T08:10:33.926 INFO:tasks.rados.rados.0.smithi118.stdout:2627: oid 23 version 33969 is already newer than 33968 2024-03-20T08:10:33.926 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 33969 (ObjNum 806 snap 277 seq_num 806) dirty exists 2024-03-20T08:10:33.926 INFO:tasks.rados.rados.0.smithi118.stdout:2627: left oid 23 (ObjNum 806 snap 277 seq_num 806) 2024-03-20T08:10:33.929 INFO:tasks.rados.rados.0.smithi118.stdout:2626: finishing write tid 5 to smithi11881747-3 2024-03-20T08:10:33.929 INFO:tasks.rados.rados.0.smithi118.stdout:2626: oid 3 updating version 0 to 34007 2024-03-20T08:10:33.929 INFO:tasks.rados.rados.0.smithi118.stdout:2626: oid 3 updating version 34007 to 34008 2024-03-20T08:10:33.929 INFO:tasks.rados.rados.0.smithi118.stdout:2626: oid 3 updating version 34008 to 34009 2024-03-20T08:10:33.929 INFO:tasks.rados.rados.0.smithi118.stdout:2626: oid 3 updating version 34009 to 34010 2024-03-20T08:10:33.930 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34010 (ObjNum 805 snap 277 seq_num 805) dirty exists 2024-03-20T08:10:33.930 INFO:tasks.rados.rados.0.smithi118.stdout:2626: left oid 3 (ObjNum 805 snap 277 seq_num 805) 2024-03-20T08:10:34.051 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.811+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.822+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.835+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.847+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.872+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.927+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.952+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:33.989+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.052 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.015+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.053 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.133 INFO:tasks.rados.rados.0.smithi118.stdout:2626: done (2 left) 2024-03-20T08:10:34.133 INFO:tasks.rados.rados.0.smithi118.stdout:2627: done (1 left) 2024-03-20T08:10:34.133 INFO:tasks.rados.rados.0.smithi118.stdout:2628: done (0 left) 2024-03-20T08:10:34.133 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 278 2024-03-20T08:10:34.133 INFO:tasks.rados.rados.0.smithi118.stdout:2629: seq_num 807 ranges {0=720896,720896=729088,1449984=73728} 2024-03-20T08:10:34.137 INFO:tasks.rados.rados.0.smithi118.stdout:2629: writing smithi11881747-15 from 0 to 720896 tid 1 2024-03-20T08:10:34.141 INFO:tasks.rados.rados.0.smithi118.stdout:2629: writing smithi11881747-15 from 720896 to 1449984 tid 2 2024-03-20T08:10:34.142 INFO:tasks.rados.rados.0.smithi118.stdout:2629: writing smithi11881747-15 from 1449984 to 1523712 tid 3 2024-03-20T08:10:34.142 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 278 2024-03-20T08:10:34.142 INFO:tasks.rados.rados.0.smithi118.stdout:2630: seq_num 808 ranges {0=737280,737280=614400,1351680=188416} 2024-03-20T08:10:34.147 INFO:tasks.rados.rados.0.smithi118.stdout:2630: writing smithi11881747-21 from 0 to 737280 tid 1 2024-03-20T08:10:34.151 INFO:tasks.rados.rados.0.smithi118.stdout:2630: writing smithi11881747-21 from 737280 to 1351680 tid 2 2024-03-20T08:10:34.152 INFO:tasks.rados.rados.0.smithi118.stdout:2630: writing smithi11881747-21 from 1351680 to 1540096 tid 3 2024-03-20T08:10:34.152 INFO:tasks.rados.rados.0.smithi118.stdout:2631: setattr oid 46 current snap is 278 2024-03-20T08:10:34.154 INFO:tasks.rados.rados.0.smithi118.stdout:2632: copy_from oid 41 from oid 40 current snap is 278 2024-03-20T08:10:34.154 INFO:tasks.rados.rados.0.smithi118.stdout:2633: snap_create 2024-03-20T08:10:34.155 INFO:tasks.rados.rados.0.smithi118.stdout:2632: finishing copy_from to smithi11881747-41 2024-03-20T08:10:34.155 INFO:tasks.rados.rados.0.smithi118.stdout:2632: got expected ENOENT (src dne) 2024-03-20T08:10:34.155 INFO:tasks.rados.rados.0.smithi118.stdout:2632: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:10:34.155 INFO:tasks.rados.rados.0.smithi118.stdout:2629: finishing write tid 1 to smithi11881747-15 2024-03-20T08:10:34.156 INFO:tasks.rados.rados.0.smithi118.stdout:2629: finishing write tid 2 to smithi11881747-15 2024-03-20T08:10:34.156 INFO:tasks.rados.rados.0.smithi118.stdout:2629: finishing write tid 3 to smithi11881747-15 2024-03-20T08:10:34.156 INFO:tasks.rados.rados.0.smithi118.stdout:2629: finishing write tid 4 to smithi11881747-15 2024-03-20T08:10:34.163 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 45577 (ObjNum 797 snap 275 seq_num 797) dirty exists 2024-03-20T08:10:34.164 INFO:tasks.rados.rados.0.smithi118.stdout:2630: finishing write tid 1 to smithi11881747-21 2024-03-20T08:10:34.164 INFO:tasks.rados.rados.0.smithi118.stdout:2629: finishing write tid 5 to smithi11881747-15 2024-03-20T08:10:34.164 INFO:tasks.rados.rados.0.smithi118.stdout:2629: oid 15 updating version 0 to 7082 2024-03-20T08:10:34.164 INFO:tasks.rados.rados.0.smithi118.stdout:2629: oid 15 version 7082 is already newer than 7081 2024-03-20T08:10:34.164 INFO:tasks.rados.rados.0.smithi118.stdout:2629: oid 15 version 7082 is already newer than 7079 2024-03-20T08:10:34.165 INFO:tasks.rados.rados.0.smithi118.stdout:2629: oid 15 version 7082 is already newer than 7080 2024-03-20T08:10:34.165 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 7082 (ObjNum 807 snap 278 seq_num 807) dirty exists 2024-03-20T08:10:34.165 INFO:tasks.rados.rados.0.smithi118.stdout:2629: left oid 15 (ObjNum 807 snap 278 seq_num 807) 2024-03-20T08:10:34.168 INFO:tasks.rados.rados.0.smithi118.stdout:2630: finishing write tid 2 to smithi11881747-21 2024-03-20T08:10:34.176 INFO:tasks.rados.rados.0.smithi118.stdout:2630: finishing write tid 3 to smithi11881747-21 2024-03-20T08:10:34.176 INFO:tasks.rados.rados.0.smithi118.stdout:2630: finishing write tid 4 to smithi11881747-21 2024-03-20T08:10:34.180 INFO:tasks.rados.rados.0.smithi118.stdout:2630: finishing write tid 5 to smithi11881747-21 2024-03-20T08:10:34.180 INFO:tasks.rados.rados.0.smithi118.stdout:2630: oid 21 updating version 0 to 39719 2024-03-20T08:10:34.180 INFO:tasks.rados.rados.0.smithi118.stdout:2630: oid 21 version 39719 is already newer than 39717 2024-03-20T08:10:34.180 INFO:tasks.rados.rados.0.smithi118.stdout:2630: oid 21 updating version 39719 to 39720 2024-03-20T08:10:34.180 INFO:tasks.rados.rados.0.smithi118.stdout:2630: oid 21 version 39720 is already newer than 39718 2024-03-20T08:10:34.180 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 39720 (ObjNum 808 snap 278 seq_num 808) dirty exists 2024-03-20T08:10:34.180 INFO:tasks.rados.rados.0.smithi118.stdout:2630: left oid 21 (ObjNum 808 snap 278 seq_num 808) 2024-03-20T08:10:34.305 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:34.160+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:34.306 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:34.162+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:34.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:34.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:34.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:34.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: osdmap e599: 8 total, 8 up, 8 in 2024-03-20T08:10:34.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:34.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:34.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:34.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:34.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:34.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:34.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:34.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:34.307 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:34.135+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:34.307 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:34.136+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.052+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.137+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.137+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.141+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.308 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.213+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.226+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.239+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.267+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.292+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:34.136+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:34.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:34.136+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:34.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:34.130+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:34.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:34.132+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:34.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:34.132+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:34.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:34.133+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:34.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:34.133+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:34.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: osdmap e599: 8 total, 8 up, 8 in 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:34.425 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:34.145+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:34.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:34.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:34.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:34.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: osdmap e599: 8 total, 8 up, 8 in 2024-03-20T08:10:34.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:34.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:34.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:34.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:34.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:34.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:34.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:34.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:34.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.331+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.345+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.566 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.395+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.452+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.466+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.479+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.491+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.503+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.528+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.552+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.607+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.621+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.662+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.690+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.705+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.720+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.733+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.760+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.789+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:34.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.847+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.874+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.888+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.903+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.916+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.957+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.970+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.982+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:34.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.024+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.039+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:2629: done (4 left) 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:2630: done (3 left) 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:2631: done (2 left) 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:2632: done (1 left) 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:2633: done (0 left) 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:2634: rollback oid 6 current snap is 279 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 272 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 279 2024-03-20T08:10:35.138 INFO:tasks.rados.rados.0.smithi118.stdout:2635: seq_num 809 ranges {1253376=753664,2007040=499712,2506752=303104} 2024-03-20T08:10:35.149 INFO:tasks.rados.rados.0.smithi118.stdout:2635: writing smithi11881747-9 from 1253376 to 2007040 tid 1 2024-03-20T08:10:35.152 INFO:tasks.rados.rados.0.smithi118.stdout:2635: writing smithi11881747-9 from 2007040 to 2506752 tid 2 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2635: writing smithi11881747-9 from 2506752 to 2809856 tid 3 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2636: copy_from oid 19 from oid 31 current snap is 279 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2634: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2634: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2634: finishing rollback tid 2 to smithi11881747-6 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 45580 (ObjNum 709 snap 239 seq_num 709) dirty exists 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2634: done (2 left) 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2637: rollback oid 34 current snap is 279 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 34 to 225 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2638: delete oid 7 current snap is 279 2024-03-20T08:10:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:2635: finishing write tid 1 to smithi11881747-9 2024-03-20T08:10:35.157 INFO:tasks.rados.rados.0.smithi118.stdout:2636: finishing copy_from to smithi11881747-19 2024-03-20T08:10:35.157 INFO:tasks.rados.rados.0.smithi118.stdout:2636: got expected ENOENT (src dne) 2024-03-20T08:10:35.158 INFO:tasks.rados.rados.0.smithi118.stdout:2636: finishing copy_from racing read to smithi11881747-19 2024-03-20T08:10:35.158 INFO:tasks.rados.rados.0.smithi118.stdout:2635: finishing write tid 2 to smithi11881747-9 2024-03-20T08:10:35.160 INFO:tasks.rados.rados.0.smithi118.stdout:2635: finishing write tid 3 to smithi11881747-9 2024-03-20T08:10:35.163 INFO:tasks.rados.rados.0.smithi118.stdout:2635: finishing write tid 4 to smithi11881747-9 2024-03-20T08:10:35.165 INFO:tasks.rados.rados.0.smithi118.stdout:2635: finishing write tid 5 to smithi11881747-9 2024-03-20T08:10:35.165 INFO:tasks.rados.rados.0.smithi118.stdout:2635: oid 9 updating version 0 to 34056 2024-03-20T08:10:35.165 INFO:tasks.rados.rados.0.smithi118.stdout:2635: oid 9 version 34056 is already newer than 34055 2024-03-20T08:10:35.165 INFO:tasks.rados.rados.0.smithi118.stdout:2635: oid 9 version 34056 is already newer than 34054 2024-03-20T08:10:35.165 INFO:tasks.rados.rados.0.smithi118.stdout:2635: oid 9 updating version 34056 to 34057 2024-03-20T08:10:35.165 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 34057 (ObjNum 809 snap 279 seq_num 809) dirty exists 2024-03-20T08:10:35.165 INFO:tasks.rados.rados.0.smithi118.stdout:2635: left oid 9 (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2635: done (3 left) 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2636: done (2 left) 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2638: done (1 left) 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2639: read oid 22 snap -1 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2639: expect (ObjNum 734 snap 250 seq_num 734) 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2640: read oid 6 snap -1 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2640: expect (ObjNum 709 snap 239 seq_num 709) 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2641: delete oid 11 current snap is 279 2024-03-20T08:10:35.166 INFO:tasks.rados.rados.0.smithi118.stdout:2637: finishing rollback tid 0 to smithi11881747-34 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:2641: done (3 left) 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:2642: read oid 31 snap -1 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:2642: expect deleted 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:2643: read oid 28 snap -1 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:2643: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:2644: rollback oid 3 current snap is 279 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 3 to 272 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:2645: read oid 23 snap -1 2024-03-20T08:10:35.170 INFO:tasks.rados.rados.0.smithi118.stdout:2645: expect (ObjNum 806 snap 277 seq_num 806) 2024-03-20T08:10:35.171 INFO:tasks.rados.rados.0.smithi118.stdout:2646: read oid 49 snap -1 2024-03-20T08:10:35.171 INFO:tasks.rados.rados.0.smithi118.stdout:2646: expect deleted 2024-03-20T08:10:35.171 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 279 2024-03-20T08:10:35.171 INFO:tasks.rados.rados.0.smithi118.stdout:2647: seq_num 812 ranges {0=802816,802816=548864,1351680=253952} 2024-03-20T08:10:35.175 INFO:tasks.rados.rados.0.smithi118.stdout:2647: writing smithi11881747-2 from 0 to 802816 tid 1 2024-03-20T08:10:35.178 INFO:tasks.rados.rados.0.smithi118.stdout:2647: writing smithi11881747-2 from 802816 to 1351680 tid 2 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:2647: writing smithi11881747-2 from 1351680 to 1605632 tid 3 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:2648: copy_from oid 37 from oid 19 current snap is 279 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:2637: finishing rollback tid 1 to smithi11881747-34 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:2637: finishing rollback tid 2 to smithi11881747-34 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 33941 (ObjNum 644 snap 211 seq_num 644) dirty exists 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:2637: done (9 left) 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:2642: done (8 left) 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:2646: done (7 left) 2024-03-20T08:10:35.181 INFO:tasks.rados.rados.0.smithi118.stdout:2649: snap_remove snap 262 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: osdmap e600: 8 total, 8 up, 8 in 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:35.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:35.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:35.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:35.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:35.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-mon[87080]: pgmap v580: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 906 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 18 op/s 2024-03-20T08:10:35.333 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:35.142+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:35.333 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:35.142+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:35.333 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:35.145+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.127+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.143+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.147+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.165+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.186+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.221+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.235+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.249+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.304+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.332+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.335 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:35.144+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:35.335 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:35.144+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:35.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:35.138+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:35.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:35.138+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:35.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:35.138+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:35.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:35.144+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:35.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:35.138+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:35.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:35.138+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:35.159+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:35.160+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:35.164+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:35.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:35.164+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:35.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:35.164+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: osdmap e600: 8 total, 8 up, 8 in 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[93121]: pgmap v580: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 906 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 18 op/s 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: osdmap e600: 8 total, 8 up, 8 in 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:35.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:35.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:35.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:35.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:35.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:35.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:35.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:35 smithi118 ceph-mon[95692]: pgmap v580: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 906 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 18 op/s 2024-03-20T08:10:35.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.587 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.360+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.417+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.445+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.460+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.488+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.501+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.557+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.588 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.572+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.839 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.587+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.617+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.632+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.645+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.658+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.672+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.684+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.696+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.764+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.789+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.800+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.840 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.826+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:35.841 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.852+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.890+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.901+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.927+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.939+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:35.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.015+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.039+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.062+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.074+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.087+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.100+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.142 INFO:tasks.rados.rados.0.smithi118.stdout:2649: done (7 left) 2024-03-20T08:10:36.142 INFO:tasks.rados.rados.0.smithi118.stdout:2650: snap_remove snap 264 2024-03-20T08:10:36.359 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:36.146+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:36.359 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:36.146+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: osdmap e601: 8 total, 8 up, 8 in 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:36.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:36.361 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:36.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.113+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.141+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.144+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:36.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.144+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:36.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.169+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.181+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.193+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.247+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.259+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.284+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.309+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.321+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.334+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.345+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.359+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:36.143+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:36.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:36.143+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:36.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:36.143+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: osdmap e601: 8 total, 8 up, 8 in 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: osdmap e601: 8 total, 8 up, 8 in 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:36.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.450+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.489+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.526+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.955 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.576+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.662+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.675+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.741+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.754+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.767+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.791+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.956 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.957 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:36.957 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.830+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.143 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:36.908+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:37.143 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:36.732+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:37.144 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:36.732+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:37.144 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:36.732+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:37.144 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:36.733+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:37.150 INFO:tasks.rados.rados.0.smithi118.stdout:2650: done (7 left) 2024-03-20T08:10:37.150 INFO:tasks.rados.rados.0.smithi118.stdout:2651: rollback oid 14 current snap is 279 2024-03-20T08:10:37.150 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 226 2024-03-20T08:10:37.150 INFO:tasks.rados.rados.0.smithi118.stdout:2639: expect (ObjNum 734 snap 250 seq_num 734) 2024-03-20T08:10:37.160 INFO:tasks.rados.rados.0.smithi118.stdout:2643: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: osdmap e602: 8 total, 8 up, 8 in 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: pgmap v583: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 915 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 6.5 MiB/s rd, 2.1 MiB/s wr, 21 op/s 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:37.289 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:37.290 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:37.150+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:37.290 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:37.150+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:37.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.843+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.855+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.868+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.965+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:36.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.170+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:37.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.176+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.190+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.341 INFO:tasks.rados.rados.0.smithi118.stdout:2644: finishing rollback tid 0 to smithi11881747-3 2024-03-20T08:10:37.342 INFO:tasks.rados.rados.0.smithi118.stdout:2644: finishing rollback tid 1 to smithi11881747-3 2024-03-20T08:10:37.342 INFO:tasks.rados.rados.0.smithi118.stdout:2639: done (7 left) 2024-03-20T08:10:37.342 INFO:tasks.rados.rados.0.smithi118.stdout:2643: done (6 left) 2024-03-20T08:10:37.342 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 279 2024-03-20T08:10:37.342 INFO:tasks.rados.rados.0.smithi118.stdout:2645: expect (ObjNum 806 snap 277 seq_num 806) 2024-03-20T08:10:37.388 INFO:tasks.rados.rados.0.smithi118.stdout:2648: finishing copy_from to smithi11881747-37 2024-03-20T08:10:37.388 INFO:tasks.rados.rados.0.smithi118.stdout:2648: got expected ENOENT (src dne) 2024-03-20T08:10:37.388 INFO:tasks.rados.rados.0.smithi118.stdout:2648: finishing copy_from racing read to smithi11881747-37 2024-03-20T08:10:37.388 INFO:tasks.rados.rados.0.smithi118.stdout:2644: finishing rollback tid 2 to smithi11881747-3 2024-03-20T08:10:37.388 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34013 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:10:37.388 INFO:tasks.rados.rados.0.smithi118.stdout:2652: seq_num 813 ranges {1523712=417792,1941504=434176,2375680=442368,2818048=327680} 2024-03-20T08:10:37.398 INFO:tasks.rados.rados.0.smithi118.stdout:2652: writing smithi11881747-15 from 1523712 to 1941504 tid 1 2024-03-20T08:10:37.401 INFO:tasks.rados.rados.0.smithi118.stdout:2652: writing smithi11881747-15 from 1941504 to 2375680 tid 2 2024-03-20T08:10:37.404 INFO:tasks.rados.rados.0.smithi118.stdout:2652: writing smithi11881747-15 from 2375680 to 2818048 tid 3 2024-03-20T08:10:37.407 INFO:tasks.rados.rados.0.smithi118.stdout:2652: writing smithi11881747-15 from 2818048 to 3145728 tid 4 2024-03-20T08:10:37.407 INFO:tasks.rados.rados.0.smithi118.stdout:2644: done (6 left) 2024-03-20T08:10:37.407 INFO:tasks.rados.rados.0.smithi118.stdout:2645: done (5 left) 2024-03-20T08:10:37.407 INFO:tasks.rados.rados.0.smithi118.stdout:2648: done (4 left) 2024-03-20T08:10:37.408 INFO:tasks.rados.rados.0.smithi118.stdout:2653: rollback oid 24 current snap is 279 2024-03-20T08:10:37.408 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 216 2024-03-20T08:10:37.408 INFO:tasks.rados.rados.0.smithi118.stdout:2640: expect (ObjNum 709 snap 239 seq_num 709) 2024-03-20T08:10:37.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:37.149+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:37.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:37.149+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:37.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:37.150+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:37.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:37.216+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:37.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:37.217+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:37.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:37.217+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:37.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:37.217+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:37.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:37.217+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: osdmap e602: 8 total, 8 up, 8 in 2024-03-20T08:10:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:37.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:37.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: pgmap v583: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 915 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 6.5 MiB/s rd, 2.1 MiB/s wr, 21 op/s 2024-03-20T08:10:37.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:37.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:37.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:37.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:37.424 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:37.147+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:37.424 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:37.147+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: osdmap e602: 8 total, 8 up, 8 in 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: pgmap v583: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 915 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 6.5 MiB/s rd, 2.1 MiB/s wr, 21 op/s 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:37.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:37.425 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:37.147+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:37.540 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:37.288+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:37.540 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:37.289+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:37.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.540 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.405+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.454+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.479+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.503+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.516+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.541 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.542 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.794 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.794 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.567+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.794 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.591+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.604+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.616+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.642+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.655+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.718+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.729+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.795 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.767+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.796 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.794+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:37.832 INFO:tasks.rados.rados.0.smithi118.stdout:2647: finishing write tid 1 to smithi11881747-2 2024-03-20T08:10:37.832 INFO:tasks.rados.rados.0.smithi118.stdout:2647: finishing write tid 2 to smithi11881747-2 2024-03-20T08:10:37.832 INFO:tasks.rados.rados.0.smithi118.stdout:2640: done (4 left) 2024-03-20T08:10:37.833 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 279 2024-03-20T08:10:37.833 INFO:tasks.rados.rados.0.smithi118.stdout:2647: finishing write tid 3 to smithi11881747-2 2024-03-20T08:10:37.833 INFO:tasks.rados.rados.0.smithi118.stdout:2654: seq_num 814 ranges {1507328=442368,1949696=770048,2719744=434176} 2024-03-20T08:10:37.843 INFO:tasks.rados.rados.0.smithi118.stdout:2654: writing smithi11881747-23 from 1507328 to 1949696 tid 1 2024-03-20T08:10:37.848 INFO:tasks.rados.rados.0.smithi118.stdout:2654: writing smithi11881747-23 from 1949696 to 2719744 tid 2 2024-03-20T08:10:37.852 INFO:tasks.rados.rados.0.smithi118.stdout:2654: writing smithi11881747-23 from 2719744 to 3153920 tid 3 2024-03-20T08:10:37.852 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 279 2024-03-20T08:10:37.852 INFO:tasks.rados.rados.0.smithi118.stdout:2647: finishing write tid 4 to smithi11881747-2 2024-03-20T08:10:37.852 INFO:tasks.rados.rados.0.smithi118.stdout:2647: finishing write tid 5 to smithi11881747-2 2024-03-20T08:10:37.852 INFO:tasks.rados.rados.0.smithi118.stdout:2647: oid 2 updating version 0 to 45582 2024-03-20T08:10:37.852 INFO:tasks.rados.rados.0.smithi118.stdout:2647: oid 2 version 45582 is already newer than 45581 2024-03-20T08:10:37.852 INFO:tasks.rados.rados.0.smithi118.stdout:2647: oid 2 updating version 45582 to 45583 2024-03-20T08:10:37.853 INFO:tasks.rados.rados.0.smithi118.stdout:2647: oid 2 updating version 45583 to 45584 2024-03-20T08:10:37.853 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45584 (ObjNum 812 snap 279 seq_num 812) dirty exists 2024-03-20T08:10:37.853 INFO:tasks.rados.rados.0.smithi118.stdout:2647: left oid 2 (ObjNum 812 snap 279 seq_num 812) 2024-03-20T08:10:37.853 INFO:tasks.rados.rados.0.smithi118.stdout:2651: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:10:37.853 INFO:tasks.rados.rados.0.smithi118.stdout:2655: seq_num 815 ranges {0=458752,458752=655360,1114112=409600,1523712=139264} 2024-03-20T08:10:37.854 INFO:tasks.rados.rados.0.smithi118.stdout:2655: writing smithi11881747-19 from 0 to 458752 tid 1 2024-03-20T08:10:37.859 INFO:tasks.rados.rados.0.smithi118.stdout:2655: writing smithi11881747-19 from 458752 to 1114112 tid 2 2024-03-20T08:10:37.861 INFO:tasks.rados.rados.0.smithi118.stdout:2655: writing smithi11881747-19 from 1114112 to 1523712 tid 3 2024-03-20T08:10:37.863 INFO:tasks.rados.rados.0.smithi118.stdout:2655: writing smithi11881747-19 from 1523712 to 1662976 tid 4 2024-03-20T08:10:37.863 INFO:tasks.rados.rados.0.smithi118.stdout:2647: done (5 left) 2024-03-20T08:10:37.863 INFO:tasks.rados.rados.0.smithi118.stdout:2656: read oid 31 snap -1 2024-03-20T08:10:37.863 INFO:tasks.rados.rados.0.smithi118.stdout:2656: expect deleted 2024-03-20T08:10:37.863 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 279 2024-03-20T08:10:37.863 INFO:tasks.rados.rados.0.smithi118.stdout:2651: finishing rollback tid 2 to smithi11881747-14 2024-03-20T08:10:37.864 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 34020 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:10:37.864 INFO:tasks.rados.rados.0.smithi118.stdout:2652: finishing write tid 1 to smithi11881747-15 2024-03-20T08:10:37.864 INFO:tasks.rados.rados.0.smithi118.stdout:2657: seq_num 816 ranges {0=475136,475136=540672,1015808=614400,1630208=49152} 2024-03-20T08:10:37.866 INFO:tasks.rados.rados.0.smithi118.stdout:2657: writing smithi11881747-36 from 0 to 475136 tid 1 2024-03-20T08:10:37.870 INFO:tasks.rados.rados.0.smithi118.stdout:2657: writing smithi11881747-36 from 475136 to 1015808 tid 2 2024-03-20T08:10:37.874 INFO:tasks.rados.rados.0.smithi118.stdout:2657: writing smithi11881747-36 from 1015808 to 1630208 tid 3 2024-03-20T08:10:37.876 INFO:tasks.rados.rados.0.smithi118.stdout:2657: writing smithi11881747-36 from 1630208 to 1679360 tid 4 2024-03-20T08:10:37.876 INFO:tasks.rados.rados.0.smithi118.stdout:2651: done (6 left) 2024-03-20T08:10:37.876 INFO:tasks.rados.rados.0.smithi118.stdout:2658: copy_from oid 13 from oid 42 current snap is 279 2024-03-20T08:10:37.876 INFO:tasks.rados.rados.0.smithi118.stdout:2652: finishing write tid 2 to smithi11881747-15 2024-03-20T08:10:37.876 INFO:tasks.rados.rados.0.smithi118.stdout:2652: finishing write tid 3 to smithi11881747-15 2024-03-20T08:10:37.876 INFO:tasks.rados.rados.0.smithi118.stdout:2652: finishing write tid 4 to smithi11881747-15 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2652: finishing write tid 5 to smithi11881747-15 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2653: finishing rollback tid 0 to smithi11881747-24 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2652: finishing write tid 6 to smithi11881747-15 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2652: oid 15 updating version 0 to 7083 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2652: oid 15 updating version 7083 to 7085 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2652: oid 15 updating version 7085 to 7087 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2652: oid 15 version 7087 is already newer than 7086 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2652: oid 15 version 7087 is already newer than 7084 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 7087 (ObjNum 813 snap 279 seq_num 813) dirty exists 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2652: left oid 15 (ObjNum 813 snap 279 seq_num 813) 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2653: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2653: finishing rollback tid 2 to smithi11881747-24 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 48688 (ObjNum 617 snap 202 seq_num 617) dirty exists 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2654: finishing write tid 1 to smithi11881747-23 2024-03-20T08:10:37.877 INFO:tasks.rados.rados.0.smithi118.stdout:2654: finishing write tid 2 to smithi11881747-23 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: finishing write tid 3 to smithi11881747-23 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: finishing write tid 4 to smithi11881747-23 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: finishing write tid 5 to smithi11881747-23 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: oid 23 updating version 0 to 33970 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: oid 23 updating version 33970 to 33973 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: oid 23 version 33973 is already newer than 33972 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: oid 23 version 33973 is already newer than 33971 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 33973 (ObjNum 814 snap 279 seq_num 814) dirty exists 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: left oid 23 (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2655: finishing write tid 1 to smithi11881747-19 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2655: finishing write tid 2 to smithi11881747-19 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2652: done (6 left) 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2653: done (5 left) 2024-03-20T08:10:37.878 INFO:tasks.rados.rados.0.smithi118.stdout:2654: done (4 left) 2024-03-20T08:10:37.879 INFO:tasks.rados.rados.0.smithi118.stdout:2659: read oid 28 snap -1 2024-03-20T08:10:37.879 INFO:tasks.rados.rados.0.smithi118.stdout:2659: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:10:37.879 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 279 2024-03-20T08:10:37.879 INFO:tasks.rados.rados.0.smithi118.stdout:2660: seq_num 817 ranges {3604480=491520,4096000=417792,4513792=425984,4939776=360448} 2024-03-20T08:10:37.899 INFO:tasks.rados.rados.0.smithi118.stdout:2660: writing smithi11881747-44 from 3604480 to 4096000 tid 1 2024-03-20T08:10:37.902 INFO:tasks.rados.rados.0.smithi118.stdout:2660: writing smithi11881747-44 from 4096000 to 4513792 tid 2 2024-03-20T08:10:37.905 INFO:tasks.rados.rados.0.smithi118.stdout:2660: writing smithi11881747-44 from 4513792 to 4939776 tid 3 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2660: writing smithi11881747-44 from 4939776 to 5300224 tid 4 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2661: rollback oid 41 current snap is 279 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 41 to 255 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2655: finishing write tid 3 to smithi11881747-19 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2655: finishing write tid 4 to smithi11881747-19 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2657: finishing write tid 1 to smithi11881747-36 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2655: finishing write tid 5 to smithi11881747-19 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2657: finishing write tid 2 to smithi11881747-36 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2657: finishing write tid 3 to smithi11881747-36 2024-03-20T08:10:37.908 INFO:tasks.rados.rados.0.smithi118.stdout:2659: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.819+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.831+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.844+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.887+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.945+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.957+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.970+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.045 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.983+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:37.996+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.103 INFO:tasks.rados.rados.0.smithi118.stdout:2658: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:10:38.103 INFO:tasks.rados.rados.0.smithi118.stdout:2657: finishing write tid 4 to smithi11881747-36 2024-03-20T08:10:38.103 INFO:tasks.rados.rados.0.smithi118.stdout:2657: finishing write tid 5 to smithi11881747-36 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2656: done (6 left) 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2659: done (5 left) 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2662: snap_create 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2655: finishing write tid 6 to smithi11881747-19 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2655: oid 19 updating version 0 to 7091 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2655: oid 19 version 7091 is already newer than 7090 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2655: oid 19 updating version 7091 to 7092 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2655: oid 19 version 7092 is already newer than 7089 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2655: oid 19 version 7092 is already newer than 7088 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 7092 (ObjNum 815 snap 279 seq_num 815) dirty exists 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2655: left oid 19 (ObjNum 815 snap 279 seq_num 815) 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2658: finishing copy_from to smithi11881747-13 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 34047 (ObjNum 796 snap 275 seq_num 796) dirty exists 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2657: finishing write tid 6 to smithi11881747-36 2024-03-20T08:10:38.104 INFO:tasks.rados.rados.0.smithi118.stdout:2657: oid 36 updating version 0 to 33945 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2657: oid 36 version 33945 is already newer than 33942 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2657: oid 36 version 33945 is already newer than 33944 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2657: oid 36 version 33945 is already newer than 33943 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2657: oid 36 updating version 33945 to 33946 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 33946 (ObjNum 816 snap 279 seq_num 816) dirty exists 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2657: left oid 36 (ObjNum 816 snap 279 seq_num 816) 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2661: finishing rollback tid 0 to smithi11881747-41 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2661: finishing rollback tid 1 to smithi11881747-41 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 33975 (ObjNum 65 snap 0 seq_num 1366321772) dirty dne 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2660: finishing write tid 1 to smithi11881747-44 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2660: finishing write tid 2 to smithi11881747-44 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2660: finishing write tid 3 to smithi11881747-44 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2660: finishing write tid 4 to smithi11881747-44 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2660: finishing write tid 5 to smithi11881747-44 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2660: finishing write tid 6 to smithi11881747-44 2024-03-20T08:10:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:2660: oid 44 updating version 0 to 48836 2024-03-20T08:10:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:2660: oid 44 updating version 48836 to 48840 2024-03-20T08:10:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:2660: oid 44 version 48840 is already newer than 48838 2024-03-20T08:10:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:2660: oid 44 version 48840 is already newer than 48839 2024-03-20T08:10:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:2660: oid 44 version 48840 is already newer than 48837 2024-03-20T08:10:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 48840 (ObjNum 817 snap 279 seq_num 817) dirty exists 2024-03-20T08:10:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:2660: left oid 44 (ObjNum 817 snap 279 seq_num 817) 2024-03-20T08:10:38.153 INFO:tasks.rados.rados.0.smithi118.stdout:2655: done (5 left) 2024-03-20T08:10:38.153 INFO:tasks.rados.rados.0.smithi118.stdout:2657: done (4 left) 2024-03-20T08:10:38.153 INFO:tasks.rados.rados.0.smithi118.stdout:2658: done (3 left) 2024-03-20T08:10:38.153 INFO:tasks.rados.rados.0.smithi118.stdout:2660: done (2 left) 2024-03-20T08:10:38.153 INFO:tasks.rados.rados.0.smithi118.stdout:2661: done (1 left) 2024-03-20T08:10:38.153 INFO:tasks.rados.rados.0.smithi118.stdout:2662: done (0 left) 2024-03-20T08:10:38.153 INFO:tasks.rados.rados.0.smithi118.stdout:2663: snap_create 2024-03-20T08:10:38.307 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:38.156+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:38.307 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:38.156+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:38.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:38.307 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: osdmap e603: 8 total, 8 up, 8 in 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:38.308 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.070+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.083+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.096+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.121+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.159+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.159+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:38.309 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.164+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.190+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.241+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.253+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.266+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.293+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.310 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:38.156+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:38.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:38.156+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:38.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:38.354+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:38.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:38.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:38.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:38.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:38.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: osdmap e603: 8 total, 8 up, 8 in 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:38.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: osdmap e603: 8 total, 8 up, 8 in 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:38.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:38.154+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:38.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.321+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.372+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.567 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.422+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.435+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.447+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.459+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.472+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.500+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.568 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.827 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:38.639+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:38.827 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:38.639+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:38.827 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:38.639+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.581+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.595+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.649+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.738+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.763+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.789+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.802+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.814+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.829 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.828+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:38.896 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:38.638+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:38.897 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:38.639+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:38.897 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:38.639+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:38.897 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:38.639+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:39.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.866+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.905+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.917+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.929+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.978+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:38.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.086+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.162 INFO:tasks.rados.rados.0.smithi118.stdout:2663: done (0 left) 2024-03-20T08:10:39.162 INFO:tasks.rados.rados.0.smithi118.stdout:2664: read oid 34 snap -1 2024-03-20T08:10:39.162 INFO:tasks.rados.rados.0.smithi118.stdout:2664: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:10:39.162 INFO:tasks.rados.rados.0.smithi118.stdout:2665: read oid 16 snap -1 2024-03-20T08:10:39.162 INFO:tasks.rados.rados.0.smithi118.stdout:2665: expect deleted 2024-03-20T08:10:39.162 INFO:tasks.rados.rados.0.smithi118.stdout:2666: read oid 48 snap -1 2024-03-20T08:10:39.162 INFO:tasks.rados.rados.0.smithi118.stdout:2666: expect deleted 2024-03-20T08:10:39.162 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 281 2024-03-20T08:10:39.163 INFO:tasks.rados.rados.0.smithi118.stdout:2667: seq_num 818 ranges {0=507904,507904=704512,1212416=499712} 2024-03-20T08:10:39.168 INFO:tasks.rados.rados.0.smithi118.stdout:2667: writing smithi11881747-38 from 0 to 507904 tid 1 2024-03-20T08:10:39.171 INFO:tasks.rados.rados.0.smithi118.stdout:2667: writing smithi11881747-38 from 507904 to 1212416 tid 2 2024-03-20T08:10:39.175 INFO:tasks.rados.rados.0.smithi118.stdout:2667: writing smithi11881747-38 from 1212416 to 1712128 tid 3 2024-03-20T08:10:39.175 INFO:tasks.rados.rados.0.smithi118.stdout:2668: read oid 35 snap -1 2024-03-20T08:10:39.175 INFO:tasks.rados.rados.0.smithi118.stdout:2668: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:10:39.175 INFO:tasks.rados.rados.0.smithi118.stdout:2669: rollback oid 50 current snap is 281 2024-03-20T08:10:39.175 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 50 to 216 2024-03-20T08:10:39.175 INFO:tasks.rados.rados.0.smithi118.stdout:2670: snap_remove snap 259 2024-03-20T08:10:39.342 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:39.180+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:39.342 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:39.183+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:39.342 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:39.162+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:39.342 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:39.162+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: osdmap e604: 8 total, 8 up, 8 in 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: pgmap v586: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 920 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 26 op/s 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]: dispatch 2024-03-20T08:10:39.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]': finished 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.098+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.110+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.122+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.147+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.172+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.173+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.207+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.244+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.268+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.294+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.306+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.330+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.346 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]: dispatch 2024-03-20T08:10:39.346 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]': finished 2024-03-20T08:10:39.346 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:39.347 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:39.348 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:39.348 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:39.348 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:39.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:39.161+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:39.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:39.164+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:39.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:39.168+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:39.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:39.168+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:39.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:39.159+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:39.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:39.159+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:39.166+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:39.168+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:39.168+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:39.168+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:39.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:39.168+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: osdmap e604: 8 total, 8 up, 8 in 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: pgmap v586: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 920 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 26 op/s 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]: dispatch 2024-03-20T08:10:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]': finished 2024-03-20T08:10:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: osdmap e604: 8 total, 8 up, 8 in 2024-03-20T08:10:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: pgmap v586: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 920 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 26 op/s 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]: dispatch 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.a"}]': finished 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]: dispatch 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]': finished 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]: dispatch 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.b"}]': finished 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.429 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:39.430 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:39.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.356+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.368+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.597 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.391+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.404+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.430+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.443+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.456+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.468+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.494+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.520+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.532+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.556+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.598 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.599 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.582+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.599 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.597+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.635+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.648+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.673+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.698+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.710+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.724+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.853 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.737+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.761+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.774+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.823+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.836+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:39.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.852+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.104 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.874+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.104 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.917+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.955+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.978+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:39.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.003+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.016+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.042+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.105 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.106 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.078+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.106 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.091+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.106 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.104+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.159 INFO:tasks.rados.rados.0.smithi118.stdout:2670: done (6 left) 2024-03-20T08:10:40.160 INFO:tasks.rados.rados.0.smithi118.stdout:2671: snap_remove snap 280 2024-03-20T08:10:40.359 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:40.164+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:40.359 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:40.164+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:40.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all mds 2024-03-20T08:10:40.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all rgw 2024-03-20T08:10:40.359 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all rbd-mirror 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all ceph-exporter 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all iscsi 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all nfs 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: Upgrade: Setting container_image for all nvmeof 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: osdmap e605: 8 total, 8 up, 8 in 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:40.360 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:40.361 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:40.361 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:40.361 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:40.361 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:40.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.130+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.155+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.166+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:40.361 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.166+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.174+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.210+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.222+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.234+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.247+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.259+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.297+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.311+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.322+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.362 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.363 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.363 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:40.164+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:40.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:40.164+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:40.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:40.161+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:40.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:40.158+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:40.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:40.158+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:40.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:40.158+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:40.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:40.158+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:40.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all mds 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all rgw 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all rbd-mirror 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all ceph-exporter 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all iscsi 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all nfs 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: Upgrade: Setting container_image for all nvmeof 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: osdmap e605: 8 total, 8 up, 8 in 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:40.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all mds 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all rgw 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all rbd-mirror 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all ceph-exporter 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all iscsi 2024-03-20T08:10:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all nfs 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: Upgrade: Setting container_image for all nvmeof 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: osdmap e605: 8 total, 8 up, 8 in 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:40.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:40.616 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.398+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.510+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.522+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.534+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.617 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.546+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.558+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.570+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.618 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.867 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:40.616+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:40.867 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:40.616+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:40.867 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:40.616+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.631+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.643+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.656+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.669+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.868 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.719+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.731+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.758+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.771+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.820+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.844+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.869 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.868+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:40.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:40.614+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.916+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.929+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.954+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.966+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:40.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.130 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.016+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.029+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.042+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.055+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.092+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.131 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.161 INFO:tasks.rados.rados.0.smithi118.stdout:2671: done (6 left) 2024-03-20T08:10:41.161 INFO:tasks.rados.rados.0.smithi118.stdout:2672: rmattr oid 18 current snap is 281 2024-03-20T08:10:41.162 INFO:tasks.rados.rados.0.smithi118.stdout:2672: done (6 left) 2024-03-20T08:10:41.162 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 281 2024-03-20T08:10:41.162 INFO:tasks.rados.rados.0.smithi118.stdout:2673: seq_num 819 ranges {1294336=524288,1818624=589824,2408448=614400} 2024-03-20T08:10:41.172 INFO:tasks.rados.rados.0.smithi118.stdout:2673: writing smithi11881747-30 from 1294336 to 1818624 tid 1 2024-03-20T08:10:41.176 INFO:tasks.rados.rados.0.smithi118.stdout:2673: writing smithi11881747-30 from 1818624 to 2408448 tid 2 2024-03-20T08:10:41.180 INFO:tasks.rados.rados.0.smithi118.stdout:2673: writing smithi11881747-30 from 2408448 to 3022848 tid 3 2024-03-20T08:10:41.181 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 281 2024-03-20T08:10:41.181 INFO:tasks.rados.rados.0.smithi118.stdout:2674: seq_num 820 ranges {5300224=540672,5840896=475136,6316032=598016,6914048=131072} 2024-03-20T08:10:41.214 INFO:tasks.rados.rados.0.smithi118.stdout:2674: writing smithi11881747-44 from 5300224 to 5840896 tid 1 2024-03-20T08:10:41.217 INFO:tasks.rados.rados.0.smithi118.stdout:2674: writing smithi11881747-44 from 5840896 to 6316032 tid 2 2024-03-20T08:10:41.221 INFO:tasks.rados.rados.0.smithi118.stdout:2674: writing smithi11881747-44 from 6316032 to 6914048 tid 3 2024-03-20T08:10:41.222 INFO:tasks.rados.rados.0.smithi118.stdout:2674: writing smithi11881747-44 from 6914048 to 7045120 tid 4 2024-03-20T08:10:41.222 INFO:tasks.rados.rados.0.smithi118.stdout:2675: read oid 4 snap -1 2024-03-20T08:10:41.222 INFO:tasks.rados.rados.0.smithi118.stdout:2675: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:10:41.222 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 281 2024-03-20T08:10:41.223 INFO:tasks.rados.rados.0.smithi118.stdout:2676: seq_num 821 ranges {0=557056,557056=761856,1318912=442368} 2024-03-20T08:10:41.226 INFO:tasks.rados.rados.0.smithi118.stdout:2676: writing smithi11881747-39 from 0 to 557056 tid 1 2024-03-20T08:10:41.230 INFO:tasks.rados.rados.0.smithi118.stdout:2676: writing smithi11881747-39 from 557056 to 1318912 tid 2 2024-03-20T08:10:41.234 INFO:tasks.rados.rados.0.smithi118.stdout:2676: writing smithi11881747-39 from 1318912 to 1761280 tid 3 2024-03-20T08:10:41.234 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 281 2024-03-20T08:10:41.234 INFO:tasks.rados.rados.0.smithi118.stdout:2677: seq_num 822 ranges {655360=573440,1228800=696320,1925120=507904} 2024-03-20T08:10:41.241 INFO:tasks.rados.rados.0.smithi118.stdout:2677: writing smithi11881747-14 from 655360 to 1228800 tid 1 2024-03-20T08:10:41.245 INFO:tasks.rados.rados.0.smithi118.stdout:2677: writing smithi11881747-14 from 1228800 to 1925120 tid 2 2024-03-20T08:10:41.249 INFO:tasks.rados.rados.0.smithi118.stdout:2677: writing smithi11881747-14 from 1925120 to 2433024 tid 3 2024-03-20T08:10:41.249 INFO:tasks.rados.rados.0.smithi118.stdout:2678: read oid 9 snap -1 2024-03-20T08:10:41.249 INFO:tasks.rados.rados.0.smithi118.stdout:2678: expect (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:10:41.249 INFO:tasks.rados.rados.0.smithi118.stdout:2679: rmattr oid 13 current snap is 281 2024-03-20T08:10:41.249 INFO:tasks.rados.rados.0.smithi118.stdout:2679: done (12 left) 2024-03-20T08:10:41.250 INFO:tasks.rados.rados.0.smithi118.stdout:2680: rollback oid 20 current snap is 281 2024-03-20T08:10:41.250 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 20 to 249 2024-03-20T08:10:41.250 INFO:tasks.rados.rados.0.smithi118.stdout:2664: expect (ObjNum 644 snap 211 seq_num 644) 2024-03-20T08:10:41.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:41.178+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:41.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:41.179+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:41.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:41.163+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:41.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:41.163+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: osdmap e606: 8 total, 8 up, 8 in 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: pgmap v589: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 933 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-03-20T08:10:41.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:41.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:41.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.165+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:41.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.190+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.203+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.216+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.247+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.278+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.302+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.354+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.366+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.378+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:41.164+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:41.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:41.164+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:41.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:41.164+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:41.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:41.161+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:41.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:41.159+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:41.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:41.159+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:41.316+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:41.316+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:41.316+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:41.316+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:41.317+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: osdmap e606: 8 total, 8 up, 8 in 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: pgmap v589: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 933 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: osdmap e606: 8 total, 8 up, 8 in 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: pgmap v589: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 933 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 9.1 MiB/s rd, 2.7 MiB/s wr, 17 op/s 2024-03-20T08:10:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:41.449 INFO:tasks.rados.rados.0.smithi118.stdout:2668: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2667: finishing write tid 1 to smithi11881747-38 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2667: finishing write tid 2 to smithi11881747-38 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2669: finishing rollback tid 0 to smithi11881747-50 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2667: finishing write tid 3 to smithi11881747-38 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2667: finishing write tid 4 to smithi11881747-38 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2664: done (12 left) 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2665: done (11 left) 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2666: done (10 left) 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:2668: done (9 left) 2024-03-20T08:10:41.475 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 281 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2669: finishing rollback tid 1 to smithi11881747-50 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 49225 (ObjNum 97 snap 0 seq_num 1) dirty dne 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2667: finishing write tid 5 to smithi11881747-38 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2667: oid 38 updating version 0 to 41517 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2667: oid 38 updating version 41517 to 41519 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2667: oid 38 updating version 41519 to 41520 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2667: oid 38 version 41520 is already newer than 41518 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 41520 (ObjNum 818 snap 281 seq_num 818) dirty exists 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2667: left oid 38 (ObjNum 818 snap 281 seq_num 818) 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2673: finishing write tid 1 to smithi11881747-30 2024-03-20T08:10:41.476 INFO:tasks.rados.rados.0.smithi118.stdout:2681: seq_num 823 ranges {1286144=589824,1875968=573440,2449408=630784} 2024-03-20T08:10:41.485 INFO:tasks.rados.rados.0.smithi118.stdout:2681: writing smithi11881747-17 from 1286144 to 1875968 tid 1 2024-03-20T08:10:41.489 INFO:tasks.rados.rados.0.smithi118.stdout:2681: writing smithi11881747-17 from 1875968 to 2449408 tid 2 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2681: writing smithi11881747-17 from 2449408 to 3080192 tid 3 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2667: done (9 left) 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2669: done (8 left) 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2682: read oid 1 snap -1 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2682: expect (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 281 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2673: finishing write tid 2 to smithi11881747-30 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2673: finishing write tid 3 to smithi11881747-30 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2673: finishing write tid 4 to smithi11881747-30 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2673: finishing write tid 5 to smithi11881747-30 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2673: oid 30 updating version 0 to 33930 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2673: oid 30 updating version 33930 to 33931 2024-03-20T08:10:41.494 INFO:tasks.rados.rados.0.smithi118.stdout:2673: oid 30 updating version 33931 to 33932 2024-03-20T08:10:41.495 INFO:tasks.rados.rados.0.smithi118.stdout:2673: oid 30 version 33932 is already newer than 33929 2024-03-20T08:10:41.495 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 33932 (ObjNum 819 snap 281 seq_num 819) dirty exists 2024-03-20T08:10:41.495 INFO:tasks.rados.rados.0.smithi118.stdout:2673: left oid 30 (ObjNum 819 snap 281 seq_num 819) 2024-03-20T08:10:41.495 INFO:tasks.rados.rados.0.smithi118.stdout:2675: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:10:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2676: finishing write tid 1 to smithi11881747-39 2024-03-20T08:10:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2676: finishing write tid 2 to smithi11881747-39 2024-03-20T08:10:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2674: finishing write tid 1 to smithi11881747-44 2024-03-20T08:10:41.504 INFO:tasks.rados.rados.0.smithi118.stdout:2683: seq_num 824 ranges {0=606208,606208=458752,1064960=573440,1638400=172032} 2024-03-20T08:10:41.510 INFO:tasks.rados.rados.0.smithi118.stdout:2683: writing smithi11881747-26 from 0 to 606208 tid 1 2024-03-20T08:10:41.510 INFO:tasks.rados.rados.0.smithi118.stdout:2683: writing smithi11881747-26 from 606208 to 1064960 tid 2 2024-03-20T08:10:41.514 INFO:tasks.rados.rados.0.smithi118.stdout:2683: writing smithi11881747-26 from 1064960 to 1638400 tid 3 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2683: writing smithi11881747-26 from 1638400 to 1810432 tid 4 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2673: done (9 left) 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2675: done (8 left) 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2684: copy_from oid 36 from oid 46 current snap is 281 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2674: finishing write tid 2 to smithi11881747-44 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2676: finishing write tid 3 to smithi11881747-39 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2676: finishing write tid 4 to smithi11881747-39 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2674: finishing write tid 3 to smithi11881747-44 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2676: finishing write tid 5 to smithi11881747-39 2024-03-20T08:10:41.517 INFO:tasks.rados.rados.0.smithi118.stdout:2676: oid 39 updating version 0 to 34043 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2676: oid 39 updating version 34043 to 34044 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2676: oid 39 version 34044 is already newer than 34042 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2676: oid 39 updating version 34044 to 34045 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 34045 (ObjNum 821 snap 281 seq_num 821) dirty exists 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2676: left oid 39 (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: finishing write tid 4 to smithi11881747-44 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: finishing write tid 5 to smithi11881747-44 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: finishing write tid 6 to smithi11881747-44 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: oid 44 updating version 0 to 49868 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: oid 44 updating version 49868 to 49870 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: oid 44 version 49870 is already newer than 49867 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: oid 44 updating version 49870 to 49871 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: oid 44 version 49871 is already newer than 49869 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 49871 (ObjNum 820 snap 281 seq_num 820) dirty exists 2024-03-20T08:10:41.518 INFO:tasks.rados.rados.0.smithi118.stdout:2674: left oid 44 (ObjNum 820 snap 281 seq_num 820) 2024-03-20T08:10:41.519 INFO:tasks.rados.rados.0.smithi118.stdout:2680: finishing rollback tid 1 to smithi11881747-20 2024-03-20T08:10:41.519 INFO:tasks.rados.rados.0.smithi118.stdout:2680: finishing rollback tid 2 to smithi11881747-20 2024-03-20T08:10:41.519 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 34047 (ObjNum 447 snap 148 seq_num 447) dirty exists 2024-03-20T08:10:41.519 INFO:tasks.rados.rados.0.smithi118.stdout:2677: finishing write tid 1 to smithi11881747-14 2024-03-20T08:10:41.519 INFO:tasks.rados.rados.0.smithi118.stdout:2677: finishing write tid 2 to smithi11881747-14 2024-03-20T08:10:41.519 INFO:tasks.rados.rados.0.smithi118.stdout:2678: expect (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:10:41.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.397+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.439+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.464+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.478+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.560+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.572+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.625+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.637+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.651+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: finishing write tid 3 to smithi11881747-14 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: finishing write tid 4 to smithi11881747-14 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: finishing write tid 5 to smithi11881747-14 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: oid 14 updating version 0 to 34022 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: oid 14 updating version 34022 to 34023 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: oid 14 updating version 34023 to 34024 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: oid 14 updating version 34024 to 34025 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 34025 (ObjNum 822 snap 281 seq_num 822) dirty exists 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: left oid 14 (ObjNum 822 snap 281 seq_num 822) 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2674: done (8 left) 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2676: done (7 left) 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2677: done (6 left) 2024-03-20T08:10:41.655 INFO:tasks.rados.rados.0.smithi118.stdout:2678: done (5 left) 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2680: done (4 left) 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2685: snap_create 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: finishing write tid 1 to smithi11881747-17 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: finishing write tid 2 to smithi11881747-17 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: finishing write tid 3 to smithi11881747-17 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: finishing write tid 4 to smithi11881747-17 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: finishing write tid 5 to smithi11881747-17 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: oid 17 updating version 0 to 7064 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: oid 17 version 7064 is already newer than 7063 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: oid 17 version 7064 is already newer than 7062 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: oid 17 updating version 7064 to 7065 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7065 (ObjNum 823 snap 281 seq_num 823) dirty exists 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2681: left oid 17 (ObjNum 823 snap 281 seq_num 823) 2024-03-20T08:10:41.656 INFO:tasks.rados.rados.0.smithi118.stdout:2682: expect (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:10:41.757 INFO:tasks.rados.rados.0.smithi118.stdout:2683: finishing write tid 1 to smithi11881747-26 2024-03-20T08:10:41.757 INFO:tasks.rados.rados.0.smithi118.stdout:2683: finishing write tid 2 to smithi11881747-26 2024-03-20T08:10:41.757 INFO:tasks.rados.rados.0.smithi118.stdout:2683: finishing write tid 3 to smithi11881747-26 2024-03-20T08:10:41.757 INFO:tasks.rados.rados.0.smithi118.stdout:2683: finishing write tid 4 to smithi11881747-26 2024-03-20T08:10:41.757 INFO:tasks.rados.rados.0.smithi118.stdout:2683: finishing write tid 5 to smithi11881747-26 2024-03-20T08:10:41.757 INFO:tasks.rados.rados.0.smithi118.stdout:2683: finishing write tid 6 to smithi11881747-26 2024-03-20T08:10:41.757 INFO:tasks.rados.rados.0.smithi118.stdout:2683: oid 26 updating version 0 to 49954 2024-03-20T08:10:41.757 INFO:tasks.rados.rados.0.smithi118.stdout:2683: oid 26 version 49954 is already newer than 49952 2024-03-20T08:10:41.758 INFO:tasks.rados.rados.0.smithi118.stdout:2683: oid 26 version 49954 is already newer than 49953 2024-03-20T08:10:41.758 INFO:tasks.rados.rados.0.smithi118.stdout:2683: oid 26 updating version 49954 to 49956 2024-03-20T08:10:41.758 INFO:tasks.rados.rados.0.smithi118.stdout:2683: oid 26 version 49956 is already newer than 49955 2024-03-20T08:10:41.758 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 49956 (ObjNum 824 snap 281 seq_num 824) dirty exists 2024-03-20T08:10:41.758 INFO:tasks.rados.rados.0.smithi118.stdout:2683: left oid 26 (ObjNum 824 snap 281 seq_num 824) 2024-03-20T08:10:41.758 INFO:tasks.rados.rados.0.smithi118.stdout:2684: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:10:41.758 INFO:tasks.rados.rados.0.smithi118.stdout:2684: finishing copy_from to smithi11881747-36 2024-03-20T08:10:41.758 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45577 (ObjNum 797 snap 275 seq_num 797) dirty exists 2024-03-20T08:10:41.906 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.906 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.678+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.691+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.717+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.729+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.794+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.833+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.907 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.845+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.870+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.882+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.895+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.908 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.906+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:41.922 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:41 smithi118 systemd[1]: Stopping Ceph node-exporter.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:10:42.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.944+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.166 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.969+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.982+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:41.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.006+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.018+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.043+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.055+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.068+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.080+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.093+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.167 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.143+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.154+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.168 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: Upgrade: Updating node-exporter.a (1/2) 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: Deploying daemon node-exporter.a on smithi118 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: osdmap e607: 8 total, 8 up, 8 in 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:42.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: Upgrade: Updating node-exporter.a (1/2) 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: Deploying daemon node-exporter.a on smithi118 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: osdmap e607: 8 total, 8 up, 8 in 2024-03-20T08:10:42.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:42.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:42.181 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:41 smithi118 podman[125233]: 2024-03-20 08:10:41.9256264 +0000 UTC m=+0.126302019 container died e22b97acea5c3f0f6a0f4da6f5e91ad57a2f7d2169ff8d2dec8cbd55293c008b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:10:42.181 INFO:tasks.rados.rados.0.smithi118.stdout:2681: done (4 left) 2024-03-20T08:10:42.181 INFO:tasks.rados.rados.0.smithi118.stdout:2682: done (3 left) 2024-03-20T08:10:42.181 INFO:tasks.rados.rados.0.smithi118.stdout:2683: done (2 left) 2024-03-20T08:10:42.181 INFO:tasks.rados.rados.0.smithi118.stdout:2684: done (1 left) 2024-03-20T08:10:42.182 INFO:tasks.rados.rados.0.smithi118.stdout:2685: done (0 left) 2024-03-20T08:10:42.182 INFO:tasks.rados.rados.0.smithi118.stdout:2686: snap_create 2024-03-20T08:10:42.423 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:10:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:10:42.424 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:42.185+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:42.424 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:42.185+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:42.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:42.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: Upgrade: Updating node-exporter.a (1/2) 2024-03-20T08:10:42.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: Deploying daemon node-exporter.a on smithi118 2024-03-20T08:10:42.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:42.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: osdmap e607: 8 total, 8 up, 8 in 2024-03-20T08:10:42.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:42.424 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:42.425 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.184+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.184+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.209+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.234+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.247+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.259+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.296+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.308+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.334+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.345+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.384+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.398+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.410+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:42.180+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:42.579 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:42.180+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:42.579 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:42 smithi118 podman[125233]: 2024-03-20 08:10:42.354379169 +0000 UTC m=+0.555054804 container cleanup e22b97acea5c3f0f6a0f4da6f5e91ad57a2f7d2169ff8d2dec8cbd55293c008b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:10:42.579 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:42 smithi118 bash[125233]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a 2024-03-20T08:10:42.580 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:42.176+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:42.580 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:42.177+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:42.580 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:42.177+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:42.580 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:42.177+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:42.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.436+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.449+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.462+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.500+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.562+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.599+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.621+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.852 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:42 smithi118 podman[125251]: 2024-03-20 08:10:42.580903684 +0000 UTC m=+0.659280116 container remove e22b97acea5c3f0f6a0f4da6f5e91ad57a2f7d2169ff8d2dec8cbd55293c008b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:10:42.852 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:42 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:10:42.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.695+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.720+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.835+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.847+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.861+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.874+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.947 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.887+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.899+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.912+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:42.948 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.944+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.113 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:43.004+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:43.113 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:42.956+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:43.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:42 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-20T08:10:43.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:42 smithi118 systemd[1]: Stopped Ceph node-exporter.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:10:43.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:42 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.a.service: Consumed 3.209s CPU time. 2024-03-20T08:10:43.114 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 systemd[1]: Starting Ceph node-exporter.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:10:43.185 INFO:tasks.rados.rados.0.smithi118.stdout:2686: done (0 left) 2024-03-20T08:10:43.185 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 283 2024-03-20T08:10:43.185 INFO:tasks.rados.rados.0.smithi118.stdout:2687: seq_num 825 ranges {1343488=622592,1966080=745472,2711552=458752} 2024-03-20T08:10:43.195 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:43.192+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:43.195 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:43.192+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:43.195 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:43.004+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:43.196 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:43.008+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:43.196 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:43.008+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:43.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:43.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:43.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:43.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:43.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: osdmap e608: 8 total, 8 up, 8 in 2024-03-20T08:10:43.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:43.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:43.196 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:43.197 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:43.197 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:43.197 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-mon[87080]: pgmap v592: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 938 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 13 op/s 2024-03-20T08:10:43.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.957+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.969+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:42.994+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.007+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.021+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.059+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.109+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.160+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.198 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.195+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:43.199 INFO:tasks.rados.rados.0.smithi118.stdout:2687: writing smithi11881747-42 from 1343488 to 1966080 tid 1 2024-03-20T08:10:43.201 INFO:tasks.rados.rados.0.smithi118.stdout:2687: writing smithi11881747-42 from 1966080 to 2711552 tid 2 2024-03-20T08:10:43.205 INFO:tasks.rados.rados.0.smithi118.stdout:2687: writing smithi11881747-42 from 2711552 to 3170304 tid 3 2024-03-20T08:10:43.205 INFO:tasks.rados.rados.0.smithi118.stdout:2688: read oid 35 snap 278 2024-03-20T08:10:43.205 INFO:tasks.rados.rados.0.smithi118.stdout:2688: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:10:43.205 INFO:tasks.rados.rados.0.smithi118.stdout:2689: read oid 9 snap -1 2024-03-20T08:10:43.205 INFO:tasks.rados.rados.0.smithi118.stdout:2689: expect (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:10:43.205 INFO:tasks.rados.rados.0.smithi118.stdout:2690: snap_remove snap 252 2024-03-20T08:10:43.449 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:43.211+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:43.449 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:43.213+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.206+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.237+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.274+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.337+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.349+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.361+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.451 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.512 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:43.190+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:43.512 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:43.190+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:43.512 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:43.191+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:43.512 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:43.181+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:43.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:43.181+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:43.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:43.183+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:43.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:43.183+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:43.513 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:43.183+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:43.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:43.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:43.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:43.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:43.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: osdmap e608: 8 total, 8 up, 8 in 2024-03-20T08:10:43.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:43.513 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[93121]: pgmap v592: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 938 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 13 op/s 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: osdmap e608: 8 total, 8 up, 8 in 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:43.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:43.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:43.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:43.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:43.515 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-mon[95692]: pgmap v592: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 938 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 7.8 MiB/s rd, 2.7 MiB/s wr, 13 op/s 2024-03-20T08:10:43.515 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:43.187+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:43.515 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:43.187+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:43.515 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:43.202+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:43.516 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 podman[125380]: 2024-03-20 08:10:43.252524503 +0000 UTC m=+0.019580672 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-20T08:10:43.516 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 podman[125380]: 2024-03-20 08:10:43.382176964 +0000 UTC m=+0.149233130 container create 768c22c1ab593caa72aa9c1634de305d43d2a7c25e2dbfc3e8c2f1729a763392 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:10:43.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.526+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.539+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.589+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.627+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.708 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.651+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.666+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.679+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.709 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:10:43] "GET /metrics HTTP/1.1" 200 36413 "" "Prometheus/2.43.0" 2024-03-20T08:10:43.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.721+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.734+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.746+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.758+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.821+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.834+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.862+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.876+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.891+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.905+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.919+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.991 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.933+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.991 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.991 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:43.991 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 podman[125380]: 2024-03-20 08:10:43.660365131 +0000 UTC m=+0.427421321 container init 768c22c1ab593caa72aa9c1634de305d43d2a7c25e2dbfc3e8c2f1729a763392 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:10:43.991 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 podman[125380]: 2024-03-20 08:10:43.666217641 +0000 UTC m=+0.433273816 container start 768c22c1ab593caa72aa9c1634de305d43d2a7c25e2dbfc3e8c2f1729a763392 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:10:43.991 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.669Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-20T08:10:43.991 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.670Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-20T08:10:43.991 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.670Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.670Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.670Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.670Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=arp 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-20T08:10:43.992 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=edac 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-20T08:10:43.993 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=os 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=stat 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-20T08:10:43.994 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=time 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=uname 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.671Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.672Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a[125394]: ts=2024-03-20T08:10:43.672Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 bash[125380]: 768c22c1ab593caa72aa9c1634de305d43d2a7c25e2dbfc3e8c2f1729a763392 2024-03-20T08:10:43.995 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:10:43 smithi118 systemd[1]: Started Ceph node-exporter.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:10:44.191 INFO:tasks.rados.rados.0.smithi118.stdout:2690: done (3 left) 2024-03-20T08:10:44.192 INFO:tasks.rados.rados.0.smithi118.stdout:2691: copy_from oid 38 from oid 29 current snap is 283 2024-03-20T08:10:44.192 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 283 2024-03-20T08:10:44.192 INFO:tasks.rados.rados.0.smithi118.stdout:2692: seq_num 826 ranges {6569984=638976,7208960=630784,7839744=573440} 2024-03-20T08:10:44.219 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:44.196+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:44.219 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:44.196+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:44.219 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: osdmap e609: 8 total, 8 up, 8 in 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:44.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:44.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:44.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:44.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:44.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:44.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:44.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:44.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:44.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:44.221 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.979+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:43.995+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.008+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.022+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.118+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.159+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.194+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:44.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.194+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:44.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.203+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.231 INFO:tasks.rados.rados.0.smithi118.stdout:2692: writing smithi11881747-25 from 6569984 to 7208960 tid 1 2024-03-20T08:10:44.235 INFO:tasks.rados.rados.0.smithi118.stdout:2692: writing smithi11881747-25 from 7208960 to 7839744 tid 2 2024-03-20T08:10:44.240 INFO:tasks.rados.rados.0.smithi118.stdout:2692: writing smithi11881747-25 from 7839744 to 8413184 tid 3 2024-03-20T08:10:44.240 INFO:tasks.rados.rados.0.smithi118.stdout:2693: rollback oid 36 current snap is 283 2024-03-20T08:10:44.240 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 36 to 209 2024-03-20T08:10:44.240 INFO:tasks.rados.rados.0.smithi118.stdout:2687: finishing write tid 1 to smithi11881747-42 2024-03-20T08:10:44.240 INFO:tasks.rados.rados.0.smithi118.stdout:2687: finishing write tid 2 to smithi11881747-42 2024-03-20T08:10:44.240 INFO:tasks.rados.rados.0.smithi118.stdout:2687: finishing write tid 3 to smithi11881747-42 2024-03-20T08:10:44.240 INFO:tasks.rados.rados.0.smithi118.stdout:2687: finishing write tid 4 to smithi11881747-42 2024-03-20T08:10:44.240 INFO:tasks.rados.rados.0.smithi118.stdout:2688: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:10:44.269 INFO:tasks.rados.rados.0.smithi118.stdout:2687: finishing write tid 5 to smithi11881747-42 2024-03-20T08:10:44.269 INFO:tasks.rados.rados.0.smithi118.stdout:2687: oid 42 updating version 0 to 34062 2024-03-20T08:10:44.269 INFO:tasks.rados.rados.0.smithi118.stdout:2687: oid 42 version 34062 is already newer than 34060 2024-03-20T08:10:44.269 INFO:tasks.rados.rados.0.smithi118.stdout:2687: oid 42 version 34062 is already newer than 34059 2024-03-20T08:10:44.269 INFO:tasks.rados.rados.0.smithi118.stdout:2687: oid 42 version 34062 is already newer than 34061 2024-03-20T08:10:44.270 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 34062 (ObjNum 825 snap 283 seq_num 825) dirty exists 2024-03-20T08:10:44.270 INFO:tasks.rados.rados.0.smithi118.stdout:2687: left oid 42 (ObjNum 825 snap 283 seq_num 825) 2024-03-20T08:10:44.270 INFO:tasks.rados.rados.0.smithi118.stdout:2689: expect (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:10:44.410 INFO:tasks.rados.rados.0.smithi118.stdout:2692: finishing write tid 1 to smithi11881747-25 2024-03-20T08:10:44.410 INFO:tasks.rados.rados.0.smithi118.stdout:2692: finishing write tid 2 to smithi11881747-25 2024-03-20T08:10:44.410 INFO:tasks.rados.rados.0.smithi118.stdout:2687: done (5 left) 2024-03-20T08:10:44.410 INFO:tasks.rados.rados.0.smithi118.stdout:2688: done (4 left) 2024-03-20T08:10:44.410 INFO:tasks.rados.rados.0.smithi118.stdout:2689: done (3 left) 2024-03-20T08:10:44.410 INFO:tasks.rados.rados.0.smithi118.stdout:2694: snap_create 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2692: finishing write tid 3 to smithi11881747-25 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2691: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2692: finishing write tid 4 to smithi11881747-25 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2692: finishing write tid 5 to smithi11881747-25 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2692: oid 25 updating version 0 to 34063 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2692: oid 25 updating version 34063 to 34064 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2692: oid 25 updating version 34064 to 34066 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2692: oid 25 version 34066 is already newer than 34065 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 34066 (ObjNum 826 snap 283 seq_num 826) dirty exists 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2692: left oid 25 (ObjNum 826 snap 283 seq_num 826) 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2693: finishing rollback tid 0 to smithi11881747-36 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2693: finishing rollback tid 1 to smithi11881747-36 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2693: finishing rollback tid 2 to smithi11881747-36 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45580 (ObjNum 33 snap 0 seq_num 1268763186) dirty exists 2024-03-20T08:10:44.411 INFO:tasks.rados.rados.0.smithi118.stdout:2691: finishing copy_from to smithi11881747-38 2024-03-20T08:10:44.412 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 33958 (ObjNum 638 snap 206 seq_num 638) dirty exists 2024-03-20T08:10:44.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.235+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.267+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.294+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.308+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.323+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.337+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.350+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.363+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.377+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.392+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.452+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:44.189+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:44.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:44.191+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:44.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:44.235+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:44.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:44.205+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:44.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:44.205+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:44.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:44.205+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:44.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:44.206+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:44.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:44.190+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:44.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: osdmap e609: 8 total, 8 up, 8 in 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:44.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: osdmap e609: 8 total, 8 up, 8 in 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:44.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:44.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:44.735 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:44.618+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:44.735 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:44.618+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:44.735 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:44.619+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:44.735 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.735 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.516+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.530+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.543+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.581+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.593+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.618+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.632+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.643+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.655+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.682+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.694+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.736 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.721+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.737 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.751+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.780+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.794+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.807+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.819+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.831+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.844+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.871+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.883+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.895+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.908+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.966+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.980+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:44.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:44.995+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.196 INFO:tasks.rados.rados.0.smithi118.stdout:2691: done (3 left) 2024-03-20T08:10:45.196 INFO:tasks.rados.rados.0.smithi118.stdout:2692: done (2 left) 2024-03-20T08:10:45.196 INFO:tasks.rados.rados.0.smithi118.stdout:2693: done (1 left) 2024-03-20T08:10:45.196 INFO:tasks.rados.rados.0.smithi118.stdout:2694: done (0 left) 2024-03-20T08:10:45.196 INFO:tasks.rados.rados.0.smithi118.stdout:2695: snap_create 2024-03-20T08:10:45.250 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:45.199+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:45.250 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:45.199+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:45.250 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: osdmap e610: 8 total, 8 up, 8 in 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: pgmap v595: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 941 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 8 op/s 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:45.251 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.037+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.082+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.095+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.121+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.133+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.145+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.252 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.203+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:45.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.250+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.290+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.304+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.331+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.369+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.381+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.445+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.470+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.482+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.495+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:45.198+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:45.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:45.198+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:45.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:45.198+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:45.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:45.194+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:45.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:45.195+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:45.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:45.195+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:45.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:45.195+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:45.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:45.195+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:45.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: osdmap e610: 8 total, 8 up, 8 in 2024-03-20T08:10:45.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:45.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: pgmap v595: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 941 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 8 op/s 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:45.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: osdmap e610: 8 total, 8 up, 8 in 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: pgmap v595: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 941 MiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 8 op/s 2024-03-20T08:10:45.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:45.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.559+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.584+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.649+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.687+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:45.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.769+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.031 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:45.916+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:46.032 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:45.918+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.812+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.885+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.899+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.914+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.930+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.033 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.944+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:45.989+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.003+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.017+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.032+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:45.916+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:46.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:45.916+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:46.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:45.916+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:2695: done (0 left) 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:2696: rollback oid 2 current snap is 285 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 258 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:2697: read oid 40 snap -1 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:2697: expect deleted 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:2698: read oid 24 snap -1 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:2698: expect (ObjNum 617 snap 202 seq_num 617) 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:2699: read oid 50 snap -1 2024-03-20T08:10:46.197 INFO:tasks.rados.rados.0.smithi118.stdout:2699: expect deleted 2024-03-20T08:10:46.198 INFO:tasks.rados.rados.0.smithi118.stdout:2700: delete oid 17 current snap is 285 2024-03-20T08:10:46.207 INFO:tasks.rados.rados.0.smithi118.stdout:2697: done (4 left) 2024-03-20T08:10:46.208 INFO:tasks.rados.rados.0.smithi118.stdout:2700: done (3 left) 2024-03-20T08:10:46.208 INFO:tasks.rados.rados.0.smithi118.stdout:2701: copy_from oid 38 from oid 23 current snap is 285 2024-03-20T08:10:46.208 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 285 2024-03-20T08:10:46.208 INFO:tasks.rados.rados.0.smithi118.stdout:2702: seq_num 828 ranges {294912=671744,966656=802816,1769472=401408} 2024-03-20T08:10:46.213 INFO:tasks.rados.rados.0.smithi118.stdout:2702: writing smithi11881747-22 from 294912 to 966656 tid 1 2024-03-20T08:10:46.218 INFO:tasks.rados.rados.0.smithi118.stdout:2702: writing smithi11881747-22 from 966656 to 1769472 tid 2 2024-03-20T08:10:46.220 INFO:tasks.rados.rados.0.smithi118.stdout:2702: writing smithi11881747-22 from 1769472 to 2170880 tid 3 2024-03-20T08:10:46.221 INFO:tasks.rados.rados.0.smithi118.stdout:2703: delete oid 36 current snap is 285 2024-03-20T08:10:46.221 INFO:tasks.rados.rados.0.smithi118.stdout:2696: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:10:46.221 INFO:tasks.rados.rados.0.smithi118.stdout:2696: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:10:46.221 INFO:tasks.rados.rados.0.smithi118.stdout:2696: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:10:46.221 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45587 (ObjNum 685 snap 225 seq_num 685) dirty exists 2024-03-20T08:10:46.225 INFO:tasks.rados.rados.0.smithi118.stdout:2698: expect (ObjNum 617 snap 202 seq_num 617) 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2701: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2702: finishing write tid 1 to smithi11881747-22 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2702: finishing write tid 2 to smithi11881747-22 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2696: done (5 left) 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2698: done (4 left) 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2699: done (3 left) 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2703: done (2 left) 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2704: read oid 16 snap 255 2024-03-20T08:10:46.248 INFO:tasks.rados.rados.0.smithi118.stdout:2704: expect deleted 2024-03-20T08:10:46.249 INFO:tasks.rados.rados.0.smithi118.stdout:2705: rmattr oid 15 current snap is 285 2024-03-20T08:10:46.249 INFO:tasks.rados.rados.0.smithi118.stdout:2705: done (3 left) 2024-03-20T08:10:46.249 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 285 2024-03-20T08:10:46.249 INFO:tasks.rados.rados.0.smithi118.stdout:2701: finishing copy_from to smithi11881747-38 2024-03-20T08:10:46.249 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 33973 (ObjNum 814 snap 279 seq_num 814) dirty exists 2024-03-20T08:10:46.249 INFO:tasks.rados.rados.0.smithi118.stdout:2706: seq_num 830 ranges {3022848=704512,3727360=614400,4341760=573440,4915200=16384} 2024-03-20T08:10:46.270 INFO:tasks.rados.rados.0.smithi118.stdout:2706: writing smithi11881747-30 from 3022848 to 3727360 tid 1 2024-03-20T08:10:46.275 INFO:tasks.rados.rados.0.smithi118.stdout:2706: writing smithi11881747-30 from 3727360 to 4341760 tid 2 2024-03-20T08:10:46.279 INFO:tasks.rados.rados.0.smithi118.stdout:2706: writing smithi11881747-30 from 4341760 to 4915200 tid 3 2024-03-20T08:10:46.279 INFO:tasks.rados.rados.0.smithi118.stdout:2706: writing smithi11881747-30 from 4915200 to 4931584 tid 4 2024-03-20T08:10:46.279 INFO:tasks.rados.rados.0.smithi118.stdout:2701: done (3 left) 2024-03-20T08:10:46.279 INFO:tasks.rados.rados.0.smithi118.stdout:2707: rollback oid 8 current snap is 285 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 8 to 283 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: finishing write tid 3 to smithi11881747-22 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: finishing write tid 4 to smithi11881747-22 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: finishing write tid 5 to smithi11881747-22 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: oid 22 updating version 0 to 44208 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: oid 22 version 44208 is already newer than 44205 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: oid 22 version 44208 is already newer than 44204 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: oid 22 updating version 44208 to 44209 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 44209 (ObjNum 828 snap 285 seq_num 828) dirty exists 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: left oid 22 (ObjNum 828 snap 285 seq_num 828) 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2706: finishing write tid 1 to smithi11881747-30 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2706: finishing write tid 2 to smithi11881747-30 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2702: done (3 left) 2024-03-20T08:10:46.280 INFO:tasks.rados.rados.0.smithi118.stdout:2704: done (2 left) 2024-03-20T08:10:46.281 INFO:tasks.rados.rados.0.smithi118.stdout:2708: setattr oid 36 current snap is 285 2024-03-20T08:10:46.281 INFO:tasks.rados.rados.0.smithi118.stdout:2709: rollback oid 12 current snap is 285 2024-03-20T08:10:46.281 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 12 to 265 2024-03-20T08:10:46.281 INFO:tasks.rados.rados.0.smithi118.stdout:2710: snap_create 2024-03-20T08:10:46.287 INFO:tasks.rados.rados.0.smithi118.stdout:2706: finishing write tid 3 to smithi11881747-30 2024-03-20T08:10:46.287 INFO:tasks.rados.rados.0.smithi118.stdout:2706: finishing write tid 4 to smithi11881747-30 2024-03-20T08:10:46.287 INFO:tasks.rados.rados.0.smithi118.stdout:2706: finishing write tid 5 to smithi11881747-30 2024-03-20T08:10:46.291 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45582 (ObjNum 65 snap 0 seq_num 1382706798) dirty exists 2024-03-20T08:10:46.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:46.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:46.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:46.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:46.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: osdmap e611: 8 total, 8 up, 8 in 2024-03-20T08:10:46.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:46.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:46.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:46.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:46.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:46.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.047+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.103+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.130+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.144+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.158+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.202+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.202+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.202+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.229+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.295 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:46.208+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:46.295 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:46.208+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:46.295 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:46.206+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:46.295 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:46.206+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:46.295 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:46.214+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:46.296 INFO:tasks.rados.rados.0.smithi118.stdout:2706: finishing write tid 6 to smithi11881747-30 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2706: oid 30 updating version 0 to 33937 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2706: oid 30 version 33937 is already newer than 33933 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2706: oid 30 version 33937 is already newer than 33936 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2706: oid 30 version 33937 is already newer than 33935 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2706: oid 30 version 33937 is already newer than 33934 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 33937 (ObjNum 830 snap 285 seq_num 830) dirty exists 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2706: left oid 30 (ObjNum 830 snap 285 seq_num 830) 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2707: finishing rollback tid 0 to smithi11881747-8 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2707: finishing rollback tid 1 to smithi11881747-8 2024-03-20T08:10:46.297 INFO:tasks.rados.rados.0.smithi118.stdout:2709: finishing rollback tid 0 to smithi11881747-12 2024-03-20T08:10:46.298 INFO:tasks.rados.rados.0.smithi118.stdout:2707: finishing rollback tid 2 to smithi11881747-8 2024-03-20T08:10:46.298 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34029 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:10:46.298 INFO:tasks.rados.rados.0.smithi118.stdout:2709: finishing rollback tid 1 to smithi11881747-12 2024-03-20T08:10:46.298 INFO:tasks.rados.rados.0.smithi118.stdout:2709: finishing rollback tid 2 to smithi11881747-12 2024-03-20T08:10:46.299 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34021 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.290+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.309+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.325+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.342+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.396+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.446+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.459+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.488+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:46.195+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:46.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:46.196+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:46.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:46.198+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:46.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:46.200+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:46.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:46.200+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:46.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:46.200+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:46.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:46.201+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:46.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: osdmap e611: 8 total, 8 up, 8 in 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:46.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: osdmap e611: 8 total, 8 up, 8 in 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:46.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:46.675 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:46.211+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:46.807 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:46 smithi196 systemd[1]: Stopping Ceph node-exporter.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:10:46.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.549+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.606+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.619+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.632+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.660+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.674+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.689+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.808 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.763+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.793+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:46.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.807+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.072 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:47 smithi196 podman[115824]: 2024-03-20 08:10:47.043816201 +0000 UTC m=+0.135724377 container died fd3b2992729c450834df853343fb76eaf82d5259eaa6e522e1d73344b819274e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:10:47.072 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.822+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.072 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.072 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.072 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.866+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.880+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.953+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:46.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.001+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.028+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.073 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.043+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.074 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.074 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.072+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:2706: done (4 left) 2024-03-20T08:10:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:2707: done (3 left) 2024-03-20T08:10:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:2708: done (2 left) 2024-03-20T08:10:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:2709: done (1 left) 2024-03-20T08:10:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:2710: done (0 left) 2024-03-20T08:10:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:2711: rollback oid 14 current snap is 286 2024-03-20T08:10:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 279 2024-03-20T08:10:47.213 INFO:tasks.rados.rados.0.smithi118.stdout:2712: snap_create 2024-03-20T08:10:47.224 INFO:tasks.rados.rados.0.smithi118.stdout:2711: finishing rollback tid 0 to smithi11881747-14 2024-03-20T08:10:47.224 INFO:tasks.rados.rados.0.smithi118.stdout:2711: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:10:47.224 INFO:tasks.rados.rados.0.smithi118.stdout:2711: finishing rollback tid 2 to smithi11881747-14 2024-03-20T08:10:47.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 34033 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:10:47.331 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:47.214+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:47.332 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:47.214+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:47.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: Upgrade: Updating node-exporter.b (2/2) 2024-03-20T08:10:47.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: Deploying daemon node-exporter.b on smithi196 2024-03-20T08:10:47.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:47.332 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: osdmap e612: 8 total, 8 up, 8 in 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:47.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:47.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:47.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-mon[87080]: pgmap v598: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 961 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 8.6 MiB/s rd, 4.9 MiB/s wr, 19 op/s 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.136+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.161+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.175+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.188+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.334 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.200+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.217+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.231+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.256+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.269+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.282+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.335 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.592 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:47 smithi196 podman[115824]: 2024-03-20 08:10:47.454753847 +0000 UTC m=+0.546662014 container cleanup fd3b2992729c450834df853343fb76eaf82d5259eaa6e522e1d73344b819274e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:10:47.592 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:47 smithi196 bash[115824]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.346+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.386+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.398+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.412+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.463+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.478+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.552+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.578+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.615 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:47.215+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:47.615 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:47.215+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:47.615 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:47.215+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:47.615 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:47.212+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:47.616 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:47.211+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:47.616 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:47.211+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:47.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: Upgrade: Updating node-exporter.b (2/2) 2024-03-20T08:10:47.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: Deploying daemon node-exporter.b on smithi196 2024-03-20T08:10:47.616 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: osdmap e612: 8 total, 8 up, 8 in 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:47.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[93121]: pgmap v598: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 961 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 8.6 MiB/s rd, 4.9 MiB/s wr, 19 op/s 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: Upgrade: Updating node-exporter.b (2/2) 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: Deploying daemon node-exporter.b on smithi196 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:47.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: osdmap e612: 8 total, 8 up, 8 in 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:47.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:47.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:47.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:47.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:47.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:47.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-mon[95692]: pgmap v598: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 961 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 8.6 MiB/s rd, 4.9 MiB/s wr, 19 op/s 2024-03-20T08:10:47.854 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:47.616+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:47.854 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:47.617+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:47.855 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:47 smithi196 podman[115841]: 2024-03-20 08:10:47.698231539 +0000 UTC m=+0.656284392 container remove fd3b2992729c450834df853343fb76eaf82d5259eaa6e522e1d73344b819274e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:10:47.855 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:47 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:10:47.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.633+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.647+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.660+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.685+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.698+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.710+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.800+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.826+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.841+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:47.615+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:47.615+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:47.615+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:47.616+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:47.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:47.616+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:48.115 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:47 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-20T08:10:48.115 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:47 smithi196 systemd[1]: Stopped Ceph node-exporter.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:10:48.115 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:47 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.b.service: Consumed 3.331s CPU time. 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.856+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.886+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.899+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.912+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.952+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.977+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:47.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.003+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.026+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.039+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.052+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.065+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.077+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.103+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.115+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.212 INFO:tasks.rados.rados.0.smithi118.stdout:2711: done (1 left) 2024-03-20T08:10:48.212 INFO:tasks.rados.rados.0.smithi118.stdout:2712: done (0 left) 2024-03-20T08:10:48.212 INFO:tasks.rados.rados.0.smithi118.stdout:2713: read oid 49 snap -1 2024-03-20T08:10:48.213 INFO:tasks.rados.rados.0.smithi118.stdout:2713: expect deleted 2024-03-20T08:10:48.213 INFO:tasks.rados.rados.0.smithi118.stdout:2714: rmattr oid 6 current snap is 287 2024-03-20T08:10:48.213 INFO:tasks.rados.rados.0.smithi118.stdout:2714: done (1 left) 2024-03-20T08:10:48.213 INFO:tasks.rados.rados.0.smithi118.stdout:2715: rollback oid 28 current snap is 287 2024-03-20T08:10:48.213 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 28 to 248 2024-03-20T08:10:48.213 INFO:tasks.rados.rados.0.smithi118.stdout:2716: copy_from oid 34 from oid 35 current snap is 287 2024-03-20T08:10:48.213 INFO:tasks.rados.rados.0.smithi118.stdout:2717: delete oid 27 current snap is 287 2024-03-20T08:10:48.223 INFO:tasks.rados.rados.0.smithi118.stdout:2713: done (3 left) 2024-03-20T08:10:48.223 INFO:tasks.rados.rados.0.smithi118.stdout:2717: done (2 left) 2024-03-20T08:10:48.223 INFO:tasks.rados.rados.0.smithi118.stdout:2718: read oid 19 snap 249 2024-03-20T08:10:48.223 INFO:tasks.rados.rados.0.smithi118.stdout:2718: expect deleted 2024-03-20T08:10:48.224 INFO:tasks.rados.rados.0.smithi118.stdout:2719: snap_create 2024-03-20T08:10:48.224 INFO:tasks.rados.rados.0.smithi118.stdout:2715: finishing rollback tid 0 to smithi11881747-28 2024-03-20T08:10:48.224 INFO:tasks.rados.rados.0.smithi118.stdout:2715: finishing rollback tid 1 to smithi11881747-28 2024-03-20T08:10:48.224 INFO:tasks.rados.rados.0.smithi118.stdout:2715: finishing rollback tid 2 to smithi11881747-28 2024-03-20T08:10:48.224 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 33978 (ObjNum 644 snap 211 seq_num 644) dirty exists 2024-03-20T08:10:48.233 INFO:tasks.rados.rados.0.smithi118.stdout:2716: finishing copy_from racing read to smithi11881747-34 2024-03-20T08:10:48.234 INFO:tasks.rados.rados.0.smithi118.stdout:2716: finishing copy_from to smithi11881747-34 2024-03-20T08:10:48.234 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 23103 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:10:48.372 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:48 smithi196 systemd[1]: Starting Ceph node-exporter.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:10:48.372 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:48.226+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:48.372 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:48.226+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:48.373 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:48.226+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: osdmap e613: 8 total, 8 up, 8 in 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:48.373 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:48.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:48.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:48.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:48.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:48.374 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:48.219+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:48.374 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:48.219+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:48.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.128+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.374 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.153+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.191+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.213+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.213+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.224+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.241+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.255+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.375 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.297+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.328+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.372+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.625 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:48 smithi196 podman[115967]: 2024-03-20 08:10:48.502552378 +0000 UTC m=+0.020887471 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.432+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.448+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.492+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.533+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.626 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.574+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.600+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.613+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.627 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:48.223+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:48.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:48.223+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:48.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:48.212+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:48.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:48.215+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:48.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:48.218+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:48.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:48.219+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:48.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:48.224+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:48.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:48.223+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: osdmap e613: 8 total, 8 up, 8 in 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:48.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: osdmap e613: 8 total, 8 up, 8 in 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 154=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:48.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:48.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:48.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:48.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:48.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:48.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.640+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.665+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.720+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.758+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.770+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.782+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.806+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.832+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.845+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.858+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.870+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:48.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.139 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 podman[115967]: 2024-03-20 08:10:49.091637602 +0000 UTC m=+0.609972683 container create d2c82f232b48b2da2603e10bb87ec286a3be82b716620c08ea4c659de42af9b9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:10:49.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.896+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.923+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.936+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.964+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.978+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:48.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.038+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.051+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.089+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.223 INFO:tasks.rados.rados.0.smithi118.stdout:2715: done (3 left) 2024-03-20T08:10:49.223 INFO:tasks.rados.rados.0.smithi118.stdout:2716: done (2 left) 2024-03-20T08:10:49.223 INFO:tasks.rados.rados.0.smithi118.stdout:2718: done (1 left) 2024-03-20T08:10:49.223 INFO:tasks.rados.rados.0.smithi118.stdout:2719: done (0 left) 2024-03-20T08:10:49.223 INFO:tasks.rados.rados.0.smithi118.stdout:2720: rollback oid 18 current snap is 288 2024-03-20T08:10:49.223 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 18 to 278 2024-03-20T08:10:49.223 INFO:tasks.rados.rados.0.smithi118.stdout:2721: copy_from oid 24 from oid 30 current snap is 288 2024-03-20T08:10:49.224 INFO:tasks.rados.rados.0.smithi118.stdout:2722: copy_from oid 10 from oid 22 current snap is 288 2024-03-20T08:10:49.224 INFO:tasks.rados.rados.0.smithi118.stdout:2723: snap_remove snap 236 2024-03-20T08:10:49.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:49.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: osdmap e614: 8 total, 8 up, 8 in 2024-03-20T08:10:49.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:49.400 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:49.401 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-mon[87080]: pgmap v601: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 970 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 19 op/s 2024-03-20T08:10:49.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.168+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.180+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.224+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.308+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.321+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.403 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 podman[115967]: 2024-03-20 08:10:49.378495828 +0000 UTC m=+0.896830927 container init d2c82f232b48b2da2603e10bb87ec286a3be82b716620c08ea4c659de42af9b9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:10:49.403 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 podman[115967]: 2024-03-20 08:10:49.385454568 +0000 UTC m=+0.903789650 container start d2c82f232b48b2da2603e10bb87ec286a3be82b716620c08ea4c659de42af9b9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:10:49.403 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.390Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-20T08:10:49.403 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.390Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-20T08:10:49.403 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.390Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-20T08:10:49.403 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.390Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-20T08:10:49.403 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.391Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-20T08:10:49.403 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.391Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-20T08:10:49.404 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.391Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-20T08:10:49.404 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.391Z caller=node_exporter.go:117 level=info collector=arp 2024-03-20T08:10:49.404 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.391Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-20T08:10:49.404 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.391Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-20T08:10:49.404 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.391Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-20T08:10:49.404 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-20T08:10:49.404 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-20T08:10:49.404 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-20T08:10:49.404 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:49.227+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:49.404 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:49.229+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:49.405 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-20T08:10:49.405 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-20T08:10:49.405 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=edac 2024-03-20T08:10:49.405 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-20T08:10:49.405 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-20T08:10:49.405 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-20T08:10:49.405 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-20T08:10:49.406 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=os 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=stat 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-20T08:10:49.407 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=time 2024-03-20T08:10:49.408 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-20T08:10:49.408 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=uname 2024-03-20T08:10:49.408 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-20T08:10:49.408 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-20T08:10:49.408 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.392Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-20T08:10:49.408 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.393Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-20T08:10:49.408 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b[115980]: ts=2024-03-20T08:10:49.393Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-20T08:10:49.650 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 bash[115967]: d2c82f232b48b2da2603e10bb87ec286a3be82b716620c08ea4c659de42af9b9 2024-03-20T08:10:49.651 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:10:49 smithi196 systemd[1]: Started Ceph node-exporter.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:10:49.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.651 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.500+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.526+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.554+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.571+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.586+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.652 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.598+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.611+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.651+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: osdmap e614: 8 total, 8 up, 8 in 2024-03-20T08:10:49.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:49.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[93121]: pgmap v601: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 970 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 19 op/s 2024-03-20T08:10:49.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:49.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: osdmap e614: 8 total, 8 up, 8 in 2024-03-20T08:10:49.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:49.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:49.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:49.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:49.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:49.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:49.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:49.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:49.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:49.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:49.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:49.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:49.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-mon[95692]: pgmap v601: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 970 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.6 MiB/s wr, 19 op/s 2024-03-20T08:10:49.675 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:49.225+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:49.675 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:49.225+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:49.675 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:49.225+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:49.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:49.222+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:49.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:49.232+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:49.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:49.232+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:49.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.665+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.694+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.722+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.751+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.767+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.798+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.841+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.856+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.869+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:49.914 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:49.840+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:49.915 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:49.840+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:50.096 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:49.839+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:50.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:49.839+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:50.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:49.840+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:50.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:49.840+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:50.097 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:49.840+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:50.097 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:10:50.169 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.924+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.939+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.968+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:49.995+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.109+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.151+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.169+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.226 INFO:tasks.rados.rados.0.smithi118.stdout:2723: done (3 left) 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2724: read oid 16 snap -1 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2724: expect deleted 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2725: rollback oid 11 current snap is 288 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 238 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2720: finishing rollback tid 1 to smithi11881747-18 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 34037 (ObjNum 33 snap 0 seq_num 322182624) dirty dne 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2722: finishing copy_from racing read to smithi11881747-10 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2722: finishing copy_from to smithi11881747-10 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 44209 (ObjNum 828 snap 285 seq_num 828) dirty exists 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2721: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2721: finishing copy_from to smithi11881747-24 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 33937 (ObjNum 830 snap 285 seq_num 830) dirty exists 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2720: done (4 left) 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2721: done (3 left) 2024-03-20T08:10:50.227 INFO:tasks.rados.rados.0.smithi118.stdout:2722: done (2 left) 2024-03-20T08:10:50.228 INFO:tasks.rados.rados.0.smithi118.stdout:2726: copy_from oid 45 from oid 9 current snap is 288 2024-03-20T08:10:50.228 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 288 2024-03-20T08:10:50.228 INFO:tasks.rados.rados.0.smithi118.stdout:2727: seq_num 832 ranges {1810432=737280,2547712=786432,3334144=417792} 2024-03-20T08:10:50.243 INFO:tasks.rados.rados.0.smithi118.stdout:2727: writing smithi11881747-26 from 1810432 to 2547712 tid 1 2024-03-20T08:10:50.248 INFO:tasks.rados.rados.0.smithi118.stdout:2727: writing smithi11881747-26 from 2547712 to 3334144 tid 2 2024-03-20T08:10:50.251 INFO:tasks.rados.rados.0.smithi118.stdout:2727: writing smithi11881747-26 from 3334144 to 3751936 tid 3 2024-03-20T08:10:50.252 INFO:tasks.rados.rados.0.smithi118.stdout:2725: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:10:50.252 INFO:tasks.rados.rados.0.smithi118.stdout:2725: finishing rollback tid 2 to smithi11881747-11 2024-03-20T08:10:50.252 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 34068 (ObjNum 704 snap 236 seq_num 704) dirty exists 2024-03-20T08:10:50.252 INFO:tasks.rados.rados.0.smithi118.stdout:2724: done (3 left) 2024-03-20T08:10:50.252 INFO:tasks.rados.rados.0.smithi118.stdout:2725: done (2 left) 2024-03-20T08:10:50.252 INFO:tasks.rados.rados.0.smithi118.stdout:2728: snap_remove snap 283 2024-03-20T08:10:50.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:50.232+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:50.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:50.236+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:50.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:50.239+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:50.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:50.225+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:50.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:50.225+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:50.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:50.226+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:50.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:50.226+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: osdmap e615: 8 total, 8 up, 8 in 2024-03-20T08:10:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: osdmap e615: 8 total, 8 up, 8 in 2024-03-20T08:10:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:50.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:50.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:50.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:50.433 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: osdmap e615: 8 total, 8 up, 8 in 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:10:50.434 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:50.435 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:50.435 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:50.435 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:50.435 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:50.435 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:50.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.229+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:50.435 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.229+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.231+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.255+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.328+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.358+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.373+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.403+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.436 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.418+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:50.232+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:50.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:50.232+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:50.533 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:10:50.533 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (10m) 46s ago 11m 27.2M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:10:50.533 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (11m) 19s ago 11m 89.6M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:10:50.533 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (50s) 46s ago 12m 12.5M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 b9e9c53270c7 2024-03-20T08:10:50.533 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (26s) 19s ago 12m 12.6M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 237cfe0be476 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (7m) 19s ago 16m 453M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (7m) 46s ago 17m 514M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (7m) 46s ago 18m 67.3M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (6m) 19s ago 16m 52.0M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (6m) 46s ago 16m 61.0M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 starting - - - - 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 starting - - - - 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (5m) 46s ago 15m 329M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (4m) 46s ago 15m 513M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (4m) 46s ago 14m 388M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (3m) 46s ago 14m 141M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (3m) 19s ago 14m 404M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:10:50.534 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (2m) 19s ago 13m 347M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 72b569d10541 2024-03-20T08:10:50.535 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (118s) 19s ago 13m 250M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 dca658056148 2024-03-20T08:10:50.535 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (74s) 19s ago 12m 117M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 52392403da44 2024-03-20T08:10:50.535 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (7m) 19s ago 11m 62.8M - 2.43.0 a07b618ecd1d 1fb280ede3dc 2024-03-20T08:10:50.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.448+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.461+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.690 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.473+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.499+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.523+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.549+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.600+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.629+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.642+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.691 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.672+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.692 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.690+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.692 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:50.618+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:50.692 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:50.619+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:50.692 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:50.619+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:50.788 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:50.618+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:50.945 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.945 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.718+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.945 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.945 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.762+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.776+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.809+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.824+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.866+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.879+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.905+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.919+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:50.946 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.945+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.118 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:10:51.118 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 8 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 15 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:10:51.119 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:10:51.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.958+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.222 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:50.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.072+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.137+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.222+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2728: done (2 left) 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2729: delete oid 40 current snap is 288 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2726: finishing copy_from racing read to smithi11881747-45 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2726: finishing copy_from to smithi11881747-45 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 34057 (ObjNum 809 snap 279 seq_num 809) dirty exists 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2727: finishing write tid 1 to smithi11881747-26 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2727: finishing write tid 2 to smithi11881747-26 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2727: finishing write tid 3 to smithi11881747-26 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2727: finishing write tid 4 to smithi11881747-26 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2727: finishing write tid 5 to smithi11881747-26 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2727: oid 26 updating version 0 to 52523 2024-03-20T08:10:51.241 INFO:tasks.rados.rados.0.smithi118.stdout:2727: oid 26 updating version 52523 to 52527 2024-03-20T08:10:51.242 INFO:tasks.rados.rados.0.smithi118.stdout:2727: oid 26 version 52527 is already newer than 52526 2024-03-20T08:10:51.242 INFO:tasks.rados.rados.0.smithi118.stdout:2727: oid 26 version 52527 is already newer than 52522 2024-03-20T08:10:51.242 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 52527 (ObjNum 832 snap 288 seq_num 832) dirty exists 2024-03-20T08:10:51.242 INFO:tasks.rados.rados.0.smithi118.stdout:2727: left oid 26 (ObjNum 832 snap 288 seq_num 832) 2024-03-20T08:10:51.251 INFO:tasks.rados.rados.0.smithi118.stdout:2726: done (2 left) 2024-03-20T08:10:51.251 INFO:tasks.rados.rados.0.smithi118.stdout:2727: done (1 left) 2024-03-20T08:10:51.251 INFO:tasks.rados.rados.0.smithi118.stdout:2729: done (0 left) 2024-03-20T08:10:51.251 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 288 2024-03-20T08:10:51.252 INFO:tasks.rados.rados.0.smithi118.stdout:2730: seq_num 834 ranges {0=778240,778240=557056,1335296=548864,1884160=98304} 2024-03-20T08:10:51.255 INFO:tasks.rados.rados.0.smithi118.stdout:2730: writing smithi11881747-43 from 0 to 778240 tid 1 2024-03-20T08:10:51.258 INFO:tasks.rados.rados.0.smithi118.stdout:2730: writing smithi11881747-43 from 778240 to 1335296 tid 2 2024-03-20T08:10:51.262 INFO:tasks.rados.rados.0.smithi118.stdout:2730: writing smithi11881747-43 from 1335296 to 1884160 tid 3 2024-03-20T08:10:51.262 INFO:tasks.rados.rados.0.smithi118.stdout:2730: writing smithi11881747-43 from 1884160 to 1982464 tid 4 2024-03-20T08:10:51.262 INFO:tasks.rados.rados.0.smithi118.stdout:2731: snap_remove snap 248 2024-03-20T08:10:51.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:51.238+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:51.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:51.242+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:51.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:51.242+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:51.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:51.242+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:51.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:51.238+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:51.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:51.238+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:51.244+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:51.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:51.244+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:51.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:51.244+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:51.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:51.244+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:51.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:51.248+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: from='client.44360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: osdmap e616: 8 total, 8 up, 8 in 2024-03-20T08:10:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: pgmap v604: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 983 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 541 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3051283222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: from='client.44360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: osdmap e616: 8 total, 8 up, 8 in 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:51.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:51.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: pgmap v604: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 983 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 541 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-03-20T08:10:51.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:51.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:51.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:51 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/3051283222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:51.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: from='client.44360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:51.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: osdmap e616: 8 total, 8 up, 8 in 2024-03-20T08:10:51.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 126=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:51.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:51.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:51.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:51.482 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: pgmap v604: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 983 MiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 541 KiB/s rd, 4.6 MiB/s wr, 8 op/s 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:51.483 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/3051283222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:10:51.483 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.237+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.244+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.350+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.365+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.393+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.484 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.438+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.454+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.485 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.485 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:51.242+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:51.485 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:51.242+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:51.739 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:51.719+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:51.739 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:51.719+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.557+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.572+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.587+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.687+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.701+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.713+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.992 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.789+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.801+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.815+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.832+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.845+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.879+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.993 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.927+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.953+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:51.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:51.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.246 INFO:tasks.rados.rados.0.smithi118.stdout:2731: done (1 left) 2024-03-20T08:10:52.246 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 288 2024-03-20T08:10:52.246 INFO:tasks.rados.rados.0.smithi118.stdout:2732: seq_num 835 ranges {1540096=794624,2334720=442368,2777088=712704,3489792=49152} 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: from='client.54349 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: osdmap e617: 8 total, 8 up, 8 in 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:52.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:52.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:52.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:52.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:52.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:52.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.086+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.140+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.154+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.167+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.180+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.194+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.206+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.231+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.249 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.247+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.258 INFO:tasks.rados.rados.0.smithi118.stdout:2732: writing smithi11881747-21 from 1540096 to 2334720 tid 1 2024-03-20T08:10:52.261 INFO:tasks.rados.rados.0.smithi118.stdout:2732: writing smithi11881747-21 from 2334720 to 2777088 tid 2 2024-03-20T08:10:52.265 INFO:tasks.rados.rados.0.smithi118.stdout:2732: writing smithi11881747-21 from 2777088 to 3489792 tid 3 2024-03-20T08:10:52.266 INFO:tasks.rados.rados.0.smithi118.stdout:2732: writing smithi11881747-21 from 3489792 to 3538944 tid 4 2024-03-20T08:10:52.266 INFO:tasks.rados.rados.0.smithi118.stdout:2733: rmattr oid 39 current snap is 288 2024-03-20T08:10:52.266 INFO:tasks.rados.rados.0.smithi118.stdout:2733: done (2 left) 2024-03-20T08:10:52.266 INFO:tasks.rados.rados.0.smithi118.stdout:2734: snap_remove snap 281 2024-03-20T08:10:52.503 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:52.246+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:52.503 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:52.246+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:52.504 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:10:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:10:52.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.250+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:52.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.250+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:52.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.504 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.330+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.404+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.419+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.434+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.450+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.466+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.504+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.534 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:52.249+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:52.534 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:52.249+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:52.535 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: from='client.54349 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:52.535 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:52.535 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:52.535 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:52.535 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: osdmap e617: 8 total, 8 up, 8 in 2024-03-20T08:10:52.535 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:52.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:52.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:52.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:52.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:52.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:52.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:52.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:52.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:52.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:52.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:52.537 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: from='client.54349 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:10:52.537 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:52.537 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:52.537 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:52.537 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: osdmap e617: 8 total, 8 up, 8 in 2024-03-20T08:10:52.537 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:52.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:52.539 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:52.245+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:52.539 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:52.244+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:52.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.517+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.530+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.555+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.568+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.606+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.619+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.632+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.644+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.658+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.673+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.686+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.699+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.747+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.764 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.762+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:52.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:52.590+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:52.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:52.591+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:52.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:52.591+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:52.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:52.591+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:53.024 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:53.005+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:53.024 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:53.006+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:53.025 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:53.007+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:53.025 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.025 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.791+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.025 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.806+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.025 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.819+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.025 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.025 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.860+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.875+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.890+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.905+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.919+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.934+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.947+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.960+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.973+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.985+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:52.997+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.026 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.250 INFO:tasks.rados.rados.0.smithi118.stdout:2734: done (2 left) 2024-03-20T08:10:53.250 INFO:tasks.rados.rados.0.smithi118.stdout:2735: snap_remove snap 276 2024-03-20T08:10:53.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:53.251+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:53.286 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:53.251+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:53.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:53.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: osdmap e618: 8 total, 8 up, 8 in 2024-03-20T08:10:53.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:53.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:53.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:53.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:53.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:53.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:53.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: pgmap v607: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 985 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 540 KiB/s rd, 2.3 MiB/s wr, 11 op/s 2024-03-20T08:10:53.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:53.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:53.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:53.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:53.287 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.038+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.287 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.052+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.287 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.065+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.287 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.079+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.092+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.130+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.158+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.169+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.182+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.195+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.253+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.261+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.288 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.274+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.548 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:53.340+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:53.548 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:53.340+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:53.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.300+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.313+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.326+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.339+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.353+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.365+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.436+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.448+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.460+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.472+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.484+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.496+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.510+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.535+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.550 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.574 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:53.252+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:53.574 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:53.252+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:53.574 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:53.252+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:53.575 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:53.249+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:53.575 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:53.249+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:53.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:53.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: osdmap e618: 8 total, 8 up, 8 in 2024-03-20T08:10:53.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:53.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:53.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:53.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:53.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:53.575 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:53.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: pgmap v607: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 985 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 540 KiB/s rd, 2.3 MiB/s wr, 11 op/s 2024-03-20T08:10:53.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:53.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:53.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:53.576 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:53.576 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:53.339+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:53.576 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:53.339+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:53.576 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:53.339+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:53.576 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:53.339+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:53.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:53.340+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:53.577 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:53.251+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:53.577 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:53.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: osdmap e618: 8 total, 8 up, 8 in 2024-03-20T08:10:53.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:53.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:53.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:53.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:53.579 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:53.579 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:53.579 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: pgmap v607: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 985 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 540 KiB/s rd, 2.3 MiB/s wr, 11 op/s 2024-03-20T08:10:53.579 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:53.579 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:53.580 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:53.580 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:53.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.628+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.882 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.654+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.681+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.730+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.883 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:10:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:10:53] "GET /metrics HTTP/1.1" 200 36413 "" "Prometheus/2.43.0" 2024-03-20T08:10:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.919+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.932+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.945+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.957+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.971+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.983+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:53.997+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.009+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.022+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.033+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.056+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.068+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.081+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.093+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.106+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.252 INFO:tasks.rados.rados.0.smithi118.stdout:2735: done (2 left) 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2736: rmattr oid 29 current snap is 288 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: finishing write tid 1 to smithi11881747-43 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: finishing write tid 2 to smithi11881747-43 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: finishing write tid 3 to smithi11881747-43 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: finishing write tid 4 to smithi11881747-43 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: finishing write tid 5 to smithi11881747-43 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: finishing write tid 6 to smithi11881747-43 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: oid 43 updating version 0 to 44214 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: oid 43 version 44214 is already newer than 44210 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: oid 43 version 44214 is already newer than 44213 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: oid 43 version 44214 is already newer than 44211 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: oid 43 version 44214 is already newer than 44212 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 44214 (ObjNum 834 snap 288 seq_num 834) dirty exists 2024-03-20T08:10:54.253 INFO:tasks.rados.rados.0.smithi118.stdout:2730: left oid 43 (ObjNum 834 snap 288 seq_num 834) 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: finishing write tid 1 to smithi11881747-21 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: finishing write tid 2 to smithi11881747-21 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: finishing write tid 3 to smithi11881747-21 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: finishing write tid 4 to smithi11881747-21 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: finishing write tid 5 to smithi11881747-21 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: finishing write tid 6 to smithi11881747-21 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: oid 21 updating version 0 to 46378 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: oid 21 version 46378 is already newer than 46377 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: oid 21 updating version 46378 to 46379 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: oid 21 version 46379 is already newer than 46376 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: oid 21 version 46379 is already newer than 46375 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 46379 (ObjNum 835 snap 288 seq_num 835) dirty exists 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2732: left oid 21 (ObjNum 835 snap 288 seq_num 835) 2024-03-20T08:10:54.254 INFO:tasks.rados.rados.0.smithi118.stdout:2730: done (2 left) 2024-03-20T08:10:54.255 INFO:tasks.rados.rados.0.smithi118.stdout:2732: done (1 left) 2024-03-20T08:10:54.255 INFO:tasks.rados.rados.0.smithi118.stdout:2737: read oid 37 snap -1 2024-03-20T08:10:54.255 INFO:tasks.rados.rados.0.smithi118.stdout:2737: expect (ObjNum 794 snap 275 seq_num 794) 2024-03-20T08:10:54.255 INFO:tasks.rados.rados.0.smithi118.stdout:2738: snap_create 2024-03-20T08:10:54.258 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33979 (ObjNum 638 snap 206 seq_num 638) dirty exists 2024-03-20T08:10:54.266 INFO:tasks.rados.rados.0.smithi118.stdout:2737: expect (ObjNum 794 snap 275 seq_num 794) 2024-03-20T08:10:54.469 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:54.253+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:54.469 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:54.253+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:54.469 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:54.439+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:54.469 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:54.445+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:54.469 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:54.445+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.229+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.256+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.256+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.470 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.278+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.303+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.315+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.328+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.355+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.368+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.471 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: osdmap e619: 8 total, 8 up, 8 in 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:54.472 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:54.473 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:54.473 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:54.473 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.473 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.405+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.473 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.473 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.430+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.473 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.473 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.473 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:54.263+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:54.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:54.263+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:54.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:54.251+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:54.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:54.252+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:54.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:54.252+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:54.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:54.252+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:54.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:54.440+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:54.674 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:54.251+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:54.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:54.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:54.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:54.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: osdmap e619: 8 total, 8 up, 8 in 2024-03-20T08:10:54.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:54.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: osdmap e619: 8 total, 8 up, 8 in 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:54.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:54.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:54.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:54.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:54.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:54.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:54.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:54 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:54.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.482+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.508+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.536+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.562+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.590+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.615+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.627+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.679+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.718+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.730+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.793+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.986 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.817+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.829+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.855+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.866+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.874+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.891+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.900+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.987 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.988 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.988 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.973+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:54.988 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.986+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:54.998+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.022+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.036+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.048+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.063+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.079+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.245 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.124+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.137+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.152+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.167+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.181+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.208+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.245+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.265 INFO:tasks.rados.rados.0.smithi118.stdout:2736: done (2 left) 2024-03-20T08:10:55.265 INFO:tasks.rados.rados.0.smithi118.stdout:2737: done (1 left) 2024-03-20T08:10:55.265 INFO:tasks.rados.rados.0.smithi118.stdout:2738: done (0 left) 2024-03-20T08:10:55.265 INFO:tasks.rados.rados.0.smithi118.stdout:2739: setattr oid 8 current snap is 289 2024-03-20T08:10:55.268 INFO:tasks.rados.rados.0.smithi118.stdout:2740: rmattr oid 27 current snap is 289 2024-03-20T08:10:55.268 INFO:tasks.rados.rados.0.smithi118.stdout:2740: done (1 left) 2024-03-20T08:10:55.268 INFO:tasks.rados.rados.0.smithi118.stdout:2741: rmattr oid 30 current snap is 289 2024-03-20T08:10:55.268 INFO:tasks.rados.rados.0.smithi118.stdout:2741: done (1 left) 2024-03-20T08:10:55.269 INFO:tasks.rados.rados.0.smithi118.stdout:2742: read oid 16 snap 279 2024-03-20T08:10:55.269 INFO:tasks.rados.rados.0.smithi118.stdout:2742: expect deleted 2024-03-20T08:10:55.269 INFO:tasks.rados.rados.0.smithi118.stdout:2743: read oid 15 snap -1 2024-03-20T08:10:55.269 INFO:tasks.rados.rados.0.smithi118.stdout:2743: expect (ObjNum 813 snap 279 seq_num 813) 2024-03-20T08:10:55.269 INFO:tasks.rados.rados.0.smithi118.stdout:2744: delete oid 5 current snap is 289 2024-03-20T08:10:55.274 INFO:tasks.rados.rados.0.smithi118.stdout:2744: done (3 left) 2024-03-20T08:10:55.275 INFO:tasks.rados.rados.0.smithi118.stdout:2745: snap_create 2024-03-20T08:10:55.277 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34035 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:10:55.285 INFO:tasks.rados.rados.0.smithi118.stdout:2743: expect (ObjNum 813 snap 279 seq_num 813) 2024-03-20T08:10:55.506 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:55.264+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:55.506 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:55.264+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: osdmap e620: 8 total, 8 up, 8 in 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: pgmap v610: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 990 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 255 B/s rd, 1.3 MiB/s wr, 8 op/s 2024-03-20T08:10:55.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:55.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:55.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:55.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:55.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:55.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.266+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:55.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.275+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.508 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.368+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.380+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.464+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.477+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:55.266+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:55.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:55.266+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:55.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:55.267+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:55.274+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:55.274+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:55.275+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:55.275+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:55.276+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:55.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:55.261+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: osdmap e620: 8 total, 8 up, 8 in 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: pgmap v610: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 990 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 255 B/s rd, 1.3 MiB/s wr, 8 op/s 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: osdmap e620: 8 total, 8 up, 8 in 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:55.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: pgmap v610: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 990 MiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 255 B/s rd, 1.3 MiB/s wr, 8 op/s 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:55.768 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:55.707+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:55.769 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:55.707+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:55.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.521+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.534+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.769 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.560+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.572+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.650+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.664+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.703+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.717+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.729+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.770 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:55.771 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.769+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.782+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.808+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.846+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.858+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.871+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.028 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.908+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.959+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.974+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:55.988+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.001+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.029 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.086 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:55.706+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:56.087 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:55.706+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:56.265 INFO:tasks.rados.rados.0.smithi118.stdout:2739: done (3 left) 2024-03-20T08:10:56.265 INFO:tasks.rados.rados.0.smithi118.stdout:2742: done (2 left) 2024-03-20T08:10:56.265 INFO:tasks.rados.rados.0.smithi118.stdout:2743: done (1 left) 2024-03-20T08:10:56.265 INFO:tasks.rados.rados.0.smithi118.stdout:2745: done (0 left) 2024-03-20T08:10:56.265 INFO:tasks.rados.rados.0.smithi118.stdout:2746: snap_remove snap 265 2024-03-20T08:10:56.287 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:56.266+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:56.287 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:56.266+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:56.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:56.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: osdmap e621: 8 total, 8 up, 8 in 2024-03-20T08:10:56.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:56.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:56.288 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.043+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.072+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.086+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.128+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.199+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.289 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.214+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.242+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.271+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:56.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.271+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:56.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.290 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.287+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:56.268+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:56.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:56.268+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:56.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:56.264+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:56.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:56.264+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:56.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:56.269+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:56.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:56.269+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: osdmap e621: 8 total, 8 up, 8 in 2024-03-20T08:10:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:56.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:56.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:56.424 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:56.264+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: osdmap e621: 8 total, 8 up, 8 in 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:56.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:56.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:56.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:56.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:56 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:56.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.301+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.314+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.328+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.354+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.381+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.396+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.423+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.438+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.452+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.479+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.493+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.520+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.534+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.622+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.637+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.664+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.678+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.803 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.718+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.760+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.775+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.789+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:56.804 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.802+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.054 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:56.980+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:57.054 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:56.980+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:57.054 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:56.980+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.815+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.827+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.867+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.925+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.939+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.055 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.984+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:56.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.026+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.040+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.056 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.271 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:57.016+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:57.275 INFO:tasks.rados.rados.0.smithi118.stdout:2746: done (0 left) 2024-03-20T08:10:57.275 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 290 2024-03-20T08:10:57.275 INFO:tasks.rados.rados.0.smithi118.stdout:2747: seq_num 837 ranges {0=425984,425984=655360,1081344=729088,1810432=221184} 2024-03-20T08:10:57.278 INFO:tasks.rados.rados.0.smithi118.stdout:2747: writing smithi11881747-31 from 0 to 425984 tid 1 2024-03-20T08:10:57.282 INFO:tasks.rados.rados.0.smithi118.stdout:2747: writing smithi11881747-31 from 425984 to 1081344 tid 2 2024-03-20T08:10:57.286 INFO:tasks.rados.rados.0.smithi118.stdout:2747: writing smithi11881747-31 from 1081344 to 1810432 tid 3 2024-03-20T08:10:57.288 INFO:tasks.rados.rados.0.smithi118.stdout:2747: writing smithi11881747-31 from 1810432 to 2031616 tid 4 2024-03-20T08:10:57.288 INFO:tasks.rados.rados.0.smithi118.stdout:2748: delete oid 49 current snap is 290 2024-03-20T08:10:57.288 INFO:tasks.rados.rados.0.smithi118.stdout:2747: finishing write tid 1 to smithi11881747-31 2024-03-20T08:10:57.288 INFO:tasks.rados.rados.0.smithi118.stdout:2747: finishing write tid 2 to smithi11881747-31 2024-03-20T08:10:57.290 INFO:tasks.rados.rados.0.smithi118.stdout:2748: done (1 left) 2024-03-20T08:10:57.290 INFO:tasks.rados.rados.0.smithi118.stdout:2749: delete oid 34 current snap is 290 2024-03-20T08:10:57.294 INFO:tasks.rados.rados.0.smithi118.stdout:2747: finishing write tid 3 to smithi11881747-31 2024-03-20T08:10:57.294 INFO:tasks.rados.rados.0.smithi118.stdout:2747: finishing write tid 4 to smithi11881747-31 2024-03-20T08:10:57.294 INFO:tasks.rados.rados.0.smithi118.stdout:2747: finishing write tid 5 to smithi11881747-31 2024-03-20T08:10:57.294 INFO:tasks.rados.rados.0.smithi118.stdout:2749: done (1 left) 2024-03-20T08:10:57.294 INFO:tasks.rados.rados.0.smithi118.stdout:2750: snap_remove snap 288 2024-03-20T08:10:57.304 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:57.281+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:57.304 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:57.283+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: osdmap e622: 8 total, 8 up, 8 in 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:57.305 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:57.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:57.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:57.306 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-mon[87080]: pgmap v613: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 994 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 767 B/s rd, 2.1 MiB/s wr, 5 op/s 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.069+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.128+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.171+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.306 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.185+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.226+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.240+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.254+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.267+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.280+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.288+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.307 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.305+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.570 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.376+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.390+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.406+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.438+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.467+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.509+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.571 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.572 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.540+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.572 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.572 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:57.409+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:57.572 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:57.411+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:57.616 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:57.278+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:57.616 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:57.279+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:57.616 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:57.279+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:57.616 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:57.407+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:57.616 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:57.407+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:57.616 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:57.407+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:57.616 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:57.407+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:57.616 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:57.407+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:57.617 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:57.277+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:57.617 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:57.277+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:57.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:57.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: osdmap e622: 8 total, 8 up, 8 in 2024-03-20T08:10:57.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:57.617 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[93121]: pgmap v613: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 994 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 767 B/s rd, 2.1 MiB/s wr, 5 op/s 2024-03-20T08:10:57.618 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: osdmap e622: 8 total, 8 up, 8 in 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:57.619 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:57.620 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-mon[95692]: pgmap v613: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 994 MiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 767 B/s rd, 2.1 MiB/s wr, 5 op/s 2024-03-20T08:10:57.620 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:57.273+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:57.823 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.570+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.586+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.602+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.618+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.633+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.684+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.730+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.745+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.761+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:57.824 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.808+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.090 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.823+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.090 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.090 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.090 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.869+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.090 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.090 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.945+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.975+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:57.991+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.007+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.024+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.042+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.058+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.283 INFO:tasks.rados.rados.0.smithi118.stdout:2750: done (1 left) 2024-03-20T08:10:58.283 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 290 2024-03-20T08:10:58.283 INFO:tasks.rados.rados.0.smithi118.stdout:2751: seq_num 840 ranges {1982464=475136,2457600=712704,3170304=499712,3670016=393216} 2024-03-20T08:10:58.296 INFO:tasks.rados.rados.0.smithi118.stdout:2751: writing smithi11881747-43 from 1982464 to 2457600 tid 1 2024-03-20T08:10:58.301 INFO:tasks.rados.rados.0.smithi118.stdout:2751: writing smithi11881747-43 from 2457600 to 3170304 tid 2 2024-03-20T08:10:58.305 INFO:tasks.rados.rados.0.smithi118.stdout:2751: writing smithi11881747-43 from 3170304 to 3670016 tid 3 2024-03-20T08:10:58.310 INFO:tasks.rados.rados.0.smithi118.stdout:2751: writing smithi11881747-43 from 3670016 to 4063232 tid 4 2024-03-20T08:10:58.310 INFO:tasks.rados.rados.0.smithi118.stdout:2747: finishing write tid 6 to smithi11881747-31 2024-03-20T08:10:58.310 INFO:tasks.rados.rados.0.smithi118.stdout:2747: oid 31 updating version 0 to 44216 2024-03-20T08:10:58.310 INFO:tasks.rados.rados.0.smithi118.stdout:2747: oid 31 updating version 44216 to 44218 2024-03-20T08:10:58.310 INFO:tasks.rados.rados.0.smithi118.stdout:2747: oid 31 updating version 44218 to 44219 2024-03-20T08:10:58.310 INFO:tasks.rados.rados.0.smithi118.stdout:2747: oid 31 version 44219 is already newer than 44217 2024-03-20T08:10:58.310 INFO:tasks.rados.rados.0.smithi118.stdout:2747: oid 31 version 44219 is already newer than 44215 2024-03-20T08:10:58.310 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 44219 (ObjNum 837 snap 290 seq_num 837) dirty exists 2024-03-20T08:10:58.311 INFO:tasks.rados.rados.0.smithi118.stdout:2747: left oid 31 (ObjNum 837 snap 290 seq_num 837) 2024-03-20T08:10:58.311 INFO:tasks.rados.rados.0.smithi118.stdout:2752: read oid 22 snap -1 2024-03-20T08:10:58.311 INFO:tasks.rados.rados.0.smithi118.stdout:2752: expect (ObjNum 828 snap 285 seq_num 828) 2024-03-20T08:10:58.311 INFO:tasks.rados.rados.0.smithi118.stdout:2747: done (2 left) 2024-03-20T08:10:58.311 INFO:tasks.rados.rados.0.smithi118.stdout:2753: rmattr oid 41 current snap is 290 2024-03-20T08:10:58.311 INFO:tasks.rados.rados.0.smithi118.stdout:2753: done (2 left) 2024-03-20T08:10:58.311 INFO:tasks.rados.rados.0.smithi118.stdout:2754: snap_remove snap 209 2024-03-20T08:10:58.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:58.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:58.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:58.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:58.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: osdmap e623: 8 total, 8 up, 8 in 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:58.344 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:58.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.345 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.125+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.161+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.180+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.196+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.230+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.292+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.292+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.297+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.346 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.328+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.347 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.347 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:58.292+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:58.347 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:58.293+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:58.595 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:58.507+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:58.595 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:58.508+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:10:58.595 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:10:58.508+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:58.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.369+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.384+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.428+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.444+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.470+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.498+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.531+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.564+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.596 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:58.293+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:58.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:58.293+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:58.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:10:58.281+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:58.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:58.284+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:58.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:58.508+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:58.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:58.508+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:58.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:58.508+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:58.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:58.509+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:58.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:58.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:58.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: osdmap e623: 8 total, 8 up, 8 in 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:58.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: osdmap e623: 8 total, 8 up, 8 in 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:58.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:58.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:58.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:58.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:58.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:58.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:58.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.622+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.635+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.854 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.708+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.740+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.769+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.786+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.801+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.828+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:58.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.854+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.881+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.909+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.964+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:58.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.018+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.089+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.103+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.117 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.290 INFO:tasks.rados.rados.0.smithi118.stdout:2751: finishing write tid 1 to smithi11881747-43 2024-03-20T08:10:59.290 INFO:tasks.rados.rados.0.smithi118.stdout:2751: finishing write tid 2 to smithi11881747-43 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: finishing write tid 3 to smithi11881747-43 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: finishing write tid 4 to smithi11881747-43 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: finishing write tid 5 to smithi11881747-43 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: finishing write tid 6 to smithi11881747-43 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: oid 43 updating version 0 to 44222 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: oid 43 updating version 44222 to 44224 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: oid 43 version 44224 is already newer than 44221 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: oid 43 version 44224 is already newer than 44223 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: oid 43 version 44224 is already newer than 44220 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 44224 (ObjNum 840 snap 290 seq_num 840) dirty exists 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2751: left oid 43 (ObjNum 840 snap 290 seq_num 840) 2024-03-20T08:10:59.291 INFO:tasks.rados.rados.0.smithi118.stdout:2752: expect (ObjNum 828 snap 285 seq_num 828) 2024-03-20T08:10:59.362 INFO:tasks.rados.rados.0.smithi118.stdout:2751: done (2 left) 2024-03-20T08:10:59.362 INFO:tasks.rados.rados.0.smithi118.stdout:2752: done (1 left) 2024-03-20T08:10:59.363 INFO:tasks.rados.rados.0.smithi118.stdout:2754: done (0 left) 2024-03-20T08:10:59.363 INFO:tasks.rados.rados.0.smithi118.stdout:2755: snap_create 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: pgmap v615: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 994 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 950 KiB/s wr, 10 op/s 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: osdmap e624: 8 total, 8 up, 8 in 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:59.374 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:59.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:59.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:59.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:59.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:59.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:59.375 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:59.375 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:59.290+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:59.375 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:10:59.290+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.145+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.159+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.203+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.217+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.244+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.283+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.376 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.293+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:59.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.299+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.313+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.325+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.337+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.362+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.377 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.425+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.478+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.490+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.528+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.541+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.567+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.592+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.604+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.616+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.630+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:59.290+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:10:59.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:59.290+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:59.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:10:59.291+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:59.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:59.286+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:59.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:59.290+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:59.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:59.290+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:10:59.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:59.290+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:59.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:10:59.290+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:59.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:59.287+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:10:59.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:10:59.287+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:59.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: pgmap v615: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 994 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 950 KiB/s wr, 10 op/s 2024-03-20T08:10:59.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:59.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: osdmap e624: 8 total, 8 up, 8 in 2024-03-20T08:10:59.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:59.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:59.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:59.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: pgmap v615: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 994 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 2.5 MiB/s rd, 950 KiB/s wr, 10 op/s 2024-03-20T08:10:59.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: osdmap e624: 8 total, 8 up, 8 in 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:10:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:10:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:10:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:10:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:10:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:10:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:10:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:10:59.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.656+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.693+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.718+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.757+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.813+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.828+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.842+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.872+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:10:59.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.884+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.134 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.898+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.924+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.936+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.948+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.960+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.973+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:10:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.985+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:10:59.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.024+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.085+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.110+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.134+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.293 INFO:tasks.rados.rados.0.smithi118.stdout:2755: done (0 left) 2024-03-20T08:11:00.293 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 291 2024-03-20T08:11:00.293 INFO:tasks.rados.rados.0.smithi118.stdout:2756: seq_num 841 ranges {2916352=491520,3407872=598016,4005888=704512,4710400=303104} 2024-03-20T08:11:00.312 INFO:tasks.rados.rados.0.smithi118.stdout:2756: writing smithi11881747-20 from 2916352 to 3407872 tid 1 2024-03-20T08:11:00.316 INFO:tasks.rados.rados.0.smithi118.stdout:2756: writing smithi11881747-20 from 3407872 to 4005888 tid 2 2024-03-20T08:11:00.321 INFO:tasks.rados.rados.0.smithi118.stdout:2756: writing smithi11881747-20 from 4005888 to 4710400 tid 3 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:2756: writing smithi11881747-20 from 4710400 to 5013504 tid 4 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:2757: read oid 9 snap 286 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:2757: expect (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:2758: read oid 30 snap -1 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:2758: expect (ObjNum 830 snap 285 seq_num 830) 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:2759: rollback oid 50 current snap is 291 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 50 to 279 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 291 2024-03-20T08:11:00.324 INFO:tasks.rados.rados.0.smithi118.stdout:2760: seq_num 842 ranges {4931584=507904,5439488=475136,5914624=516096,6430720=499712,6930432=114688} 2024-03-20T08:11:00.356 INFO:tasks.rados.rados.0.smithi118.stdout:2760: writing smithi11881747-24 from 4931584 to 5439488 tid 1 2024-03-20T08:11:00.359 INFO:tasks.rados.rados.0.smithi118.stdout:2760: writing smithi11881747-24 from 5439488 to 5914624 tid 2 2024-03-20T08:11:00.362 INFO:tasks.rados.rados.0.smithi118.stdout:2760: writing smithi11881747-24 from 5914624 to 6430720 tid 3 2024-03-20T08:11:00.366 INFO:tasks.rados.rados.0.smithi118.stdout:2760: writing smithi11881747-24 from 6430720 to 6930432 tid 4 2024-03-20T08:11:00.367 INFO:tasks.rados.rados.0.smithi118.stdout:2760: writing smithi11881747-24 from 6930432 to 7045120 tid 5 2024-03-20T08:11:00.367 INFO:tasks.rados.rados.0.smithi118.stdout:2761: snap_remove snap 261 2024-03-20T08:11:00.385 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:00.209+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:00.385 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:00.210+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:00.385 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:00.292+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:00.385 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:00.293+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:00.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: osdmap e625: 8 total, 8 up, 8 in 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:00.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.149+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.163+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.207+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.264+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.294+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.294+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:00.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.305+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.388 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.356+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:00.421+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:00.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:00.421+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:00.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:00.422+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:00.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.408+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.452+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.465+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.479+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.493+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.519+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.532+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.560+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.573+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.600+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.627+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:00.319+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:00.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:00.290+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:00.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:00.291+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:00.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:00.208+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:00.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:00.320+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:00.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:00.419+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:00.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:00.420+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:00.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:00.420+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:00.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:00.420+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:00.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:00.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: osdmap e625: 8 total, 8 up, 8 in 2024-03-20T08:11:00.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:00.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:00.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:00.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:00.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:00.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:00.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:00.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:00.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: osdmap e625: 8 total, 8 up, 8 in 2024-03-20T08:11:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:00.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:00.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:00.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:00.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:00.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.655+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.670+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.683+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.892 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.697+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.711+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.725+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.752+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.766+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.780+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.792+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.806+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.893 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.833+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.894 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.894 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.894 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.878+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:00.894 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.143 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.144 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.144 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.144 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.144 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.144 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.977+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.144 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:00.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.020+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.033+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.047+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.145 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.298 INFO:tasks.rados.rados.0.smithi118.stdout:2761: done (5 left) 2024-03-20T08:11:01.298 INFO:tasks.rados.rados.0.smithi118.stdout:2762: rollback oid 18 current snap is 291 2024-03-20T08:11:01.298 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 18 to 244 2024-03-20T08:11:01.298 INFO:tasks.rados.rados.0.smithi118.stdout:2756: finishing write tid 1 to smithi11881747-20 2024-03-20T08:11:01.298 INFO:tasks.rados.rados.0.smithi118.stdout:2756: finishing write tid 2 to smithi11881747-20 2024-03-20T08:11:01.298 INFO:tasks.rados.rados.0.smithi118.stdout:2756: finishing write tid 3 to smithi11881747-20 2024-03-20T08:11:01.298 INFO:tasks.rados.rados.0.smithi118.stdout:2756: finishing write tid 4 to smithi11881747-20 2024-03-20T08:11:01.298 INFO:tasks.rados.rados.0.smithi118.stdout:2757: expect (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:11:01.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:01.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:01.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:01.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: pgmap v618: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 998 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 4.8 MiB/s rd, 999 KiB/s wr, 11 op/s 2024-03-20T08:11:01.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:01.394 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: osdmap e626: 8 total, 8 up, 8 in 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:01.395 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.143+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.186+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.201+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.228+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.242+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.256+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.284+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.396 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.300+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.301+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.345+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.374+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:01.303+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:01.397 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:01.303+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:01.424 INFO:tasks.rados.rados.0.smithi118.stdout:2756: finishing write tid 5 to smithi11881747-20 2024-03-20T08:11:01.424 INFO:tasks.rados.rados.0.smithi118.stdout:2757: done (5 left) 2024-03-20T08:11:01.424 INFO:tasks.rados.rados.0.smithi118.stdout:2763: delete oid 13 current snap is 291 2024-03-20T08:11:01.424 INFO:tasks.rados.rados.0.smithi118.stdout:2756: finishing write tid 6 to smithi11881747-20 2024-03-20T08:11:01.424 INFO:tasks.rados.rados.0.smithi118.stdout:2756: oid 20 updating version 0 to 34052 2024-03-20T08:11:01.425 INFO:tasks.rados.rados.0.smithi118.stdout:2756: oid 20 updating version 34052 to 34053 2024-03-20T08:11:01.425 INFO:tasks.rados.rados.0.smithi118.stdout:2756: oid 20 version 34053 is already newer than 34051 2024-03-20T08:11:01.425 INFO:tasks.rados.rados.0.smithi118.stdout:2756: oid 20 version 34053 is already newer than 34050 2024-03-20T08:11:01.425 INFO:tasks.rados.rados.0.smithi118.stdout:2756: oid 20 version 34053 is already newer than 34049 2024-03-20T08:11:01.425 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 34053 (ObjNum 841 snap 291 seq_num 841) dirty exists 2024-03-20T08:11:01.425 INFO:tasks.rados.rados.0.smithi118.stdout:2756: left oid 20 (ObjNum 841 snap 291 seq_num 841) 2024-03-20T08:11:01.425 INFO:tasks.rados.rados.0.smithi118.stdout:2758: expect (ObjNum 830 snap 285 seq_num 830) 2024-03-20T08:11:01.657 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.658 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.417+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.658 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.431+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.658 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.447+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.658 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.658 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.658 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.658 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.562+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.575+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.614+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.627+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.643+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:01.299+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:01.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:01.299+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:01.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:01.300+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:01.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:01.302+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:01.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:01.298+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:01.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:01.298+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:01.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:01.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:01.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: pgmap v618: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 998 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 4.8 MiB/s rd, 999 KiB/s wr, 11 op/s 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: osdmap e626: 8 total, 8 up, 8 in 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:01.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: pgmap v618: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 998 MiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 4.8 MiB/s rd, 999 KiB/s wr, 11 op/s 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: osdmap e626: 8 total, 8 up, 8 in 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:2759: finishing rollback tid 1 to smithi11881747-50 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:2759: finishing rollback tid 2 to smithi11881747-50 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 55367 (ObjNum 663 snap 216 seq_num 663) dirty exists 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:2760: finishing write tid 1 to smithi11881747-24 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:2760: finishing write tid 2 to smithi11881747-24 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:2756: done (5 left) 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:2758: done (4 left) 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:2759: done (3 left) 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:2763: done (2 left) 2024-03-20T08:11:01.688 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 291 2024-03-20T08:11:01.689 INFO:tasks.rados.rados.0.smithi118.stdout:2760: finishing write tid 3 to smithi11881747-24 2024-03-20T08:11:01.689 INFO:tasks.rados.rados.0.smithi118.stdout:2760: finishing write tid 4 to smithi11881747-24 2024-03-20T08:11:01.689 INFO:tasks.rados.rados.0.smithi118.stdout:2764: seq_num 844 ranges {0=540672,540672=696320,1236992=475136,1712128=434176} 2024-03-20T08:11:01.691 INFO:tasks.rados.rados.0.smithi118.stdout:2764: writing smithi11881747-7 from 0 to 540672 tid 1 2024-03-20T08:11:01.695 INFO:tasks.rados.rados.0.smithi118.stdout:2764: writing smithi11881747-7 from 540672 to 1236992 tid 2 2024-03-20T08:11:01.698 INFO:tasks.rados.rados.0.smithi118.stdout:2764: writing smithi11881747-7 from 1236992 to 1712128 tid 3 2024-03-20T08:11:01.701 INFO:tasks.rados.rados.0.smithi118.stdout:2764: writing smithi11881747-7 from 1712128 to 2146304 tid 4 2024-03-20T08:11:01.702 INFO:tasks.rados.rados.0.smithi118.stdout:2765: snap_remove snap 284 2024-03-20T08:11:01.917 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:01.697+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:01.917 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:01.697+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:01.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.671+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.687+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.703+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.717+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.918 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.744+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.757+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.784+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.796+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.832+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.844+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.894+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:01.919 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.918+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:01.677+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:02.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:01.677+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:02.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:01.677+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:02.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:01.677+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:02.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:01.678+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:02.179 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.931+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.179 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.943+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.969+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:01.995+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.008+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.020+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.180 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.070+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.082+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.113+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.127+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.155+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.181 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.305 INFO:tasks.rados.rados.0.smithi118.stdout:2765: done (3 left) 2024-03-20T08:11:02.305 INFO:tasks.rados.rados.0.smithi118.stdout:2766: read oid 27 snap -1 2024-03-20T08:11:02.305 INFO:tasks.rados.rados.0.smithi118.stdout:2766: expect deleted 2024-03-20T08:11:02.305 INFO:tasks.rados.rados.0.smithi118.stdout:2767: rollback oid 29 current snap is 291 2024-03-20T08:11:02.306 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 225 2024-03-20T08:11:02.306 INFO:tasks.rados.rados.0.smithi118.stdout:2760: finishing write tid 5 to smithi11881747-24 2024-03-20T08:11:02.306 INFO:tasks.rados.rados.0.smithi118.stdout:2760: finishing write tid 6 to smithi11881747-24 2024-03-20T08:11:02.306 INFO:tasks.rados.rados.0.smithi118.stdout:2760: finishing write tid 7 to smithi11881747-24 2024-03-20T08:11:02.306 INFO:tasks.rados.rados.0.smithi118.stdout:2760: oid 24 updating version 0 to 55376 2024-03-20T08:11:02.306 INFO:tasks.rados.rados.0.smithi118.stdout:2760: oid 24 updating version 55376 to 55380 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:2760: oid 24 version 55380 is already newer than 55375 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:2760: oid 24 version 55380 is already newer than 55377 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:2760: oid 24 updating version 55380 to 55381 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:2760: oid 24 updating version 55381 to 55382 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 55382 (ObjNum 842 snap 291 seq_num 842) dirty exists 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:2760: left oid 24 (ObjNum 842 snap 291 seq_num 842) 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:2762: finishing rollback tid 1 to smithi11881747-18 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 34037 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:11:02.307 INFO:tasks.rados.rados.0.smithi118.stdout:2764: finishing write tid 1 to smithi11881747-7 2024-03-20T08:11:02.308 INFO:tasks.rados.rados.0.smithi118.stdout:2764: finishing write tid 2 to smithi11881747-7 2024-03-20T08:11:02.308 INFO:tasks.rados.rados.0.smithi118.stdout:2764: finishing write tid 3 to smithi11881747-7 2024-03-20T08:11:02.308 INFO:tasks.rados.rados.0.smithi118.stdout:2764: finishing write tid 4 to smithi11881747-7 2024-03-20T08:11:02.308 INFO:tasks.rados.rados.0.smithi118.stdout:2764: finishing write tid 5 to smithi11881747-7 2024-03-20T08:11:02.308 INFO:tasks.rados.rados.0.smithi118.stdout:2764: finishing write tid 6 to smithi11881747-7 2024-03-20T08:11:02.308 INFO:tasks.rados.rados.0.smithi118.stdout:2764: oid 7 updating version 0 to 34025 2024-03-20T08:11:02.308 INFO:tasks.rados.rados.0.smithi118.stdout:2764: oid 7 version 34025 is already newer than 34024 2024-03-20T08:11:02.309 INFO:tasks.rados.rados.0.smithi118.stdout:2764: oid 7 updating version 34025 to 34026 2024-03-20T08:11:02.309 INFO:tasks.rados.rados.0.smithi118.stdout:2764: oid 7 version 34026 is already newer than 34023 2024-03-20T08:11:02.309 INFO:tasks.rados.rados.0.smithi118.stdout:2764: oid 7 version 34026 is already newer than 34022 2024-03-20T08:11:02.309 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 34026 (ObjNum 844 snap 291 seq_num 844) dirty exists 2024-03-20T08:11:02.309 INFO:tasks.rados.rados.0.smithi118.stdout:2764: left oid 7 (ObjNum 844 snap 291 seq_num 844) 2024-03-20T08:11:02.309 INFO:tasks.rados.rados.0.smithi118.stdout:2760: done (4 left) 2024-03-20T08:11:02.309 INFO:tasks.rados.rados.0.smithi118.stdout:2762: done (3 left) 2024-03-20T08:11:02.310 INFO:tasks.rados.rados.0.smithi118.stdout:2764: done (2 left) 2024-03-20T08:11:02.310 INFO:tasks.rados.rados.0.smithi118.stdout:2768: setattr oid 49 current snap is 291 2024-03-20T08:11:02.310 INFO:tasks.rados.rados.0.smithi118.stdout:2769: copy_from oid 50 from oid 39 current snap is 291 2024-03-20T08:11:02.311 INFO:tasks.rados.rados.0.smithi118.stdout:2770: rmattr oid 46 current snap is 291 2024-03-20T08:11:02.311 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 291 2024-03-20T08:11:02.311 INFO:tasks.rados.rados.0.smithi118.stdout:2771: seq_num 845 ranges {7045120=557056,7602176=581632,8183808=688128,8871936=335872} 2024-03-20T08:11:02.349 INFO:tasks.rados.rados.0.smithi118.stdout:2771: writing smithi11881747-44 from 7045120 to 7602176 tid 1 2024-03-20T08:11:02.353 INFO:tasks.rados.rados.0.smithi118.stdout:2771: writing smithi11881747-44 from 7602176 to 8183808 tid 2 2024-03-20T08:11:02.357 INFO:tasks.rados.rados.0.smithi118.stdout:2771: writing smithi11881747-44 from 8183808 to 8871936 tid 3 2024-03-20T08:11:02.360 INFO:tasks.rados.rados.0.smithi118.stdout:2771: writing smithi11881747-44 from 8871936 to 9207808 tid 4 2024-03-20T08:11:02.360 INFO:tasks.rados.rados.0.smithi118.stdout:2772: delete oid 18 current snap is 291 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:2767: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 34054 (ObjNum 161 snap 0 seq_num 0) dirty exists 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:2767: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 45588 (ObjNum 797 snap 275 seq_num 797) dirty exists 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:2767: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33982 (ObjNum 659 snap 215 seq_num 659) dirty exists 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:2769: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:2769: finishing copy_from to smithi11881747-50 2024-03-20T08:11:02.361 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 34045 (ObjNum 821 snap 281 seq_num 821) dirty exists 2024-03-20T08:11:02.362 INFO:tasks.rados.rados.0.smithi118.stdout:2766: done (6 left) 2024-03-20T08:11:02.362 INFO:tasks.rados.rados.0.smithi118.stdout:2767: done (5 left) 2024-03-20T08:11:02.362 INFO:tasks.rados.rados.0.smithi118.stdout:2768: done (4 left) 2024-03-20T08:11:02.362 INFO:tasks.rados.rados.0.smithi118.stdout:2769: done (3 left) 2024-03-20T08:11:02.363 INFO:tasks.rados.rados.0.smithi118.stdout:2770: done (2 left) 2024-03-20T08:11:02.363 INFO:tasks.rados.rados.0.smithi118.stdout:2772: done (1 left) 2024-03-20T08:11:02.363 INFO:tasks.rados.rados.0.smithi118.stdout:2773: snap_remove snap 225 2024-03-20T08:11:02.442 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:02.306+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:02.442 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:02.307+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:02.443 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:11:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: osdmap e627: 8 total, 8 up, 8 in 2024-03-20T08:11:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:02.443 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:02.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.445 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.217+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.245+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.271+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.284+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.298+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.305+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.305+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.318+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.336+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.351+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.388+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.402+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.416+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.428+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.447 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:02.314+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:02.447 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:02.315+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:02.447 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:02.317+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:02.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:02.310+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:02.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:02.320+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:02.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:02.308+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:02.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:02.312+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:02.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:02.314+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:02.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:02.315+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:02.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: osdmap e627: 8 total, 8 up, 8 in 2024-03-20T08:11:02.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:02.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:02.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:02.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:02.675 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:02.311+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:02.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: osdmap e627: 8 total, 8 up, 8 in 2024-03-20T08:11:02.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:02.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 151=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:02.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:02.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:02.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.678 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:02.303+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:02.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:02.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:02.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:02.705 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.458+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.472+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.500+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.530+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.544+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.557+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.583+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.706 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.611+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.624+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.651+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.707 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.705+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.966 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.732+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.746+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.762+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.810+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.824+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.838+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.850+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.877+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.967 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.907+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.968 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.968 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.968 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.951+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:02.968 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.966+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:02.995+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.008+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.046+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.058+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.083+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.095+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.119+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.225 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.182+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.195+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.226 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.221+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:2771: finishing write tid 1 to smithi11881747-44 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: finishing write tid 2 to smithi11881747-44 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: finishing write tid 3 to smithi11881747-44 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: finishing write tid 4 to smithi11881747-44 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: finishing write tid 5 to smithi11881747-44 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: finishing write tid 6 to smithi11881747-44 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: oid 44 updating version 0 to 55961 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: oid 44 updating version 55961 to 55962 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: oid 44 version 55962 is already newer than 55960 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: oid 44 version 55962 is already newer than 55959 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: oid 44 updating version 55962 to 55963 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 55963 (ObjNum 845 snap 291 seq_num 845) dirty exists 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: left oid 44 (ObjNum 845 snap 291 seq_num 845) 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2771: done (1 left) 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2773: done (0 left) 2024-03-20T08:11:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:2774: read oid 29 snap -1 2024-03-20T08:11:03.310 INFO:tasks.rados.rados.0.smithi118.stdout:2774: expect (ObjNum 659 snap 215 seq_num 659) 2024-03-20T08:11:03.310 INFO:tasks.rados.rados.0.smithi118.stdout:2775: read oid 3 snap -1 2024-03-20T08:11:03.310 INFO:tasks.rados.rados.0.smithi118.stdout:2775: expect (ObjNum 740 snap 253 seq_num 740) 2024-03-20T08:11:03.310 INFO:tasks.rados.rados.0.smithi118.stdout:2776: snap_remove snap 285 2024-03-20T08:11:03.484 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: pgmap v621: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 998 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 4.3 MiB/s rd, 1004 KiB/s wr, 5 op/s 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: osdmap e628: 8 total, 8 up, 8 in 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.485 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:03.486 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.487 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:03.311+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:03.487 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:03.311+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:03.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.235+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.266+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.280+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.487 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.309+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.352+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.377+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.415+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.430+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.456+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.488 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.470+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.577 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:03.307+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:03.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:03.310+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:03.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:03.310+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:03.577 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:03.311+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:03.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:03.311+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:03.577 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:03.316+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:03.578 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:03.316+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:03.578 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:03.322+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:03.578 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:03.322+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:03.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: pgmap v621: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 998 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 4.3 MiB/s rd, 1004 KiB/s wr, 5 op/s 2024-03-20T08:11:03.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: osdmap e628: 8 total, 8 up, 8 in 2024-03-20T08:11:03.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:03.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: pgmap v621: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 998 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 4.3 MiB/s rd, 1004 KiB/s wr, 5 op/s 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: osdmap e628: 8 total, 8 up, 8 in 2024-03-20T08:11:03.580 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:03.581 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.582 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:03.582 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.582 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:03.582 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.582 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:03.582 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:03.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.483+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.530+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.544+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.558+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.745 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.615+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.627+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.641+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.746 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.693+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.718+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.730+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.747 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.840 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:11:03] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:11:03.994 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:03.839+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:03.994 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:03.840+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:03.994 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.779+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.791+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.803+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.816+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.843+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.874+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.899+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.912+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.925+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.980+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:03.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:03.993+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:03.840+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:04.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:03.840+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:04.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.019+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.046+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.253 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.085+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.097+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.109+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.122+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.135+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.148+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.160+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.173+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.186+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.214+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.254 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.239+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.255 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.313 INFO:tasks.rados.rados.0.smithi118.stdout:2776: done (2 left) 2024-03-20T08:11:04.313 INFO:tasks.rados.rados.0.smithi118.stdout:2777: copy_from oid 8 from oid 25 current snap is 291 2024-03-20T08:11:04.313 INFO:tasks.rados.rados.0.smithi118.stdout:2774: expect (ObjNum 659 snap 215 seq_num 659) 2024-03-20T08:11:04.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:04.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: osdmap e629: 8 total, 8 up, 8 in 2024-03-20T08:11:04.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:04.503 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:04.504 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-mon[87080]: pgmap v624: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1001 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 3.6 MiB/s rd, 524 KiB/s wr, 4 op/s 2024-03-20T08:11:04.504 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:04.340+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:04.505 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:04.341+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:04.505 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:04.341+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:04.505 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:04.312+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:04.505 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:04.312+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:04.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.505 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.292+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.314+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.314+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.323+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.340+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.381+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.396+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.506 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.475+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.507 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.489+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:04.312+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:04.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:04.316+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:04.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:04.317+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:04.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:04.343+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:04.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:04.343+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:04.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:04.343+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:04.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:04.344+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:04.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:04.344+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: osdmap e629: 8 total, 8 up, 8 in 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:04.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[93121]: pgmap v624: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1001 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 3.6 MiB/s rd, 524 KiB/s wr, 4 op/s 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: osdmap e629: 8 total, 8 up, 8 in 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:04.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:04.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:04.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:04.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:04.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:04.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:04.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:04 smithi118 ceph-mon[95692]: pgmap v624: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1001 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 3.6 MiB/s rd, 524 KiB/s wr, 4 op/s 2024-03-20T08:11:04.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.502+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.515+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.568+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.581+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.608+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.621+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.647+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.762 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.659+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.672+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.686+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.700+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.730+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.746+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.763 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.760+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:2775: expect (ObjNum 740 snap 253 seq_num 740) 2024-03-20T08:11:05.020 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.775+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.020 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.790+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.020 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.804+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.819+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.834+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.848+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.862+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.877+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.893+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.021 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:04.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.022 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.005+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.022 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.276 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.048+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.063+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.090+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.103+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.161+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.175+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.190+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.277 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.204+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.218+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.245+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.261+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:05.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:05.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: osdmap e630: 8 total, 8 up, 8 in 2024-03-20T08:11:05.528 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:05.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:05.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:05.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:05.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:05.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:05.529 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:05.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:05.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:05.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:05.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:05.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:05.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: Upgrade: Updating prometheus.a 2024-03-20T08:11:05.530 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-mon[87080]: Deploying daemon prometheus.a on smithi196 2024-03-20T08:11:05.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.289+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.362+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.388+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.428+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.441+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.453+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.533 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.533 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.476+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.533 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.489+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.533 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.533 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.514+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.533 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: osdmap e630: 8 total, 8 up, 8 in 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:05.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: Upgrade: Updating prometheus.a 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[93121]: Deploying daemon prometheus.a on smithi196 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: osdmap e630: 8 total, 8 up, 8 in 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:05.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:05.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:05.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:05.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:05.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:05.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:05.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:05.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: Upgrade: Updating prometheus.a 2024-03-20T08:11:05.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:05 smithi118 ceph-mon[95692]: Deploying daemon prometheus.a on smithi196 2024-03-20T08:11:05.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.540+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.590+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.788 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.615+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.627+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.640+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.665+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.709+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.721+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.733+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.745+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.757+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.770+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.789 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:05.901 INFO:tasks.rados.rados.0.smithi118.stdout:2777: finishing copy_from racing read to smithi11881747-8 2024-03-20T08:11:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:2777: finishing copy_from to smithi11881747-8 2024-03-20T08:11:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34066 (ObjNum 826 snap 283 seq_num 826) dirty exists 2024-03-20T08:11:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:2774: done (2 left) 2024-03-20T08:11:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:2775: done (1 left) 2024-03-20T08:11:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:2777: done (0 left) 2024-03-20T08:11:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:2778: copy_from oid 2 from oid 19 current snap is 291 2024-03-20T08:11:05.902 INFO:tasks.rados.rados.0.smithi118.stdout:2779: snap_create 2024-03-20T08:11:05.913 INFO:tasks.rados.rados.0.smithi118.stdout:2778: finishing copy_from racing read to smithi11881747-2 2024-03-20T08:11:05.913 INFO:tasks.rados.rados.0.smithi118.stdout:2778: finishing copy_from to smithi11881747-2 2024-03-20T08:11:05.914 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 7092 (ObjNum 815 snap 279 seq_num 815) dirty exists 2024-03-20T08:11:06.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.798+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.041 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.813+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.828+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.841+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.879+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.890+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.903+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.917+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.974+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:05.989+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.042 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.001+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.043 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.043 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.027+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.043 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.040+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.092+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.293 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.130+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.168+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.180+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.207+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.219+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.231+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.243+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.255+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.267+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.294 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.292+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.318 INFO:tasks.rados.rados.0.smithi118.stdout:2778: done (1 left) 2024-03-20T08:11:06.318 INFO:tasks.rados.rados.0.smithi118.stdout:2779: done (0 left) 2024-03-20T08:11:06.318 INFO:tasks.rados.rados.0.smithi118.stdout:2780: snap_remove snap 244 2024-03-20T08:11:06.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-mon[87080]: pgmap v626: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1009 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 12 op/s 2024-03-20T08:11:06.551 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:06.322+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:06.551 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:06.322+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:06.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.306+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.319+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:06.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.337+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.365+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.377+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.401+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.427+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.440+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.452+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.465+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.477+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.489+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.512+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-mon[93121]: pgmap v626: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1009 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 12 op/s 2024-03-20T08:11:06.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-mon[95692]: pgmap v626: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1009 MiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 12 op/s 2024-03-20T08:11:06.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:06.321+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:06.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:06.321+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:06.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:06.321+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:06.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:06.317+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:06.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:06.317+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:06.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:06.318+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:06.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:06.318+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:06.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:06.318+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.589+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.639+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.809 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.704+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.717+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.730+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.767+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:06.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.845+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.862+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.877+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.893+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.908+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.937+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.950+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.979+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:06.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.069 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.017+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.029+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.070 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:07.003+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:07.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:06.908+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:07.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:06.908+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:07.324 INFO:tasks.rados.rados.0.smithi118.stdout:2780: done (0 left) 2024-03-20T08:11:07.325 INFO:tasks.rados.rados.0.smithi118.stdout:2781: snap_remove snap 274 2024-03-20T08:11:07.325 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:07.325 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:07.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:07.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:07.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: osdmap e631: 8 total, 8 up, 8 in 2024-03-20T08:11:07.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:07.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:07.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:07.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:07.326 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.079+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.092+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.156+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.327 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.234+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.259+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.284+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.297+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.309+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.328 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.325+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.578 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:07.362+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:07.579 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:07.364+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:07.579 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:07.328+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:07.579 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:07.329+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:07.579 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.331+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.331+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.343+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.358+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.375+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.416+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.429+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.442+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.455+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.481+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.494+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.507+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.580 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.520+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.581 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.581 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.581 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.565+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.581 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:07.328+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:07.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:07.329+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:07.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:07.325+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: osdmap e631: 8 total, 8 up, 8 in 2024-03-20T08:11:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:07.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:07.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:07.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:07.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:07.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:07.322+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:07.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:07.323+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:07.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:07.324+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:07.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:07.324+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: osdmap e631: 8 total, 8 up, 8 in 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:07.675 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:07.420+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:07.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.630+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.643+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.831 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.656+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.668+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.683+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.694+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.719+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.744+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.756+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.817+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:07.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.830+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.843+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.868+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.879+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.894+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.908+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.933+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.947+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.984+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:07.998+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.023+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.035+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.088 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.085+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.330 INFO:tasks.rados.rados.0.smithi118.stdout:2781: done (0 left) 2024-03-20T08:11:08.330 INFO:tasks.rados.rados.0.smithi118.stdout:2782: rollback oid 35 current snap is 292 2024-03-20T08:11:08.330 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 35 to 263 2024-03-20T08:11:08.330 INFO:tasks.rados.rados.0.smithi118.stdout:2783: snap_remove snap 267 2024-03-20T08:11:08.339 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:08.339 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: osdmap e632: 8 total, 8 up, 8 in 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:08.340 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:08.341 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:08.341 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-mon[87080]: pgmap v629: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1017 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 3.2 MiB/s wr, 12 op/s 2024-03-20T08:11:08.341 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:08.333+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:08.341 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:08.333+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:08.341 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:08.335+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:08.341 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:08.335+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.099+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.124+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.137+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.162+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.175+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.188+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.215+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.227+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.240+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.265+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.316+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.332+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:08.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.592 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:08.339+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:08.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.348+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.592 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.374+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.398+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.423+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.495+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.520+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.534+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.547+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.561+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.576+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:08.328+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:08.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:08.330+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:08.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:08.330+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:08.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:08.331+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:08.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:08.521+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:08.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:08.521+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:08.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:08.521+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:08.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:08.522+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:08.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:08.522+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:08.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:08.523+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:08.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:08.523+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:08.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:08.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:08.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:08.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:08.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:08.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: osdmap e632: 8 total, 8 up, 8 in 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:08.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[93121]: pgmap v629: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1017 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 3.2 MiB/s wr, 12 op/s 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: osdmap e632: 8 total, 8 up, 8 in 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:08.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:08.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:08.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:08 smithi118 ceph-mon[95692]: pgmap v629: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1017 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 3.2 MiB/s wr, 12 op/s 2024-03-20T08:11:08.855 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.615+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.630+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.658+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.701+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.740+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.767+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.782+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.856 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.796+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.823+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.857 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:08.857 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:08 smithi196 systemd[1]: Stopping Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.855+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.884+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.898+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.911+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.926+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.940+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.953+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.969+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.984+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:08.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.014+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.043+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.056+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.070+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.083+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.095+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.109 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.107+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.328 INFO:tasks.rados.rados.0.smithi118.stdout:2783: done (1 left) 2024-03-20T08:11:09.328 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 292 2024-03-20T08:11:09.329 INFO:tasks.rados.rados.0.smithi118.stdout:2784: seq_num 847 ranges {0=589824,589824=753664,1343488=655360,1998848=196608} 2024-03-20T08:11:09.331 INFO:tasks.rados.rados.0.smithi118.stdout:2784: writing smithi11881747-13 from 0 to 589824 tid 1 2024-03-20T08:11:09.337 INFO:tasks.rados.rados.0.smithi118.stdout:2784: writing smithi11881747-13 from 589824 to 1343488 tid 2 2024-03-20T08:11:09.341 INFO:tasks.rados.rados.0.smithi118.stdout:2784: writing smithi11881747-13 from 1343488 to 1998848 tid 3 2024-03-20T08:11:09.343 INFO:tasks.rados.rados.0.smithi118.stdout:2784: writing smithi11881747-13 from 1998848 to 2195456 tid 4 2024-03-20T08:11:09.343 INFO:tasks.rados.rados.0.smithi118.stdout:2785: setattr oid 41 current snap is 292 2024-03-20T08:11:09.343 INFO:tasks.rados.rados.0.smithi118.stdout:2782: finishing rollback tid 0 to smithi11881747-35 2024-03-20T08:11:09.343 INFO:tasks.rados.rados.0.smithi118.stdout:2782: finishing rollback tid 1 to smithi11881747-35 2024-03-20T08:11:09.343 INFO:tasks.rados.rados.0.smithi118.stdout:2782: finishing rollback tid 2 to smithi11881747-35 2024-03-20T08:11:09.343 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 45591 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:11:09.343 INFO:tasks.rados.rados.0.smithi118.stdout:2784: finishing write tid 1 to smithi11881747-13 2024-03-20T08:11:09.344 INFO:tasks.rados.rados.0.smithi118.stdout:2784: finishing write tid 2 to smithi11881747-13 2024-03-20T08:11:09.346 INFO:tasks.rados.rados.0.smithi118.stdout:2782: done (2 left) 2024-03-20T08:11:09.346 INFO:tasks.rados.rados.0.smithi118.stdout:2786: read oid 49 snap -1 2024-03-20T08:11:09.346 INFO:tasks.rados.rados.0.smithi118.stdout:2786: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:11:09.346 INFO:tasks.rados.rados.0.smithi118.stdout:2787: read oid 11 snap -1 2024-03-20T08:11:09.346 INFO:tasks.rados.rados.0.smithi118.stdout:2787: expect (ObjNum 704 snap 236 seq_num 704) 2024-03-20T08:11:09.346 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 292 2024-03-20T08:11:09.346 INFO:tasks.rados.rados.0.smithi118.stdout:2788: seq_num 848 ranges {7077888=606208,7684096=630784,8314880=458752,8773632=516096} 2024-03-20T08:11:09.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:09.335+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:09.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:09.335+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:09.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:09.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: osdmap e633: 8 total, 8 up, 8 in 2024-03-20T08:11:09.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:09.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:09.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:09.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:09.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:09.366 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:09.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:09.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:09.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:09.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:09.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:09.367 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.137Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-20T08:11:09.367 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-20T08:11:09.367 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-20T08:11:09.367 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-20T08:11:09.367 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-20T08:11:09.367 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-20T08:11:09.368 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-20T08:11:09.368 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-20T08:11:09.368 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-20T08:11:09.368 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-20T08:11:09.368 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-20T08:11:09.368 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[78104]: ts=2024-03-20T08:11:09.138Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-20T08:11:09.368 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 podman[120024]: 2024-03-20 08:11:09.149310167 +0000 UTC m=+0.141039373 container died 1fb280ede3dcb0830e57079bc46ab0ed065bf9dd1a3e466e3fb391bd7ac9ee21 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:09.368 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.133+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.147+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.164+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.177+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.203+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.369 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.218+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.247+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.262+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.278+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.314+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.327+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.333+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.333+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.365+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.389 INFO:tasks.rados.rados.0.smithi118.stdout:2788: writing smithi11881747-6 from 7077888 to 7684096 tid 1 2024-03-20T08:11:09.392 INFO:tasks.rados.rados.0.smithi118.stdout:2788: writing smithi11881747-6 from 7684096 to 8314880 tid 2 2024-03-20T08:11:09.395 INFO:tasks.rados.rados.0.smithi118.stdout:2788: writing smithi11881747-6 from 8314880 to 8773632 tid 3 2024-03-20T08:11:09.398 INFO:tasks.rados.rados.0.smithi118.stdout:2788: writing smithi11881747-6 from 8773632 to 9289728 tid 4 2024-03-20T08:11:09.398 INFO:tasks.rados.rados.0.smithi118.stdout:2789: snap_create 2024-03-20T08:11:09.398 INFO:tasks.rados.rados.0.smithi118.stdout:2784: finishing write tid 3 to smithi11881747-13 2024-03-20T08:11:09.398 INFO:tasks.rados.rados.0.smithi118.stdout:2784: finishing write tid 4 to smithi11881747-13 2024-03-20T08:11:09.399 INFO:tasks.rados.rados.0.smithi118.stdout:2786: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:11:09.402 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 33983 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-03-20T08:11:09.402 INFO:tasks.rados.rados.0.smithi118.stdout:2784: finishing write tid 5 to smithi11881747-13 2024-03-20T08:11:09.403 INFO:tasks.rados.rados.0.smithi118.stdout:2787: expect (ObjNum 704 snap 236 seq_num 704) 2024-03-20T08:11:09.476 INFO:tasks.rados.rados.0.smithi118.stdout:2784: finishing write tid 6 to smithi11881747-13 2024-03-20T08:11:09.476 INFO:tasks.rados.rados.0.smithi118.stdout:2784: oid 13 updating version 0 to 34072 2024-03-20T08:11:09.476 INFO:tasks.rados.rados.0.smithi118.stdout:2784: oid 13 updating version 34072 to 34074 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2784: oid 13 version 34074 is already newer than 34070 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2784: oid 13 version 34074 is already newer than 34071 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2784: oid 13 version 34074 is already newer than 34073 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 34074 (ObjNum 847 snap 292 seq_num 847) dirty exists 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2784: left oid 13 (ObjNum 847 snap 292 seq_num 847) 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: finishing write tid 1 to smithi11881747-6 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: finishing write tid 2 to smithi11881747-6 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: finishing write tid 3 to smithi11881747-6 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: finishing write tid 4 to smithi11881747-6 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: finishing write tid 5 to smithi11881747-6 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: finishing write tid 6 to smithi11881747-6 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: oid 6 updating version 0 to 45593 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: oid 6 version 45593 is already newer than 45592 2024-03-20T08:11:09.477 INFO:tasks.rados.rados.0.smithi118.stdout:2788: oid 6 updating version 45593 to 45596 2024-03-20T08:11:09.478 INFO:tasks.rados.rados.0.smithi118.stdout:2788: oid 6 version 45596 is already newer than 45595 2024-03-20T08:11:09.478 INFO:tasks.rados.rados.0.smithi118.stdout:2788: oid 6 version 45596 is already newer than 45594 2024-03-20T08:11:09.478 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 45596 (ObjNum 848 snap 292 seq_num 848) dirty exists 2024-03-20T08:11:09.478 INFO:tasks.rados.rados.0.smithi118.stdout:2788: left oid 6 (ObjNum 848 snap 292 seq_num 848) 2024-03-20T08:11:09.623 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:09.395+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:09.623 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:09.397+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:09.624 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 podman[120024]: 2024-03-20 08:11:09.547059317 +0000 UTC m=+0.538788517 container cleanup 1fb280ede3dcb0830e57079bc46ab0ed065bf9dd1a3e466e3fb391bd7ac9ee21 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:09.624 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 bash[120024]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a 2024-03-20T08:11:09.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.380+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.395+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.422+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.434+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.624 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.447+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.460+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.538+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.550+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.564+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.579+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.625 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:09.328+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:09.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:09.331+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:09.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:09.338+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:09.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:09.350+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:09.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:09.351+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:09.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:09.352+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:09.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:09.353+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:09.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:09.352+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:09.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: osdmap e633: 8 total, 8 up, 8 in 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:09.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: osdmap e633: 8 total, 8 up, 8 in 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:09.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:09.875 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:09 smithi196 podman[120043]: 2024-03-20 08:11:09.841638768 +0000 UTC m=+0.692749370 container remove 1fb280ede3dcb0830e57079bc46ab0ed065bf9dd1a3e466e3fb391bd7ac9ee21 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:09.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.669+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.682+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.696+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.709+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.723+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.736+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.748+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.774+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.798+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.822+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.835+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.848+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.862+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:09.876 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.875+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.127 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:10 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-20T08:11:10.127 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:10 smithi196 systemd[1]: Stopped Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:11:10.127 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:10 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service: Consumed 3.188s CPU time. 2024-03-20T08:11:10.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.905+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.127 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.934+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:09.987+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.002+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.015+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.128 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.043+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.058+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.088+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.101+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.129 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.343 INFO:tasks.rados.rados.0.smithi118.stdout:2784: done (5 left) 2024-03-20T08:11:10.343 INFO:tasks.rados.rados.0.smithi118.stdout:2785: done (4 left) 2024-03-20T08:11:10.343 INFO:tasks.rados.rados.0.smithi118.stdout:2786: done (3 left) 2024-03-20T08:11:10.343 INFO:tasks.rados.rados.0.smithi118.stdout:2787: done (2 left) 2024-03-20T08:11:10.343 INFO:tasks.rados.rados.0.smithi118.stdout:2788: done (1 left) 2024-03-20T08:11:10.343 INFO:tasks.rados.rados.0.smithi118.stdout:2789: done (0 left) 2024-03-20T08:11:10.343 INFO:tasks.rados.rados.0.smithi118.stdout:2790: setattr oid 17 current snap is 293 2024-03-20T08:11:10.344 INFO:tasks.rados.rados.0.smithi118.stdout:2791: read oid 6 snap -1 2024-03-20T08:11:10.344 INFO:tasks.rados.rados.0.smithi118.stdout:2791: expect (ObjNum 848 snap 292 seq_num 848) 2024-03-20T08:11:10.344 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 293 2024-03-20T08:11:10.345 INFO:tasks.rados.rados.0.smithi118.stdout:2792: seq_num 849 ranges {3751936=622592,4374528=516096,4890624=663552,5554176=425984} 2024-03-20T08:11:10.370 INFO:tasks.rados.rados.0.smithi118.stdout:2792: writing smithi11881747-26 from 3751936 to 4374528 tid 1 2024-03-20T08:11:10.374 INFO:tasks.rados.rados.0.smithi118.stdout:2792: writing smithi11881747-26 from 4374528 to 4890624 tid 2 2024-03-20T08:11:10.379 INFO:tasks.rados.rados.0.smithi118.stdout:2792: writing smithi11881747-26 from 4890624 to 5554176 tid 3 2024-03-20T08:11:10.382 INFO:tasks.rados.rados.0.smithi118.stdout:2792: writing smithi11881747-26 from 5554176 to 5980160 tid 4 2024-03-20T08:11:10.382 INFO:tasks.rados.rados.0.smithi118.stdout:2793: snap_remove snap 275 2024-03-20T08:11:10.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.154+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.167+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.181+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.208+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.222+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.236+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.250+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.264+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.277+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.304+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.392 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.335+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.393 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.343+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:10.393 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.357+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.393 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:10.348+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:10.394 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:10.349+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:10.394 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:10.350+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:10.395 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:10.350+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:10.395 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:10.357+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:10.396 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:10 smithi196 systemd[1]: Starting Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:11:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:10.396 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: osdmap e634: 8 total, 8 up, 8 in 2024-03-20T08:11:10.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:10.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:10.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:10.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:10.397 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:10.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:10.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:10.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:10.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:10.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:10.398 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:10.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:10.399 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-mon[87080]: pgmap v632: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1020 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-03-20T08:11:10.658 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:10 smithi196 podman[120165]: 2024-03-20 08:11:10.475252663 +0000 UTC m=+0.024688060 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.397+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.417+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.433+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.449+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.463+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.478+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.491+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.503+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.515+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.659 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.565+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.603+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.617+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.631+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.644+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.660 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:10.340+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:10.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:10.340+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:10.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:10.341+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:10.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:10.352+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:10.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:10.353+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:10.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:10.353+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:10.353+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:10.353+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: osdmap e634: 8 total, 8 up, 8 in 2024-03-20T08:11:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[95692]: pgmap v632: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1020 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-03-20T08:11:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: osdmap e634: 8 total, 8 up, 8 in 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 158=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:10.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:10.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-mon[93121]: pgmap v632: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1020 MiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 2.9 MiB/s wr, 5 op/s 2024-03-20T08:11:10.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:10.343+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:10.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:10.343+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:10.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.673+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.684+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.739+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.765+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.789+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.801+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.813+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.849+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.861+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.874+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.887+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.899+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:10.912 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.911+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.923+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.965+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.978+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:10.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.043+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.055+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.119+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.173 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.338 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:10.978+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:11.346 INFO:tasks.rados.rados.0.smithi118.stdout:2793: done (3 left) 2024-03-20T08:11:11.346 INFO:tasks.rados.rados.0.smithi118.stdout:2794: rollback oid 42 current snap is 293 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 287 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7067 (ObjNum 65 snap 0 seq_num 1934712148) dirty exists 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: finishing write tid 1 to smithi11881747-26 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: finishing write tid 2 to smithi11881747-26 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: finishing write tid 3 to smithi11881747-26 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: finishing write tid 4 to smithi11881747-26 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: finishing write tid 5 to smithi11881747-26 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: finishing write tid 6 to smithi11881747-26 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: oid 26 updating version 0 to 58325 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: oid 26 version 58325 is already newer than 58323 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: oid 26 version 58325 is already newer than 58324 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: oid 26 updating version 58325 to 58326 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: oid 26 updating version 58326 to 58327 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 58327 (ObjNum 849 snap 293 seq_num 849) dirty exists 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2792: left oid 26 (ObjNum 849 snap 293 seq_num 849) 2024-03-20T08:11:11.347 INFO:tasks.rados.rados.0.smithi118.stdout:2791: expect (ObjNum 848 snap 292 seq_num 848) 2024-03-20T08:11:11.443 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:11.346+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:11.443 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:11.348+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: osdmap e635: 8 total, 8 up, 8 in 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:11.444 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:11.445 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:11.354+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:11.445 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:11.354+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:11.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.207+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.445 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.220+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.233+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.245+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.257+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.269+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.282+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.294+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.307+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.319+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.333+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.350+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.350+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.350+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.365+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.446 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.378+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.402+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.447 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.415+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.611 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:11.346+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:11.611 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:11.346+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:11.611 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:11.611 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:11.611 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: osdmap e635: 8 total, 8 up, 8 in 2024-03-20T08:11:11.611 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:11.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:11.612 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:11.351+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:11.613 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:11.520+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:11.613 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:11.613 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:11.613 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: osdmap e635: 8 total, 8 up, 8 in 2024-03-20T08:11:11.613 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:11.613 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:11.613 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:11.613 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:11.613 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:11.614 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:11.614 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:11.614 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:11.614 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:11.614 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:11.702 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:11 smithi196 podman[120165]: 2024-03-20 08:11:11.574438789 +0000 UTC m=+1.123874162 container create bf82125604d28a2622d14af478df505067512cf315941aa4b44a7e5244e73721 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:11.702 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.428+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.439+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.460+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.473+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.485+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.550+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.576+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.589+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.613+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.626+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.639+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.652+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.665+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.678+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.702+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:11.610+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:11.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:11.611+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:11.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:11.611+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:11.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:11.611+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:11.960 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.728+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.755+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.780+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.806+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.818+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.830+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.961 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.855+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.865+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.890+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.899+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.936+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.962 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.960+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:11.974 INFO:tasks.rados.rados.0.smithi118.stdout:2794: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2794: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2794: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 34077 (ObjNum 825 snap 283 seq_num 825) dirty exists 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2790: done (3 left) 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2791: done (2 left) 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2792: done (1 left) 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2794: done (0 left) 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2795: read oid 24 snap -1 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2795: expect (ObjNum 842 snap 291 seq_num 842) 2024-03-20T08:11:11.975 INFO:tasks.rados.rados.0.smithi118.stdout:2796: snap_create 2024-03-20T08:11:12.012 INFO:tasks.rados.rados.0.smithi118.stdout:2795: expect (ObjNum 842 snap 291 seq_num 842) 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:11.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.012+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.063+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.102+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.126+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.223 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.180+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.209+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.224 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.222+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.414 INFO:tasks.rados.rados.0.smithi118.stdout:2795: done (1 left) 2024-03-20T08:11:12.414 INFO:tasks.rados.rados.0.smithi118.stdout:2796: done (0 left) 2024-03-20T08:11:12.414 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 294 2024-03-20T08:11:12.414 INFO:tasks.rados.rados.0.smithi118.stdout:2797: seq_num 850 ranges {9756672=638976,10395648=802816,11198464=475136,11673600=327680} 2024-03-20T08:11:12.471 INFO:tasks.rados.rados.0.smithi118.stdout:2797: writing smithi11881747-12 from 9756672 to 10395648 tid 1 2024-03-20T08:11:12.476 INFO:tasks.rados.rados.0.smithi118.stdout:2797: writing smithi11881747-12 from 10395648 to 11198464 tid 2 2024-03-20T08:11:12.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.235+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.261+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.274+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.299+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.311+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.326+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.362+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.364+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.378+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.394+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.411+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.427+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.472+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.480 INFO:tasks.rados.rados.0.smithi118.stdout:2797: writing smithi11881747-12 from 11198464 to 11673600 tid 3 2024-03-20T08:11:12.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:12.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:12.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:12.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:12.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: osdmap e636: 8 total, 8 up, 8 in 2024-03-20T08:11:12.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:12.481 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-mon[87080]: pgmap v635: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1023 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 5 op/s 2024-03-20T08:11:12.481 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:12.361+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:12.482 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:12.361+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 podman[120165]: 2024-03-20 08:11:12.313286818 +0000 UTC m=+1.862722208 container init bf82125604d28a2622d14af478df505067512cf315941aa4b44a7e5244e73721 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 podman[120165]: 2024-03-20 08:11:12.321291323 +0000 UTC m=+1.870726710 container start bf82125604d28a2622d14af478df505067512cf315941aa4b44a7e5244e73721 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.352Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.352Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.352Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi196 (none))" 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.352Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.352Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.354Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-20T08:11:12.482 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.355Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-20T08:11:12.483 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.356Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-20T08:11:12.483 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.356Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-20T08:11:12.483 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.360Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-20T08:11:12.483 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.361Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=365.52µs 2024-03-20T08:11:12.483 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.361Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-20T08:11:12.483 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.365Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=4 2024-03-20T08:11:12.483 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.375Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=4 2024-03-20T08:11:12.483 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.384Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=4 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.394Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=4 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.394Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=4 maxSegment=4 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.394Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=47.244µs wal_replay_duration=33.227666ms wbl_replay_duration=125ns total_replay_duration=33.910252ms 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.398Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.399Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.399Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 bash[120165]: bf82125604d28a2622d14af478df505067512cf315941aa4b44a7e5244e73721 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 systemd[1]: Started Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.427Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=28.689081ms db_storage=1.521µs remote_storage=1.703µs web_handler=579ns query_engine=1.03µs scrape=784.817µs scrape_sd=151.347µs notify=16.678µs notify_sd=19.879µs rules=24.187131ms tracing=8.402µs 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.428Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-20T08:11:12.484 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:12.428Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-20T08:11:12.485 INFO:tasks.rados.rados.0.smithi118.stdout:2797: writing smithi11881747-12 from 11673600 to 12001280 tid 4 2024-03-20T08:11:12.485 INFO:tasks.rados.rados.0.smithi118.stdout:2798: rollback oid 47 current snap is 294 2024-03-20T08:11:12.485 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 286 2024-03-20T08:11:12.485 INFO:tasks.rados.rados.0.smithi118.stdout:2799: snap_create 2024-03-20T08:11:12.485 INFO:tasks.rados.rados.0.smithi118.stdout:2798: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:11:12.485 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 33962 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-03-20T08:11:12.490 INFO:tasks.rados.rados.0.smithi118.stdout:2797: finishing write tid 1 to smithi11881747-12 2024-03-20T08:11:12.500 INFO:tasks.rados.rados.0.smithi118.stdout:2797: finishing write tid 2 to smithi11881747-12 2024-03-20T08:11:12.500 INFO:tasks.rados.rados.0.smithi118.stdout:2797: finishing write tid 3 to smithi11881747-12 2024-03-20T08:11:12.500 INFO:tasks.rados.rados.0.smithi118.stdout:2797: finishing write tid 4 to smithi11881747-12 2024-03-20T08:11:12.501 INFO:tasks.rados.rados.0.smithi118.stdout:2797: finishing write tid 5 to smithi11881747-12 2024-03-20T08:11:12.505 INFO:tasks.rados.rados.0.smithi118.stdout:2797: finishing write tid 6 to smithi11881747-12 2024-03-20T08:11:12.505 INFO:tasks.rados.rados.0.smithi118.stdout:2797: oid 12 updating version 0 to 34028 2024-03-20T08:11:12.505 INFO:tasks.rados.rados.0.smithi118.stdout:2797: oid 12 updating version 34028 to 34031 2024-03-20T08:11:12.505 INFO:tasks.rados.rados.0.smithi118.stdout:2797: oid 12 version 34031 is already newer than 34029 2024-03-20T08:11:12.505 INFO:tasks.rados.rados.0.smithi118.stdout:2797: oid 12 version 34031 is already newer than 34027 2024-03-20T08:11:12.505 INFO:tasks.rados.rados.0.smithi118.stdout:2797: oid 12 version 34031 is already newer than 34030 2024-03-20T08:11:12.505 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34031 (ObjNum 850 snap 294 seq_num 850) dirty exists 2024-03-20T08:11:12.505 INFO:tasks.rados.rados.0.smithi118.stdout:2797: left oid 12 (ObjNum 850 snap 294 seq_num 850) 2024-03-20T08:11:12.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:12.357+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:12.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:12.357+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:12.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:12.358+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:12.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:12.355+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:12.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:12.355+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:12.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:12.474+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:12.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:12.474+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:12.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:12.475+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:12.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:12.475+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:12.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:12.480+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:12.674 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:12.352+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:12.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:12.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:12.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:12.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:12.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: osdmap e636: 8 total, 8 up, 8 in 2024-03-20T08:11:12.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:12.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:12.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[93121]: pgmap v635: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1023 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 5 op/s 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:12.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: osdmap e636: 8 total, 8 up, 8 in 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:12.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:12 smithi118 ceph-mon[95692]: pgmap v635: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1023 MiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 5 op/s 2024-03-20T08:11:12.742 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:12.478+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:12.742 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:12.479+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:12.742 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:12.479+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.506+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.521+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.534+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.549+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.563+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.590+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.602+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.616+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.743 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.631+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.645+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.659+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.672+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.686+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.727+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.744 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.742+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.995 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.757+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.769+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.811+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.869+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.884+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.996 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.925+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.940+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.954+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.969+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.982+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:12.997 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:12.995+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.009+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.052+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.246 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.066+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.095+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.110+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.126+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.141+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.173+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.188+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.216+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.248 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.246+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:2797: done (2 left) 2024-03-20T08:11:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:2798: done (1 left) 2024-03-20T08:11:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:2799: done (0 left) 2024-03-20T08:11:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 295 2024-03-20T08:11:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:2800: seq_num 851 ranges {5013504=655360,5668864=688128,6356992=679936,7036928=237568} 2024-03-20T08:11:13.391 INFO:tasks.rados.rados.0.smithi118.stdout:2800: writing smithi11881747-20 from 5013504 to 5668864 tid 1 2024-03-20T08:11:13.395 INFO:tasks.rados.rados.0.smithi118.stdout:2800: writing smithi11881747-20 from 5668864 to 6356992 tid 2 2024-03-20T08:11:13.400 INFO:tasks.rados.rados.0.smithi118.stdout:2800: writing smithi11881747-20 from 6356992 to 7036928 tid 3 2024-03-20T08:11:13.402 INFO:tasks.rados.rados.0.smithi118.stdout:2800: writing smithi11881747-20 from 7036928 to 7274496 tid 4 2024-03-20T08:11:13.402 INFO:tasks.rados.rados.0.smithi118.stdout:2801: read oid 17 snap 272 2024-03-20T08:11:13.402 INFO:tasks.rados.rados.0.smithi118.stdout:2801: expect deleted 2024-03-20T08:11:13.402 INFO:tasks.rados.rados.0.smithi118.stdout:2802: setattr oid 26 current snap is 295 2024-03-20T08:11:13.402 INFO:tasks.rados.rados.0.smithi118.stdout:2800: finishing write tid 1 to smithi11881747-20 2024-03-20T08:11:13.402 INFO:tasks.rados.rados.0.smithi118.stdout:2800: finishing write tid 2 to smithi11881747-20 2024-03-20T08:11:13.404 INFO:tasks.rados.rados.0.smithi118.stdout:2801: done (2 left) 2024-03-20T08:11:13.404 INFO:tasks.rados.rados.0.smithi118.stdout:2803: read oid 30 snap -1 2024-03-20T08:11:13.404 INFO:tasks.rados.rados.0.smithi118.stdout:2803: expect (ObjNum 830 snap 285 seq_num 830) 2024-03-20T08:11:13.404 INFO:tasks.rados.rados.0.smithi118.stdout:2804: copy_from oid 2 from oid 5 current snap is 295 2024-03-20T08:11:13.404 INFO:tasks.rados.rados.0.smithi118.stdout:2805: snap_create 2024-03-20T08:11:13.405 INFO:tasks.rados.rados.0.smithi118.stdout:2800: finishing write tid 3 to smithi11881747-20 2024-03-20T08:11:13.405 INFO:tasks.rados.rados.0.smithi118.stdout:2800: finishing write tid 4 to smithi11881747-20 2024-03-20T08:11:13.405 INFO:tasks.rados.rados.0.smithi118.stdout:2804: finishing copy_from to smithi11881747-2 2024-03-20T08:11:13.405 INFO:tasks.rados.rados.0.smithi118.stdout:2804: got expected ENOENT (src dne) 2024-03-20T08:11:13.405 INFO:tasks.rados.rados.0.smithi118.stdout:2804: finishing copy_from racing read to smithi11881747-2 2024-03-20T08:11:13.406 INFO:tasks.rados.rados.0.smithi118.stdout:2800: finishing write tid 5 to smithi11881747-20 2024-03-20T08:11:13.407 INFO:tasks.rados.rados.0.smithi118.stdout:2800: finishing write tid 6 to smithi11881747-20 2024-03-20T08:11:13.407 INFO:tasks.rados.rados.0.smithi118.stdout:2800: oid 20 updating version 0 to 34056 2024-03-20T08:11:13.407 INFO:tasks.rados.rados.0.smithi118.stdout:2800: oid 20 updating version 34056 to 34058 2024-03-20T08:11:13.407 INFO:tasks.rados.rados.0.smithi118.stdout:2800: oid 20 updating version 34058 to 34059 2024-03-20T08:11:13.407 INFO:tasks.rados.rados.0.smithi118.stdout:2800: oid 20 version 34059 is already newer than 34057 2024-03-20T08:11:13.407 INFO:tasks.rados.rados.0.smithi118.stdout:2800: oid 20 version 34059 is already newer than 34055 2024-03-20T08:11:13.407 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 34059 (ObjNum 851 snap 295 seq_num 851) dirty exists 2024-03-20T08:11:13.407 INFO:tasks.rados.rados.0.smithi118.stdout:2800: left oid 20 (ObjNum 851 snap 295 seq_num 851) 2024-03-20T08:11:13.418 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 59215 (ObjNum 849 snap 293 seq_num 849) dirty exists 2024-03-20T08:11:13.427 INFO:tasks.rados.rados.0.smithi118.stdout:2803: expect (ObjNum 830 snap 285 seq_num 830) 2024-03-20T08:11:13.506 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:13.359+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:13.506 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:13.359+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: osdmap e637: 8 total, 8 up, 8 in 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:13.507 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:13.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:13.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:13.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:13.508 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:13.363+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:13.508 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:13.363+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:13.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:13.508 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:13.509 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.321+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.360+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:13.509 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.360+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.377+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.389+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.426+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.480+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.510 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.507+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:13.361+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:13.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:13.361+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:13.675 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:13.362+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:13.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:13.359+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:13.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:13.360+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:13.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:13.361+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:13.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:13.361+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:13.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:13.358+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:13.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:13.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:13.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: osdmap e637: 8 total, 8 up, 8 in 2024-03-20T08:11:13.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:13.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:13.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:13.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:13.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:13.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: osdmap e637: 8 total, 8 up, 8 in 2024-03-20T08:11:13.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:13.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:13.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:13.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:13.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:13.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.533+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.595+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.610+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.652+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.766 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.666+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.680+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.709+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.725+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.740+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:13.767 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.766+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.779+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.791+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.805+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.818+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.831+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.845+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.857+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.883+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.896+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.908+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.921+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.960+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:13.989+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.017+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.043+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.059+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.273 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.089+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.105+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.152+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.167+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.182+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.228+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.274 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.273+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.364 INFO:tasks.rados.rados.0.smithi118.stdout:2800: done (4 left) 2024-03-20T08:11:14.364 INFO:tasks.rados.rados.0.smithi118.stdout:2802: done (3 left) 2024-03-20T08:11:14.364 INFO:tasks.rados.rados.0.smithi118.stdout:2803: done (2 left) 2024-03-20T08:11:14.364 INFO:tasks.rados.rados.0.smithi118.stdout:2804: done (1 left) 2024-03-20T08:11:14.365 INFO:tasks.rados.rados.0.smithi118.stdout:2805: done (0 left) 2024-03-20T08:11:14.365 INFO:tasks.rados.rados.0.smithi118.stdout:2806: delete oid 22 current snap is 296 2024-03-20T08:11:14.382 INFO:tasks.rados.rados.0.smithi118.stdout:2806: done (0 left) 2024-03-20T08:11:14.382 INFO:tasks.rados.rados.0.smithi118.stdout:2807: rollback oid 19 current snap is 296 2024-03-20T08:11:14.382 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 19 to 272 2024-03-20T08:11:14.382 INFO:tasks.rados.rados.0.smithi118.stdout:2808: rmattr oid 23 current snap is 296 2024-03-20T08:11:14.382 INFO:tasks.rados.rados.0.smithi118.stdout:2808: done (1 left) 2024-03-20T08:11:14.382 INFO:tasks.rados.rados.0.smithi118.stdout:2809: read oid 50 snap -1 2024-03-20T08:11:14.382 INFO:tasks.rados.rados.0.smithi118.stdout:2809: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:14.382 INFO:tasks.rados.rados.0.smithi118.stdout:2810: delete oid 10 current snap is 296 2024-03-20T08:11:14.391 INFO:tasks.rados.rados.0.smithi118.stdout:2807: finishing rollback tid 0 to smithi11881747-19 2024-03-20T08:11:14.392 INFO:tasks.rados.rados.0.smithi118.stdout:2807: finishing rollback tid 1 to smithi11881747-19 2024-03-20T08:11:14.392 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 7094 (ObjNum 49 snap 0 seq_num 1415733603) dirty dne 2024-03-20T08:11:14.395 INFO:tasks.rados.rados.0.smithi118.stdout:2807: done (2 left) 2024-03-20T08:11:14.395 INFO:tasks.rados.rados.0.smithi118.stdout:2810: done (1 left) 2024-03-20T08:11:14.396 INFO:tasks.rados.rados.0.smithi118.stdout:2811: rmattr oid 23 current snap is 296 2024-03-20T08:11:14.396 INFO:tasks.rados.rados.0.smithi118.stdout:2811: done (1 left) 2024-03-20T08:11:14.396 INFO:tasks.rados.rados.0.smithi118.stdout:2812: snap_remove snap 292 2024-03-20T08:11:14.531 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:14.512+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:14.532 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:14.512+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:14.532 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:14.512+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:14.532 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: osdmap e638: 8 total, 8 up, 8 in 2024-03-20T08:11:14.532 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:14.532 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:14.532 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:14.532 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:14.532 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:14.533 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-mon[87080]: pgmap v638: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-03-20T08:11:14.533 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:14.369+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:14.533 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:14.369+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.288+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.335+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.350+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.368+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.369+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.408+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.439+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.455+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.471+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.502+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.519+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.532+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:14.363+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:14.363+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:14.363+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: osdmap e638: 8 total, 8 up, 8 in 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[93121]: pgmap v638: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-03-20T08:11:14.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:14.510+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:14.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:14.510+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:14.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:14.511+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:14.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:14.511+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:14.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:14.511+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: osdmap e638: 8 total, 8 up, 8 in 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-mon[95692]: pgmap v638: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-03-20T08:11:14.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:14.387+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:14.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:14.364+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:14.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:14.364+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.546+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.559+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.587+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.601+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.614+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.785 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.650+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.676+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.702+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.715+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.741+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.753+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:14.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.785+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.801+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.816+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.832+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.877+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.890+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.904+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.916+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.039 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.954+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.967+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.980+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:14.992+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.006+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.040 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.032+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.046+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.059+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.297 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.087+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.113+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.126+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.139+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.152+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.166+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.193+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.206+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.219+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.298 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.232+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.299 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.299 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.299 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.299 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.299 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.297+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.371 INFO:tasks.rados.rados.0.smithi118.stdout:2812: done (1 left) 2024-03-20T08:11:15.371 INFO:tasks.rados.rados.0.smithi118.stdout:2813: rollback oid 2 current snap is 296 2024-03-20T08:11:15.371 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 255 2024-03-20T08:11:15.371 INFO:tasks.rados.rados.0.smithi118.stdout:2809: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:15.432 INFO:tasks.rados.rados.0.smithi118.stdout:2813: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2813: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2813: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45599 (ObjNum 685 snap 225 seq_num 685) dirty exists 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2809: done (1 left) 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2813: done (0 left) 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2814: copy_from oid 12 from oid 39 current snap is 296 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2815: read oid 10 snap -1 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2815: expect deleted 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2816: copy_from oid 23 from oid 34 current snap is 296 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2817: read oid 5 snap -1 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2817: expect deleted 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 296 2024-03-20T08:11:15.433 INFO:tasks.rados.rados.0.smithi118.stdout:2818: seq_num 854 ranges {9207808=712704,9920512=794624,10715136=450560,11165696=360448} 2024-03-20T08:11:15.487 INFO:tasks.rados.rados.0.smithi118.stdout:2818: writing smithi11881747-44 from 9207808 to 9920512 tid 1 2024-03-20T08:11:15.492 INFO:tasks.rados.rados.0.smithi118.stdout:2818: writing smithi11881747-44 from 9920512 to 10715136 tid 2 2024-03-20T08:11:15.495 INFO:tasks.rados.rados.0.smithi118.stdout:2818: writing smithi11881747-44 from 10715136 to 11165696 tid 3 2024-03-20T08:11:15.497 INFO:tasks.rados.rados.0.smithi118.stdout:2818: writing smithi11881747-44 from 11165696 to 11526144 tid 4 2024-03-20T08:11:15.498 INFO:tasks.rados.rados.0.smithi118.stdout:2819: setattr oid 38 current snap is 296 2024-03-20T08:11:15.498 INFO:tasks.rados.rados.0.smithi118.stdout:2816: finishing copy_from to smithi11881747-23 2024-03-20T08:11:15.498 INFO:tasks.rados.rados.0.smithi118.stdout:2816: got expected ENOENT (src dne) 2024-03-20T08:11:15.498 INFO:tasks.rados.rados.0.smithi118.stdout:2816: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:11:15.498 INFO:tasks.rados.rados.0.smithi118.stdout:2814: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:11:15.498 INFO:tasks.rados.rados.0.smithi118.stdout:2814: finishing copy_from to smithi11881747-12 2024-03-20T08:11:15.498 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34045 (ObjNum 821 snap 281 seq_num 821) dirty exists 2024-03-20T08:11:15.502 INFO:tasks.rados.rados.0.smithi118.stdout:2814: done (5 left) 2024-03-20T08:11:15.502 INFO:tasks.rados.rados.0.smithi118.stdout:2815: done (4 left) 2024-03-20T08:11:15.502 INFO:tasks.rados.rados.0.smithi118.stdout:2816: done (3 left) 2024-03-20T08:11:15.502 INFO:tasks.rados.rados.0.smithi118.stdout:2817: done (2 left) 2024-03-20T08:11:15.502 INFO:tasks.rados.rados.0.smithi118.stdout:2820: read oid 10 snap -1 2024-03-20T08:11:15.502 INFO:tasks.rados.rados.0.smithi118.stdout:2820: expect deleted 2024-03-20T08:11:15.502 INFO:tasks.rados.rados.0.smithi118.stdout:2821: rmattr oid 29 current snap is 296 2024-03-20T08:11:15.502 INFO:tasks.rados.rados.0.smithi118.stdout:2822: snap_create 2024-03-20T08:11:15.503 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33984 (ObjNum 659 snap 215 seq_num 659) dirty exists 2024-03-20T08:11:15.518 INFO:tasks.rados.rados.0.smithi118.stdout:2818: finishing write tid 1 to smithi11881747-44 2024-03-20T08:11:15.519 INFO:tasks.rados.rados.0.smithi118.stdout:2818: finishing write tid 2 to smithi11881747-44 2024-03-20T08:11:15.519 INFO:tasks.rados.rados.0.smithi118.stdout:2818: finishing write tid 3 to smithi11881747-44 2024-03-20T08:11:15.519 INFO:tasks.rados.rados.0.smithi118.stdout:2818: finishing write tid 4 to smithi11881747-44 2024-03-20T08:11:15.522 INFO:tasks.rados.rados.0.smithi118.stdout:2818: finishing write tid 5 to smithi11881747-44 2024-03-20T08:11:15.524 INFO:tasks.rados.rados.0.smithi118.stdout:2818: finishing write tid 6 to smithi11881747-44 2024-03-20T08:11:15.524 INFO:tasks.rados.rados.0.smithi118.stdout:2818: oid 44 updating version 0 to 59809 2024-03-20T08:11:15.524 INFO:tasks.rados.rados.0.smithi118.stdout:2818: oid 44 updating version 59809 to 59810 2024-03-20T08:11:15.524 INFO:tasks.rados.rados.0.smithi118.stdout:2818: oid 44 updating version 59810 to 59812 2024-03-20T08:11:15.524 INFO:tasks.rados.rados.0.smithi118.stdout:2818: oid 44 version 59812 is already newer than 59811 2024-03-20T08:11:15.524 INFO:tasks.rados.rados.0.smithi118.stdout:2818: oid 44 updating version 59812 to 59813 2024-03-20T08:11:15.524 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 59813 (ObjNum 854 snap 296 seq_num 854) dirty exists 2024-03-20T08:11:15.524 INFO:tasks.rados.rados.0.smithi118.stdout:2818: left oid 44 (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:11:15.525 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 54775 (ObjNum 814 snap 279 seq_num 814) dirty exists 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.309+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.321+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.334+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.346+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.360+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.374+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.374+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.375+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.391+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.561 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.405+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.448+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.476+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.528+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.544+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.562 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:15.377+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:15.563 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:15.380+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:15.563 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:15.378+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:15.563 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:15.378+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:15.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:15.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:15.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:15.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: osdmap e639: 8 total, 8 up, 8 in 2024-03-20T08:11:15.563 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:15.564 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:15.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:15.372+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:15.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:15.372+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:15.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:15.371+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:15.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:15.379+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:15.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:15.435+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:15.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:15.435+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:15.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:15.436+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:15.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:15.436+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: osdmap e639: 8 total, 8 up, 8 in 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: osdmap e639: 8 total, 8 up, 8 in 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 152=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.577+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.608+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.637+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.662+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.675+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.689+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.703+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.816 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.716+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.728+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.742+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.782+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.795+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:15.817 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.812+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.066 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.839+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.854+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.885+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.901+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.930+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.946+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.962+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.977+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:15.991+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.036+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.066+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.082+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.096+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.109+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.123+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.135+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.150+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.164+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.178+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.190+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.203+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.215+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.230+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.283+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.296+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.376 INFO:tasks.rados.rados.0.smithi118.stdout:2818: done (4 left) 2024-03-20T08:11:16.376 INFO:tasks.rados.rados.0.smithi118.stdout:2819: done (3 left) 2024-03-20T08:11:16.376 INFO:tasks.rados.rados.0.smithi118.stdout:2820: done (2 left) 2024-03-20T08:11:16.376 INFO:tasks.rados.rados.0.smithi118.stdout:2821: done (1 left) 2024-03-20T08:11:16.376 INFO:tasks.rados.rados.0.smithi118.stdout:2822: done (0 left) 2024-03-20T08:11:16.376 INFO:tasks.rados.rados.0.smithi118.stdout:2823: delete oid 33 current snap is 297 2024-03-20T08:11:16.385 INFO:tasks.rados.rados.0.smithi118.stdout:2823: done (0 left) 2024-03-20T08:11:16.385 INFO:tasks.rados.rados.0.smithi118.stdout:2824: snap_remove snap 250 2024-03-20T08:11:16.580 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:16.380+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:16.580 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:16.380+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:16.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: osdmap e640: 8 total, 8 up, 8 in 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:16.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:16.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-mon[87080]: pgmap v641: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 16 MiB/s rd, 1.6 MiB/s wr, 24 op/s 2024-03-20T08:11:16.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.337+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.349+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.365+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.377+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:16.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.382+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.410+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.422+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.451+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.474+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.488+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.514+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.527+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.540+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.581+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:16.375+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:16.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:16.375+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:16.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:16.376+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:16.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:16.376+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:16.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:16.377+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:16.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:16.383+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:16.379+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:16.379+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:16.380+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:16.385+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:16.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:16.385+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:16.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: osdmap e640: 8 total, 8 up, 8 in 2024-03-20T08:11:16.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:16.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:16.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:16.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:16.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:16.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:16.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[95692]: pgmap v641: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 16 MiB/s rd, 1.6 MiB/s wr, 24 op/s 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: osdmap e640: 8 total, 8 up, 8 in 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:16.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:16 smithi118 ceph-mon[93121]: pgmap v641: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 16 MiB/s rd, 1.6 MiB/s wr, 24 op/s 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.607+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.620+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.634+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.648+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.677+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.690+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.705+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.717+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.730+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.835 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.743+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.794+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.808+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.821+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:16.836 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.834+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.085 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:16.878+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:17.085 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:16.878+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:17.086 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:16.878+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:17.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.847+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.860+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.086 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.901+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.915+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.928+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.954+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.982+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:16.995+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.007+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.020+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.034+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.047+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.060+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.087 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.086+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.341 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.099+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.112+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.155+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.168+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.195+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.207+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.342 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.248+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.276+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.289+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.302+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.314+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.327+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.343 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.341+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.381 INFO:tasks.rados.rados.0.smithi118.stdout:2824: done (0 left) 2024-03-20T08:11:17.381 INFO:tasks.rados.rados.0.smithi118.stdout:2825: snap_remove snap 291 2024-03-20T08:11:17.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:17.386+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:17.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:17.386+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:17.592 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:17.482+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:17.592 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:17.494+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:17.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:17.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: osdmap e641: 8 total, 8 up, 8 in 2024-03-20T08:11:17.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:17.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:17.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:17.592 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:17.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:17.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:17.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:17.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:17.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:17.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:17.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:17.593 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:17.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.354+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.593 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.366+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.381+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.385+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.385+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.462+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.472+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.486+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.526+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.539+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.552+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.565+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.595 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:17.382+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:17.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:17.383+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:17.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:17.383+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: osdmap e641: 8 total, 8 up, 8 in 2024-03-20T08:11:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:17.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: osdmap e641: 8 total, 8 up, 8 in 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:17.850 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.605+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.632+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.646+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.658+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.672+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.699+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.714+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.730+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.851 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.746+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.809+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.824+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:17.852 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.850+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.876+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.915+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.942+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.968+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:17.994+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.008+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.047+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.063+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.093+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.108 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.122+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.181+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.194+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.207+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.359 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.236+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.262+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.275+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.303+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.316+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.330+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.360 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.357+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.383 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:17.923+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:18.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:17.923+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:18.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:17.923+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:18.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:17.923+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:18.383 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:17.923+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:18.389 INFO:tasks.rados.rados.0.smithi118.stdout:2825: done (0 left) 2024-03-20T08:11:18.390 INFO:tasks.rados.rados.0.smithi118.stdout:2826: read oid 40 snap -1 2024-03-20T08:11:18.390 INFO:tasks.rados.rados.0.smithi118.stdout:2826: expect deleted 2024-03-20T08:11:18.390 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 297 2024-03-20T08:11:18.390 INFO:tasks.rados.rados.0.smithi118.stdout:2827: seq_num 856 ranges {0=745472,745472=557056,1302528=417792,1720320=434176,2154496=196608} 2024-03-20T08:11:18.394 INFO:tasks.rados.rados.0.smithi118.stdout:2827: writing smithi11881747-16 from 0 to 745472 tid 1 2024-03-20T08:11:18.397 INFO:tasks.rados.rados.0.smithi118.stdout:2827: writing smithi11881747-16 from 745472 to 1302528 tid 2 2024-03-20T08:11:18.400 INFO:tasks.rados.rados.0.smithi118.stdout:2827: writing smithi11881747-16 from 1302528 to 1720320 tid 3 2024-03-20T08:11:18.403 INFO:tasks.rados.rados.0.smithi118.stdout:2827: writing smithi11881747-16 from 1720320 to 2154496 tid 4 2024-03-20T08:11:18.405 INFO:tasks.rados.rados.0.smithi118.stdout:2827: writing smithi11881747-16 from 2154496 to 2351104 tid 5 2024-03-20T08:11:18.405 INFO:tasks.rados.rados.0.smithi118.stdout:2828: delete oid 9 current snap is 297 2024-03-20T08:11:18.405 INFO:tasks.rados.rados.0.smithi118.stdout:2827: finishing write tid 1 to smithi11881747-16 2024-03-20T08:11:18.405 INFO:tasks.rados.rados.0.smithi118.stdout:2827: finishing write tid 2 to smithi11881747-16 2024-03-20T08:11:18.405 INFO:tasks.rados.rados.0.smithi118.stdout:2827: finishing write tid 3 to smithi11881747-16 2024-03-20T08:11:18.409 INFO:tasks.rados.rados.0.smithi118.stdout:2826: done (2 left) 2024-03-20T08:11:18.409 INFO:tasks.rados.rados.0.smithi118.stdout:2828: done (1 left) 2024-03-20T08:11:18.409 INFO:tasks.rados.rados.0.smithi118.stdout:2829: rollback oid 33 current snap is 297 2024-03-20T08:11:18.409 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 213 2024-03-20T08:11:18.410 INFO:tasks.rados.rados.0.smithi118.stdout:2830: rollback oid 3 current snap is 297 2024-03-20T08:11:18.410 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 3 to 277 2024-03-20T08:11:18.410 INFO:tasks.rados.rados.0.smithi118.stdout:2831: copy_from oid 47 from oid 34 current snap is 297 2024-03-20T08:11:18.410 INFO:tasks.rados.rados.0.smithi118.stdout:2832: copy_from oid 13 from oid 17 current snap is 297 2024-03-20T08:11:18.410 INFO:tasks.rados.rados.0.smithi118.stdout:2833: snap_remove snap 268 2024-03-20T08:11:18.617 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:18.390+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:18.617 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:18.390+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:18.618 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:18.413+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:18.618 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:18.415+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:18.618 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:18.415+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:18.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: osdmap e642: 8 total, 8 up, 8 in 2024-03-20T08:11:18.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:18.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:18.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:18.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:18.618 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:18.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:18.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:18.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:18.619 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-mon[87080]: pgmap v644: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 28 op/s 2024-03-20T08:11:18.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.393+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:18.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.403+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.424+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.619 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.454+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.467+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.494+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.506+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.519+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.546+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.559+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.588+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.603+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.620 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.617+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:18.388+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:18.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:18.388+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:18.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:18.388+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:18.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:18.386+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:18.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:18.387+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:18.392+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:18.392+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:18.393+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:18.393+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:18.393+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: osdmap e642: 8 total, 8 up, 8 in 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[95692]: pgmap v644: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 28 op/s 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: osdmap e642: 8 total, 8 up, 8 in 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-mon[93121]: pgmap v644: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 28 op/s 2024-03-20T08:11:18.675 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:18.409+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.632+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.691+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.706+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.720+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.750+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.873 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.779+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.874 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.794+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.874 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.808+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.874 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.824+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.874 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.874 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.854+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:18.874 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.887+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.906+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.962+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:18.989+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.003+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.018+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.049+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.064+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.078+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.092+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.106+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.121+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.136+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.151+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.166+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.181+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.194+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.252+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.266+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.279+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.293+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.306+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.319+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.334+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.348+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.378+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: osdmap e643: 8 total, 8 up, 8 in 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:19.394 INFO:tasks.rados.rados.0.smithi118.stdout:2833: done (5 left) 2024-03-20T08:11:19.394 INFO:tasks.rados.rados.0.smithi118.stdout:2834: delete oid 42 current snap is 297 2024-03-20T08:11:19.394 INFO:tasks.rados.rados.0.smithi118.stdout:2827: finishing write tid 4 to smithi11881747-16 2024-03-20T08:11:19.394 INFO:tasks.rados.rados.0.smithi118.stdout:2827: finishing write tid 5 to smithi11881747-16 2024-03-20T08:11:19.394 INFO:tasks.rados.rados.0.smithi118.stdout:2827: finishing write tid 6 to smithi11881747-16 2024-03-20T08:11:19.394 INFO:tasks.rados.rados.0.smithi118.stdout:2831: finishing copy_from to smithi11881747-47 2024-03-20T08:11:19.394 INFO:tasks.rados.rados.0.smithi118.stdout:2831: got expected ENOENT (src dne) 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2831: finishing copy_from racing read to smithi11881747-47 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2827: finishing write tid 7 to smithi11881747-16 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2827: oid 16 updating version 0 to 33990 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2827: oid 16 version 33990 is already newer than 33986 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2827: oid 16 version 33990 is already newer than 33987 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2827: oid 16 version 33990 is already newer than 33988 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2827: oid 16 version 33990 is already newer than 33985 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2827: oid 16 version 33990 is already newer than 33989 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 33990 (ObjNum 856 snap 297 seq_num 856) dirty exists 2024-03-20T08:11:19.395 INFO:tasks.rados.rados.0.smithi118.stdout:2827: left oid 16 (ObjNum 856 snap 297 seq_num 856) 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:2832: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:2832: finishing copy_from to smithi11881747-13 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 7067 (ObjNum 945 snap 0 seq_num 1890261328) dirty exists 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:2829: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:2829: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34016 (ObjNum 618 snap 202 seq_num 618) dirty exists 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:2830: finishing rollback tid 0 to smithi11881747-3 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:2830: finishing rollback tid 1 to smithi11881747-3 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:2830: finishing rollback tid 2 to smithi11881747-3 2024-03-20T08:11:19.396 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34019 (ObjNum 805 snap 277 seq_num 805) dirty exists 2024-03-20T08:11:19.404 INFO:tasks.rados.rados.0.smithi118.stdout:2827: done (5 left) 2024-03-20T08:11:19.405 INFO:tasks.rados.rados.0.smithi118.stdout:2829: done (4 left) 2024-03-20T08:11:19.405 INFO:tasks.rados.rados.0.smithi118.stdout:2830: done (3 left) 2024-03-20T08:11:19.405 INFO:tasks.rados.rados.0.smithi118.stdout:2831: done (2 left) 2024-03-20T08:11:19.405 INFO:tasks.rados.rados.0.smithi118.stdout:2832: done (1 left) 2024-03-20T08:11:19.405 INFO:tasks.rados.rados.0.smithi118.stdout:2834: done (0 left) 2024-03-20T08:11:19.405 INFO:tasks.rados.rados.0.smithi118.stdout:2835: delete oid 21 current snap is 297 2024-03-20T08:11:19.416 INFO:tasks.rados.rados.0.smithi118.stdout:2835: done (0 left) 2024-03-20T08:11:19.416 INFO:tasks.rados.rados.0.smithi118.stdout:2836: copy_from oid 13 from oid 17 current snap is 297 2024-03-20T08:11:19.416 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 297 2024-03-20T08:11:19.416 INFO:tasks.rados.rados.0.smithi118.stdout:2837: seq_num 860 ranges {5980160=8192} 2024-03-20T08:11:19.449 INFO:tasks.rados.rados.0.smithi118.stdout:2837: writing smithi11881747-26 from 5980160 to 5988352 tid 1 2024-03-20T08:11:19.450 INFO:tasks.rados.rados.0.smithi118.stdout:2838: snap_create 2024-03-20T08:11:19.450 INFO:tasks.rados.rados.0.smithi118.stdout:2836: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:11:19.450 INFO:tasks.rados.rados.0.smithi118.stdout:2836: finishing copy_from to smithi11881747-13 2024-03-20T08:11:19.450 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 7067 (ObjNum 945 snap 0 seq_num 1890261328) dirty exists 2024-03-20T08:11:19.461 INFO:tasks.rados.rados.0.smithi118.stdout:2837: finishing write tid 1 to smithi11881747-26 2024-03-20T08:11:19.461 INFO:tasks.rados.rados.0.smithi118.stdout:2837: finishing write tid 2 to smithi11881747-26 2024-03-20T08:11:19.464 INFO:tasks.rados.rados.0.smithi118.stdout:2837: finishing write tid 3 to smithi11881747-26 2024-03-20T08:11:19.464 INFO:tasks.rados.rados.0.smithi118.stdout:2837: oid 26 updating version 0 to 60938 2024-03-20T08:11:19.465 INFO:tasks.rados.rados.0.smithi118.stdout:2837: oid 26 version 60938 is already newer than 60937 2024-03-20T08:11:19.465 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 60938 (ObjNum 860 snap 297 seq_num 860) dirty exists 2024-03-20T08:11:19.465 INFO:tasks.rados.rados.0.smithi118.stdout:2837: left oid 26 (ObjNum 860 snap 297 seq_num 860) 2024-03-20T08:11:19.652 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:19.395+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:19.652 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:19.396+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:19.402+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:19.402+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.397+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.399+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.399+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.415+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.443+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.461+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.653 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.465+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.481+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.494+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.508+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.537+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.565+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.581+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.624+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.639+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.654 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.653+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:19.397+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:19.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:19.399+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:19.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:19.394+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:19.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: osdmap e643: 8 total, 8 up, 8 in 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:19.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:19.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: osdmap e643: 8 total, 8 up, 8 in 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:19.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 134=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:19.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:19.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:19.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:19.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:19 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:19.909 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.667+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.909 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.695+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.709+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.738+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.752+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.767+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.781+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.796+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.810+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.824+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.853+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.868+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.881+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:19.911 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.924+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.938+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.953+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.966+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.981+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:19.996+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.041+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.058+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.073+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.089+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.104+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.117+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.129+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.142+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.157+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.171+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.400 INFO:tasks.rados.rados.0.smithi118.stdout:2836: done (2 left) 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2837: done (1 left) 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2838: done (0 left) 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2839: copy_from oid 24 from oid 35 current snap is 298 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2840: read oid 32 snap 286 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2840: expect (ObjNum 785 snap 274 seq_num 785) 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2841: read oid 38 snap -1 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2841: expect (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2842: read oid 41 snap -1 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2842: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:11:20.401 INFO:tasks.rados.rados.0.smithi118.stdout:2843: setattr oid 9 current snap is 298 2024-03-20T08:11:20.402 INFO:tasks.rados.rados.0.smithi118.stdout:2842: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:11:20.407 INFO:tasks.rados.rados.0.smithi118.stdout:2842: done (4 left) 2024-03-20T08:11:20.407 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 298 2024-03-20T08:11:20.407 INFO:tasks.rados.rados.0.smithi118.stdout:2844: seq_num 861 ranges {0=24576} 2024-03-20T08:11:20.407 INFO:tasks.rados.rados.0.smithi118.stdout:2844: writing smithi11881747-48 from 0 to 24576 tid 1 2024-03-20T08:11:20.407 INFO:tasks.rados.rados.0.smithi118.stdout:2845: copy_from oid 21 from oid 42 current snap is 298 2024-03-20T08:11:20.407 INFO:tasks.rados.rados.0.smithi118.stdout:2846: rollback oid 25 current snap is 298 2024-03-20T08:11:20.407 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 25 to 277 2024-03-20T08:11:20.407 INFO:tasks.rados.rados.0.smithi118.stdout:2847: setattr oid 5 current snap is 298 2024-03-20T08:11:20.408 INFO:tasks.rados.rados.0.smithi118.stdout:2848: snap_create 2024-03-20T08:11:20.413 INFO:tasks.rados.rados.0.smithi118.stdout:2840: expect (ObjNum 785 snap 274 seq_num 785) 2024-03-20T08:11:20.436 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:20.419+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:20.437 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:20.419+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:20.437 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:20.419+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:20.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:20.407+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:20.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:20.407+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:20.437 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: osdmap e644: 8 total, 8 up, 8 in 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:20.438 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-mon[87080]: pgmap v647: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 998 KiB/s wr, 12 op/s 2024-03-20T08:11:20.438 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.202+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.215+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.230+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.245+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.258+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.272+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.287+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.302+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.316+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.330+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.344+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.359+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.375+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.439 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.392+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.408+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:20.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.440 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.459 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 34080 (ObjNum 65 snap 0 seq_num 1651340386) dirty exists 2024-03-20T08:11:20.459 INFO:tasks.rados.rados.0.smithi118.stdout:2846: finishing rollback tid 0 to smithi11881747-25 2024-03-20T08:11:20.459 INFO:tasks.rados.rados.0.smithi118.stdout:2846: finishing rollback tid 1 to smithi11881747-25 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:2846: finishing rollback tid 2 to smithi11881747-25 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 34083 (ObjNum 798 snap 275 seq_num 798) dirty exists 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 34084 (ObjNum 353 snap 0 seq_num 323982592) dirty exists 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:2844: finishing write tid 1 to smithi11881747-48 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:2844: finishing write tid 2 to smithi11881747-48 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:2839: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:2839: finishing copy_from to smithi11881747-24 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 45591 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:11:20.460 INFO:tasks.rados.rados.0.smithi118.stdout:2841: expect (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:11:20.592 INFO:tasks.rados.rados.0.smithi118.stdout:2844: finishing write tid 3 to smithi11881747-48 2024-03-20T08:11:20.593 INFO:tasks.rados.rados.0.smithi118.stdout:2844: oid 48 updating version 0 to 56531 2024-03-20T08:11:20.593 INFO:tasks.rados.rados.0.smithi118.stdout:2844: oid 48 version 56531 is already newer than 56530 2024-03-20T08:11:20.593 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 56531 (ObjNum 861 snap 298 seq_num 861) dirty exists 2024-03-20T08:11:20.593 INFO:tasks.rados.rados.0.smithi118.stdout:2844: left oid 48 (ObjNum 861 snap 298 seq_num 861) 2024-03-20T08:11:20.593 INFO:tasks.rados.rados.0.smithi118.stdout:2845: finishing copy_from to smithi11881747-21 2024-03-20T08:11:20.593 INFO:tasks.rados.rados.0.smithi118.stdout:2845: got expected ENOENT (src dne) 2024-03-20T08:11:20.593 INFO:tasks.rados.rados.0.smithi118.stdout:2845: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:11:20.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:20.408+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:20.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:20.409+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:20.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:20.409+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: osdmap e644: 8 total, 8 up, 8 in 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[93121]: pgmap v647: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 998 KiB/s wr, 12 op/s 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.278+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.278+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.278+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.279+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.399+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.399+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.399+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.400+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:20.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:20.400+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: osdmap e644: 8 total, 8 up, 8 in 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:20.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-mon[95692]: pgmap v647: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 998 KiB/s wr, 12 op/s 2024-03-20T08:11:20.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:20.408+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:20.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:20.358+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:20.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:20.402+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:20.676 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:20.402+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:20.698 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.453+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.467+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.483+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.526+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.540+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.555+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.570+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.583+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.597+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.610+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.699 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.640+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.655+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.669+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.684+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.700 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.698+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.713+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.729+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.745+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.777+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.950 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.809+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.824+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.841+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.856+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.870+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.883+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.895+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.922+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.936+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:20.951 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.950+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.965+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.977+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:20.992+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.005+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.019+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.203 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.057+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.070+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.083+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.095+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.109+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.123+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.135+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.149+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.162+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.176+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.189+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.204 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.203+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2839: done (8 left) 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2840: done (7 left) 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2841: done (6 left) 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2843: done (5 left) 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2844: done (4 left) 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2845: done (3 left) 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2846: done (2 left) 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2847: done (1 left) 2024-03-20T08:11:21.407 INFO:tasks.rados.rados.0.smithi118.stdout:2848: done (0 left) 2024-03-20T08:11:21.408 INFO:tasks.rados.rados.0.smithi118.stdout:2849: read oid 18 snap -1 2024-03-20T08:11:21.408 INFO:tasks.rados.rados.0.smithi118.stdout:2849: expect deleted 2024-03-20T08:11:21.408 INFO:tasks.rados.rados.0.smithi118.stdout:2850: read oid 38 snap -1 2024-03-20T08:11:21.408 INFO:tasks.rados.rados.0.smithi118.stdout:2850: expect (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:11:21.408 INFO:tasks.rados.rados.0.smithi118.stdout:2851: setattr oid 40 current snap is 299 2024-03-20T08:11:21.409 INFO:tasks.rados.rados.0.smithi118.stdout:2852: read oid 50 snap -1 2024-03-20T08:11:21.409 INFO:tasks.rados.rados.0.smithi118.stdout:2852: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:21.409 INFO:tasks.rados.rados.0.smithi118.stdout:2853: rollback oid 7 current snap is 299 2024-03-20T08:11:21.409 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 7 to 290 2024-03-20T08:11:21.409 INFO:tasks.rados.rados.0.smithi118.stdout:2849: done (4 left) 2024-03-20T08:11:21.409 INFO:tasks.rados.rados.0.smithi118.stdout:2854: delete oid 45 current snap is 299 2024-03-20T08:11:21.423 INFO:tasks.rados.rados.0.smithi118.stdout:2853: finishing rollback tid 0 to smithi11881747-7 2024-03-20T08:11:21.423 INFO:tasks.rados.rados.0.smithi118.stdout:2853: finishing rollback tid 1 to smithi11881747-7 2024-03-20T08:11:21.424 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 34047 (ObjNum 65 snap 0 seq_num 1365401433) dirty dne 2024-03-20T08:11:21.425 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34058 (ObjNum 33 snap 0 seq_num 323266688) dirty exists 2024-03-20T08:11:21.425 INFO:tasks.rados.rados.0.smithi118.stdout:2851: done (4 left) 2024-03-20T08:11:21.425 INFO:tasks.rados.rados.0.smithi118.stdout:2853: done (3 left) 2024-03-20T08:11:21.425 INFO:tasks.rados.rados.0.smithi118.stdout:2854: done (2 left) 2024-03-20T08:11:21.425 INFO:tasks.rados.rados.0.smithi118.stdout:2855: rollback oid 17 current snap is 299 2024-03-20T08:11:21.425 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 297 2024-03-20T08:11:21.425 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 299 2024-03-20T08:11:21.426 INFO:tasks.rados.rados.0.smithi118.stdout:2856: seq_num 863 ranges {0=57344} 2024-03-20T08:11:21.426 INFO:tasks.rados.rados.0.smithi118.stdout:2856: writing smithi11881747-19 from 0 to 57344 tid 1 2024-03-20T08:11:21.426 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 299 2024-03-20T08:11:21.426 INFO:tasks.rados.rados.0.smithi118.stdout:2857: seq_num 864 ranges {1761280=73728} 2024-03-20T08:11:21.437 INFO:tasks.rados.rados.0.smithi118.stdout:2857: writing smithi11881747-12 from 1761280 to 1835008 tid 1 2024-03-20T08:11:21.437 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 299 2024-03-20T08:11:21.437 INFO:tasks.rados.rados.0.smithi118.stdout:2852: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:21.463 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:21.417+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:21.463 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:21.418+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:21.464 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:21.408+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:21.464 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:21.409+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:21.464 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:21.464 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:21.464 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:21.464 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: osdmap e645: 8 total, 8 up, 8 in 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:21.465 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.217+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.230+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.243+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.257+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.284+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.297+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.310+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.466 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.352+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.365+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.380+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.408+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.409+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.409+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.433+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.450+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.467 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.464+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.503 INFO:tasks.rados.rados.0.smithi118.stdout:2850: expect (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:11:21.601 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:11:21.628 INFO:tasks.rados.rados.0.smithi118.stdout:2855: finishing rollback tid 0 to smithi11881747-17 2024-03-20T08:11:21.629 INFO:tasks.rados.rados.0.smithi118.stdout:2855: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:11:21.629 INFO:tasks.rados.rados.0.smithi118.stdout:2855: finishing rollback tid 2 to smithi11881747-17 2024-03-20T08:11:21.629 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7070 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-03-20T08:11:21.629 INFO:tasks.rados.rados.0.smithi118.stdout:2856: finishing write tid 1 to smithi11881747-19 2024-03-20T08:11:21.629 INFO:tasks.rados.rados.0.smithi118.stdout:2858: seq_num 865 ranges {0=90112} 2024-03-20T08:11:21.630 INFO:tasks.rados.rados.0.smithi118.stdout:2858: writing smithi11881747-36 from 0 to 90112 tid 1 2024-03-20T08:11:21.630 INFO:tasks.rados.rados.0.smithi118.stdout:2850: done (5 left) 2024-03-20T08:11:21.630 INFO:tasks.rados.rados.0.smithi118.stdout:2852: done (4 left) 2024-03-20T08:11:21.630 INFO:tasks.rados.rados.0.smithi118.stdout:2855: done (3 left) 2024-03-20T08:11:21.630 INFO:tasks.rados.rados.0.smithi118.stdout:2859: snap_create 2024-03-20T08:11:21.630 INFO:tasks.rados.rados.0.smithi118.stdout:2856: finishing write tid 2 to smithi11881747-19 2024-03-20T08:11:21.630 INFO:tasks.rados.rados.0.smithi118.stdout:2856: finishing write tid 3 to smithi11881747-19 2024-03-20T08:11:21.630 INFO:tasks.rados.rados.0.smithi118.stdout:2856: oid 19 updating version 0 to 7095 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:2856: oid 19 updating version 7095 to 7096 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 7096 (ObjNum 863 snap 299 seq_num 863) dirty exists 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:2856: left oid 19 (ObjNum 863 snap 299 seq_num 863) 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:2857: finishing write tid 1 to smithi11881747-12 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:2857: finishing write tid 2 to smithi11881747-12 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:2857: finishing write tid 3 to smithi11881747-12 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:2857: oid 12 updating version 0 to 34049 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:2857: oid 12 version 34049 is already newer than 34048 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34049 (ObjNum 864 snap 299 seq_num 864) dirty exists 2024-03-20T08:11:21.631 INFO:tasks.rados.rados.0.smithi118.stdout:2857: left oid 12 (ObjNum 864 snap 299 seq_num 864) 2024-03-20T08:11:21.637 INFO:tasks.rados.rados.0.smithi118.stdout:2858: finishing write tid 1 to smithi11881747-36 2024-03-20T08:11:21.637 INFO:tasks.rados.rados.0.smithi118.stdout:2858: finishing write tid 2 to smithi11881747-36 2024-03-20T08:11:21.641 INFO:tasks.rados.rados.0.smithi118.stdout:2858: finishing write tid 3 to smithi11881747-36 2024-03-20T08:11:21.641 INFO:tasks.rados.rados.0.smithi118.stdout:2858: oid 36 updating version 0 to 45585 2024-03-20T08:11:21.641 INFO:tasks.rados.rados.0.smithi118.stdout:2858: oid 36 version 45585 is already newer than 45584 2024-03-20T08:11:21.641 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45585 (ObjNum 865 snap 299 seq_num 865) dirty exists 2024-03-20T08:11:21.641 INFO:tasks.rados.rados.0.smithi118.stdout:2858: left oid 36 (ObjNum 865 snap 299 seq_num 865) 2024-03-20T08:11:21.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:21.410+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:21.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:21.406+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:21.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:21.406+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:21.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:21.417+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:21.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:21.423+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:21.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:21.423+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:21.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:21.424+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:21.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:21.424+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: osdmap e645: 8 total, 8 up, 8 in 2024-03-20T08:11:21.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:21.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15b=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: osdmap e645: 8 total, 8 up, 8 in 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:21.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:21.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:21.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:21.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:21.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:21.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:21.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.479+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.493+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.509+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.525+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.556+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.569+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.596+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.721 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.623+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.651+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.722 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.721+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.972 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.747+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.773+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.786+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.799+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.813+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.826+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.840+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.868+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.880+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.892+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.973 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.920+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.933+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.947+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:21.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.002 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:11:22.002 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (11m) 28s ago 12m 27.3M - 0.25.0 c8568f914cd2 39fe7da5bc35 2024-03-20T08:11:22.002 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (11m) 3s ago 11m 87.9M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:11:22.002 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (81s) 28s ago 13m 14.0M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 b9e9c53270c7 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (58s) 3s ago 13m 14.0M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 237cfe0be476 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (7m) 3s ago 16m 453M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (8m) 28s ago 18m 515M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (7m) 28s ago 18m 69.3M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (6m) 3s ago 17m 58.8M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (7m) 28s ago 16m 61.5M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (38s) 28s ago 11m 3653k - 1.5.0 0da6a335fe13 768c22c1ab59 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (33s) 3s ago 11m 8648k - 1.5.0 0da6a335fe13 d2c82f232b48 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (6m) 28s ago 16m 402M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (5m) 28s ago 15m 525M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (4m) 28s ago 15m 372M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (4m) 28s ago 14m 148M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (3m) 3s ago 14m 381M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (3m) 3s ago 14m 413M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 72b569d10541 2024-03-20T08:11:22.003 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (2m) 3s ago 13m 228M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 dca658056148 2024-03-20T08:11:22.004 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (105s) 3s ago 13m 127M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 52392403da44 2024-03-20T08:11:22.004 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (11s) 3s ago 12m 35.6M - 2.43.0 a07b618ecd1d bf82125604d2 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:21.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.026+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.039+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.054+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.067+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.080+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.227 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.107+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.121+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.136+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.174+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.201+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.214+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.228 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.226+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.415 INFO:tasks.rados.rados.0.smithi118.stdout:2856: done (3 left) 2024-03-20T08:11:22.415 INFO:tasks.rados.rados.0.smithi118.stdout:2857: done (2 left) 2024-03-20T08:11:22.415 INFO:tasks.rados.rados.0.smithi118.stdout:2858: done (1 left) 2024-03-20T08:11:22.415 INFO:tasks.rados.rados.0.smithi118.stdout:2859: done (0 left) 2024-03-20T08:11:22.415 INFO:tasks.rados.rados.0.smithi118.stdout:2860: copy_from oid 48 from oid 43 current snap is 300 2024-03-20T08:11:22.415 INFO:tasks.rados.rados.0.smithi118.stdout:2861: setattr oid 7 current snap is 300 2024-03-20T08:11:22.417 INFO:tasks.rados.rados.0.smithi118.stdout:2862: read oid 36 snap 258 2024-03-20T08:11:22.417 INFO:tasks.rados.rados.0.smithi118.stdout:2862: expect (ObjNum 667 snap 217 seq_num 667) 2024-03-20T08:11:22.417 INFO:tasks.rados.rados.0.smithi118.stdout:2863: snap_create 2024-03-20T08:11:22.426 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 34050 (ObjNum 81 snap 0 seq_num 1273390728) dirty exists 2024-03-20T08:11:22.427 INFO:tasks.rados.rados.0.smithi118.stdout:2862: expect (ObjNum 667 snap 217 seq_num 667) 2024-03-20T08:11:22.478 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:22.424+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:22.478 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:22.424+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:22.478 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:22.425+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:22.478 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:11:22.479 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:22.415+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:22.479 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:22.415+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:22.479 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:22.479 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:22.479 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: osdmap e646: 8 total, 8 up, 8 in 2024-03-20T08:11:22.479 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: from='client.44375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: from='client.44381 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:22.480 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-mon[87080]: pgmap v650: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 907 KiB/s rd, 568 KiB/s wr, 15 op/s 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.240+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.254+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.267+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.292+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.332+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.481 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.345+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.359+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.373+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.386+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.399+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.418+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.437+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.451+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.465+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.482 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.478+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.483 INFO:tasks.rados.rados.0.smithi118.stdout:2860: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:11:22.483 INFO:tasks.rados.rados.0.smithi118.stdout:2860: finishing copy_from to smithi11881747-48 2024-03-20T08:11:22.483 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 44224 (ObjNum 840 snap 290 seq_num 840) dirty exists 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 8 2024-03-20T08:11:22.517 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:11:22.518 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:11:22.518 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:11:22.518 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:11:22.518 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:11:22.518 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 15 2024-03-20T08:11:22.518 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:11:22.518 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:11:22.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:22.418+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:22.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:22.418+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:22.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:22.420+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:22.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:22.411+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:22.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:22.414+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:22.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:22.414+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:22.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: osdmap e646: 8 total, 8 up, 8 in 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:22.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: from='client.44375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: from='client.44381 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[93121]: pgmap v650: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 907 KiB/s rd, 568 KiB/s wr, 15 op/s 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: osdmap e646: 8 total, 8 up, 8 in 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:22.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 160=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: from='client.44375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: from='client.44381 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:22.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-mon[95692]: pgmap v650: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 907 KiB/s rd, 568 KiB/s wr, 15 op/s 2024-03-20T08:11:22.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:22.421+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:22.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:22.421+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:22.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:22.422+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:22.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:22.422+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:22.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:22.427+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.491+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.505+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.520+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.534+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.548+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.562+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.577+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.593+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.622+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.648+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.740 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.661+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.674+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.700+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.713+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.725+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:22.741 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.768+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.810+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.823+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.837+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.002 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.865+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.880+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.909+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.924+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.937+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.950+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.976+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:22.988+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.001+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.016+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.263 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.032+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.076+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.091+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.106+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.133+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.164+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.178+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.192+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.205+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.219+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.234+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.264 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.249+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:2860: done (3 left) 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:2861: done (2 left) 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:2862: done (1 left) 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:2863: done (0 left) 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:2864: read oid 19 snap 242 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:2864: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 301 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:2865: seq_num 866 ranges {0=106496} 2024-03-20T08:11:23.413 INFO:tasks.rados.rados.0.smithi118.stdout:2865: writing smithi11881747-42 from 0 to 106496 tid 1 2024-03-20T08:11:23.414 INFO:tasks.rados.rados.0.smithi118.stdout:2866: snap_remove snap 271 2024-03-20T08:11:23.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:23.516 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: osdmap e647: 8 total, 8 up, 8 in 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:23.517 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/994731446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.293+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.309+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.324+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.352+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.366+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.518 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.381+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.395+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.417+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.417+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.429+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.458+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.472+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.487+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.501+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.516+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:23.422+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:23.519 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:23.422+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:23.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:11:23] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:11:23.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:23.415+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:23.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:23.419+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:23.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:23.413+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:23.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:23.412+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:23.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: osdmap e647: 8 total, 8 up, 8 in 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:23.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/994731446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: osdmap e647: 8 total, 8 up, 8 in 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:23.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:23.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:23.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:23.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/994731446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:23.780 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.532+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.780 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.546+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.559+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.575+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.589+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.603+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.633+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.647+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.679+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.692+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.707+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.722+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.737+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.751+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.765+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:23.782 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.780+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.045 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:23.860+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:24.045 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:23.860+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.796+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.810+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.854+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.869+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.898+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.912+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.046 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.928+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.942+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.957+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.972+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:23.987+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.001+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.015+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.047 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.030+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:23.858+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:24.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:23.858+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:24.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:23.859+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:24.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:23.859+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:24.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.045+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.061+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.076+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.089+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.300 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.129+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.144+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.156+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.183+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.197+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.210+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.223+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.235+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.248+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.273+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.286+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.299+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2866: done (2 left) 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2867: delete oid 11 current snap is 301 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2865: finishing write tid 1 to smithi11881747-42 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2865: finishing write tid 2 to smithi11881747-42 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2865: finishing write tid 3 to smithi11881747-42 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2865: oid 42 updating version 0 to 34086 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2865: oid 42 version 34086 is already newer than 34085 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 34086 (ObjNum 866 snap 301 seq_num 866) dirty exists 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2865: left oid 42 (ObjNum 866 snap 301 seq_num 866) 2024-03-20T08:11:24.427 INFO:tasks.rados.rados.0.smithi118.stdout:2864: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:11:24.473 INFO:tasks.rados.rados.0.smithi118.stdout:2864: done (2 left) 2024-03-20T08:11:24.473 INFO:tasks.rados.rados.0.smithi118.stdout:2865: done (1 left) 2024-03-20T08:11:24.473 INFO:tasks.rados.rados.0.smithi118.stdout:2867: done (0 left) 2024-03-20T08:11:24.473 INFO:tasks.rados.rados.0.smithi118.stdout:2868: snap_remove snap 286 2024-03-20T08:11:24.550 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:24.424+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:24.551 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:24.425+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:24.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: osdmap e648: 8 total, 8 up, 8 in 2024-03-20T08:11:24.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:24.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:24.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:24.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:24.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:24.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:24.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:24.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:24.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:24.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:24.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:24.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:24.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:24.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-mon[87080]: pgmap v653: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 5.4 MiB/s rd, 598 KiB/s wr, 31 op/s 2024-03-20T08:11:24.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.314+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.552 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.328+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.384+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.397+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.411+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.428+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.431+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.446+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.459+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.472+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.497+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.511+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.553 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.524+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.538+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.554 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.551+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:24.426+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:24.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:24.426+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:24.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:24.427+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:24.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:24.423+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:24.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:24.423+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:24.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:24.423+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:24.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:24.423+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:24.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:24.423+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:24.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:24.433+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: osdmap e648: 8 total, 8 up, 8 in 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:24.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[95692]: pgmap v653: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 5.4 MiB/s rd, 598 KiB/s wr, 31 op/s 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: osdmap e648: 8 total, 8 up, 8 in 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:24.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:24.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-mon[93121]: pgmap v653: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 5.4 MiB/s rd, 598 KiB/s wr, 31 op/s 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.564+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.578+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.591+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.604+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.618+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.631+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.644+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.657+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.670+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.683+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.806 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.696+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.709+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.723+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.735+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.749+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.764+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.778+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:24.807 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.805+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.066 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:24.979+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:25.066 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:24.979+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:25.067 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:24.979+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:25.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.820+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.835+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.849+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.863+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.877+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.067 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.889+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.903+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.918+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.932+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.945+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.972+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:24.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.013+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.025+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.039+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.068 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:24.829+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:25.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:24.829+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.081+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.094+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.107+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.120+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.134+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.148+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.161+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.324 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.174+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.187+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.199+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.213+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.240+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.268+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.281+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.295+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.309+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.325 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.323+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.432 INFO:tasks.rados.rados.0.smithi118.stdout:2868: done (0 left) 2024-03-20T08:11:25.432 INFO:tasks.rados.rados.0.smithi118.stdout:2869: read oid 45 snap 282 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2869: expect (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2870: read oid 9 snap 266 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2870: expect (ObjNum 751 snap 256 seq_num 751) 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2871: read oid 30 snap -1 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2871: expect (ObjNum 830 snap 285 seq_num 830) 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2872: copy_from oid 48 from oid 49 current snap is 301 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 301 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2873: seq_num 868 ranges {0=139264} 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2873: writing smithi11881747-7 from 0 to 139264 tid 1 2024-03-20T08:11:25.433 INFO:tasks.rados.rados.0.smithi118.stdout:2874: delete oid 36 current snap is 301 2024-03-20T08:11:25.446 INFO:tasks.rados.rados.0.smithi118.stdout:2870: expect (ObjNum 751 snap 256 seq_num 751) 2024-03-20T08:11:25.522 INFO:tasks.rados.rados.0.smithi118.stdout:2872: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:11:25.522 INFO:tasks.rados.rados.0.smithi118.stdout:2873: finishing write tid 1 to smithi11881747-7 2024-03-20T08:11:25.522 INFO:tasks.rados.rados.0.smithi118.stdout:2869: expect (ObjNum 685 snap 225 seq_num 685) 2024-03-20T08:11:25.579 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:25.452+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:25.579 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:25.453+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:25.580 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:25.434+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:25.580 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:25.434+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:25.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:25.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:25.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:25.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: osdmap e649: 8 total, 8 up, 8 in 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.581 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.582 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.351+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.364+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.379+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.393+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.407+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.420+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.438+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:25.582 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.438+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:25.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.448+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.467+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.496+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.509+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.583 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.524+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.537+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.584 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.553+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.584 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.567+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.585 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.580+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:25.430+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:25.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:25.431+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:25.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:25.429+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:25.442+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:25.455+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:25.456+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:25.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:25.457+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: osdmap e649: 8 total, 8 up, 8 in 2024-03-20T08:11:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:25.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:25.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:25.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:25.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: osdmap e649: 8 total, 8 up, 8 in 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:25.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:25.681 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:25.437+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:25.681 INFO:tasks.rados.rados.0.smithi118.stdout:2873: finishing write tid 2 to smithi11881747-7 2024-03-20T08:11:25.681 INFO:tasks.rados.rados.0.smithi118.stdout:2871: expect (ObjNum 830 snap 285 seq_num 830) 2024-03-20T08:11:25.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.594+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.606+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.832 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.619+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.632+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.645+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.659+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.685+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.712+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.724+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.752+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.765+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.778+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.791+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.833 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.805+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.834 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.818+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.834 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.832+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:25.927 INFO:tasks.rados.rados.0.smithi118.stdout:2872: finishing copy_from to smithi11881747-48 2024-03-20T08:11:25.927 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 34054 (ObjNum 33 snap 0 seq_num 322532912) dirty exists 2024-03-20T08:11:25.927 INFO:tasks.rados.rados.0.smithi118.stdout:2873: finishing write tid 3 to smithi11881747-7 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2873: oid 7 updating version 0 to 34052 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2873: oid 7 version 34052 is already newer than 34051 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 34052 (ObjNum 868 snap 301 seq_num 868) dirty exists 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2873: left oid 7 (ObjNum 868 snap 301 seq_num 868) 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2869: done (5 left) 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2870: done (4 left) 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2871: done (3 left) 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2872: done (2 left) 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2873: done (1 left) 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2874: done (0 left) 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2875: rollback oid 8 current snap is 301 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 8 to 279 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2876: copy_from oid 10 from oid 47 current snap is 301 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2877: rollback oid 12 current snap is 301 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 12 to 169 2024-03-20T08:11:25.928 INFO:tasks.rados.rados.0.smithi118.stdout:2878: delete oid 43 current snap is 301 2024-03-20T08:11:25.932 INFO:tasks.rados.rados.0.smithi118.stdout:2876: finishing copy_from to smithi11881747-10 2024-03-20T08:11:25.932 INFO:tasks.rados.rados.0.smithi118.stdout:2876: got expected ENOENT (src dne) 2024-03-20T08:11:25.932 INFO:tasks.rados.rados.0.smithi118.stdout:2876: finishing copy_from racing read to smithi11881747-10 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2876: done (3 left) 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2878: done (2 left) 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2879: read oid 45 snap -1 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2879: expect deleted 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2880: read oid 19 snap -1 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2880: expect (ObjNum 863 snap 299 seq_num 863) 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2881: read oid 34 snap -1 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2881: expect deleted 2024-03-20T08:11:25.934 INFO:tasks.rados.rados.0.smithi118.stdout:2882: snap_create 2024-03-20T08:11:25.938 INFO:tasks.rados.rados.0.smithi118.stdout:2880: expect (ObjNum 863 snap 299 seq_num 863) 2024-03-20T08:11:25.940 INFO:tasks.rados.rados.0.smithi118.stdout:2875: finishing rollback tid 0 to smithi11881747-8 2024-03-20T08:11:25.941 INFO:tasks.rados.rados.0.smithi118.stdout:2877: finishing rollback tid 0 to smithi11881747-12 2024-03-20T08:11:25.941 INFO:tasks.rados.rados.0.smithi118.stdout:2875: finishing rollback tid 1 to smithi11881747-8 2024-03-20T08:11:25.941 INFO:tasks.rados.rados.0.smithi118.stdout:2877: finishing rollback tid 1 to smithi11881747-12 2024-03-20T08:11:25.941 INFO:tasks.rados.rados.0.smithi118.stdout:2875: finishing rollback tid 2 to smithi11881747-8 2024-03-20T08:11:25.941 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34069 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:11:25.941 INFO:tasks.rados.rados.0.smithi118.stdout:2877: finishing rollback tid 2 to smithi11881747-12 2024-03-20T08:11:25.942 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34055 (ObjNum 399 snap 135 seq_num 399) dirty exists 2024-03-20T08:11:26.091 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.845+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.859+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.873+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.886+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.899+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.912+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.926+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.945+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.959+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.972+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.092 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.985+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:25.998+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.011+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.037+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.077+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.093 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.091+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.104+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.116+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.129+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.142+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.157+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.170+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.184+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.197+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.355 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.211+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.224+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.239+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.251+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.264+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.276+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.290+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.302+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.315+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.328+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.341+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.356 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.354+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:2875: done (5 left) 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:2877: done (4 left) 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:2879: done (3 left) 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:2880: done (2 left) 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:2881: done (1 left) 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:2882: done (0 left) 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:2883: rollback oid 3 current snap is 302 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 3 to 296 2024-03-20T08:11:26.435 INFO:tasks.rados.rados.0.smithi118.stdout:2884: read oid 39 snap -1 2024-03-20T08:11:26.436 INFO:tasks.rados.rados.0.smithi118.stdout:2884: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:26.436 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 302 2024-03-20T08:11:26.436 INFO:tasks.rados.rados.0.smithi118.stdout:2885: seq_num 871 ranges {57344=188416} 2024-03-20T08:11:26.436 INFO:tasks.rados.rados.0.smithi118.stdout:2885: writing smithi11881747-19 from 57344 to 245760 tid 1 2024-03-20T08:11:26.436 INFO:tasks.rados.rados.0.smithi118.stdout:2886: read oid 26 snap -1 2024-03-20T08:11:26.437 INFO:tasks.rados.rados.0.smithi118.stdout:2886: expect (ObjNum 860 snap 297 seq_num 860) 2024-03-20T08:11:26.437 INFO:tasks.rados.rados.0.smithi118.stdout:2887: snap_remove snap 299 2024-03-20T08:11:26.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:26.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:26.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:26.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:26.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:26.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: osdmap e650: 8 total, 8 up, 8 in 2024-03-20T08:11:26.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:26.605 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:26.606 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-mon[87080]: pgmap v656: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 9.1 MiB/s rd, 840 KiB/s wr, 25 op/s 2024-03-20T08:11:26.606 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.368+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.383+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.396+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.409+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.423+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.438+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.447+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.471+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.486+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.498+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.512+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.538+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.551+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.566+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.579+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.608 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.592+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.608 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:26.447+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:26.608 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:26.447+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:26.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.605+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.618+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.632+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.861 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.645+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.671+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.685+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.698+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.713+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.726+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.739+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.752+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.767+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.808+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.862 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.820+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.863 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.834+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.863 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.847+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.863 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.861+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:26.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:26.433+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:26.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:26.433+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:26.887 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:26.433+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:26.888 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:26.446+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:26.888 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:26.437+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:26.888 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:26.437+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:26.888 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:26.438+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:26.888 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:26.451+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:26.888 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:26.451+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:26.889 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:26.441+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:26.889 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:26.441+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:26.889 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: osdmap e650: 8 total, 8 up, 8 in 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:26.890 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[93121]: pgmap v656: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 9.1 MiB/s rd, 840 KiB/s wr, 25 op/s 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:26.891 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: osdmap e650: 8 total, 8 up, 8 in 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:26.892 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:26 smithi118 ceph-mon[95692]: pgmap v656: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 9.1 MiB/s rd, 840 KiB/s wr, 25 op/s 2024-03-20T08:11:27.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.875+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.889+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.915+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.928+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.955+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.969+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.981+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:26.994+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.007+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.021+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.035+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.048+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.074+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.115+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.369 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:27.220+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:27.369 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:27.220+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:27.369 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:27.221+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.128+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.141+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.154+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.182+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.196+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.209+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.223+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.370 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.237+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.249+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.264+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.279+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.292+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.304+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.318+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.330+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.355+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.371 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.370+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.436 INFO:tasks.rados.rados.0.smithi118.stdout:2887: done (4 left) 2024-03-20T08:11:27.436 INFO:tasks.rados.rados.0.smithi118.stdout:2888: rollback oid 22 current snap is 302 2024-03-20T08:11:27.436 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 22 to 289 2024-03-20T08:11:27.436 INFO:tasks.rados.rados.0.smithi118.stdout:2884: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2885: finishing write tid 1 to smithi11881747-19 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2885: finishing write tid 2 to smithi11881747-19 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2883: finishing rollback tid 0 to smithi11881747-3 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2883: finishing rollback tid 1 to smithi11881747-3 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2885: finishing write tid 3 to smithi11881747-19 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2885: oid 19 updating version 0 to 7098 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2885: oid 19 version 7098 is already newer than 7097 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 7098 (ObjNum 871 snap 302 seq_num 871) dirty exists 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2885: left oid 19 (ObjNum 871 snap 302 seq_num 871) 2024-03-20T08:11:27.497 INFO:tasks.rados.rados.0.smithi118.stdout:2883: finishing rollback tid 2 to smithi11881747-3 2024-03-20T08:11:27.498 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34022 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:11:27.498 INFO:tasks.rados.rados.0.smithi118.stdout:2883: done (4 left) 2024-03-20T08:11:27.498 INFO:tasks.rados.rados.0.smithi118.stdout:2884: done (3 left) 2024-03-20T08:11:27.498 INFO:tasks.rados.rados.0.smithi118.stdout:2885: done (2 left) 2024-03-20T08:11:27.498 INFO:tasks.rados.rados.0.smithi118.stdout:2889: copy_from oid 39 from oid 33 current snap is 302 2024-03-20T08:11:27.498 INFO:tasks.rados.rados.0.smithi118.stdout:2886: expect (ObjNum 860 snap 297 seq_num 860) 2024-03-20T08:11:27.627 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:27.437+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:27.628 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:27.437+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: osdmap e651: 8 total, 8 up, 8 in 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:27.628 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:27.629 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:27.629 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:27.629 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:27.629 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: Upgrade: Updating alertmanager.a 2024-03-20T08:11:27.629 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-mon[87080]: Deploying daemon alertmanager.a on smithi118 2024-03-20T08:11:27.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.399+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.413+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.427+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.629 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.442+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.442+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.444+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.471+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.485+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.499+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.517+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.534+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.550+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.563+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.576+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.589+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.630 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.602+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.631 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.628+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.739 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:27.439+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:27.739 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:27.440+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:27.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:27.499+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:27.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:27.500+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:27.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:27.500+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:27.740 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:27.501+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:27.740 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:27.740 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: osdmap e651: 8 total, 8 up, 8 in 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: Upgrade: Updating alertmanager.a 2024-03-20T08:11:27.741 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[93121]: Deploying daemon alertmanager.a on smithi118 2024-03-20T08:11:27.742 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:27.435+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:27.742 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:27.742 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:27.742 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:27.742 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: osdmap e651: 8 total, 8 up, 8 in 2024-03-20T08:11:27.742 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:27.742 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:27.742 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:27.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:27.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:27.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:27.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:27.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:27.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: Upgrade: Updating alertmanager.a 2024-03-20T08:11:27.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-mon[95692]: Deploying daemon alertmanager.a on smithi118 2024-03-20T08:11:27.743 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:27.434+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:27.883 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:27.720+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:27.883 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:27.721+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:27.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.654+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.667+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.680+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.694+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.884 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.722+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.736+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.748+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.761+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.774+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.788+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.802+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.816+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.843+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.856+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.869+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.885 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.884+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:27.915 INFO:tasks.rados.rados.0.smithi118.stdout:2888: finishing rollback tid 1 to smithi11881747-22 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2888: finishing rollback tid 2 to smithi11881747-22 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 59109 (ObjNum 828 snap 285 seq_num 828) dirty exists 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2889: finishing copy_from racing read to smithi11881747-39 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2889: finishing copy_from to smithi11881747-39 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 34016 (ObjNum 618 snap 202 seq_num 618) dirty exists 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2886: done (2 left) 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2888: done (1 left) 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2889: done (0 left) 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2890: read oid 24 snap 301 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2890: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 302 2024-03-20T08:11:27.916 INFO:tasks.rados.rados.0.smithi118.stdout:2891: seq_num 872 ranges {6807552=212992} 2024-03-20T08:11:27.954 INFO:tasks.rados.rados.0.smithi118.stdout:2891: writing smithi11881747-29 from 6807552 to 7020544 tid 1 2024-03-20T08:11:27.955 INFO:tasks.rados.rados.0.smithi118.stdout:2892: read oid 19 snap -1 2024-03-20T08:11:27.955 INFO:tasks.rados.rados.0.smithi118.stdout:2892: expect (ObjNum 871 snap 302 seq_num 871) 2024-03-20T08:11:27.955 INFO:tasks.rados.rados.0.smithi118.stdout:2893: snap_create 2024-03-20T08:11:27.955 INFO:tasks.rados.rados.0.smithi118.stdout:2890: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:11:27.981 INFO:tasks.rados.rados.0.smithi118.stdout:2892: expect (ObjNum 871 snap 302 seq_num 871) 2024-03-20T08:11:27.990 INFO:tasks.rados.rados.0.smithi118.stdout:2891: finishing write tid 1 to smithi11881747-29 2024-03-20T08:11:27.990 INFO:tasks.rados.rados.0.smithi118.stdout:2891: finishing write tid 2 to smithi11881747-29 2024-03-20T08:11:27.990 INFO:tasks.rados.rados.0.smithi118.stdout:2891: finishing write tid 3 to smithi11881747-29 2024-03-20T08:11:27.990 INFO:tasks.rados.rados.0.smithi118.stdout:2891: oid 29 updating version 0 to 33991 2024-03-20T08:11:27.990 INFO:tasks.rados.rados.0.smithi118.stdout:2891: oid 29 updating version 33991 to 33992 2024-03-20T08:11:27.991 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 33992 (ObjNum 872 snap 302 seq_num 872) dirty exists 2024-03-20T08:11:27.991 INFO:tasks.rados.rados.0.smithi118.stdout:2891: left oid 29 (ObjNum 872 snap 302 seq_num 872) 2024-03-20T08:11:28.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.897+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.910+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.928+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.135 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.943+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.956+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.970+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.984+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:27.997+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.136 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.037+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.050+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.076+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.137 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.104+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.119+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.138 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.134+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.149+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.164+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.385 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.194+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.208+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.224+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.238+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.253+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.268+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.284+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.298+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.312+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.326+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.386 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.342+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.356+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.371+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.387 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.385+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.437 INFO:tasks.rados.rados.0.smithi118.stdout:2890: done (3 left) 2024-03-20T08:11:28.437 INFO:tasks.rados.rados.0.smithi118.stdout:2891: done (2 left) 2024-03-20T08:11:28.437 INFO:tasks.rados.rados.0.smithi118.stdout:2892: done (1 left) 2024-03-20T08:11:28.437 INFO:tasks.rados.rados.0.smithi118.stdout:2893: done (0 left) 2024-03-20T08:11:28.437 INFO:tasks.rados.rados.0.smithi118.stdout:2894: read oid 32 snap -1 2024-03-20T08:11:28.437 INFO:tasks.rados.rados.0.smithi118.stdout:2894: expect (ObjNum 785 snap 274 seq_num 785) 2024-03-20T08:11:28.438 INFO:tasks.rados.rados.0.smithi118.stdout:2895: snap_remove snap 301 2024-03-20T08:11:28.638 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:28.440+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:28.638 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:28.440+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:28.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:28.638 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: osdmap e652: 8 total, 8 up, 8 in 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:28.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.399+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.413+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.427+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.438+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.443+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.457+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.470+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.484+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.498+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.513+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.541+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.555+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.570+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.584+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.598+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.642 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.611+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.642 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.625+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.642 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.638+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:28.435+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:28.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:28.435+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:28.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:28.436+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:28.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:28.442+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: osdmap e652: 8 total, 8 up, 8 in 2024-03-20T08:11:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: osdmap e652: 8 total, 8 up, 8 in 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:28.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:28.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:28.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:28.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:28.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:28.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:28.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:28.844+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:28.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:28.844+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:28.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:28.844+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.653+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.666+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.679+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.692+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.706+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.720+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.745+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.772+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.785+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.798+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.812+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.838+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.852+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.866+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:28.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.878+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.893+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.921+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.961+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.974+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.146 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:28.987+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.000+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.014+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.027+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.040+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.053+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.067+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.080+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.094+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.108+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.120+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.147 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.146+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:28.842+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:29.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:28.843+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:29.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:28.844+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:29.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:28.844+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:29.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:28.845+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:29.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:28.845+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:29.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:28.845+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:29.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.159+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.172+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.199+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.212+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.401 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.252+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.266+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.280+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.293+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.306+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.320+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.334+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.360+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.387+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.400+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.440 INFO:tasks.rados.rados.0.smithi118.stdout:2895: done (1 left) 2024-03-20T08:11:29.440 INFO:tasks.rados.rados.0.smithi118.stdout:2896: setattr oid 49 current snap is 303 2024-03-20T08:11:29.440 INFO:tasks.rados.rados.0.smithi118.stdout:2894: expect (ObjNum 785 snap 274 seq_num 785) 2024-03-20T08:11:29.480 INFO:tasks.rados.rados.0.smithi118.stdout:2894: done (1 left) 2024-03-20T08:11:29.622 INFO:tasks.rados.rados.0.smithi118.stdout:2897: rollback oid 10 current snap is 303 2024-03-20T08:11:29.623 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 213 2024-03-20T08:11:29.623 INFO:tasks.rados.rados.0.smithi118.stdout:2898: snap_create 2024-03-20T08:11:29.624 INFO:tasks.rados.rados.0.smithi118.stdout:2897: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:11:29.624 INFO:tasks.rados.rados.0.smithi118.stdout:2897: finishing rollback tid 2 to smithi11881747-10 2024-03-20T08:11:29.624 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 44228 (ObjNum 472 snap 156 seq_num 472) dirty exists 2024-03-20T08:11:29.624 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 34060 (ObjNum 129 snap 0 seq_num 322538432) dirty exists 2024-03-20T08:11:29.663 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:29.622+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:29.663 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:29.623+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:29.663 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:29.663 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:29.663 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: osdmap e653: 8 total, 8 up, 8 in 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: pgmap v659: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:29.664 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:29.665 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:29.665 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:29.444+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:29.665 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:29.444+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:29.665 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.412+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.425+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.440+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.440+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.441+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.456+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.500+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.541+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.666 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.568+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.581+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.596+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.609+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.623+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.636+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.650+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.667 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.663+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:29.443+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:29.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:29.444+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:29.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:29.439+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:29.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:29.439+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:29.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:29.440+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:29.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:29.440+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:29.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: osdmap e653: 8 total, 8 up, 8 in 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: pgmap v659: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:29.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:29.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:29.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:29.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:29.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: osdmap e653: 8 total, 8 up, 8 in 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: pgmap v659: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.0 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 6.7 MiB/s rd, 1.2 MiB/s wr, 11 op/s 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:29.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:29.677 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:29.487+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:29.677 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:29.487+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:29.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.677+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.691+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.704+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.717+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.731+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.745+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.915 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.786+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.825+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.842+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.857+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.887+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.902+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:29.916 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.915+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.170 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.929+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.943+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.957+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.970+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.984+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:29.998+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.010+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.023+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.036+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.049+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.075+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.090+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.171 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.104+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.117+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.131+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.145+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.158+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.172 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.170+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.422 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.185+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.422 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.198+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.211+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.224+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.264+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.291+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.304+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.317+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.343+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.358+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.370+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.383+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.423 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.396+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.409+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.422+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.441 INFO:tasks.rados.rados.0.smithi118.stdout:2896: done (2 left) 2024-03-20T08:11:30.442 INFO:tasks.rados.rados.0.smithi118.stdout:2897: done (1 left) 2024-03-20T08:11:30.442 INFO:tasks.rados.rados.0.smithi118.stdout:2898: done (0 left) 2024-03-20T08:11:30.442 INFO:tasks.rados.rados.0.smithi118.stdout:2899: rollback oid 30 current snap is 304 2024-03-20T08:11:30.442 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 294 2024-03-20T08:11:30.442 INFO:tasks.rados.rados.0.smithi118.stdout:2900: snap_remove snap 258 2024-03-20T08:11:30.680 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:30.448+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:30.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:30.448+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:30.681 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:30.448+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:30.681 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:30.443+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:30.681 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:30.444+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: osdmap e654: 8 total, 8 up, 8 in 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:30.682 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:30.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:30.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:30.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:30.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:30.683 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:30.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.449+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:30.683 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.456+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.468+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.482+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.496+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.509+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.522+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.536+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.548+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.561+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.574+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.684 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.587+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.599+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.612+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.626+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.685 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.641+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.686 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.656+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.686 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.686 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.681+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:30.622+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:30.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:30.441+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:30.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:30.442+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:30.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:30.442+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:30.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:30.621+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:30.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:30.621+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:30.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:30.622+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:30.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:30.622+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:30.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:30.622+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:30.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: osdmap e654: 8 total, 8 up, 8 in 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:30.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:30.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:30.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:30.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:30.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:30.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:30.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:30.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:30.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:30.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:30.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:30.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: osdmap e654: 8 total, 8 up, 8 in 2024-03-20T08:11:30.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:30.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:30.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:30.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:30.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:30.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:30.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:30.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:30.928 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:30.447+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:30.928 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:30.447+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:30.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.694+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.708+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.722+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.734+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.746+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.759+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.771+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.942 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.786+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.800+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.813+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.827+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.840+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.855+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.874+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.888+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.901+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.943 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.914+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.927+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:30.944 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.941+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.193 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.954+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.967+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.980+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:30.993+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.006+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.020+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.033+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.194 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.045+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.059+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.072+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.084+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.097+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.111+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.124+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.195 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.138+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.196 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.152+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.196 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.167+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.196 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.179+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.196 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.192+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.446 INFO:tasks.rados.rados.0.smithi118.stdout:2900: done (1 left) 2024-03-20T08:11:31.446 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 304 2024-03-20T08:11:31.446 INFO:tasks.rados.rados.0.smithi118.stdout:2899: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:11:31.447 INFO:tasks.rados.rados.0.smithi118.stdout:2901: seq_num 873 ranges {0=229376} 2024-03-20T08:11:31.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:31.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:31.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:31.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: osdmap e655: 8 total, 8 up, 8 in 2024-03-20T08:11:31.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:31.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:31.447 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:31.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:31.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:31.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:31.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: pgmap v663: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 16 op/s 2024-03-20T08:11:31.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:31.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:31.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:31.448 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:31.448 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.206+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.448 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.220+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.233+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.246+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.260+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.285+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.299+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.313+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.326+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.340+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.358+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.372+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.387+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.401+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.449 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.414+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.428+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.442+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.446+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:31.450 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.446+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:31.450 INFO:tasks.rados.rados.0.smithi118.stdout:2901: writing smithi11881747-49 from 0 to 229376 tid 1 2024-03-20T08:11:31.450 INFO:tasks.rados.rados.0.smithi118.stdout:2902: snap_remove snap 293 2024-03-20T08:11:31.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:31.452+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:31.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:31.452+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:31.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:31.458+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:31.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:31.446+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:31.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:31.447+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:31.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:31.447+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:31.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:31.448+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:31.674 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:31 smithi118 systemd[1]: Stopping Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:11:31.674 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:31.459+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: osdmap e655: 8 total, 8 up, 8 in 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:31.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: pgmap v663: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 16 op/s 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: osdmap e655: 8 total, 8 up, 8 in 2024-03-20T08:11:31.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: pgmap v663: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 16 op/s 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:31.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:31.701 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:31.451+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:31.702 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:31.451+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:31.702 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:31.460+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:31.702 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:31.461+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.459+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.487+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.501+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.527+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.542+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.555+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.569+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.582+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.595+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.622+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.703 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.636+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.650+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.675+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.688+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.704 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.702+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.957 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.716+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.957 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.729+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.957 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.957 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.769+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.783+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.797+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.824+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.837+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.864+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.878+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.891+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.905+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.918+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.958 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.932+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.959 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.944+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:31.959 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.957+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.172 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[78399]: ts=2024-03-20T08:11:31.846Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-20T08:11:32.172 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:31 smithi118 podman[129595]: 2024-03-20 08:11:31.887195196 +0000 UTC m=+0.173150864 container died 39fe7da5bc35889e340ba127b7098e634a1086af63d3a83be8f268b4cd71babb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:32.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.970+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.983+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:31.996+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.010+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.024+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.218 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.035+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.048+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.061+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.075+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.088+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.101+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.114+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.128+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.141+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.155+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.168+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.180+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.205+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.219 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.218+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.428 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:32 smithi118 podman[129595]: 2024-03-20 08:11:32.369253717 +0000 UTC m=+0.655209382 container cleanup 39fe7da5bc35889e340ba127b7098e634a1086af63d3a83be8f268b4cd71babb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:32.428 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:32 smithi118 bash[129595]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a 2024-03-20T08:11:32.450 INFO:tasks.rados.rados.0.smithi118.stdout:2902: done (2 left) 2024-03-20T08:11:32.450 INFO:tasks.rados.rados.0.smithi118.stdout:2903: copy_from oid 12 from oid 33 current snap is 304 2024-03-20T08:11:32.451 INFO:tasks.rados.rados.0.smithi118.stdout:2904: delete oid 9 current snap is 304 2024-03-20T08:11:32.451 INFO:tasks.rados.rados.0.smithi118.stdout:2899: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:11:32.451 INFO:tasks.rados.rados.0.smithi118.stdout:2899: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:11:32.451 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 33940 (ObjNum 830 snap 285 seq_num 830) dirty exists 2024-03-20T08:11:32.451 INFO:tasks.rados.rados.0.smithi118.stdout:2901: finishing write tid 1 to smithi11881747-49 2024-03-20T08:11:32.451 INFO:tasks.rados.rados.0.smithi118.stdout:2901: finishing write tid 2 to smithi11881747-49 2024-03-20T08:11:32.452 INFO:tasks.rados.rados.0.smithi118.stdout:2901: finishing write tid 3 to smithi11881747-49 2024-03-20T08:11:32.452 INFO:tasks.rados.rados.0.smithi118.stdout:2901: oid 49 updating version 0 to 34061 2024-03-20T08:11:32.452 INFO:tasks.rados.rados.0.smithi118.stdout:2901: oid 49 updating version 34061 to 34062 2024-03-20T08:11:32.452 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 34062 (ObjNum 873 snap 304 seq_num 873) dirty exists 2024-03-20T08:11:32.452 INFO:tasks.rados.rados.0.smithi118.stdout:2901: left oid 49 (ObjNum 873 snap 304 seq_num 873) 2024-03-20T08:11:32.456 INFO:tasks.rados.rados.0.smithi118.stdout:2899: done (3 left) 2024-03-20T08:11:32.456 INFO:tasks.rados.rados.0.smithi118.stdout:2901: done (2 left) 2024-03-20T08:11:32.456 INFO:tasks.rados.rados.0.smithi118.stdout:2904: done (1 left) 2024-03-20T08:11:32.456 INFO:tasks.rados.rados.0.smithi118.stdout:2905: copy_from oid 30 from oid 39 current snap is 304 2024-03-20T08:11:32.457 INFO:tasks.rados.rados.0.smithi118.stdout:2906: setattr oid 10 current snap is 304 2024-03-20T08:11:32.457 INFO:tasks.rados.rados.0.smithi118.stdout:2907: rollback oid 21 current snap is 304 2024-03-20T08:11:32.457 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 277 2024-03-20T08:11:32.458 INFO:tasks.rados.rados.0.smithi118.stdout:2908: rmattr oid 24 current snap is 304 2024-03-20T08:11:32.458 INFO:tasks.rados.rados.0.smithi118.stdout:2908: done (4 left) 2024-03-20T08:11:32.458 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 304 2024-03-20T08:11:32.458 INFO:tasks.rados.rados.0.smithi118.stdout:2909: seq_num 875 ranges {0=262144} 2024-03-20T08:11:32.459 INFO:tasks.rados.rados.0.smithi118.stdout:2909: writing smithi11881747-13 from 0 to 262144 tid 1 2024-03-20T08:11:32.459 INFO:tasks.rados.rados.0.smithi118.stdout:2910: delete oid 4 current snap is 304 2024-03-20T08:11:32.461 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 44229 (ObjNum 472 snap 156 seq_num 472) dirty exists 2024-03-20T08:11:32.464 INFO:tasks.rados.rados.0.smithi118.stdout:2906: done (5 left) 2024-03-20T08:11:32.464 INFO:tasks.rados.rados.0.smithi118.stdout:2910: done (4 left) 2024-03-20T08:11:32.464 INFO:tasks.rados.rados.0.smithi118.stdout:2911: copy_from oid 49 from oid 47 current snap is 304 2024-03-20T08:11:32.464 INFO:tasks.rados.rados.0.smithi118.stdout:2912: delete oid 28 current snap is 304 2024-03-20T08:11:32.469 INFO:tasks.rados.rados.0.smithi118.stdout:2911: finishing copy_from to smithi11881747-49 2024-03-20T08:11:32.469 INFO:tasks.rados.rados.0.smithi118.stdout:2911: got expected ENOENT (src dne) 2024-03-20T08:11:32.469 INFO:tasks.rados.rados.0.smithi118.stdout:2909: finishing write tid 1 to smithi11881747-13 2024-03-20T08:11:32.469 INFO:tasks.rados.rados.0.smithi118.stdout:2911: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:11:32.469 INFO:tasks.rados.rados.0.smithi118.stdout:2909: finishing write tid 2 to smithi11881747-13 2024-03-20T08:11:32.474 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:32.465+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:32.475 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:32.466+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:32.475 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:32.466+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:32.475 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:32.475 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:32.475 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: osdmap e656: 8 total, 8 up, 8 in 2024-03-20T08:11:32.475 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:32.475 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:32.475 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:32.475 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:32.476 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:32.476 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:32.476 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:32.476 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:32.476 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:32.476 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:32.476 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:32.476 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:32.477 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:11:32.477 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:32.454+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:32.477 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:32.455+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:32.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.232+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.477 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.244+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.272+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.285+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.297+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.312+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.325+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.338+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.351+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.363+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.376+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.390+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.403+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.417+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.478 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.431+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.452+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:32.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.454+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.479 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.479 INFO:tasks.rados.rados.0.smithi118.stdout:2909: finishing write tid 3 to smithi11881747-13 2024-03-20T08:11:32.479 INFO:tasks.rados.rados.0.smithi118.stdout:2909: oid 13 updating version 0 to 34090 2024-03-20T08:11:32.479 INFO:tasks.rados.rados.0.smithi118.stdout:2909: oid 13 version 34090 is already newer than 34089 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 34090 (ObjNum 875 snap 304 seq_num 875) dirty exists 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2909: left oid 13 (ObjNum 875 snap 304 seq_num 875) 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2909: done (5 left) 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2911: done (4 left) 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2912: done (3 left) 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2913: setattr oid 13 current snap is 304 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2914: rollback oid 23 current snap is 304 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 23 to 302 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2915: copy_from oid 9 from oid 1 current snap is 304 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2916: rollback oid 20 current snap is 304 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 20 to 279 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2917: read oid 44 snap -1 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2917: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:11:32.480 INFO:tasks.rados.rados.0.smithi118.stdout:2918: snap_create 2024-03-20T08:11:32.481 INFO:tasks.rados.rados.0.smithi118.stdout:2916: finishing rollback tid 0 to smithi11881747-20 2024-03-20T08:11:32.481 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 34091 (ObjNum 875 snap 304 seq_num 875) dirty exists 2024-03-20T08:11:32.484 INFO:tasks.rados.rados.0.smithi118.stdout:2916: finishing rollback tid 1 to smithi11881747-20 2024-03-20T08:11:32.484 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 34064 (ObjNum 65 snap 0 seq_num 1332891739) dirty dne 2024-03-20T08:11:32.484 INFO:tasks.rados.rados.0.smithi118.stdout:2914: finishing rollback tid 0 to smithi11881747-23 2024-03-20T08:11:32.484 INFO:tasks.rados.rados.0.smithi118.stdout:2914: finishing rollback tid 1 to smithi11881747-23 2024-03-20T08:11:32.486 INFO:tasks.rados.rados.0.smithi118.stdout:2914: finishing rollback tid 2 to smithi11881747-23 2024-03-20T08:11:32.486 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 33996 (ObjNum 814 snap 279 seq_num 814) dirty exists 2024-03-20T08:11:32.492 INFO:tasks.rados.rados.0.smithi118.stdout:2907: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:11:32.492 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 56181 (ObjNum 129 snap 0 seq_num 322084720) dirty dne 2024-03-20T08:11:32.495 INFO:tasks.rados.rados.0.smithi118.stdout:2903: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:11:32.495 INFO:tasks.rados.rados.0.smithi118.stdout:2915: finishing copy_from racing read to smithi11881747-9 2024-03-20T08:11:32.500 INFO:tasks.rados.rados.0.smithi118.stdout:2915: finishing copy_from to smithi11881747-9 2024-03-20T08:11:32.500 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 34044 (ObjNum 792 snap 274 seq_num 792) dirty exists 2024-03-20T08:11:32.503 INFO:tasks.rados.rados.0.smithi118.stdout:2905: finishing copy_from racing read to smithi11881747-30 2024-03-20T08:11:32.516 INFO:tasks.rados.rados.0.smithi118.stdout:2903: finishing copy_from to smithi11881747-12 2024-03-20T08:11:32.516 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34016 (ObjNum 618 snap 202 seq_num 618) dirty exists 2024-03-20T08:11:32.519 INFO:tasks.rados.rados.0.smithi118.stdout:2905: finishing copy_from to smithi11881747-30 2024-03-20T08:11:32.519 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 34016 (ObjNum 618 snap 202 seq_num 618) dirty exists 2024-03-20T08:11:32.547 INFO:tasks.rados.rados.0.smithi118.stdout:2917: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:11:32.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.538+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.728 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.553+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.582+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.594+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.729 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.607+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.620+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.633+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.646+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.659+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.673+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.687+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.700+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.731 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.714+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.732 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.771 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:32.447+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:32.771 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:32.447+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:32.771 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:32.449+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:32.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:32.453+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:32.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:32.453+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:32.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:32.453+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:32.771 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:32.453+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:32.772 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:32.455+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:32.772 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:32 smithi118 podman[129614]: 2024-03-20 08:11:32.662949579 +0000 UTC m=+0.773384695 container remove 39fe7da5bc35889e340ba127b7098e634a1086af63d3a83be8f268b4cd71babb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:32.772 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:32.449+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:32.772 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:32.449+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:32.772 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:32.455+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: osdmap e656: 8 total, 8 up, 8 in 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:32.773 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: osdmap e656: 8 total, 8 up, 8 in 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:32.774 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:32.775 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:32.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.742+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.989 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.768+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.781+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.795+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.808+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.822+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.836+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.850+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.863+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.878+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.892+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.906+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.920+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.935+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.990 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.948+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.991 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.991 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:32.991 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:32.989+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.028 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:32 smithi118 podman[129614]: 2024-03-20 08:11:32.772380227 +0000 UTC m=+0.882815351 volume remove 839383f78a10d31de2a2c2ef160af5db0a47d82eac323d9e6bf6264db05c36ac 2024-03-20T08:11:33.028 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:33 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-20T08:11:33.028 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:33 smithi118 systemd[1]: Stopped Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:11:33.028 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:33 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@alertmanager.a.service: Consumed 1.856s CPU time. 2024-03-20T08:11:33.241 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.003+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.241 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.241 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.031+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.241 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.241 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.057+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.241 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.070+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.241 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.092+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.241 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.106+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.119+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.132+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.145+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.159+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.173+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.187+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.200+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.227+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.242 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.241+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.422 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:33 smithi118 systemd[1]: Starting Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:11:33.452 INFO:tasks.rados.rados.0.smithi118.stdout:2903: done (8 left) 2024-03-20T08:11:33.452 INFO:tasks.rados.rados.0.smithi118.stdout:2905: done (7 left) 2024-03-20T08:11:33.452 INFO:tasks.rados.rados.0.smithi118.stdout:2907: done (6 left) 2024-03-20T08:11:33.452 INFO:tasks.rados.rados.0.smithi118.stdout:2913: done (5 left) 2024-03-20T08:11:33.452 INFO:tasks.rados.rados.0.smithi118.stdout:2914: done (4 left) 2024-03-20T08:11:33.452 INFO:tasks.rados.rados.0.smithi118.stdout:2915: done (3 left) 2024-03-20T08:11:33.452 INFO:tasks.rados.rados.0.smithi118.stdout:2916: done (2 left) 2024-03-20T08:11:33.452 INFO:tasks.rados.rados.0.smithi118.stdout:2917: done (1 left) 2024-03-20T08:11:33.453 INFO:tasks.rados.rados.0.smithi118.stdout:2918: done (0 left) 2024-03-20T08:11:33.453 INFO:tasks.rados.rados.0.smithi118.stdout:2919: copy_from oid 2 from oid 5 current snap is 305 2024-03-20T08:11:33.453 INFO:tasks.rados.rados.0.smithi118.stdout:2920: delete oid 31 current snap is 305 2024-03-20T08:11:33.460 INFO:tasks.rados.rados.0.smithi118.stdout:2920: done (1 left) 2024-03-20T08:11:33.460 INFO:tasks.rados.rados.0.smithi118.stdout:2921: copy_from oid 16 from oid 33 current snap is 305 2024-03-20T08:11:33.460 INFO:tasks.rados.rados.0.smithi118.stdout:2922: snap_remove snap 169 2024-03-20T08:11:33.500 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:33.459+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:33.500 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:33.461+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:33.500 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:33.459+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:33.500 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:33.460+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: osdmap e657: 8 total, 8 up, 8 in 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:33.501 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:33.502 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:33.502 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:33.502 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:33.502 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:33.502 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-mon[87080]: pgmap v666: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-03-20T08:11:33.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.254+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.267+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.282+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.294+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.502 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.309+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.323+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.336+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.351+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.365+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.379+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.392+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.406+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.418+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.432+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.447+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.457+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.457+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.469+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.484+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.503 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.500+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.514+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.528+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.541+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.554+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.567+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.759 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.581+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.595+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.607+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.621+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.634+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.648+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.663+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.677+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.690+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.705+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.718+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.745+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.760 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.759+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:33.854 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:33.453+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:33.855 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:33.453+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:33.855 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:33 smithi118 podman[129743]: 2024-03-20 08:11:33.397076693 +0000 UTC m=+0.024071318 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-20T08:11:33.855 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:33 smithi118 podman[129743]: 2024-03-20 08:11:33.523750804 +0000 UTC m=+0.150745410 volume create 33f49e902662488d1a4f740318bae0621d42b6deebd0cb7728c90ba06b663908 2024-03-20T08:11:33.855 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:33 smithi118 podman[129743]: 2024-03-20 08:11:33.632645366 +0000 UTC m=+0.259639980 container create 68a6e855b9aafc6c1e26ceb62524492a7f3e4be8aea4f202bbb1458d55f51f4a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:33.855 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:11:33] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:11:33.856 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:33.465+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:33.856 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:33.463+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:33.856 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:33.463+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:33.856 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:33.463+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:33.856 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:33.464+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:33.856 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:33.456+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: osdmap e657: 8 total, 8 up, 8 in 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:33.857 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[93121]: pgmap v666: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: osdmap e657: 8 total, 8 up, 8 in 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:33.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:33.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:33.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:33.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:33.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:33.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:33.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:33.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:33.859 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:33 smithi118 ceph-mon[95692]: pgmap v666: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-03-20T08:11:34.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.772+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.786+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.017 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.799+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.813+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.825+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.839+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.853+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.867+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.882+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.895+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.018 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.908+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.922+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.935+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.949+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.963+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.977+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:33.990+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.004+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.019 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.017+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.044+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.057+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.071+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.085+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.100+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.113+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.278 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.127+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.139+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.152+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.166+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.179+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.192+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.212+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.225+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.238+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.252+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.265+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.279 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.277+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.422 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 podman[129743]: 2024-03-20 08:11:34.052613213 +0000 UTC m=+0.679607831 container init 68a6e855b9aafc6c1e26ceb62524492a7f3e4be8aea4f202bbb1458d55f51f4a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:34.422 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 podman[129743]: 2024-03-20 08:11:34.058336668 +0000 UTC m=+0.685331282 container start 68a6e855b9aafc6c1e26ceb62524492a7f3e4be8aea4f202bbb1458d55f51f4a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:11:34.422 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:34.070Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-20T08:11:34.422 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:34.070Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-20T08:11:34.423 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:34.072Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.118 port=9094 2024-03-20T08:11:34.423 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:34.073Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-20T08:11:34.423 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:34.102Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-20T08:11:34.423 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:34.102Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-20T08:11:34.423 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:34.106Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-20T08:11:34.423 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:34.106Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-20T08:11:34.458 INFO:tasks.rados.rados.0.smithi118.stdout:2922: done (2 left) 2024-03-20T08:11:34.458 INFO:tasks.rados.rados.0.smithi118.stdout:2923: rmattr oid 13 current snap is 305 2024-03-20T08:11:34.458 INFO:tasks.rados.rados.0.smithi118.stdout:2919: finishing copy_from racing read to smithi11881747-2 2024-03-20T08:11:34.459 INFO:tasks.rados.rados.0.smithi118.stdout:2924: setattr oid 29 current snap is 305 2024-03-20T08:11:34.459 INFO:tasks.rados.rados.0.smithi118.stdout:2919: finishing copy_from to smithi11881747-2 2024-03-20T08:11:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 34084 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:11:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:2921: finishing copy_from racing read to smithi11881747-16 2024-03-20T08:11:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:2921: finishing copy_from to smithi11881747-16 2024-03-20T08:11:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 34016 (ObjNum 618 snap 202 seq_num 618) dirty exists 2024-03-20T08:11:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:2919: done (3 left) 2024-03-20T08:11:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:2921: done (2 left) 2024-03-20T08:11:34.460 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 305 2024-03-20T08:11:34.461 INFO:tasks.rados.rados.0.smithi118.stdout:2925: seq_num 879 ranges {0=327680} 2024-03-20T08:11:34.463 INFO:tasks.rados.rados.0.smithi118.stdout:2925: writing smithi11881747-27 from 0 to 327680 tid 1 2024-03-20T08:11:34.463 INFO:tasks.rados.rados.0.smithi118.stdout:2926: read oid 33 snap -1 2024-03-20T08:11:34.463 INFO:tasks.rados.rados.0.smithi118.stdout:2926: expect (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:11:34.463 INFO:tasks.rados.rados.0.smithi118.stdout:2927: snap_remove snap 238 2024-03-20T08:11:34.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:34.487+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:34.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:34.457+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:34.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:34.457+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: osdmap e658: 8 total, 8 up, 8 in 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:34.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.291+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.306+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.320+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.334+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.347+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.361+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.408+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.421+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.437+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.450+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.460+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.474+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap da after trimming was started 2024-03-20T08:11:34.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:34.482+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 164=[]:{ea=[e8],fa=[fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],fd=[fd,fc],105=[105,104,103,102,101,fe],129=[129,128,127,126,125,124,122,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a],15b=[15b,15a,159,158,157,156,155],160=[160,15f,15e,15d],162=[162,161],164=[164,163]} for object 5:79a2dcb6:::smithi11881747-26:e5 2024-03-20T08:11:34.922 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 bash[129743]: 68a6e855b9aafc6c1e26ceb62524492a7f3e4be8aea4f202bbb1458d55f51f4a 2024-03-20T08:11:34.922 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:34 smithi118 systemd[1]: Started Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:11:34.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:34.458+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:34.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:34.459+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:34.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:34.459+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:34.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:34.455+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:34.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:34.455+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:34.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:34.456+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:34.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:34.456+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:34.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:34.457+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:34.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:34.459+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:34.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:34.459+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:34.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:34.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: osdmap e658: 8 total, 8 up, 8 in 2024-03-20T08:11:34.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:34.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:34.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:34.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:34.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:34.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:34.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:34.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:34.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:34.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:34.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:34.925 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:34.464+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:34.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: osdmap e658: 8 total, 8 up, 8 in 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{ea=[e5],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:f6 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:34.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:2927: done (4 left) 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 305 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 34092 (ObjNum 875 snap 304 seq_num 875) dirty exists 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 34017 (ObjNum 872 snap 302 seq_num 872) dirty exists 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:2925: finishing write tid 1 to smithi11881747-27 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:2925: finishing write tid 2 to smithi11881747-27 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:2925: finishing write tid 3 to smithi11881747-27 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:2925: oid 27 updating version 0 to 34065 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:2925: oid 27 updating version 34065 to 34066 2024-03-20T08:11:35.459 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 34066 (ObjNum 879 snap 305 seq_num 879) dirty exists 2024-03-20T08:11:35.460 INFO:tasks.rados.rados.0.smithi118.stdout:2925: left oid 27 (ObjNum 879 snap 305 seq_num 879) 2024-03-20T08:11:35.460 INFO:tasks.rados.rados.0.smithi118.stdout:2926: expect (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:11:35.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:35.459+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:35.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:35.482+0000 7ffa6f970700 -1 osd.5 pg_epoch: 660 pg[5.es1( v 659'65236 (635'58615,659'65236] local-lis/les=470/471 n=51 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [4,5,0]p4(0) r=1 lpr=470 luod=0'0 crt=659'65236 mlcod 659'65236 active mbc={} ps=82] update_snap_map remove_oid 5:70e08578:::smithi11881747-24:9e failed with -2 2024-03-20T08:11:35.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:35.483+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:35.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:35.483+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:35.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:35.324+0000 7ffb41f18700 -1 osd.7 pg_epoch: 659 pg[5.9s0( v 658'1215 (0'0,658'1215] local-lis/les=551/552 n=35 ec=73/73 lis/c=551/551 les/c/f=552/552/0 sis=551) [7,6,1]p7(0) r=0 lpr=551 crt=658'1215 lcod 640'1213 mlcod 640'1213 active+clean+snaptrim trimq=60 ps=121] update_snap_map remove_oid 5:902426e2:::smithi11881747-46:ea failed with -2 2024-03-20T08:11:35.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:35.324+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:35.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:35.327+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:35.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:35.480+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:35.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:35.481+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:35.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:35.481+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:35.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:35.481+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:35.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:35.481+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:35.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:35.481+0000 7fa5cf030700 -1 osd.4 pg_epoch: 660 pg[5.es0( v 659'65236 (635'58615,659'65236] local-lis/les=470/471 n=51 ec=73/73 lis/c=470/470 les/c/f=471/471/0 sis=470) [4,5,0]p4(0) r=0 lpr=470 crt=659'65236 lcod 658'65234 mlcod 658'65234 active+clean+snaptrim trimq=100 ps=82] update_snap_map remove_oid 5:70e08578:::smithi11881747-24:9e failed with -2 2024-03-20T08:11:35.641 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:35.493+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:35.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:35.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:35.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:35.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:35.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: osdmap e659: 8 total, 8 up, 8 in 2024-03-20T08:11:35.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 164=[]:{ea=[e8],fa=[fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],fd=[fd,fc],105=[105,104,103,102,101,fe],129=[129,128,127,126,125,124,122,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a],15b=[15b,15a,159,158,157,156,155],160=[160,15f,15e,15d],162=[162,161],164=[164,163]} for object 5:79a2dcb6:::smithi11881747-26:e5 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: pgmap v669: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 3 op/s 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:35.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:35.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:35 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:35.682 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: osdmap e659: 8 total, 8 up, 8 in 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:35.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 164=[]:{ea=[e8],fa=[fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],fd=[fd,fc],105=[105,104,103,102,101,fe],129=[129,128,127,126,125,124,122,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a],15b=[15b,15a,159,158,157,156,155],160=[160,15f,15e,15d],162=[162,161],164=[164,163]} for object 5:79a2dcb6:::smithi11881747-26:e5 2024-03-20T08:11:35.684 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:35.480+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:35.684 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:35.481+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:35.684 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:35.458+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:35.684 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:35.458+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:35.684 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:35.459+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:35.684 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:35.460+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:35.685 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:35.459+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:35.685 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:35.685 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: pgmap v669: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 3 op/s 2024-03-20T08:11:35.685 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:35.685 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:35.685 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: osdmap e659: 8 total, 8 up, 8 in 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:35.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:35.687 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:35.687 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:35.687 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:35.687 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 164=[]:{ea=[e8],fa=[fa,f8,f7,f5,f4,f3,f2,f1,f0,eb],fd=[fd,fc],105=[105,104,103,102,101,fe],129=[129,128,127,126,125,124,122,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a],15b=[15b,15a,159,158,157,156,155],160=[160,15f,15e,15d],162=[162,161],164=[164,163]} for object 5:79a2dcb6:::smithi11881747-26:e5 2024-03-20T08:11:35.687 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:35.482+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:35.687 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:35.687 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: pgmap v669: 121 pgs: 2 active+clean+snaptrim, 13 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 3 op/s 2024-03-20T08:11:35.687 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:35.687 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:35.688 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:35 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:36.221 INFO:tasks.rados.rados.0.smithi118.stdout:2928: seq_num 880 ranges {0=344064} 2024-03-20T08:11:36.223 INFO:tasks.rados.rados.0.smithi118.stdout:2928: writing smithi11881747-20 from 0 to 344064 tid 1 2024-03-20T08:11:36.224 INFO:tasks.rados.rados.0.smithi118.stdout:2923: done (4 left) 2024-03-20T08:11:36.224 INFO:tasks.rados.rados.0.smithi118.stdout:2924: done (3 left) 2024-03-20T08:11:36.224 INFO:tasks.rados.rados.0.smithi118.stdout:2925: done (2 left) 2024-03-20T08:11:36.224 INFO:tasks.rados.rados.0.smithi118.stdout:2926: done (1 left) 2024-03-20T08:11:36.224 INFO:tasks.rados.rados.0.smithi118.stdout:2929: snap_remove snap 282 2024-03-20T08:11:36.422 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:36.074Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000808222s 2024-03-20T08:11:36.465 INFO:tasks.rados.rados.0.smithi118.stdout:2929: done (1 left) 2024-03-20T08:11:36.465 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 305 2024-03-20T08:11:36.465 INFO:tasks.rados.rados.0.smithi118.stdout:2928: finishing write tid 1 to smithi11881747-20 2024-03-20T08:11:36.465 INFO:tasks.rados.rados.0.smithi118.stdout:2930: seq_num 881 ranges {0=360448} 2024-03-20T08:11:36.466 INFO:tasks.rados.rados.0.smithi118.stdout:2930: writing smithi11881747-18 from 0 to 360448 tid 1 2024-03-20T08:11:36.466 INFO:tasks.rados.rados.0.smithi118.stdout:2931: snap_create 2024-03-20T08:11:36.466 INFO:tasks.rados.rados.0.smithi118.stdout:2928: finishing write tid 2 to smithi11881747-20 2024-03-20T08:11:36.467 INFO:tasks.rados.rados.0.smithi118.stdout:2928: finishing write tid 3 to smithi11881747-20 2024-03-20T08:11:36.467 INFO:tasks.rados.rados.0.smithi118.stdout:2928: oid 20 updating version 0 to 34067 2024-03-20T08:11:36.467 INFO:tasks.rados.rados.0.smithi118.stdout:2928: oid 20 updating version 34067 to 34068 2024-03-20T08:11:36.467 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 34068 (ObjNum 880 snap 305 seq_num 880) dirty exists 2024-03-20T08:11:36.467 INFO:tasks.rados.rados.0.smithi118.stdout:2928: left oid 20 (ObjNum 880 snap 305 seq_num 880) 2024-03-20T08:11:36.471 INFO:tasks.rados.rados.0.smithi118.stdout:2930: finishing write tid 1 to smithi11881747-18 2024-03-20T08:11:36.471 INFO:tasks.rados.rados.0.smithi118.stdout:2930: finishing write tid 2 to smithi11881747-18 2024-03-20T08:11:36.472 INFO:tasks.rados.rados.0.smithi118.stdout:2930: finishing write tid 3 to smithi11881747-18 2024-03-20T08:11:36.472 INFO:tasks.rados.rados.0.smithi118.stdout:2930: oid 18 updating version 0 to 34069 2024-03-20T08:11:36.472 INFO:tasks.rados.rados.0.smithi118.stdout:2930: oid 18 updating version 34069 to 34070 2024-03-20T08:11:36.472 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 34070 (ObjNum 881 snap 305 seq_num 881) dirty exists 2024-03-20T08:11:36.472 INFO:tasks.rados.rados.0.smithi118.stdout:2930: left oid 18 (ObjNum 881 snap 305 seq_num 881) 2024-03-20T08:11:36.741 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:36.464+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:36.742 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:36.472+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:36.742 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:36.472+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:36.742 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:36.742 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:36.742 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:36.742 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: osdmap e660: 8 total, 8 up, 8 in 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:36.743 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: osdmap e660: 8 total, 8 up, 8 in 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:36.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:36.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:36.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:36.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:36.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:36.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:36.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:36.745 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:36.746 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:36.746 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:36.761 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:36.465+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:36.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:36.465+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:36.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:36.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:36.761 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:36.466+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: osdmap e660: 8 total, 8 up, 8 in 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 14d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:36.762 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:36.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:36.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:36.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:36.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:36.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:36.763 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:37.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:36.761+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:37.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:36.762+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:37.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:36.761+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:37.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:36.762+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:37.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:36.761+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:37.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:36.761+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:37.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:36.761+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:37.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:36.760+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:37.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:36.760+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:37.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:36.761+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:37.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:36.761+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:37.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:36.761+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:37.470 INFO:tasks.rados.rados.0.smithi118.stdout:2928: done (2 left) 2024-03-20T08:11:37.470 INFO:tasks.rados.rados.0.smithi118.stdout:2930: done (1 left) 2024-03-20T08:11:37.470 INFO:tasks.rados.rados.0.smithi118.stdout:2931: done (0 left) 2024-03-20T08:11:37.470 INFO:tasks.rados.rados.0.smithi118.stdout:2932: read oid 30 snap -1 2024-03-20T08:11:37.470 INFO:tasks.rados.rados.0.smithi118.stdout:2932: expect (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:11:37.470 INFO:tasks.rados.rados.0.smithi118.stdout:2933: delete oid 49 current snap is 306 2024-03-20T08:11:37.476 INFO:tasks.rados.rados.0.smithi118.stdout:2933: done (1 left) 2024-03-20T08:11:37.476 INFO:tasks.rados.rados.0.smithi118.stdout:2934: rollback oid 33 current snap is 306 2024-03-20T08:11:37.476 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 304 2024-03-20T08:11:37.476 INFO:tasks.rados.rados.0.smithi118.stdout:2935: copy_from oid 25 from oid 40 current snap is 306 2024-03-20T08:11:37.477 INFO:tasks.rados.rados.0.smithi118.stdout:2936: read oid 8 snap -1 2024-03-20T08:11:37.477 INFO:tasks.rados.rados.0.smithi118.stdout:2936: expect (ObjNum 740 snap 253 seq_num 740) 2024-03-20T08:11:37.477 INFO:tasks.rados.rados.0.smithi118.stdout:2937: snap_remove snap 279 2024-03-20T08:11:37.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:37.470+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:37.786 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:37.471+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:37.786 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:37.471+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:37.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:37.473+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:37.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:37.478+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:37.786 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:37.478+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: osdmap e661: 8 total, 8 up, 8 in 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: pgmap v672: 121 pgs: 1 active+clean+snaptrim, 14 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.4 MiB/s wr, 22 op/s 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:37.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:37.788 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:37.788 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:37.788 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:37.788 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:37.788 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:37.788 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:37.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:37.480+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:37.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:37.482+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:37.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:37.469+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:37.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:37.475+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:37.480+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:37.480+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:37.481+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:37.500+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: osdmap e661: 8 total, 8 up, 8 in 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: pgmap v672: 121 pgs: 1 active+clean+snaptrim, 14 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.4 MiB/s wr, 22 op/s 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:37.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:11:37.462+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 14 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: osdmap e661: 8 total, 8 up, 8 in 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:37.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: pgmap v672: 121 pgs: 1 active+clean+snaptrim, 14 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.4 MiB/s wr, 22 op/s 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:37.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:38.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:37.785+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:38.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:37.785+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:38.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:37.785+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:38.471 INFO:tasks.rados.rados.0.smithi118.stdout:2937: done (4 left) 2024-03-20T08:11:38.471 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 306 2024-03-20T08:11:38.471 INFO:tasks.rados.rados.0.smithi118.stdout:2935: finishing copy_from racing read to smithi11881747-25 2024-03-20T08:11:38.471 INFO:tasks.rados.rados.0.smithi118.stdout:2938: seq_num 883 ranges {9289728=393216} 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:2938: writing smithi11881747-6 from 9289728 to 9682944 tid 1 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:2939: delete oid 9 current snap is 306 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:2935: finishing copy_from to smithi11881747-25 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 34058 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:2934: finishing rollback tid 0 to smithi11881747-33 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:2934: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:2934: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34025 (ObjNum 618 snap 202 seq_num 618) dirty exists 2024-03-20T08:11:38.523 INFO:tasks.rados.rados.0.smithi118.stdout:2932: expect (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:11:38.822 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:38.526+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:38.822 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:38.526+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:38.822 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:38.528+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:38.823 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:38.471+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:38.823 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:38.823 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 14 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:11:38.823 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: osdmap e662: 8 total, 8 up, 8 in 2024-03-20T08:11:38.823 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:38.824 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 14 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: osdmap e662: 8 total, 8 up, 8 in 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:38.825 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:38.826 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:38.826 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:38.826 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:38.826 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:38.826 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:38.826 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:38.826 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:38.827 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:38.827 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:38.827 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:38.827 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:38.474+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:38.827 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:38.474+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:38.827 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:38.470+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:38.828 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:38.527+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:38.828 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:38.531+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:38.530+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:38.530+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:38.828 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:38.530+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 14 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: osdmap e662: 8 total, 8 up, 8 in 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:38.829 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:38.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:38.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:38.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:38.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:38.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:38.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:38.830 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:39.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:38.822+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:39.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:38.822+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:39.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:38.821+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:39.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:38.821+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:39.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:38.822+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:39.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:38.822+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:39.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:38.822+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:39.229 INFO:tasks.rados.rados.0.smithi118.stdout:2936: expect (ObjNum 740 snap 253 seq_num 740) 2024-03-20T08:11:39.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:39.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:39.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:39.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: osdmap e663: 8 total, 8 up, 8 in 2024-03-20T08:11:39.777 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: pgmap v675: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 27 op/s 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:39.778 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: osdmap e663: 8 total, 8 up, 8 in 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: pgmap v675: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 27 op/s 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:39.779 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:39.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:39.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:39.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:39.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:39.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:39.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:39.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:39.780 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:39.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:39.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:39.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:39.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:39 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:39.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: osdmap e663: 8 total, 8 up, 8 in 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: pgmap v675: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 27 op/s 2024-03-20T08:11:39.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:39.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:39.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:39.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:39.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:39.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:39.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:39.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:39 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:40.293 INFO:tasks.rados.rados.0.smithi118.stdout:2938: finishing write tid 1 to smithi11881747-6 2024-03-20T08:11:40.293 INFO:tasks.rados.rados.0.smithi118.stdout:2938: finishing write tid 2 to smithi11881747-6 2024-03-20T08:11:40.293 INFO:tasks.rados.rados.0.smithi118.stdout:2932: done (5 left) 2024-03-20T08:11:40.293 INFO:tasks.rados.rados.0.smithi118.stdout:2934: done (4 left) 2024-03-20T08:11:40.293 INFO:tasks.rados.rados.0.smithi118.stdout:2935: done (3 left) 2024-03-20T08:11:40.293 INFO:tasks.rados.rados.0.smithi118.stdout:2936: done (2 left) 2024-03-20T08:11:40.294 INFO:tasks.rados.rados.0.smithi118.stdout:2939: done (1 left) 2024-03-20T08:11:40.294 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 306 2024-03-20T08:11:40.294 INFO:tasks.rados.rados.0.smithi118.stdout:2938: finishing write tid 3 to smithi11881747-6 2024-03-20T08:11:40.294 INFO:tasks.rados.rados.0.smithi118.stdout:2938: oid 6 updating version 0 to 45600 2024-03-20T08:11:40.294 INFO:tasks.rados.rados.0.smithi118.stdout:2938: oid 6 updating version 45600 to 45601 2024-03-20T08:11:40.294 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 45601 (ObjNum 883 snap 306 seq_num 883) dirty exists 2024-03-20T08:11:40.294 INFO:tasks.rados.rados.0.smithi118.stdout:2938: left oid 6 (ObjNum 883 snap 306 seq_num 883) 2024-03-20T08:11:40.294 INFO:tasks.rados.rados.0.smithi118.stdout:2940: seq_num 885 ranges {0=425984} 2024-03-20T08:11:40.295 INFO:tasks.rados.rados.0.smithi118.stdout:2940: writing smithi11881747-31 from 0 to 425984 tid 1 2024-03-20T08:11:40.295 INFO:tasks.rados.rados.0.smithi118.stdout:2938: done (1 left) 2024-03-20T08:11:40.295 INFO:tasks.rados.rados.0.smithi118.stdout:2941: snap_remove snap 295 2024-03-20T08:11:40.473 INFO:tasks.rados.rados.0.smithi118.stdout:2941: done (1 left) 2024-03-20T08:11:40.473 INFO:tasks.rados.rados.0.smithi118.stdout:2942: rmattr oid 26 current snap is 306 2024-03-20T08:11:40.473 INFO:tasks.rados.rados.0.smithi118.stdout:2940: finishing write tid 1 to smithi11881747-31 2024-03-20T08:11:40.473 INFO:tasks.rados.rados.0.smithi118.stdout:2940: finishing write tid 2 to smithi11881747-31 2024-03-20T08:11:40.473 INFO:tasks.rados.rados.0.smithi118.stdout:2940: finishing write tid 3 to smithi11881747-31 2024-03-20T08:11:40.474 INFO:tasks.rados.rados.0.smithi118.stdout:2940: oid 31 updating version 0 to 44232 2024-03-20T08:11:40.474 INFO:tasks.rados.rados.0.smithi118.stdout:2940: oid 31 version 44232 is already newer than 44231 2024-03-20T08:11:40.474 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 44232 (ObjNum 885 snap 306 seq_num 885) dirty exists 2024-03-20T08:11:40.474 INFO:tasks.rados.rados.0.smithi118.stdout:2940: left oid 31 (ObjNum 885 snap 306 seq_num 885) 2024-03-20T08:11:40.474 INFO:tasks.rados.rados.0.smithi118.stdout:2940: done (1 left) 2024-03-20T08:11:40.474 INFO:tasks.rados.rados.0.smithi118.stdout:2943: delete oid 37 current snap is 306 2024-03-20T08:11:40.481 INFO:tasks.rados.rados.0.smithi118.stdout:2943: done (1 left) 2024-03-20T08:11:40.481 INFO:tasks.rados.rados.0.smithi118.stdout:2944: rollback oid 28 current snap is 306 2024-03-20T08:11:40.481 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 28 to 213 2024-03-20T08:11:40.482 INFO:tasks.rados.rados.0.smithi118.stdout:2945: read oid 14 snap -1 2024-03-20T08:11:40.482 INFO:tasks.rados.rados.0.smithi118.stdout:2945: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:11:40.482 INFO:tasks.rados.rados.0.smithi118.stdout:2946: setattr oid 36 current snap is 306 2024-03-20T08:11:40.483 INFO:tasks.rados.rados.0.smithi118.stdout:2947: rmattr oid 24 current snap is 306 2024-03-20T08:11:40.483 INFO:tasks.rados.rados.0.smithi118.stdout:2947: done (4 left) 2024-03-20T08:11:40.484 INFO:tasks.rados.rados.0.smithi118.stdout:2948: snap_create 2024-03-20T08:11:40.484 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65242 (ObjNum 860 snap 297 seq_num 860) dirty exists 2024-03-20T08:11:40.485 INFO:tasks.rados.rados.0.smithi118.stdout:2944: finishing rollback tid 1 to smithi11881747-28 2024-03-20T08:11:40.487 INFO:tasks.rados.rados.0.smithi118.stdout:2944: finishing rollback tid 2 to smithi11881747-28 2024-03-20T08:11:40.487 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 34019 (ObjNum 644 snap 211 seq_num 644) dirty exists 2024-03-20T08:11:40.488 INFO:tasks.rados.rados.0.smithi118.stdout:2945: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:11:40.518 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45587 (ObjNum 33 snap 0 seq_num 323217712) dirty exists 2024-03-20T08:11:40.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:40.475+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:40.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:40.478+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:40.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:40.478+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:40.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:40.483+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:40.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:40.624+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:40.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:40.624+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:40.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:40.624+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:40.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:40.482+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:40.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:40.482+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:40.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:40.475+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:40.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:40.472+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:40.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:40.473+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:40.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:40.481+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:40.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:40.481+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:40.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:40.481+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:40.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:40.482+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:40.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:40.479+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:41.479 INFO:tasks.rados.rados.0.smithi118.stdout:2942: done (4 left) 2024-03-20T08:11:41.479 INFO:tasks.rados.rados.0.smithi118.stdout:2944: done (3 left) 2024-03-20T08:11:41.479 INFO:tasks.rados.rados.0.smithi118.stdout:2945: done (2 left) 2024-03-20T08:11:41.479 INFO:tasks.rados.rados.0.smithi118.stdout:2946: done (1 left) 2024-03-20T08:11:41.479 INFO:tasks.rados.rados.0.smithi118.stdout:2948: done (0 left) 2024-03-20T08:11:41.479 INFO:tasks.rados.rados.0.smithi118.stdout:2949: snap_remove snap 277 2024-03-20T08:11:41.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:41.477+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:41.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:41.477+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:41.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:41.477+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:41.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:41.478+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:41.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:41.478+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:41.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:41.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:41.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: osdmap e664: 8 total, 8 up, 8 in 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:41.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: pgmap v677: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 10 op/s 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: osdmap e664: 8 total, 8 up, 8 in 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:41.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: pgmap v677: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 10 op/s 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:41.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:41.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:41.479+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:41.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:41.478+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:41.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:41.478+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:41.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:41.479+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:41.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:41.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:41.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: osdmap e664: 8 total, 8 up, 8 in 2024-03-20T08:11:41.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:41.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:41.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:41.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:41.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: pgmap v677: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 10 op/s 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:42.047 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:41.701+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:42.047 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:41.701+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:42.047 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:41.701+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:42.314 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:42.047+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:42.314 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:42.047+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:42.314 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:42.047+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:42.314 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:42.048+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:42.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:42.047+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:42.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:42.047+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:42.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:42.047+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:42.479 INFO:tasks.rados.rados.0.smithi118.stdout:2949: done (0 left) 2024-03-20T08:11:42.480 INFO:tasks.rados.rados.0.smithi118.stdout:2950: setattr oid 32 current snap is 307 2024-03-20T08:11:42.482 INFO:tasks.rados.rados.0.smithi118.stdout:2951: copy_from oid 38 from oid 50 current snap is 307 2024-03-20T08:11:42.482 INFO:tasks.rados.rados.0.smithi118.stdout:2952: read oid 1 snap 266 2024-03-20T08:11:42.482 INFO:tasks.rados.rados.0.smithi118.stdout:2952: expect (ObjNum 637 snap 205 seq_num 637) 2024-03-20T08:11:42.482 INFO:tasks.rados.rados.0.smithi118.stdout:2953: read oid 40 snap 213 2024-03-20T08:11:42.482 INFO:tasks.rados.rados.0.smithi118.stdout:2953: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:11:42.482 INFO:tasks.rados.rados.0.smithi118.stdout:2954: setattr oid 27 current snap is 307 2024-03-20T08:11:42.483 INFO:tasks.rados.rados.0.smithi118.stdout:2955: rmattr oid 37 current snap is 307 2024-03-20T08:11:42.483 INFO:tasks.rados.rados.0.smithi118.stdout:2955: done (5 left) 2024-03-20T08:11:42.484 INFO:tasks.rados.rados.0.smithi118.stdout:2956: copy_from oid 22 from oid 37 current snap is 307 2024-03-20T08:11:42.484 INFO:tasks.rados.rados.0.smithi118.stdout:2957: snap_remove snap 235 2024-03-20T08:11:42.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:42.479+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:11:42.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:42.489+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:42.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:42.489+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:42.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:42.490+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:42.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:42.492+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:42.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:42.492+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:42.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:42.514+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:42.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:42.514+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:42.641 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:42.514+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:42.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:42.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:42.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:42.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:42.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:42.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: osdmap e665: 8 total, 8 up, 8 in 2024-03-20T08:11:42.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:42.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:42.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:42.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:42.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:42.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:42.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:42.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:42.742 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:42.486+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:42.742 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:42.488+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:42.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:42.478+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:42.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:42.478+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:42.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:42.480+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:42.742 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:42.480+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: osdmap e665: 8 total, 8 up, 8 in 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:42.743 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: osdmap e665: 8 total, 8 up, 8 in 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:42.744 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:42.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:42.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:42.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:42.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:42.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:42.745 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:42.745 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:42.480+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:42.746 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:42.493+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:43.491 INFO:tasks.rados.rados.0.smithi118.stdout:2957: done (6 left) 2024-03-20T08:11:43.492 INFO:tasks.rados.rados.0.smithi118.stdout:2958: rollback oid 17 current snap is 307 2024-03-20T08:11:43.492 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 17 to 263 2024-03-20T08:11:43.492 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 34072 (ObjNum 879 snap 305 seq_num 879) dirty exists 2024-03-20T08:11:43.492 INFO:tasks.rados.rados.0.smithi118.stdout:2953: expect (ObjNum 374 snap 127 seq_num 374) 2024-03-20T08:11:43.531 INFO:tasks.rados.rados.0.smithi118.stdout:2956: finishing copy_from to smithi11881747-22 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:2956: got expected ENOENT (src dne) 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:2956: finishing copy_from racing read to smithi11881747-22 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:2953: done (6 left) 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:2954: done (5 left) 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:2956: done (4 left) 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:2959: rollback oid 21 current snap is 307 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 302 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34070 (ObjNum 785 snap 274 seq_num 785) dirty exists 2024-03-20T08:11:43.532 INFO:tasks.rados.rados.0.smithi118.stdout:2952: expect (ObjNum 637 snap 205 seq_num 637) 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:2951: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:2951: finishing copy_from to smithi11881747-38 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 34045 (ObjNum 821 snap 281 seq_num 821) dirty exists 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:2958: finishing rollback tid 0 to smithi11881747-17 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:2958: finishing rollback tid 1 to smithi11881747-17 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:2950: done (4 left) 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:2951: done (3 left) 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:2952: done (2 left) 2024-03-20T08:11:43.573 INFO:tasks.rados.rados.0.smithi118.stdout:2960: rollback oid 29 current snap is 307 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 303 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:2958: finishing rollback tid 2 to smithi11881747-17 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 7073 (ObjNum 568 snap 185 seq_num 568) dirty exists 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:2959: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 56181 (ObjNum 33 snap 0 seq_num 291747632) dirty dne 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:2958: done (2 left) 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:2959: done (1 left) 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 307 2024-03-20T08:11:43.574 INFO:tasks.rados.rados.0.smithi118.stdout:2961: seq_num 887 ranges {0=458752} 2024-03-20T08:11:43.575 INFO:tasks.rados.rados.0.smithi118.stdout:2961: writing smithi11881747-48 from 0 to 458752 tid 1 2024-03-20T08:11:43.575 INFO:tasks.rados.rados.0.smithi118.stdout:2962: read oid 10 snap -1 2024-03-20T08:11:43.575 INFO:tasks.rados.rados.0.smithi118.stdout:2962: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:11:43.575 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 307 2024-03-20T08:11:43.575 INFO:tasks.rados.rados.0.smithi118.stdout:2963: seq_num 888 ranges {327680=475136} 2024-03-20T08:11:43.580 INFO:tasks.rados.rados.0.smithi118.stdout:2963: writing smithi11881747-27 from 327680 to 802816 tid 1 2024-03-20T08:11:43.580 INFO:tasks.rados.rados.0.smithi118.stdout:2964: setattr oid 43 current snap is 307 2024-03-20T08:11:43.580 INFO:tasks.rados.rados.0.smithi118.stdout:2960: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:11:43.580 INFO:tasks.rados.rados.0.smithi118.stdout:2960: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:11:43.580 INFO:tasks.rados.rados.0.smithi118.stdout:2960: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:11:43.580 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 34022 (ObjNum 872 snap 302 seq_num 872) dirty exists 2024-03-20T08:11:43.581 INFO:tasks.rados.rados.0.smithi118.stdout:2960: done (4 left) 2024-03-20T08:11:43.581 INFO:tasks.rados.rados.0.smithi118.stdout:2965: delete oid 39 current snap is 307 2024-03-20T08:11:43.585 INFO:tasks.rados.rados.0.smithi118.stdout:2963: finishing write tid 1 to smithi11881747-27 2024-03-20T08:11:43.585 INFO:tasks.rados.rados.0.smithi118.stdout:2963: finishing write tid 2 to smithi11881747-27 2024-03-20T08:11:43.585 INFO:tasks.rados.rados.0.smithi118.stdout:2961: finishing write tid 1 to smithi11881747-48 2024-03-20T08:11:43.585 INFO:tasks.rados.rados.0.smithi118.stdout:2961: finishing write tid 2 to smithi11881747-48 2024-03-20T08:11:43.586 INFO:tasks.rados.rados.0.smithi118.stdout:2965: done (4 left) 2024-03-20T08:11:43.586 INFO:tasks.rados.rados.0.smithi118.stdout:2966: snap_remove snap 206 2024-03-20T08:11:43.738 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:43.487+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:43.738 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:43.494+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:43.739 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:43.498+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:43.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:43.525+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:43.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:43.525+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:43.739 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:43.525+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:43.739 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:43.497+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:43.739 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:43.497+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:43.740 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:43.740 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:43.740 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:43.740 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:43.740 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:43.740 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: osdmap e666: 8 total, 8 up, 8 in 2024-03-20T08:11:43.740 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: pgmap v680: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 13 MiB/s rd, 261 KiB/s wr, 10 op/s 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.741 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.742 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.743 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.743 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.743 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.743 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.743 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.743 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.743 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:11:43] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:11:43.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:43.488+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:43.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:43.488+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:43.537+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:43.537+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:43.537+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:43.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:43.497+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:43.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:43.497+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:43.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:43.497+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:43.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:43.498+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:43.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:43.498+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:43.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:43.488+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: osdmap e666: 8 total, 8 up, 8 in 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 163=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: osdmap e666: 8 total, 8 up, 8 in 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:43.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:43.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: pgmap v680: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 13 MiB/s rd, 261 KiB/s wr, 10 op/s 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:43.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: pgmap v680: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 13 MiB/s rd, 261 KiB/s wr, 10 op/s 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.932 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.932 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.932 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.932 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:43.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:43.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:44.422 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:11:44.076Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002815973s 2024-03-20T08:11:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:2966: done (4 left) 2024-03-20T08:11:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:2967: read oid 17 snap -1 2024-03-20T08:11:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:2967: expect (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:11:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:2968: setattr oid 15 current snap is 307 2024-03-20T08:11:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:2963: finishing write tid 3 to smithi11881747-27 2024-03-20T08:11:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:2963: oid 27 updating version 0 to 34074 2024-03-20T08:11:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:2963: oid 27 version 34074 is already newer than 34073 2024-03-20T08:11:44.494 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 34074 (ObjNum 888 snap 307 seq_num 888) dirty exists 2024-03-20T08:11:44.495 INFO:tasks.rados.rados.0.smithi118.stdout:2963: left oid 27 (ObjNum 888 snap 307 seq_num 888) 2024-03-20T08:11:44.495 INFO:tasks.rados.rados.0.smithi118.stdout:2961: finishing write tid 3 to smithi11881747-48 2024-03-20T08:11:44.495 INFO:tasks.rados.rados.0.smithi118.stdout:2961: oid 48 updating version 0 to 61695 2024-03-20T08:11:44.495 INFO:tasks.rados.rados.0.smithi118.stdout:2961: oid 48 updating version 61695 to 61696 2024-03-20T08:11:44.495 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 61696 (ObjNum 887 snap 307 seq_num 887) dirty exists 2024-03-20T08:11:44.495 INFO:tasks.rados.rados.0.smithi118.stdout:2961: left oid 48 (ObjNum 887 snap 307 seq_num 887) 2024-03-20T08:11:44.495 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 44233 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:11:44.495 INFO:tasks.rados.rados.0.smithi118.stdout:2962: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:11:44.830 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:44.493+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:44.861 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:44.493+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:44.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:44.500+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:44.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:44.501+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:44.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:44.501+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:44.864 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:44.501+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: osdmap e667: 8 total, 8 up, 8 in 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:44.867 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: osdmap e667: 8 total, 8 up, 8 in 2024-03-20T08:11:44.868 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:44.869 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:44.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:44.494+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:44.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:44.499+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:44.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:44.499+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:44.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:44.498+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:44.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:44.502+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:44.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:44.502+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:44.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: osdmap e667: 8 total, 8 up, 8 in 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:44.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:44.973 INFO:tasks.rados.rados.0.smithi118.stdout:2967: expect (ObjNum 568 snap 185 seq_num 568) 2024-03-20T08:11:45.124 INFO:tasks.rados.rados.0.smithi118.stdout:2961: done (5 left) 2024-03-20T08:11:45.124 INFO:tasks.rados.rados.0.smithi118.stdout:2962: done (4 left) 2024-03-20T08:11:45.124 INFO:tasks.rados.rados.0.smithi118.stdout:2963: done (3 left) 2024-03-20T08:11:45.124 INFO:tasks.rados.rados.0.smithi118.stdout:2964: done (2 left) 2024-03-20T08:11:45.124 INFO:tasks.rados.rados.0.smithi118.stdout:2967: done (1 left) 2024-03-20T08:11:45.124 INFO:tasks.rados.rados.0.smithi118.stdout:2969: delete oid 9 current snap is 307 2024-03-20T08:11:45.126 INFO:tasks.rados.rados.0.smithi118.stdout:2969: done (1 left) 2024-03-20T08:11:45.126 INFO:tasks.rados.rados.0.smithi118.stdout:2970: copy_from oid 8 from oid 41 current snap is 307 2024-03-20T08:11:45.126 INFO:tasks.rados.rados.0.smithi118.stdout:2971: snap_remove snap 256 2024-03-20T08:11:45.128 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:11:44.830+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 15 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:11:45.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:45.127+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:45.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:45.127+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:45.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:45.128+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:45.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:45.225+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:45.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:45.129+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:2971: done (2 left) 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:2972: rmattr oid 40 current snap is 307 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 7099 (ObjNum 813 snap 279 seq_num 813) dirty exists 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:2970: finishing copy_from racing read to smithi11881747-8 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:2970: finishing copy_from to smithi11881747-8 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 33983 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:2968: done (2 left) 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:2970: done (1 left) 2024-03-20T08:11:45.501 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 307 2024-03-20T08:11:45.502 INFO:tasks.rados.rados.0.smithi118.stdout:2973: seq_num 891 ranges {7520256=532480} 2024-03-20T08:11:45.544 INFO:tasks.rados.rados.0.smithi118.stdout:2973: writing smithi11881747-12 from 7520256 to 8052736 tid 1 2024-03-20T08:11:45.544 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 307 2024-03-20T08:11:45.544 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34061 (ObjNum 81 snap 0 seq_num 1) dirty exists 2024-03-20T08:11:45.544 INFO:tasks.rados.rados.0.smithi118.stdout:2974: seq_num 892 ranges {0=548864} 2024-03-20T08:11:45.547 INFO:tasks.rados.rados.0.smithi118.stdout:2974: writing smithi11881747-34 from 0 to 548864 tid 1 2024-03-20T08:11:45.547 INFO:tasks.rados.rados.0.smithi118.stdout:2972: done (2 left) 2024-03-20T08:11:45.547 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 307 2024-03-20T08:11:45.547 INFO:tasks.rados.rados.0.smithi118.stdout:2975: seq_num 893 ranges {0=565248} 2024-03-20T08:11:45.551 INFO:tasks.rados.rados.0.smithi118.stdout:2975: writing smithi11881747-21 from 0 to 565248 tid 1 2024-03-20T08:11:45.551 INFO:tasks.rados.rados.0.smithi118.stdout:2976: copy_from oid 5 from oid 37 current snap is 307 2024-03-20T08:11:45.551 INFO:tasks.rados.rados.0.smithi118.stdout:2977: snap_create 2024-03-20T08:11:45.552 INFO:tasks.rados.rados.0.smithi118.stdout:2976: finishing copy_from to smithi11881747-5 2024-03-20T08:11:45.552 INFO:tasks.rados.rados.0.smithi118.stdout:2976: got expected ENOENT (src dne) 2024-03-20T08:11:45.552 INFO:tasks.rados.rados.0.smithi118.stdout:2976: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:11:45.554 INFO:tasks.rados.rados.0.smithi118.stdout:2973: finishing write tid 1 to smithi11881747-12 2024-03-20T08:11:45.554 INFO:tasks.rados.rados.0.smithi118.stdout:2973: finishing write tid 2 to smithi11881747-12 2024-03-20T08:11:45.554 INFO:tasks.rados.rados.0.smithi118.stdout:2974: finishing write tid 1 to smithi11881747-34 2024-03-20T08:11:45.554 INFO:tasks.rados.rados.0.smithi118.stdout:2974: finishing write tid 2 to smithi11881747-34 2024-03-20T08:11:45.557 INFO:tasks.rados.rados.0.smithi118.stdout:2974: finishing write tid 3 to smithi11881747-34 2024-03-20T08:11:45.557 INFO:tasks.rados.rados.0.smithi118.stdout:2974: oid 34 updating version 0 to 45589 2024-03-20T08:11:45.557 INFO:tasks.rados.rados.0.smithi118.stdout:2974: oid 34 version 45589 is already newer than 45588 2024-03-20T08:11:45.557 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45589 (ObjNum 892 snap 307 seq_num 892) dirty exists 2024-03-20T08:11:45.558 INFO:tasks.rados.rados.0.smithi118.stdout:2974: left oid 34 (ObjNum 892 snap 307 seq_num 892) 2024-03-20T08:11:45.558 INFO:tasks.rados.rados.0.smithi118.stdout:2973: finishing write tid 3 to smithi11881747-12 2024-03-20T08:11:45.558 INFO:tasks.rados.rados.0.smithi118.stdout:2973: oid 12 updating version 0 to 34056 2024-03-20T08:11:45.558 INFO:tasks.rados.rados.0.smithi118.stdout:2973: oid 12 updating version 34056 to 34057 2024-03-20T08:11:45.558 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34057 (ObjNum 891 snap 307 seq_num 891) dirty exists 2024-03-20T08:11:45.559 INFO:tasks.rados.rados.0.smithi118.stdout:2973: left oid 12 (ObjNum 891 snap 307 seq_num 891) 2024-03-20T08:11:45.559 INFO:tasks.rados.rados.0.smithi118.stdout:2975: finishing write tid 1 to smithi11881747-21 2024-03-20T08:11:45.560 INFO:tasks.rados.rados.0.smithi118.stdout:2975: finishing write tid 2 to smithi11881747-21 2024-03-20T08:11:45.563 INFO:tasks.rados.rados.0.smithi118.stdout:2975: finishing write tid 3 to smithi11881747-21 2024-03-20T08:11:45.563 INFO:tasks.rados.rados.0.smithi118.stdout:2975: oid 21 updating version 0 to 61697 2024-03-20T08:11:45.563 INFO:tasks.rados.rados.0.smithi118.stdout:2975: oid 21 updating version 61697 to 61698 2024-03-20T08:11:45.563 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 61698 (ObjNum 893 snap 307 seq_num 893) dirty exists 2024-03-20T08:11:45.563 INFO:tasks.rados.rados.0.smithi118.stdout:2975: left oid 21 (ObjNum 893 snap 307 seq_num 893) 2024-03-20T08:11:45.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:45.552+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:45.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:45.549+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:45.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:45.549+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:45.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:45.500+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:45.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:45.500+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:45.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:45.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:45.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: osdmap e668: 8 total, 8 up, 8 in 2024-03-20T08:11:45.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:45.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:45.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:45.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: Upgrade: Updating grafana.a 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: Deploying daemon grafana.a on smithi196 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: pgmap v683: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 8.5 MiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: Health check update: Possible data damage: 15 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:45.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:45.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:45.499+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:45.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:45.499+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:45.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:45.500+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:45.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:45.504+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:45.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:45.505+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:45.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:45.505+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:45.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:45.505+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:45.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:45.510+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:45.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:45.503+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:45.925 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:45.503+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:45.925 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:45.508+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:45.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:45.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:45.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: osdmap e668: 8 total, 8 up, 8 in 2024-03-20T08:11:45.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:45.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:45.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:45.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:45.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:45.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:45.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:45.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:45.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:45.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: Upgrade: Updating grafana.a 2024-03-20T08:11:45.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: Deploying daemon grafana.a on smithi196 2024-03-20T08:11:45.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: pgmap v683: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 8.5 MiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T08:11:45.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: Health check update: Possible data damage: 15 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:11:45.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:45.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:45.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:45.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:45.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:45.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:45.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: osdmap e668: 8 total, 8 up, 8 in 2024-03-20T08:11:45.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:45.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:45.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:45.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:45.933 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:45.933 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:45.933 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:45.933 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:45.933 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:45.933 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: Upgrade: Updating grafana.a 2024-03-20T08:11:45.934 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: Deploying daemon grafana.a on smithi196 2024-03-20T08:11:45.934 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: pgmap v683: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 8.5 MiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T08:11:45.934 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: Health check update: Possible data damage: 15 pgs snaptrim_error (PG_DAMAGED) 2024-03-20T08:11:45.934 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:45.934 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:45.934 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:45.934 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:46.498 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:46.146+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:46.498 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:46.146+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:46.498 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:46.146+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:46.505 INFO:tasks.rados.rados.0.smithi118.stdout:2973: done (4 left) 2024-03-20T08:11:46.505 INFO:tasks.rados.rados.0.smithi118.stdout:2974: done (3 left) 2024-03-20T08:11:46.505 INFO:tasks.rados.rados.0.smithi118.stdout:2975: done (2 left) 2024-03-20T08:11:46.506 INFO:tasks.rados.rados.0.smithi118.stdout:2976: done (1 left) 2024-03-20T08:11:46.506 INFO:tasks.rados.rados.0.smithi118.stdout:2977: done (0 left) 2024-03-20T08:11:46.506 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 308 2024-03-20T08:11:46.506 INFO:tasks.rados.rados.0.smithi118.stdout:2978: seq_num 894 ranges {262144=581632} 2024-03-20T08:11:46.510 INFO:tasks.rados.rados.0.smithi118.stdout:2978: writing smithi11881747-13 from 262144 to 843776 tid 1 2024-03-20T08:11:46.510 INFO:tasks.rados.rados.0.smithi118.stdout:2979: rollback oid 43 current snap is 308 2024-03-20T08:11:46.510 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 43 to 304 2024-03-20T08:11:46.510 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 308 2024-03-20T08:11:46.510 INFO:tasks.rados.rados.0.smithi118.stdout:2980: seq_num 895 ranges {7520256=598016} 2024-03-20T08:11:46.554 INFO:tasks.rados.rados.0.smithi118.stdout:2980: writing smithi11881747-30 from 7520256 to 8118272 tid 1 2024-03-20T08:11:46.554 INFO:tasks.rados.rados.0.smithi118.stdout:2981: snap_create 2024-03-20T08:11:46.554 INFO:tasks.rados.rados.0.smithi118.stdout:2979: finishing rollback tid 0 to smithi11881747-43 2024-03-20T08:11:46.554 INFO:tasks.rados.rados.0.smithi118.stdout:2979: finishing rollback tid 1 to smithi11881747-43 2024-03-20T08:11:46.554 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 44235 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-03-20T08:11:46.555 INFO:tasks.rados.rados.0.smithi118.stdout:2978: finishing write tid 1 to smithi11881747-13 2024-03-20T08:11:46.555 INFO:tasks.rados.rados.0.smithi118.stdout:2978: finishing write tid 2 to smithi11881747-13 2024-03-20T08:11:46.555 INFO:tasks.rados.rados.0.smithi118.stdout:2978: finishing write tid 3 to smithi11881747-13 2024-03-20T08:11:46.555 INFO:tasks.rados.rados.0.smithi118.stdout:2978: oid 13 updating version 0 to 34094 2024-03-20T08:11:46.555 INFO:tasks.rados.rados.0.smithi118.stdout:2978: oid 13 updating version 34094 to 34095 2024-03-20T08:11:46.555 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 34095 (ObjNum 894 snap 308 seq_num 894) dirty exists 2024-03-20T08:11:46.555 INFO:tasks.rados.rados.0.smithi118.stdout:2978: left oid 13 (ObjNum 894 snap 308 seq_num 894) 2024-03-20T08:11:46.557 INFO:tasks.rados.rados.0.smithi118.stdout:2980: finishing write tid 1 to smithi11881747-30 2024-03-20T08:11:46.557 INFO:tasks.rados.rados.0.smithi118.stdout:2980: finishing write tid 2 to smithi11881747-30 2024-03-20T08:11:46.559 INFO:tasks.rados.rados.0.smithi118.stdout:2980: finishing write tid 3 to smithi11881747-30 2024-03-20T08:11:46.559 INFO:tasks.rados.rados.0.smithi118.stdout:2980: oid 30 updating version 0 to 34018 2024-03-20T08:11:46.559 INFO:tasks.rados.rados.0.smithi118.stdout:2980: oid 30 version 34018 is already newer than 34017 2024-03-20T08:11:46.559 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 34018 (ObjNum 895 snap 308 seq_num 895) dirty exists 2024-03-20T08:11:46.559 INFO:tasks.rados.rados.0.smithi118.stdout:2980: left oid 30 (ObjNum 895 snap 308 seq_num 895) 2024-03-20T08:11:46.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:46.506+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:46.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:46.517+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:46.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:46.517+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:46.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:46.505+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:46.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:46.505+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:46.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:46.506+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:46.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:46.515+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:46.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:46.516+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:46.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:46.516+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:46.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:46.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:46.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:46.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:46.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: osdmap e669: 8 total, 8 up, 8 in 2024-03-20T08:11:46.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:46.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:46.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:46.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:46.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:46.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:46.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:46.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:46.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:46.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:46.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:46.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:46.517+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:46.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:46.513+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:46.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:46.518+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:46.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:46.516+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:46.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:46.505+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:46.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:46.505+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:46.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:46.505+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:46.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:46.505+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:46.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:46.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:46.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:46.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:46.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: osdmap e669: 8 total, 8 up, 8 in 2024-03-20T08:11:46.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:46.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:46.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:46.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:46.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:46.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:46.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:46.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:46.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:46.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:46.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:46.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:46.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:46.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:46.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:46.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: osdmap e669: 8 total, 8 up, 8 in 2024-03-20T08:11:46.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:46.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:46.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:46.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:46.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:46.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:46.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:46.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:46.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:46.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:46.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:47.510 INFO:tasks.rados.rados.0.smithi118.stdout:2978: done (3 left) 2024-03-20T08:11:47.510 INFO:tasks.rados.rados.0.smithi118.stdout:2979: done (2 left) 2024-03-20T08:11:47.510 INFO:tasks.rados.rados.0.smithi118.stdout:2980: done (1 left) 2024-03-20T08:11:47.510 INFO:tasks.rados.rados.0.smithi118.stdout:2981: done (0 left) 2024-03-20T08:11:47.511 INFO:tasks.rados.rados.0.smithi118.stdout:2982: snap_remove snap 305 2024-03-20T08:11:47.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:47.508+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:47.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: osdmap e670: 8 total, 8 up, 8 in 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:47.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:47.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:47 smithi196 ceph-mon[87080]: pgmap v686: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 9.3 MiB/s rd, 354 KiB/s wr, 19 op/s 2024-03-20T08:11:47.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:47.509+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:47.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:47.511+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:47.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:47.511+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: osdmap e670: 8 total, 8 up, 8 in 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:47.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[95692]: pgmap v686: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 9.3 MiB/s rd, 354 KiB/s wr, 19 op/s 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:47.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: osdmap e670: 8 total, 8 up, 8 in 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:47.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:47.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:47.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:47 smithi118 ceph-mon[93121]: pgmap v686: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 9.3 MiB/s rd, 354 KiB/s wr, 19 op/s 2024-03-20T08:11:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:2982: done (0 left) 2024-03-20T08:11:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:2983: setattr oid 40 current snap is 309 2024-03-20T08:11:48.512 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:48.186+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:48.512 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:48.186+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:48.515 INFO:tasks.rados.rados.0.smithi118.stdout:2984: read oid 41 snap 257 2024-03-20T08:11:48.515 INFO:tasks.rados.rados.0.smithi118.stdout:2984: expect deleted 2024-03-20T08:11:48.515 INFO:tasks.rados.rados.0.smithi118.stdout:2985: snap_remove snap 297 2024-03-20T08:11:48.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:48.511+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:48.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:48.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:48.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-mon[87080]: osdmap e671: 8 total, 8 up, 8 in 2024-03-20T08:11:48.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:48.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:48.591+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:48.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:48.591+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:48.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:48.591+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:48.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:48.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:48.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-mon[93121]: osdmap e671: 8 total, 8 up, 8 in 2024-03-20T08:11:48.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:48.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:48.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:48.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-mon[95692]: osdmap e671: 8 total, 8 up, 8 in 2024-03-20T08:11:48.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:48.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:48.524+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:48.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:48.524+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:48.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:48.524+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:48.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:48.519+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:48.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:48.525+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:48.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:48.525+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:48.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:48.525+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:48.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:48.526+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:48.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:48.512+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:48.925 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:48.510+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:49.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:48.981+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:49.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:48.981+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:49.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:48.980+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:49.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:48.981+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:49.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:48.981+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:49.525 INFO:tasks.rados.rados.0.smithi118.stdout:2985: done (2 left) 2024-03-20T08:11:49.525 INFO:tasks.rados.rados.0.smithi118.stdout:2986: copy_from oid 27 from oid 44 current snap is 309 2024-03-20T08:11:49.525 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34062 (ObjNum 129 snap 0 seq_num 321654496) dirty exists 2024-03-20T08:11:49.525 INFO:tasks.rados.rados.0.smithi118.stdout:2983: done (2 left) 2024-03-20T08:11:49.525 INFO:tasks.rados.rados.0.smithi118.stdout:2984: done (1 left) 2024-03-20T08:11:49.525 INFO:tasks.rados.rados.0.smithi118.stdout:2987: rmattr oid 48 current snap is 309 2024-03-20T08:11:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:2988: rmattr oid 39 current snap is 309 2024-03-20T08:11:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:2988: done (2 left) 2024-03-20T08:11:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:2989: read oid 33 snap 213 2024-03-20T08:11:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:2989: expect (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:11:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 309 2024-03-20T08:11:49.526 INFO:tasks.rados.rados.0.smithi118.stdout:2990: seq_num 896 ranges {3145728=614400} 2024-03-20T08:11:49.546 INFO:tasks.rados.rados.0.smithi118.stdout:2990: writing smithi11881747-15 from 3145728 to 3760128 tid 1 2024-03-20T08:11:49.546 INFO:tasks.rados.rados.0.smithi118.stdout:2991: copy_from oid 5 from oid 6 current snap is 309 2024-03-20T08:11:49.546 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 61700 (ObjNum 887 snap 307 seq_num 887) dirty exists 2024-03-20T08:11:49.546 INFO:tasks.rados.rados.0.smithi118.stdout:2987: done (4 left) 2024-03-20T08:11:49.546 INFO:tasks.rados.rados.0.smithi118.stdout:2992: delete oid 48 current snap is 309 2024-03-20T08:11:49.554 INFO:tasks.rados.rados.0.smithi118.stdout:2992: done (4 left) 2024-03-20T08:11:49.554 INFO:tasks.rados.rados.0.smithi118.stdout:2993: copy_from oid 21 from oid 28 current snap is 309 2024-03-20T08:11:49.555 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 309 2024-03-20T08:11:49.555 INFO:tasks.rados.rados.0.smithi118.stdout:2994: seq_num 898 ranges {2170880=647168} 2024-03-20T08:11:49.572 INFO:tasks.rados.rados.0.smithi118.stdout:2994: writing smithi11881747-22 from 2170880 to 2818048 tid 1 2024-03-20T08:11:49.572 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 309 2024-03-20T08:11:49.572 INFO:tasks.rados.rados.0.smithi118.stdout:2990: finishing write tid 1 to smithi11881747-15 2024-03-20T08:11:49.572 INFO:tasks.rados.rados.0.smithi118.stdout:2995: seq_num 899 ranges {7020544=663552} 2024-03-20T08:11:49.615 INFO:tasks.rados.rados.0.smithi118.stdout:2995: writing smithi11881747-29 from 7020544 to 7684096 tid 1 2024-03-20T08:11:49.615 INFO:tasks.rados.rados.0.smithi118.stdout:2996: copy_from oid 17 from oid 37 current snap is 309 2024-03-20T08:11:49.615 INFO:tasks.rados.rados.0.smithi118.stdout:2989: expect (ObjNum 618 snap 202 seq_num 618) 2024-03-20T08:11:49.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:49.522+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:49.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:49.529+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:49.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:49.529+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:49.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:49.550+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:49.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:49.550+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:49.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:49.527+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:49.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:49.527+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:49.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:49.531+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: osdmap e672: 8 total, 8 up, 8 in 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:49.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:49.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: pgmap v689: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 29 op/s 2024-03-20T08:11:49.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:49.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:49.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:49.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:49.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:49.529+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:49.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:49.529+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:49.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:49.554+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:49.524+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:49.524+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:49.524+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:49.528+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:49.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:49.522+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:49.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:49.522+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: osdmap e672: 8 total, 8 up, 8 in 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: pgmap v689: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 29 op/s 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:49.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: osdmap e672: 8 total, 8 up, 8 in 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:49.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: pgmap v689: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 29 op/s 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:50.354 INFO:tasks.rados.rados.0.smithi118.stdout:2990: finishing write tid 2 to smithi11881747-15 2024-03-20T08:11:50.354 INFO:tasks.rados.rados.0.smithi118.stdout:2990: finishing write tid 3 to smithi11881747-15 2024-03-20T08:11:50.354 INFO:tasks.rados.rados.0.smithi118.stdout:2990: oid 15 updating version 0 to 7100 2024-03-20T08:11:50.354 INFO:tasks.rados.rados.0.smithi118.stdout:2990: oid 15 updating version 7100 to 7101 2024-03-20T08:11:50.354 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 7101 (ObjNum 896 snap 309 seq_num 896) dirty exists 2024-03-20T08:11:50.354 INFO:tasks.rados.rados.0.smithi118.stdout:2990: left oid 15 (ObjNum 896 snap 309 seq_num 896) 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2989: done (7 left) 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2990: done (6 left) 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2997: rollback oid 11 current snap is 309 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 11 to 242 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2993: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2986: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2993: finishing copy_from to smithi11881747-21 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34019 (ObjNum 644 snap 211 seq_num 644) dirty exists 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2994: finishing write tid 1 to smithi11881747-22 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2991: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2986: finishing copy_from to smithi11881747-27 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 59813 (ObjNum 854 snap 296 seq_num 854) dirty exists 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2994: finishing write tid 2 to smithi11881747-22 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:2991: finishing copy_from to smithi11881747-5 2024-03-20T08:11:50.355 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 45601 (ObjNum 883 snap 306 seq_num 883) dirty exists 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2994: finishing write tid 3 to smithi11881747-22 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2994: oid 22 updating version 0 to 61706 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2994: oid 22 version 61706 is already newer than 61705 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 61706 (ObjNum 898 snap 309 seq_num 898) dirty exists 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2994: left oid 22 (ObjNum 898 snap 309 seq_num 898) 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2996: finishing copy_from to smithi11881747-17 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2996: got expected ENOENT (src dne) 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2996: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2995: finishing write tid 1 to smithi11881747-29 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2995: finishing write tid 2 to smithi11881747-29 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2995: finishing write tid 3 to smithi11881747-29 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2995: oid 29 updating version 0 to 34024 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2995: oid 29 version 34024 is already newer than 34023 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 34024 (ObjNum 899 snap 309 seq_num 899) dirty exists 2024-03-20T08:11:50.356 INFO:tasks.rados.rados.0.smithi118.stdout:2995: left oid 29 (ObjNum 899 snap 309 seq_num 899) 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2986: done (6 left) 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2991: done (5 left) 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2993: done (4 left) 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2994: done (3 left) 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2995: done (2 left) 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2996: done (1 left) 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2998: read oid 38 snap -1 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2998: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:2999: rollback oid 40 current snap is 309 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 249 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:3000: copy_from oid 41 from oid 5 current snap is 309 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 309 2024-03-20T08:11:50.357 INFO:tasks.rados.rados.0.smithi118.stdout:3001: seq_num 900 ranges {7520256=679936} 2024-03-20T08:11:50.398 INFO:tasks.rados.rados.0.smithi118.stdout:3001: writing smithi11881747-16 from 7520256 to 8200192 tid 1 2024-03-20T08:11:50.399 INFO:tasks.rados.rados.0.smithi118.stdout:3002: snap_remove snap 290 2024-03-20T08:11:50.528 INFO:tasks.rados.rados.0.smithi118.stdout:3002: done (5 left) 2024-03-20T08:11:50.529 INFO:tasks.rados.rados.0.smithi118.stdout:3003: rmattr oid 34 current snap is 309 2024-03-20T08:11:50.529 INFO:tasks.rados.rados.0.smithi118.stdout:3003: done (5 left) 2024-03-20T08:11:50.529 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 309 2024-03-20T08:11:50.529 INFO:tasks.rados.rados.0.smithi118.stdout:2997: finishing rollback tid 1 to smithi11881747-11 2024-03-20T08:11:50.529 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 34087 (ObjNum 161 snap 0 seq_num 0) dirty dne 2024-03-20T08:11:50.529 INFO:tasks.rados.rados.0.smithi118.stdout:2998: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:50.575 INFO:tasks.rados.rados.0.smithi118.stdout:2999: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:11:50.575 INFO:tasks.rados.rados.0.smithi118.stdout:2999: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:11:50.575 INFO:tasks.rados.rados.0.smithi118.stdout:3004: seq_num 901 ranges {0=696320} 2024-03-20T08:11:50.578 INFO:tasks.rados.rados.0.smithi118.stdout:3004: writing smithi11881747-48 from 0 to 696320 tid 1 2024-03-20T08:11:50.578 INFO:tasks.rados.rados.0.smithi118.stdout:2997: done (5 left) 2024-03-20T08:11:50.578 INFO:tasks.rados.rados.0.smithi118.stdout:2998: done (4 left) 2024-03-20T08:11:50.578 INFO:tasks.rados.rados.0.smithi118.stdout:3005: read oid 4 snap -1 2024-03-20T08:11:50.578 INFO:tasks.rados.rados.0.smithi118.stdout:3005: expect deleted 2024-03-20T08:11:50.578 INFO:tasks.rados.rados.0.smithi118.stdout:3006: copy_from oid 17 from oid 36 current snap is 309 2024-03-20T08:11:50.579 INFO:tasks.rados.rados.0.smithi118.stdout:2999: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:11:50.579 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34065 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:11:50.579 INFO:tasks.rados.rados.0.smithi118.stdout:3000: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:11:50.579 INFO:tasks.rados.rados.0.smithi118.stdout:3000: finishing copy_from to smithi11881747-41 2024-03-20T08:11:50.579 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 45601 (ObjNum 883 snap 306 seq_num 883) dirty exists 2024-03-20T08:11:50.579 INFO:tasks.rados.rados.0.smithi118.stdout:3001: finishing write tid 1 to smithi11881747-16 2024-03-20T08:11:50.579 INFO:tasks.rados.rados.0.smithi118.stdout:3001: finishing write tid 2 to smithi11881747-16 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:2999: done (5 left) 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:3000: done (4 left) 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:3007: copy_from oid 21 from oid 29 current snap is 309 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:3001: finishing write tid 3 to smithi11881747-16 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:3001: oid 16 updating version 0 to 45603 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:3001: oid 16 version 45603 is already newer than 45602 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 45603 (ObjNum 900 snap 309 seq_num 900) dirty exists 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:3001: left oid 16 (ObjNum 900 snap 309 seq_num 900) 2024-03-20T08:11:50.580 INFO:tasks.rados.rados.0.smithi118.stdout:3001: done (4 left) 2024-03-20T08:11:50.581 INFO:tasks.rados.rados.0.smithi118.stdout:3008: snap_remove snap 289 2024-03-20T08:11:50.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:50.527+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:50.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:50.585+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:50.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:50.585+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:50.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:50.582+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:50.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:50.582+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:50.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:50.582+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:50.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:50.582+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:50.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:50.584+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:50.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:50.584+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: osdmap e673: 8 total, 8 up, 8 in 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:50.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:50.528+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:50.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:50.528+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:50.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:50.529+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:50.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:50.616+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:50.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:50.528+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:50.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:50.586+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:50.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:50.586+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:50.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:50.586+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:50.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:50.586+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:50.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:50.586+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: osdmap e673: 8 total, 8 up, 8 in 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: osdmap e673: 8 total, 8 up, 8 in 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:50.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:50.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:50.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:50.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:50.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:50.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:50.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:51.534 INFO:tasks.rados.rados.0.smithi118.stdout:3008: done (4 left) 2024-03-20T08:11:51.534 INFO:tasks.rados.rados.0.smithi118.stdout:3009: rmattr oid 1 current snap is 309 2024-03-20T08:11:51.534 INFO:tasks.rados.rados.0.smithi118.stdout:3009: done (4 left) 2024-03-20T08:11:51.534 INFO:tasks.rados.rados.0.smithi118.stdout:3010: snap_remove snap 216 2024-03-20T08:11:51.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:51.715+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:51.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:51.715+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:51.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:51.531+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:51.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:51.532+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: osdmap e674: 8 total, 8 up, 8 in 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: pgmap v692: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:51.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:51.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:51.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:51.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:51.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:51.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:51.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:51.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:51.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:51.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:51.532+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:51.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:51.532+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:51.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:51.534+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:51.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:51.534+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: osdmap e674: 8 total, 8 up, 8 in 2024-03-20T08:11:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: pgmap v692: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-03-20T08:11:51.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:51.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:51.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:51.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:51.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: osdmap e674: 8 total, 8 up, 8 in 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: pgmap v692: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.8 MiB/s wr, 18 op/s 2024-03-20T08:11:51.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:51.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:51.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:52.530 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:52.223+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:52.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.223+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:52.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.224+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:52.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.224+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:52.530 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.224+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:52.531 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:52.224+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:52.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:52.223+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:52.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:52.223+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:52.531 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:52.224+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:52.531 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:11:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:11:52.539 INFO:tasks.rados.rados.0.smithi118.stdout:3010: done (4 left) 2024-03-20T08:11:52.539 INFO:tasks.rados.rados.0.smithi118.stdout:3011: rollback oid 47 current snap is 309 2024-03-20T08:11:52.539 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 226 2024-03-20T08:11:52.539 INFO:tasks.rados.rados.0.smithi118.stdout:3006: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:11:52.539 INFO:tasks.rados.rados.0.smithi118.stdout:3006: finishing copy_from to smithi11881747-17 2024-03-20T08:11:52.539 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 45587 (ObjNum 33 snap 0 seq_num 291747632) dirty exists 2024-03-20T08:11:52.539 INFO:tasks.rados.rados.0.smithi118.stdout:3004: finishing write tid 1 to smithi11881747-48 2024-03-20T08:11:52.539 INFO:tasks.rados.rados.0.smithi118.stdout:3004: finishing write tid 2 to smithi11881747-48 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3004: finishing write tid 3 to smithi11881747-48 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3004: oid 48 updating version 0 to 61708 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3004: oid 48 version 61708 is already newer than 61707 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 61708 (ObjNum 901 snap 309 seq_num 901) dirty exists 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3004: left oid 48 (ObjNum 901 snap 309 seq_num 901) 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3007: finishing copy_from racing read to smithi11881747-21 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3004: done (4 left) 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3005: done (3 left) 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3006: done (2 left) 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 309 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3007: finishing copy_from to smithi11881747-21 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 34024 (ObjNum 899 snap 309 seq_num 899) dirty exists 2024-03-20T08:11:52.540 INFO:tasks.rados.rados.0.smithi118.stdout:3012: seq_num 902 ranges {0=712704} 2024-03-20T08:11:52.544 INFO:tasks.rados.rados.0.smithi118.stdout:3012: writing smithi11881747-4 from 0 to 712704 tid 1 2024-03-20T08:11:52.544 INFO:tasks.rados.rados.0.smithi118.stdout:3007: done (2 left) 2024-03-20T08:11:52.544 INFO:tasks.rados.rados.0.smithi118.stdout:3013: read oid 19 snap -1 2024-03-20T08:11:52.545 INFO:tasks.rados.rados.0.smithi118.stdout:3013: expect (ObjNum 871 snap 302 seq_num 871) 2024-03-20T08:11:52.545 INFO:tasks.rados.rados.0.smithi118.stdout:3014: delete oid 32 current snap is 309 2024-03-20T08:11:52.545 INFO:tasks.rados.rados.0.smithi118.stdout:3011: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:11:52.545 INFO:tasks.rados.rados.0.smithi118.stdout:3011: finishing rollback tid 2 to smithi11881747-47 2024-03-20T08:11:52.545 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 45605 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:11:52.551 INFO:tasks.rados.rados.0.smithi118.stdout:3011: done (3 left) 2024-03-20T08:11:52.551 INFO:tasks.rados.rados.0.smithi118.stdout:3014: done (2 left) 2024-03-20T08:11:52.551 INFO:tasks.rados.rados.0.smithi118.stdout:3015: rollback oid 33 current snap is 309 2024-03-20T08:11:52.551 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 278 2024-03-20T08:11:52.551 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 309 2024-03-20T08:11:52.551 INFO:tasks.rados.rados.0.smithi118.stdout:3016: seq_num 904 ranges {0=745472} 2024-03-20T08:11:52.556 INFO:tasks.rados.rados.0.smithi118.stdout:3016: writing smithi11881747-43 from 0 to 745472 tid 1 2024-03-20T08:11:52.556 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 309 2024-03-20T08:11:52.556 INFO:tasks.rados.rados.0.smithi118.stdout:3017: seq_num 905 ranges {0=761856} 2024-03-20T08:11:52.561 INFO:tasks.rados.rados.0.smithi118.stdout:3017: writing smithi11881747-2 from 0 to 761856 tid 1 2024-03-20T08:11:52.561 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 309 2024-03-20T08:11:52.561 INFO:tasks.rados.rados.0.smithi118.stdout:3018: seq_num 906 ranges {425984=778240} 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:3018: writing smithi11881747-31 from 425984 to 1204224 tid 1 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:3019: read oid 50 snap -1 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:3019: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:3020: read oid 10 snap -1 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:3020: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:3021: rollback oid 8 current snap is 309 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 8 to 213 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:3012: finishing write tid 1 to smithi11881747-4 2024-03-20T08:11:52.570 INFO:tasks.rados.rados.0.smithi118.stdout:3012: finishing write tid 2 to smithi11881747-4 2024-03-20T08:11:52.571 INFO:tasks.rados.rados.0.smithi118.stdout:3013: expect (ObjNum 871 snap 302 seq_num 871) 2024-03-20T08:11:52.581 INFO:tasks.rados.rados.0.smithi118.stdout:3012: finishing write tid 3 to smithi11881747-4 2024-03-20T08:11:52.581 INFO:tasks.rados.rados.0.smithi118.stdout:3012: oid 4 updating version 0 to 34019 2024-03-20T08:11:52.581 INFO:tasks.rados.rados.0.smithi118.stdout:3012: oid 4 updating version 34019 to 34020 2024-03-20T08:11:52.581 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 34020 (ObjNum 902 snap 309 seq_num 902) dirty exists 2024-03-20T08:11:52.581 INFO:tasks.rados.rados.0.smithi118.stdout:3012: left oid 4 (ObjNum 902 snap 309 seq_num 902) 2024-03-20T08:11:52.582 INFO:tasks.rados.rados.0.smithi118.stdout:3012: done (8 left) 2024-03-20T08:11:52.582 INFO:tasks.rados.rados.0.smithi118.stdout:3013: done (7 left) 2024-03-20T08:11:52.582 INFO:tasks.rados.rados.0.smithi118.stdout:3022: read oid 29 snap -1 2024-03-20T08:11:52.582 INFO:tasks.rados.rados.0.smithi118.stdout:3022: expect (ObjNum 899 snap 309 seq_num 899) 2024-03-20T08:11:52.582 INFO:tasks.rados.rados.0.smithi118.stdout:3023: read oid 44 snap -1 2024-03-20T08:11:52.582 INFO:tasks.rados.rados.0.smithi118.stdout:3023: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:11:52.582 INFO:tasks.rados.rados.0.smithi118.stdout:3024: snap_remove snap 249 2024-03-20T08:11:52.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:52.537+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:52.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:52.550+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:52.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:52.552+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:52.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:52.544+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:52.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:52.545+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:52.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:52.545+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:52.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:52.551+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:52.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:52.552+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:52.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:52.552+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:52.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:52.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:52.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:52.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:52.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: osdmap e675: 8 total, 8 up, 8 in 2024-03-20T08:11:52.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:52.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:52.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:52.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:52.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:52.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:52.900 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:52.539+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:52.901 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:52.548+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:52.901 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:52.549+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:52.901 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:52.549+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:52.901 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.542+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:52.901 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.544+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:52.901 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.544+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:52.901 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.544+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:52.901 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:52.545+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:52.902 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:11:52.902 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:52.902 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:52.902 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:52.902 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:52.902 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: osdmap e675: 8 total, 8 up, 8 in 2024-03-20T08:11:52.902 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:52.902 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: osdmap e675: 8 total, 8 up, 8 in 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:52.903 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:52.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:52.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:52.904 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:52.904 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:52.539+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:53.227 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:11:53.227 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (19s) 15s ago 12m 15.6M - 0.25.0 c8568f914cd2 68a6e855b9aa 2024-03-20T08:11:53.227 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (12m) 34s ago 12m 87.9M - 9.4.7 954c08fa6188 c58566782642 2024-03-20T08:11:53.227 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (113s) 15s ago 13m 14.5M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 b9e9c53270c7 2024-03-20T08:11:53.227 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (89s) 34s ago 13m 14.0M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 237cfe0be476 2024-03-20T08:11:53.227 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (8m) 34s ago 17m 453M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (8m) 15s ago 18m 517M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (8m) 15s ago 19m 71.8M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (7m) 34s ago 17m 58.8M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (7m) 15s ago 17m 61.8M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (69s) 15s ago 11m 8070k - 1.5.0 0da6a335fe13 768c22c1ab59 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (64s) 34s ago 11m 8648k - 1.5.0 0da6a335fe13 d2c82f232b48 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (6m) 15s ago 16m 519M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (5m) 15s ago 16m 535M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (5m) 15s ago 15m 380M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (4m) 15s ago 15m 162M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (4m) 34s ago 15m 381M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (3m) 34s ago 14m 413M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 72b569d10541 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (3m) 34s ago 14m 228M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 dca658056148 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (2m) 34s ago 13m 127M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 52392403da44 2024-03-20T08:11:53.228 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (42s) 34s ago 13m 35.6M - 2.43.0 a07b618ecd1d bf82125604d2 2024-03-20T08:11:53.543 INFO:tasks.rados.rados.0.smithi118.stdout:3024: done (9 left) 2024-03-20T08:11:53.543 INFO:tasks.rados.rados.0.smithi118.stdout:3025: snap_remove snap 298 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 8 2024-03-20T08:11:53.643 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:11:53.644 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:11:53.644 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:11:53.644 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:11:53.644 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:11:53.644 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 15 2024-03-20T08:11:53.644 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:11:53.644 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:11:53.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:53.542+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:53.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:53.549+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:53.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:53.550+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:53.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:53.548+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:53.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:53.548+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:53.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:53.549+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:53.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:53.550+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:53.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:53.550+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: osdmap e676: 8 total, 8 up, 8 in 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:53.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: pgmap v695: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.3 MiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: from='client.44393 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:53.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:53 smithi196 ceph-mon[87080]: from='client.34479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:53.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:53.543+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:53.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:53.543+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:53.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:53.543+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:53.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:53.543+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:53.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:11:53] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:11:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:53.548+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:53.549+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:53.549+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:53.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:53.549+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:53.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:53.549+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: osdmap e676: 8 total, 8 up, 8 in 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:53.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: pgmap v695: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.3 MiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: from='client.44393 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:53.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[93121]: from='client.34479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: osdmap e676: 8 total, 8 up, 8 in 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:53.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: pgmap v695: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.3 MiB/s rd, 8.4 MiB/s wr, 15 op/s 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: from='client.44393 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:53.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:53 smithi118 ceph-mon[95692]: from='client.34479 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:11:54.543 INFO:tasks.rados.rados.0.smithi118.stdout:3025: done (9 left) 2024-03-20T08:11:54.543 INFO:tasks.rados.rados.0.smithi118.stdout:3026: read oid 13 snap -1 2024-03-20T08:11:54.543 INFO:tasks.rados.rados.0.smithi118.stdout:3026: expect (ObjNum 894 snap 308 seq_num 894) 2024-03-20T08:11:54.543 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 309 2024-03-20T08:11:54.543 INFO:tasks.rados.rados.0.smithi118.stdout:3027: seq_num 907 ranges {737280=794624} 2024-03-20T08:11:54.552 INFO:tasks.rados.rados.0.smithi118.stdout:3027: writing smithi11881747-35 from 737280 to 1531904 tid 1 2024-03-20T08:11:54.552 INFO:tasks.rados.rados.0.smithi118.stdout:3028: snap_remove snap 263 2024-03-20T08:11:54.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:54.542+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:54.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:54.556+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:54.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:54.556+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:54.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:54.560+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: osdmap e677: 8 total, 8 up, 8 in 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:54.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:54.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:54.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:54.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:54.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:54.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:54.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:54 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/2097983818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:54.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:54.542+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:54.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:54.549+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:54.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:54.549+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:54.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:54.552+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:54.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:54.552+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:54.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:54.542+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:54.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:54.543+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:54.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:54.543+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:54.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:54.543+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:54.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:54.544+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: osdmap e677: 8 total, 8 up, 8 in 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:54.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/2097983818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: osdmap e677: 8 total, 8 up, 8 in 2024-03-20T08:11:54.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:54.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:54 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/2097983818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:11:55.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:55.024+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:55.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:55.025+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:55.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:55.025+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:55.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:55.025+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:55.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:55.025+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:55.555 INFO:tasks.rados.rados.0.smithi118.stdout:3028: done (11 left) 2024-03-20T08:11:55.555 INFO:tasks.rados.rados.0.smithi118.stdout:3029: rollback oid 36 current snap is 309 2024-03-20T08:11:55.555 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 36 to 266 2024-03-20T08:11:55.555 INFO:tasks.rados.rados.0.smithi118.stdout:3016: finishing write tid 1 to smithi11881747-43 2024-03-20T08:11:55.555 INFO:tasks.rados.rados.0.smithi118.stdout:3016: finishing write tid 2 to smithi11881747-43 2024-03-20T08:11:55.555 INFO:tasks.rados.rados.0.smithi118.stdout:3016: finishing write tid 3 to smithi11881747-43 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3016: oid 43 updating version 0 to 44237 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3016: oid 43 version 44237 is already newer than 44236 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 44237 (ObjNum 904 snap 309 seq_num 904) dirty exists 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3016: left oid 43 (ObjNum 904 snap 309 seq_num 904) 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3015: finishing rollback tid 0 to smithi11881747-33 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3015: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3015: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34028 (ObjNum 691 snap 227 seq_num 691) dirty exists 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3017: finishing write tid 1 to smithi11881747-2 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3017: finishing write tid 2 to smithi11881747-2 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3017: finishing write tid 3 to smithi11881747-2 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3017: oid 2 updating version 0 to 45602 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:3017: oid 2 updating version 45602 to 45603 2024-03-20T08:11:55.556 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45603 (ObjNum 905 snap 309 seq_num 905) dirty exists 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3017: left oid 2 (ObjNum 905 snap 309 seq_num 905) 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3018: finishing write tid 1 to smithi11881747-31 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3015: done (11 left) 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3016: done (10 left) 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3017: done (9 left) 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3030: setattr oid 47 current snap is 309 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3018: finishing write tid 2 to smithi11881747-31 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3021: finishing rollback tid 0 to smithi11881747-8 2024-03-20T08:11:55.557 INFO:tasks.rados.rados.0.smithi118.stdout:3019: expect (ObjNum 821 snap 281 seq_num 821) 2024-03-20T08:11:55.627 INFO:tasks.rados.rados.0.smithi118.stdout:3021: finishing rollback tid 1 to smithi11881747-8 2024-03-20T08:11:55.627 INFO:tasks.rados.rados.0.smithi118.stdout:3021: finishing rollback tid 2 to smithi11881747-8 2024-03-20T08:11:55.627 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34074 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:11:55.627 INFO:tasks.rados.rados.0.smithi118.stdout:3022: expect (ObjNum 899 snap 309 seq_num 899) 2024-03-20T08:11:55.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:55.552+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:55.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:55.627+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:55.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:55.627+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:55.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:55.627+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:55.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:55.627+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:55.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:55.628+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:55.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:55.627+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:55.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:55.628+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:55.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:55.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: osdmap e678: 8 total, 8 up, 8 in 2024-03-20T08:11:55.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: pgmap v698: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 18 op/s 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:55.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:55.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:55.552+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:55.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:55.553+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:55.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:55.627+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:55.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:55.627+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:55.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:55.627+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:55.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:55.627+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:55.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:55.626+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:55.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:55.551+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:55.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:55.551+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: osdmap e678: 8 total, 8 up, 8 in 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:55.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: pgmap v698: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 18 op/s 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: osdmap e678: 8 total, 8 up, 8 in 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:55.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: pgmap v698: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 18 op/s 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:55.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:55.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:56.272 INFO:tasks.rados.rados.0.smithi118.stdout:3020: expect (ObjNum 472 snap 156 seq_num 472) 2024-03-20T08:11:56.760 INFO:tasks.rados.rados.0.smithi118.stdout:3018: finishing write tid 3 to smithi11881747-31 2024-03-20T08:11:56.760 INFO:tasks.rados.rados.0.smithi118.stdout:3018: oid 31 updating version 0 to 44239 2024-03-20T08:11:56.760 INFO:tasks.rados.rados.0.smithi118.stdout:3018: oid 31 version 44239 is already newer than 44238 2024-03-20T08:11:56.761 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 44239 (ObjNum 906 snap 309 seq_num 906) dirty exists 2024-03-20T08:11:56.761 INFO:tasks.rados.rados.0.smithi118.stdout:3018: left oid 31 (ObjNum 906 snap 309 seq_num 906) 2024-03-20T08:11:56.761 INFO:tasks.rados.rados.0.smithi118.stdout:3023: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:11:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: osdmap e679: 8 total, 8 up, 8 in 2024-03-20T08:11:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:56.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:56.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:56.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:56.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:56.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: osdmap e679: 8 total, 8 up, 8 in 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:56.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: osdmap e679: 8 total, 8 up, 8 in 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:56.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:56.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:56.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:56.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:56.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:56.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:57.580 INFO:tasks.rados.rados.0.smithi118.stdout:3026: expect (ObjNum 894 snap 308 seq_num 894) 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3027: finishing write tid 1 to smithi11881747-35 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3027: finishing write tid 2 to smithi11881747-35 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3027: finishing write tid 3 to smithi11881747-35 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3027: oid 35 updating version 0 to 45605 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3027: oid 35 version 45605 is already newer than 45604 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 45605 (ObjNum 907 snap 309 seq_num 907) dirty exists 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3027: left oid 35 (ObjNum 907 snap 309 seq_num 907) 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3029: finishing rollback tid 0 to smithi11881747-36 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3029: finishing rollback tid 1 to smithi11881747-36 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3029: finishing rollback tid 2 to smithi11881747-36 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45592 (ObjNum 667 snap 217 seq_num 667) dirty exists 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3018: done (9 left) 2024-03-20T08:11:57.610 INFO:tasks.rados.rados.0.smithi118.stdout:3019: done (8 left) 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3020: done (7 left) 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3021: done (6 left) 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3022: done (5 left) 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3023: done (4 left) 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3026: done (3 left) 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3027: done (2 left) 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3029: done (1 left) 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3031: copy_from oid 31 from oid 29 current snap is 309 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 309 2024-03-20T08:11:57.611 INFO:tasks.rados.rados.0.smithi118.stdout:3032: seq_num 908 ranges {8118272=409600,8527872=401408} 2024-03-20T08:11:57.656 INFO:tasks.rados.rados.0.smithi118.stdout:3032: writing smithi11881747-30 from 8118272 to 8527872 tid 1 2024-03-20T08:11:57.658 INFO:tasks.rados.rados.0.smithi118.stdout:3032: writing smithi11881747-30 from 8527872 to 8929280 tid 2 2024-03-20T08:11:57.658 INFO:tasks.rados.rados.0.smithi118.stdout:3033: copy_from oid 22 from oid 7 current snap is 309 2024-03-20T08:11:57.658 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 45606 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:11:57.658 INFO:tasks.rados.rados.0.smithi118.stdout:3031: finishing copy_from racing read to smithi11881747-31 2024-03-20T08:11:57.659 INFO:tasks.rados.rados.0.smithi118.stdout:3031: finishing copy_from to smithi11881747-31 2024-03-20T08:11:57.659 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 34024 (ObjNum 899 snap 309 seq_num 899) dirty exists 2024-03-20T08:11:57.659 INFO:tasks.rados.rados.0.smithi118.stdout:3030: done (3 left) 2024-03-20T08:11:57.659 INFO:tasks.rados.rados.0.smithi118.stdout:3031: done (2 left) 2024-03-20T08:11:57.659 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 309 2024-03-20T08:11:57.659 INFO:tasks.rados.rados.0.smithi118.stdout:3034: seq_num 909 ranges {0=425984,425984=401408} 2024-03-20T08:11:57.660 INFO:tasks.rados.rados.0.smithi118.stdout:3034: writing smithi11881747-37 from 0 to 425984 tid 1 2024-03-20T08:11:57.663 INFO:tasks.rados.rados.0.smithi118.stdout:3034: writing smithi11881747-37 from 425984 to 827392 tid 2 2024-03-20T08:11:57.663 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 309 2024-03-20T08:11:57.663 INFO:tasks.rados.rados.0.smithi118.stdout:3035: seq_num 910 ranges {7684096=442368,8126464=401408} 2024-03-20T08:11:57.707 INFO:tasks.rados.rados.0.smithi118.stdout:3035: writing smithi11881747-21 from 7684096 to 8126464 tid 1 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3035: writing smithi11881747-21 from 8126464 to 8527872 tid 2 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3036: read oid 27 snap -1 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3036: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3037: copy_from oid 26 from oid 39 current snap is 309 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3032: finishing write tid 1 to smithi11881747-30 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3032: finishing write tid 2 to smithi11881747-30 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3032: finishing write tid 3 to smithi11881747-30 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3032: finishing write tid 4 to smithi11881747-30 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3032: oid 30 updating version 0 to 34022 2024-03-20T08:11:57.711 INFO:tasks.rados.rados.0.smithi118.stdout:3032: oid 30 version 34022 is already newer than 34021 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3032: oid 30 updating version 34022 to 34023 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 34023 (ObjNum 908 snap 309 seq_num 908) dirty exists 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3032: left oid 30 (ObjNum 908 snap 309 seq_num 908) 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3033: finishing copy_from racing read to smithi11881747-22 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3033: finishing copy_from to smithi11881747-22 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 34052 (ObjNum 868 snap 301 seq_num 868) dirty exists 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3034: finishing write tid 1 to smithi11881747-37 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3032: done (5 left) 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3033: done (4 left) 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3038: rollback oid 31 current snap is 309 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 31 to 306 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3034: finishing write tid 2 to smithi11881747-37 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3034: finishing write tid 3 to smithi11881747-37 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3034: finishing write tid 4 to smithi11881747-37 2024-03-20T08:11:57.712 INFO:tasks.rados.rados.0.smithi118.stdout:3034: oid 37 updating version 0 to 34067 2024-03-20T08:11:57.713 INFO:tasks.rados.rados.0.smithi118.stdout:3034: oid 37 version 34067 is already newer than 34066 2024-03-20T08:11:57.713 INFO:tasks.rados.rados.0.smithi118.stdout:3034: oid 37 updating version 34067 to 34068 2024-03-20T08:11:57.713 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 34068 (ObjNum 909 snap 309 seq_num 909) dirty exists 2024-03-20T08:11:57.713 INFO:tasks.rados.rados.0.smithi118.stdout:3034: left oid 37 (ObjNum 909 snap 309 seq_num 909) 2024-03-20T08:11:57.713 INFO:tasks.rados.rados.0.smithi118.stdout:3034: done (4 left) 2024-03-20T08:11:57.713 INFO:tasks.rados.rados.0.smithi118.stdout:3039: snap_remove snap 255 2024-03-20T08:11:57.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:57 smithi196 ceph-mon[87080]: pgmap v700: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 25 MiB/s rd, 3.5 MiB/s wr, 19 op/s 2024-03-20T08:11:57.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:57 smithi118 ceph-mon[95692]: pgmap v700: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 25 MiB/s rd, 3.5 MiB/s wr, 19 op/s 2024-03-20T08:11:57.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:57 smithi118 ceph-mon[93121]: pgmap v700: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 25 MiB/s rd, 3.5 MiB/s wr, 19 op/s 2024-03-20T08:11:58.562 INFO:tasks.rados.rados.0.smithi118.stdout:3039: done (4 left) 2024-03-20T08:11:58.562 INFO:tasks.rados.rados.0.smithi118.stdout:3040: snap_remove snap 300 2024-03-20T08:11:58.727 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:58.727 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:58.562+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:58.727 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:58.563+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:58.727 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:58.563+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:58.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:58.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:58.748+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:58.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:58.562+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:11:58.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:58.562+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:58.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:58.562+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:58.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:58.749+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:58.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:11:58.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:58.750+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:58.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:58.750+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:58.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:58.750+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:11:58.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:58.750+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:58.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:58.750+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:59.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:58.727+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:59.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:58.727+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:59.059 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:58.727+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:59.059 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:58.748+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:59.059 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:58.748+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:11:59.059 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:58.748+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:59.570 INFO:tasks.rados.rados.0.smithi118.stdout:3040: done (4 left) 2024-03-20T08:11:59.570 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 309 2024-03-20T08:11:59.570 INFO:tasks.rados.rados.0.smithi118.stdout:3038: finishing rollback tid 0 to smithi11881747-31 2024-03-20T08:11:59.570 INFO:tasks.rados.rados.0.smithi118.stdout:3041: seq_num 911 ranges {8052736=466944,8519680=401408} 2024-03-20T08:11:59.616 INFO:tasks.rados.rados.0.smithi118.stdout:3041: writing smithi11881747-12 from 8052736 to 8519680 tid 1 2024-03-20T08:11:59.618 INFO:tasks.rados.rados.0.smithi118.stdout:3041: writing smithi11881747-12 from 8519680 to 8921088 tid 2 2024-03-20T08:11:59.618 INFO:tasks.rados.rados.0.smithi118.stdout:3042: rollback oid 49 current snap is 309 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 49 to 303 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3038: finishing rollback tid 1 to smithi11881747-31 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3038: finishing rollback tid 2 to smithi11881747-31 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 44242 (ObjNum 885 snap 306 seq_num 885) dirty exists 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3035: finishing write tid 1 to smithi11881747-21 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3035: finishing write tid 2 to smithi11881747-21 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3035: finishing write tid 3 to smithi11881747-21 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3035: finishing write tid 4 to smithi11881747-21 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3035: oid 21 updating version 0 to 61711 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3035: oid 21 updating version 61711 to 61713 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3035: oid 21 version 61713 is already newer than 61712 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 61713 (ObjNum 910 snap 309 seq_num 910) dirty exists 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3035: left oid 21 (ObjNum 910 snap 309 seq_num 910) 2024-03-20T08:11:59.619 INFO:tasks.rados.rados.0.smithi118.stdout:3037: finishing copy_from to smithi11881747-26 2024-03-20T08:11:59.620 INFO:tasks.rados.rados.0.smithi118.stdout:3037: got expected ENOENT (src dne) 2024-03-20T08:11:59.620 INFO:tasks.rados.rados.0.smithi118.stdout:3037: finishing copy_from racing read to smithi11881747-26 2024-03-20T08:11:59.620 INFO:tasks.rados.rados.0.smithi118.stdout:3036: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: osdmap e680: 8 total, 8 up, 8 in 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: pgmap v702: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 896 KiB/s wr, 13 op/s 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:59.623 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:59.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:59.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:59.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:59.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:59.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:59.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:59.624 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:59.624 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:11:59.567+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:59.889 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:11:59 smithi196 systemd[1]: Stopping Ceph grafana.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:11:59.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:59.623+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:59.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:11:59.624+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:59.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:59.624+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:59.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:59.625+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:59.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:11:59.626+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:59.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:59.622+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:59.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:11:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:11:59.622+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:59.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:59.626+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:59.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:11:59.626+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:59.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:59.567+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:11:59.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:11:59.567+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:59.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:59.566+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:59.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:59.566+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:59.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:59.567+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:59.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:11:59.567+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:59.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:11:59.623+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:59.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:59.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:59.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:59.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: osdmap e680: 8 total, 8 up, 8 in 2024-03-20T08:11:59.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:59.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: pgmap v702: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 896 KiB/s wr, 13 op/s 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:59.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: osdmap e680: 8 total, 8 up, 8 in 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: pgmap v702: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 12 MiB/s rd, 896 KiB/s wr, 13 op/s 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:11:59.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:11:59.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:11:59.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:11:59.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:11:59.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:11:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:00.377 INFO:tasks.rados.rados.0.smithi118.stdout:3042: finishing rollback tid 1 to smithi11881747-49 2024-03-20T08:12:00.377 INFO:tasks.rados.rados.0.smithi118.stdout:3035: done (5 left) 2024-03-20T08:12:00.377 INFO:tasks.rados.rados.0.smithi118.stdout:3036: done (4 left) 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3037: done (3 left) 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3038: done (2 left) 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3043: rollback oid 33 current snap is 309 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 33 to 272 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3042: finishing rollback tid 2 to smithi11881747-49 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 59815 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3041: finishing write tid 1 to smithi11881747-12 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3041: finishing write tid 2 to smithi11881747-12 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3041: finishing write tid 3 to smithi11881747-12 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3041: finishing write tid 4 to smithi11881747-12 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3041: oid 12 updating version 0 to 34058 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3041: oid 12 updating version 34058 to 34059 2024-03-20T08:12:00.378 INFO:tasks.rados.rados.0.smithi118.stdout:3041: oid 12 updating version 34059 to 34060 2024-03-20T08:12:00.379 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 34060 (ObjNum 911 snap 309 seq_num 911) dirty exists 2024-03-20T08:12:00.379 INFO:tasks.rados.rados.0.smithi118.stdout:3041: left oid 12 (ObjNum 911 snap 309 seq_num 911) 2024-03-20T08:12:00.379 INFO:tasks.rados.rados.0.smithi118.stdout:3041: done (2 left) 2024-03-20T08:12:00.379 INFO:tasks.rados.rados.0.smithi118.stdout:3042: done (1 left) 2024-03-20T08:12:00.379 INFO:tasks.rados.rados.0.smithi118.stdout:3044: snap_remove snap 242 2024-03-20T08:12:00.381 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:11:59 smithi196 podman[124302]: 2024-03-20 08:11:59.924737984 +0000 UTC m=+0.145650995 container died c58566782642a692ae9130a99b6d7efd87a0b30287cc7f64bd4f26bb71ba501a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, vcs-type=git, io.openshift.tags=base rhel8, distribution-scope=public, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., description=Ceph Grafana Container, version=8.5, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897) 2024-03-20T08:12:00.572 INFO:tasks.rados.rados.0.smithi118.stdout:3044: done (1 left) 2024-03-20T08:12:00.573 INFO:tasks.rados.rados.0.smithi118.stdout:3045: read oid 15 snap -1 2024-03-20T08:12:00.573 INFO:tasks.rados.rados.0.smithi118.stdout:3045: expect (ObjNum 896 snap 309 seq_num 896) 2024-03-20T08:12:00.573 INFO:tasks.rados.rados.0.smithi118.stdout:3046: delete oid 4 current snap is 309 2024-03-20T08:12:00.573 INFO:tasks.rados.rados.0.smithi118.stdout:3043: finishing rollback tid 0 to smithi11881747-33 2024-03-20T08:12:00.573 INFO:tasks.rados.rados.0.smithi118.stdout:3043: finishing rollback tid 1 to smithi11881747-33 2024-03-20T08:12:00.573 INFO:tasks.rados.rados.0.smithi118.stdout:3043: finishing rollback tid 2 to smithi11881747-33 2024-03-20T08:12:00.573 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34031 (ObjNum 691 snap 227 seq_num 691) dirty exists 2024-03-20T08:12:00.575 INFO:tasks.rados.rados.0.smithi118.stdout:3043: done (2 left) 2024-03-20T08:12:00.575 INFO:tasks.rados.rados.0.smithi118.stdout:3046: done (1 left) 2024-03-20T08:12:00.575 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 309 2024-03-20T08:12:00.575 INFO:tasks.rados.rados.0.smithi118.stdout:3047: seq_num 913 ranges {9093120=499712,9592832=401408} 2024-03-20T08:12:00.629 INFO:tasks.rados.rados.0.smithi118.stdout:3047: writing smithi11881747-33 from 9093120 to 9592832 tid 1 2024-03-20T08:12:00.632 INFO:tasks.rados.rados.0.smithi118.stdout:3047: writing smithi11881747-33 from 9592832 to 9994240 tid 2 2024-03-20T08:12:00.632 INFO:tasks.rados.rados.0.smithi118.stdout:3048: read oid 19 snap -1 2024-03-20T08:12:00.632 INFO:tasks.rados.rados.0.smithi118.stdout:3048: expect (ObjNum 871 snap 302 seq_num 871) 2024-03-20T08:12:00.632 INFO:tasks.rados.rados.0.smithi118.stdout:3049: rollback oid 35 current snap is 309 2024-03-20T08:12:00.632 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 35 to 294 2024-03-20T08:12:00.633 INFO:tasks.rados.rados.0.smithi118.stdout:3045: expect (ObjNum 896 snap 309 seq_num 896) 2024-03-20T08:12:00.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:00.572+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:00.639 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:00 smithi196 podman[124302]: 2024-03-20 08:12:00.473157545 +0000 UTC m=+0.694070539 container cleanup c58566782642a692ae9130a99b6d7efd87a0b30287cc7f64bd4f26bb71ba501a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, description=Ceph Grafana Container, io.openshift.tags=base rhel8, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc.) 2024-03-20T08:12:00.639 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:00 smithi196 bash[124302]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a 2024-03-20T08:12:00.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:00.572+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:00.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:00.573+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:00.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:00.573+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:00.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:00.577+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:00.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:00.577+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:00.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:00.577+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:00.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:00.581+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:00.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:00.581+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:00.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:00.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: osdmap e681: 8 total, 8 up, 8 in 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:00.813 INFO:tasks.rados.rados.0.smithi118.stdout:3048: expect (ObjNum 871 snap 302 seq_num 871) 2024-03-20T08:12:00.822 INFO:tasks.rados.rados.0.smithi118.stdout:3049: finishing rollback tid 0 to smithi11881747-35 2024-03-20T08:12:00.822 INFO:tasks.rados.rados.0.smithi118.stdout:3049: finishing rollback tid 1 to smithi11881747-35 2024-03-20T08:12:00.822 INFO:tasks.rados.rados.0.smithi118.stdout:3049: finishing rollback tid 2 to smithi11881747-35 2024-03-20T08:12:00.822 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 45608 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:12:00.822 INFO:tasks.rados.rados.0.smithi118.stdout:3047: finishing write tid 1 to smithi11881747-33 2024-03-20T08:12:00.822 INFO:tasks.rados.rados.0.smithi118.stdout:3047: finishing write tid 2 to smithi11881747-33 2024-03-20T08:12:00.822 INFO:tasks.rados.rados.0.smithi118.stdout:3047: finishing write tid 3 to smithi11881747-33 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3047: finishing write tid 4 to smithi11881747-33 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3047: oid 33 updating version 0 to 34033 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3047: oid 33 updating version 34033 to 34034 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3047: oid 33 version 34034 is already newer than 34032 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34034 (ObjNum 913 snap 309 seq_num 913) dirty exists 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3047: left oid 33 (ObjNum 913 snap 309 seq_num 913) 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3045: done (3 left) 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3047: done (2 left) 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3048: done (1 left) 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3049: done (0 left) 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3050: copy_from oid 38 from oid 3 current snap is 309 2024-03-20T08:12:00.823 INFO:tasks.rados.rados.0.smithi118.stdout:3051: copy_from oid 18 from oid 23 current snap is 309 2024-03-20T08:12:00.824 INFO:tasks.rados.rados.0.smithi118.stdout:3052: read oid 41 snap -1 2024-03-20T08:12:00.824 INFO:tasks.rados.rados.0.smithi118.stdout:3052: expect (ObjNum 883 snap 306 seq_num 883) 2024-03-20T08:12:00.824 INFO:tasks.rados.rados.0.smithi118.stdout:3053: copy_from oid 6 from oid 9 current snap is 309 2024-03-20T08:12:00.824 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 309 2024-03-20T08:12:00.824 INFO:tasks.rados.rados.0.smithi118.stdout:3054: seq_num 914 ranges {0=516096,516096=401408} 2024-03-20T08:12:00.824 INFO:tasks.rados.rados.0.smithi118.stdout:3054: writing smithi11881747-4 from 0 to 516096 tid 1 2024-03-20T08:12:00.827 INFO:tasks.rados.rados.0.smithi118.stdout:3054: writing smithi11881747-4 from 516096 to 917504 tid 2 2024-03-20T08:12:00.827 INFO:tasks.rados.rados.0.smithi118.stdout:3055: snap_create 2024-03-20T08:12:00.827 INFO:tasks.rados.rados.0.smithi118.stdout:3053: finishing copy_from to smithi11881747-6 2024-03-20T08:12:00.827 INFO:tasks.rados.rados.0.smithi118.stdout:3053: got expected ENOENT (src dne) 2024-03-20T08:12:00.827 INFO:tasks.rados.rados.0.smithi118.stdout:3053: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:12:00.830 INFO:tasks.rados.rados.0.smithi118.stdout:3054: finishing write tid 1 to smithi11881747-4 2024-03-20T08:12:00.830 INFO:tasks.rados.rados.0.smithi118.stdout:3054: finishing write tid 2 to smithi11881747-4 2024-03-20T08:12:00.830 INFO:tasks.rados.rados.0.smithi118.stdout:3054: finishing write tid 3 to smithi11881747-4 2024-03-20T08:12:00.832 INFO:tasks.rados.rados.0.smithi118.stdout:3054: finishing write tid 4 to smithi11881747-4 2024-03-20T08:12:00.832 INFO:tasks.rados.rados.0.smithi118.stdout:3054: oid 4 updating version 0 to 34025 2024-03-20T08:12:00.832 INFO:tasks.rados.rados.0.smithi118.stdout:3054: oid 4 updating version 34025 to 34027 2024-03-20T08:12:00.832 INFO:tasks.rados.rados.0.smithi118.stdout:3054: oid 4 version 34027 is already newer than 34026 2024-03-20T08:12:00.832 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 34027 (ObjNum 914 snap 309 seq_num 914) dirty exists 2024-03-20T08:12:00.832 INFO:tasks.rados.rados.0.smithi118.stdout:3054: left oid 4 (ObjNum 914 snap 309 seq_num 914) 2024-03-20T08:12:00.862 INFO:tasks.rados.rados.0.smithi118.stdout:3052: expect (ObjNum 883 snap 306 seq_num 883) 2024-03-20T08:12:00.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:00.726+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:00.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:00.726+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:00.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:00.726+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: osdmap e681: 8 total, 8 up, 8 in 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:00.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:00.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:00.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:00.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:00.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:00.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:00.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:00.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:00.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:00.571+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:00.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:00.572+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:00.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:00.572+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:00.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:00.572+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:00.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:00.572+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:00.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:00.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:00.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:00.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: osdmap e681: 8 total, 8 up, 8 in 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:00.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:00.927 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:00.575+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:00.927 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:00.580+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:01.080 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:00 smithi196 podman[124321]: 2024-03-20 08:12:00.766535943 +0000 UTC m=+0.842737149 container remove c58566782642a692ae9130a99b6d7efd87a0b30287cc7f64bd4f26bb71ba501a (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, name=ubi8, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., distribution-scope=public, io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, architecture=x86_64, release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , vcs-type=git) 2024-03-20T08:12:01.080 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:00 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:12:01.081 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:00 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-03-20T08:12:01.081 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:00 smithi196 systemd[1]: Stopped Ceph grafana.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:12:01.081 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:00 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@grafana.a.service: Consumed 11.443s CPU time. 2024-03-20T08:12:01.081 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 systemd[1]: Starting Ceph grafana.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:12:01.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 podman[124455]: 2024-03-20 08:12:01.293098384 +0000 UTC m=+0.113903037 container create b2e2f67e151687ae234bdf0fe449763e56ca444d4838648e2fd74c3e8dbee07f (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, vendor=Red Hat, Inc., description=Ceph Grafana Container, io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, architecture=x86_64, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, version=8.5) 2024-03-20T08:12:01.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 podman[124455]: 2024-03-20 08:12:01.196289115 +0000 UTC m=+0.017093767 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-03-20T08:12:01.550 INFO:tasks.rados.rados.0.smithi118.stdout:3051: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:12:01.550 INFO:tasks.rados.rados.0.smithi118.stdout:3051: finishing copy_from to smithi11881747-18 2024-03-20T08:12:01.550 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 33996 (ObjNum 814 snap 279 seq_num 814) dirty exists 2024-03-20T08:12:01.550 INFO:tasks.rados.rados.0.smithi118.stdout:3050: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:12:01.550 INFO:tasks.rados.rados.0.smithi118.stdout:3050: finishing copy_from to smithi11881747-38 2024-03-20T08:12:01.550 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 34022 (ObjNum 740 snap 253 seq_num 740) dirty exists 2024-03-20T08:12:01.578 INFO:tasks.rados.rados.0.smithi118.stdout:3050: done (5 left) 2024-03-20T08:12:01.579 INFO:tasks.rados.rados.0.smithi118.stdout:3051: done (4 left) 2024-03-20T08:12:01.579 INFO:tasks.rados.rados.0.smithi118.stdout:3052: done (3 left) 2024-03-20T08:12:01.579 INFO:tasks.rados.rados.0.smithi118.stdout:3053: done (2 left) 2024-03-20T08:12:01.579 INFO:tasks.rados.rados.0.smithi118.stdout:3054: done (1 left) 2024-03-20T08:12:01.579 INFO:tasks.rados.rados.0.smithi118.stdout:3055: done (0 left) 2024-03-20T08:12:01.579 INFO:tasks.rados.rados.0.smithi118.stdout:3056: delete oid 35 current snap is 310 2024-03-20T08:12:01.582 INFO:tasks.rados.rados.0.smithi118.stdout:3056: done (0 left) 2024-03-20T08:12:01.582 INFO:tasks.rados.rados.0.smithi118.stdout:3057: delete oid 27 current snap is 310 2024-03-20T08:12:01.587 INFO:tasks.rados.rados.0.smithi118.stdout:3057: done (0 left) 2024-03-20T08:12:01.587 INFO:tasks.rados.rados.0.smithi118.stdout:3058: snap_remove snap 257 2024-03-20T08:12:01.799 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:01.577+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:01.799 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:01.582+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:01.799 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:01.583+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:01.800 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 podman[124455]: 2024-03-20 08:12:01.529165128 +0000 UTC m=+0.349969792 container init b2e2f67e151687ae234bdf0fe449763e56ca444d4838648e2fd74c3e8dbee07f (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, version=8.5, vcs-type=git, description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, name=ubi8, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, distribution-scope=public) 2024-03-20T08:12:01.800 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 podman[124455]: 2024-03-20 08:12:01.535182265 +0000 UTC m=+0.355986918 container start b2e2f67e151687ae234bdf0fe449763e56ca444d4838648e2fd74c3e8dbee07f (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, maintainer=Paul Cuzner , description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5) 2024-03-20T08:12:01.800 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 bash[124455]: b2e2f67e151687ae234bdf0fe449763e56ca444d4838648e2fd74c3e8dbee07f 2024-03-20T08:12:01.800 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 systemd[1]: Started Ceph grafana.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:12:01.800 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694463416Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-03-20T08:12:01.800 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694646468Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-03-20T08:12:01.800 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694662968Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-03-20T08:12:01.800 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694669229Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-03-20T08:12:01.800 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: osdmap e682: 8 total, 8 up, 8 in 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: pgmap v705: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 7.1 MiB/s rd, 1.8 MiB/s wr, 9 op/s 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:01.801 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:01.802 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:01.589+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:01.802 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:01.589+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:01.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:01.587+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:01.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:01.587+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:01.802 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:01.587+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:01.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:01.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:01.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:01.803 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:01.803 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694674705Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-03-20T08:12:01.803 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694680127Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-03-20T08:12:01.803 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694685421Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-03-20T08:12:01.803 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694690668Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-03-20T08:12:01.803 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694696328Z level=info msg="Path Home" path=/usr/share/grafana 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.69470152Z level=info msg="Path Data" path=/var/lib/grafana 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694706468Z level=info msg="Path Logs" path=/var/log/grafana 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694711455Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694716579Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=settings t=2024-03-20T08:12:01.694721767Z level=info msg="App mode production" 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=sqlstore t=2024-03-20T08:12:01.694780482Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=sqlstore t=2024-03-20T08:12:01.694803368Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=migrator t=2024-03-20T08:12:01.708348368Z level=info msg="Starting DB migrations" 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=migrator t=2024-03-20T08:12:01.712998189Z level=info msg="migrations completed" performed=0 skipped=481 duration=576.559µs 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=sqlstore t=2024-03-20T08:12:01.713367171Z level=info msg="Created default organization" 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=plugin.loader t=2024-03-20T08:12:01.771432125Z level=info msg="Plugin registered" pluginID=input 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=plugin.loader t=2024-03-20T08:12:01.800239092Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=plugin.loader t=2024-03-20T08:12:01.800277168Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-03-20T08:12:01.804 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=secrets t=2024-03-20T08:12:01.800333072Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-03-20T08:12:01.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:01.589+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:01.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:01.589+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:01.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:01.577+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:01.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:01.578+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:01.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:01.578+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:01.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:01.590+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:01.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:01.590+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:01.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:01.591+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:01.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:01.591+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:01.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:01.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:01.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:01.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:01.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:01.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: osdmap e682: 8 total, 8 up, 8 in 2024-03-20T08:12:01.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: pgmap v705: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 7.1 MiB/s rd, 1.8 MiB/s wr, 9 op/s 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:01.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16e=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: osdmap e682: 8 total, 8 up, 8 in 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 16c=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: pgmap v705: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 7.1 MiB/s rd, 1.8 MiB/s wr, 9 op/s 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:01.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:01.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:01.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:01.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:02.139 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=query_data t=2024-03-20T08:12:01.802732724Z level=info msg="Query Service initialization" 2024-03-20T08:12:02.139 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=live.push_http t=2024-03-20T08:12:01.805926517Z level=info msg="Live Push Gateway initialization" 2024-03-20T08:12:02.575 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:12:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:12:02.575 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=infra.usagestats.collector t=2024-03-20T08:12:02.468989336Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-03-20T08:12:02.575 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=provisioning.datasources t=2024-03-20T08:12:02.471316664Z level=info msg="deleted correlations based on configuration" ds_name=Dashboard1 2024-03-20T08:12:02.575 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=provisioning.datasources t=2024-03-20T08:12:02.47133372Z level=info msg="deleted datasource based on configuration" name=Dashboard1 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=provisioning.datasources t=2024-03-20T08:12:02.471473169Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=provisioning.alerting t=2024-03-20T08:12:02.492915865Z level=info msg="starting to provision alerting" 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=provisioning.alerting t=2024-03-20T08:12:02.492951393Z level=info msg="finished to provision alerting" 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=ngalert.state.manager t=2024-03-20T08:12:02.493045297Z level=info msg="Warming state cache for startup" 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=grafanaStorageLogger t=2024-03-20T08:12:02.493112783Z level=info msg="storage starting" 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=ngalert.state.manager t=2024-03-20T08:12:02.493278106Z level=info msg="State cache has been initialized" states=0 duration=231.836µs 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=ticker t=2024-03-20T08:12:02.493341481Z level=info msg=starting first_tick=2024-03-20T08:12:10Z 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=ngalert.multiorg.alertmanager t=2024-03-20T08:12:02.49336366Z level=info msg="starting MultiOrg Alertmanager" 2024-03-20T08:12:02.576 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=http.server t=2024-03-20T08:12:02.495355823Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-03-20T08:12:02.583 INFO:tasks.rados.rados.0.smithi118.stdout:3058: done (0 left) 2024-03-20T08:12:02.583 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 310 2024-03-20T08:12:02.583 INFO:tasks.rados.rados.0.smithi118.stdout:3059: seq_num 917 ranges {0=565248,565248=401408} 2024-03-20T08:12:02.586 INFO:tasks.rados.rados.0.smithi118.stdout:3059: writing smithi11881747-17 from 0 to 565248 tid 1 2024-03-20T08:12:02.588 INFO:tasks.rados.rados.0.smithi118.stdout:3059: writing smithi11881747-17 from 565248 to 966656 tid 2 2024-03-20T08:12:02.588 INFO:tasks.rados.rados.0.smithi118.stdout:3060: snap_remove snap 306 2024-03-20T08:12:02.830 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:02.583+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:02.830 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:02.583+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:02.831 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:02.583+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:02.831 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:02.590+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:02.831 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:02.592+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:02.831 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:02.592+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: osdmap e683: 8 total, 8 up, 8 in 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:02.832 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:02.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:02.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:02.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:02.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:02.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:02.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:02.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:02.833 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:02.853 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:02.749+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:02.853 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:02.749+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:02.853 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:02.749+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:02.853 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:02.853 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:02.853 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:02.853 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: osdmap e683: 8 total, 8 up, 8 in 2024-03-20T08:12:02.853 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:02.853 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:02.854 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:02.854 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:02.854 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:02.854 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:02.854 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:02.854 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:02.854 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:02.854 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:02.582+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:02.854 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:02.582+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:02.855 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:02.582+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:02.855 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:02.582+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:02.855 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:02.582+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:02.855 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:02.855 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:02.855 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:02.856 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:02.856 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:02.856 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:02.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:02.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:02.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:02.856 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: osdmap e683: 8 total, 8 up, 8 in 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:02.857 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:02.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:02.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:02.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:02.858 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:03.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:02.830+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:03.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:02.830+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:03.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:02.830+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:03.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:03.082+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:03.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:02.853+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:03.584 INFO:tasks.rados.rados.0.smithi118.stdout:3060: done (1 left) 2024-03-20T08:12:03.584 INFO:tasks.rados.rados.0.smithi118.stdout:3061: snap_create 2024-03-20T08:12:03.584 INFO:tasks.rados.rados.0.smithi118.stdout:3059: finishing write tid 1 to smithi11881747-17 2024-03-20T08:12:03.584 INFO:tasks.rados.rados.0.smithi118.stdout:3059: finishing write tid 2 to smithi11881747-17 2024-03-20T08:12:03.584 INFO:tasks.rados.rados.0.smithi118.stdout:3059: finishing write tid 3 to smithi11881747-17 2024-03-20T08:12:03.584 INFO:tasks.rados.rados.0.smithi118.stdout:3059: finishing write tid 4 to smithi11881747-17 2024-03-20T08:12:03.584 INFO:tasks.rados.rados.0.smithi118.stdout:3059: oid 17 updating version 0 to 45589 2024-03-20T08:12:03.585 INFO:tasks.rados.rados.0.smithi118.stdout:3059: oid 17 updating version 45589 to 45590 2024-03-20T08:12:03.585 INFO:tasks.rados.rados.0.smithi118.stdout:3059: oid 17 version 45590 is already newer than 45588 2024-03-20T08:12:03.585 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 45590 (ObjNum 917 snap 310 seq_num 917) dirty exists 2024-03-20T08:12:03.585 INFO:tasks.rados.rados.0.smithi118.stdout:3059: left oid 17 (ObjNum 917 snap 310 seq_num 917) 2024-03-20T08:12:03.843 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:03.583+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:03.843 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:03.583+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:03.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: pgmap v707: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.6 MiB/s wr, 5 op/s 2024-03-20T08:12:03.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:03.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:03.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:03.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:03.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: osdmap e684: 8 total, 8 up, 8 in 2024-03-20T08:12:03.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:03.845 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:03.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:12:03] "GET /metrics HTTP/1.1" 200 36407 "" "Prometheus/2.43.0" 2024-03-20T08:12:03.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:03.732+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:03.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:03.733+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:03.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:03.584+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:03.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:03.584+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:03.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:03.584+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:03.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:03.584+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:03.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:03.584+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:03.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:03.584+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:03.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: pgmap v707: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.6 MiB/s wr, 5 op/s 2024-03-20T08:12:03.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:03.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:03.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:03.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:03.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: osdmap e684: 8 total, 8 up, 8 in 2024-03-20T08:12:03.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:03.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:03.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:03.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:03.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:03.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:03.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:03.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:03.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:03.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:03.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:03.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: pgmap v707: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.1 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 8.3 MiB/s rd, 4.6 MiB/s wr, 5 op/s 2024-03-20T08:12:03.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:03.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:03.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:03.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: osdmap e684: 8 total, 8 up, 8 in 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:03.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:03.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:03.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:03.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:04.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:04.035+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:04.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:04.035+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:04.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:04.227+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:04.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:04.227+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:04.520 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:04.227+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:04.521 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:04.227+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:04.591 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:04.226+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:04.594 INFO:tasks.rados.rados.0.smithi118.stdout:3059: done (1 left) 2024-03-20T08:12:04.594 INFO:tasks.rados.rados.0.smithi118.stdout:3061: done (0 left) 2024-03-20T08:12:04.594 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 311 2024-03-20T08:12:04.594 INFO:tasks.rados.rados.0.smithi118.stdout:3062: seq_num 918 ranges {548864=581632,1130496=401408} 2024-03-20T08:12:04.601 INFO:tasks.rados.rados.0.smithi118.stdout:3062: writing smithi11881747-34 from 548864 to 1130496 tid 1 2024-03-20T08:12:04.606 INFO:tasks.rados.rados.0.smithi118.stdout:3062: writing smithi11881747-34 from 1130496 to 1531904 tid 2 2024-03-20T08:12:04.606 INFO:tasks.rados.rados.0.smithi118.stdout:3063: setattr oid 41 current snap is 311 2024-03-20T08:12:04.608 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 40 current snap is 311 2024-03-20T08:12:04.608 INFO:tasks.rados.rados.0.smithi118.stdout:3064: seq_num 919 ranges {270336=598016,868352=401408} 2024-03-20T08:12:04.613 INFO:tasks.rados.rados.0.smithi118.stdout:3064: writing smithi11881747-40 from 270336 to 868352 tid 1 2024-03-20T08:12:04.617 INFO:tasks.rados.rados.0.smithi118.stdout:3064: writing smithi11881747-40 from 868352 to 1269760 tid 2 2024-03-20T08:12:04.617 INFO:tasks.rados.rados.0.smithi118.stdout:3065: read oid 47 snap 213 2024-03-20T08:12:04.617 INFO:tasks.rados.rados.0.smithi118.stdout:3065: expect (ObjNum 657 snap 213 seq_num 657) 2024-03-20T08:12:04.618 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 311 2024-03-20T08:12:04.618 INFO:tasks.rados.rados.0.smithi118.stdout:3066: seq_num 920 ranges {827392=614400,1441792=401408} 2024-03-20T08:12:04.625 INFO:tasks.rados.rados.0.smithi118.stdout:3066: writing smithi11881747-37 from 827392 to 1441792 tid 1 2024-03-20T08:12:04.628 INFO:tasks.rados.rados.0.smithi118.stdout:3066: writing smithi11881747-37 from 1441792 to 1843200 tid 2 2024-03-20T08:12:04.628 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 311 2024-03-20T08:12:04.628 INFO:tasks.rados.rados.0.smithi118.stdout:3062: finishing write tid 1 to smithi11881747-34 2024-03-20T08:12:04.628 INFO:tasks.rados.rados.0.smithi118.stdout:3067: seq_num 921 ranges {0=630784,630784=401408} 2024-03-20T08:12:04.632 INFO:tasks.rados.rados.0.smithi118.stdout:3067: writing smithi11881747-32 from 0 to 630784 tid 1 2024-03-20T08:12:04.635 INFO:tasks.rados.rados.0.smithi118.stdout:3067: writing smithi11881747-32 from 630784 to 1032192 tid 2 2024-03-20T08:12:04.635 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 311 2024-03-20T08:12:04.635 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 45607 (ObjNum 883 snap 306 seq_num 883) dirty exists 2024-03-20T08:12:04.635 INFO:tasks.rados.rados.0.smithi118.stdout:3062: finishing write tid 2 to smithi11881747-34 2024-03-20T08:12:04.635 INFO:tasks.rados.rados.0.smithi118.stdout:3062: finishing write tid 3 to smithi11881747-34 2024-03-20T08:12:04.635 INFO:tasks.rados.rados.0.smithi118.stdout:3068: seq_num 922 ranges {9682944=647168,10330112=401408} 2024-03-20T08:12:04.691 INFO:tasks.rados.rados.0.smithi118.stdout:3068: writing smithi11881747-6 from 9682944 to 10330112 tid 1 2024-03-20T08:12:04.694 INFO:tasks.rados.rados.0.smithi118.stdout:3068: writing smithi11881747-6 from 10330112 to 10731520 tid 2 2024-03-20T08:12:04.694 INFO:tasks.rados.rados.0.smithi118.stdout:3063: done (6 left) 2024-03-20T08:12:04.694 INFO:tasks.rados.rados.0.smithi118.stdout:3069: snap_create 2024-03-20T08:12:04.694 INFO:tasks.rados.rados.0.smithi118.stdout:3062: finishing write tid 4 to smithi11881747-34 2024-03-20T08:12:04.694 INFO:tasks.rados.rados.0.smithi118.stdout:3062: oid 34 updating version 0 to 45593 2024-03-20T08:12:04.695 INFO:tasks.rados.rados.0.smithi118.stdout:3062: oid 34 updating version 45593 to 45595 2024-03-20T08:12:04.695 INFO:tasks.rados.rados.0.smithi118.stdout:3062: oid 34 version 45595 is already newer than 45594 2024-03-20T08:12:04.695 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45595 (ObjNum 918 snap 311 seq_num 918) dirty exists 2024-03-20T08:12:04.695 INFO:tasks.rados.rados.0.smithi118.stdout:3062: left oid 34 (ObjNum 918 snap 311 seq_num 918) 2024-03-20T08:12:04.695 INFO:tasks.rados.rados.0.smithi118.stdout:3065: expect (ObjNum 657 snap 213 seq_num 657) 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3064: finishing write tid 1 to smithi11881747-40 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3064: finishing write tid 2 to smithi11881747-40 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3067: finishing write tid 1 to smithi11881747-32 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3064: finishing write tid 3 to smithi11881747-40 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3066: finishing write tid 1 to smithi11881747-37 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3067: finishing write tid 2 to smithi11881747-32 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3067: finishing write tid 3 to smithi11881747-32 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3066: finishing write tid 2 to smithi11881747-37 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3066: finishing write tid 3 to smithi11881747-37 2024-03-20T08:12:04.765 INFO:tasks.rados.rados.0.smithi118.stdout:3067: finishing write tid 4 to smithi11881747-32 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3067: oid 32 updating version 0 to 34076 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3067: oid 32 version 34076 is already newer than 34075 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3067: oid 32 updating version 34076 to 34077 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34077 (ObjNum 921 snap 311 seq_num 921) dirty exists 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3067: left oid 32 (ObjNum 921 snap 311 seq_num 921) 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3064: finishing write tid 4 to smithi11881747-40 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3064: oid 40 updating version 0 to 34070 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3064: oid 40 version 34070 is already newer than 34069 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3064: oid 40 updating version 34070 to 34071 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34071 (ObjNum 919 snap 311 seq_num 919) dirty exists 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3064: left oid 40 (ObjNum 919 snap 311 seq_num 919) 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3066: finishing write tid 4 to smithi11881747-37 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3066: oid 37 updating version 0 to 34073 2024-03-20T08:12:04.766 INFO:tasks.rados.rados.0.smithi118.stdout:3066: oid 37 updating version 34073 to 34074 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3066: oid 37 version 34074 is already newer than 34072 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 34074 (ObjNum 920 snap 311 seq_num 920) dirty exists 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3066: left oid 37 (ObjNum 920 snap 311 seq_num 920) 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3068: finishing write tid 1 to smithi11881747-6 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3068: finishing write tid 2 to smithi11881747-6 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3068: finishing write tid 3 to smithi11881747-6 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3068: finishing write tid 4 to smithi11881747-6 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3068: oid 6 updating version 0 to 45610 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3068: oid 6 updating version 45610 to 45612 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3068: oid 6 version 45612 is already newer than 45611 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 45612 (ObjNum 922 snap 311 seq_num 922) dirty exists 2024-03-20T08:12:04.767 INFO:tasks.rados.rados.0.smithi118.stdout:3068: left oid 6 (ObjNum 922 snap 311 seq_num 922) 2024-03-20T08:12:04.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:04.594+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:04.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:04.615+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:04.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:04.615+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:04.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:04.615+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:04.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:04.594+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:04.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:04.594+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:04.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:04.695+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:04.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:04.696+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:04.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:04.697+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:04.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: osdmap e685: 8 total, 8 up, 8 in 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:04.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:04.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:04.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:04.606+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:04.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:04.610+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:04.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:04.610+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:04.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:04.609+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:04.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:04.594+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:04.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:04.611+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:04.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:04.612+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:04.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:04.613+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:04.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:04.613+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:04.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:04.615+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: osdmap e685: 8 total, 8 up, 8 in 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:04.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:04.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: osdmap e685: 8 total, 8 up, 8 in 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:04.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:05.594 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:05.594 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: pgmap v710: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 6.9 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-03-20T08:12:05.594 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: osdmap e686: 8 total, 8 up, 8 in 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:05.595 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3062: done (6 left) 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3064: done (5 left) 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3065: done (4 left) 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3066: done (3 left) 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3067: done (2 left) 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3068: done (1 left) 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3069: done (0 left) 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3070: rmattr oid 44 current snap is 312 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3071: rmattr oid 36 current snap is 312 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3072: read oid 23 snap -1 2024-03-20T08:12:05.603 INFO:tasks.rados.rados.0.smithi118.stdout:3072: expect (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:12:05.604 INFO:tasks.rados.rados.0.smithi118.stdout:3073: rmattr oid 5 current snap is 312 2024-03-20T08:12:05.604 INFO:tasks.rados.rados.0.smithi118.stdout:3073: done (3 left) 2024-03-20T08:12:05.604 INFO:tasks.rados.rados.0.smithi118.stdout:3074: snap_create 2024-03-20T08:12:05.611 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45596 (ObjNum 667 snap 217 seq_num 667) dirty exists 2024-03-20T08:12:05.613 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 65244 (ObjNum 854 snap 296 seq_num 854) dirty exists 2024-03-20T08:12:05.614 INFO:tasks.rados.rados.0.smithi118.stdout:3072: expect (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:12:05.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:05.603+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:05.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:05.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:05.607+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:05.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:05.607+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:05.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:05.612+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:05.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:05.611+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:05.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:05.611+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:05.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:05.618+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:05.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:05.618+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:05.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:05.607+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:05.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:05.611+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:05.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:05.618+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:05.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:05.605+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:05.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:05.605+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:05.602+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:05.602+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:05.602+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:05.602+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: pgmap v710: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 6.9 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: osdmap e686: 8 total, 8 up, 8 in 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:05.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:05.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:05.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: pgmap v710: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 6.9 MiB/s rd, 3.0 MiB/s wr, 5 op/s 2024-03-20T08:12:05.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:05.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:05.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:05.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:05.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: osdmap e686: 8 total, 8 up, 8 in 2024-03-20T08:12:05.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:05.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:06.608 INFO:tasks.rados.rados.0.smithi118.stdout:3070: done (3 left) 2024-03-20T08:12:06.608 INFO:tasks.rados.rados.0.smithi118.stdout:3071: done (2 left) 2024-03-20T08:12:06.608 INFO:tasks.rados.rados.0.smithi118.stdout:3072: done (1 left) 2024-03-20T08:12:06.608 INFO:tasks.rados.rados.0.smithi118.stdout:3074: done (0 left) 2024-03-20T08:12:06.608 INFO:tasks.rados.rados.0.smithi118.stdout:3075: snap_remove snap 296 2024-03-20T08:12:06.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:06.608+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:06.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:06.609+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:06.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:06.609+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:06.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:06.609+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: osdmap e687: 8 total, 8 up, 8 in 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:06.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:06.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:06.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:06.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:06.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:06.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:06.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:06 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:06.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:06.607+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:06.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:06.608+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:06.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:06.608+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:06.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:06.608+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:06.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:06.608+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:06.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:06.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:06.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:06.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:06.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: osdmap e687: 8 total, 8 up, 8 in 2024-03-20T08:12:06.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:06.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:06.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: osdmap e687: 8 total, 8 up, 8 in 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:06.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:06.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:06.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:06.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:06.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:06 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:07.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:07.183+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:07.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:07.182+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:07.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:07.127+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:07.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:07.127+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:07.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:07.127+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:07.480 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:07.182+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:07.481 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:07.183+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:07.481 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:07.183+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:07.481 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:07.183+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:07.481 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:07.183+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:07.610 INFO:tasks.rados.rados.0.smithi118.stdout:3075: done (0 left) 2024-03-20T08:12:07.610 INFO:tasks.rados.rados.0.smithi118.stdout:3076: read oid 1 snap -1 2024-03-20T08:12:07.611 INFO:tasks.rados.rados.0.smithi118.stdout:3076: expect (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:12:07.611 INFO:tasks.rados.rados.0.smithi118.stdout:3077: copy_from oid 12 from oid 10 current snap is 313 2024-03-20T08:12:07.611 INFO:tasks.rados.rados.0.smithi118.stdout:3078: read oid 45 snap -1 2024-03-20T08:12:07.611 INFO:tasks.rados.rados.0.smithi118.stdout:3078: expect deleted 2024-03-20T08:12:07.611 INFO:tasks.rados.rados.0.smithi118.stdout:3079: copy_from oid 28 from oid 16 current snap is 313 2024-03-20T08:12:07.611 INFO:tasks.rados.rados.0.smithi118.stdout:3080: read oid 2 snap 311 2024-03-20T08:12:07.611 INFO:tasks.rados.rados.0.smithi118.stdout:3080: expect (ObjNum 905 snap 309 seq_num 905) 2024-03-20T08:12:07.611 INFO:tasks.rados.rados.0.smithi118.stdout:3081: snap_remove snap 278 2024-03-20T08:12:07.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:07.610+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:07.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:07.619+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:07.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:07.619+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:07.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:07.620+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:07.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:07.629+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:07.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:07.631+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:07.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:07.617+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:07.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:07.620+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:07.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: pgmap v713: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 9.8 MiB/s rd, 2.1 MiB/s wr, 15 op/s 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: osdmap e688: 8 total, 8 up, 8 in 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:07.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:07.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:07.609+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:07.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:07.609+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:07.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:07.612+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:07.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:07.612+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:07.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:07.727+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:07.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:07.619+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:07.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:07.619+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:07.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:07.622+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:07.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:07.622+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: pgmap v713: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 9.8 MiB/s rd, 2.1 MiB/s wr, 15 op/s 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: osdmap e688: 8 total, 8 up, 8 in 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:07.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: pgmap v713: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 9.8 MiB/s rd, 2.1 MiB/s wr, 15 op/s 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: osdmap e688: 8 total, 8 up, 8 in 2024-03-20T08:12:07.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:07.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:07.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:07.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:07.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:07.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:07.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:07.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:08.616 INFO:tasks.rados.rados.0.smithi118.stdout:3081: done (5 left) 2024-03-20T08:12:08.616 INFO:tasks.rados.rados.0.smithi118.stdout:3082: read oid 25 snap 304 2024-03-20T08:12:08.616 INFO:tasks.rados.rados.0.smithi118.stdout:3082: expect (ObjNum 798 snap 275 seq_num 798) 2024-03-20T08:12:08.616 INFO:tasks.rados.rados.0.smithi118.stdout:3083: rollback oid 40 current snap is 313 2024-03-20T08:12:08.616 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 309 2024-03-20T08:12:08.616 INFO:tasks.rados.rados.0.smithi118.stdout:3080: expect (ObjNum 905 snap 309 seq_num 905) 2024-03-20T08:12:08.655 INFO:tasks.rados.rados.0.smithi118.stdout:3076: expect (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:3079: finishing copy_from racing read to smithi11881747-28 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:3079: finishing copy_from to smithi11881747-28 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 45603 (ObjNum 900 snap 309 seq_num 900) dirty exists 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:3077: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:3077: finishing copy_from to smithi11881747-12 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 44229 (ObjNum 472 snap 156 seq_num 472) dirty exists 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:3076: done (6 left) 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:3077: done (5 left) 2024-03-20T08:12:08.739 INFO:tasks.rados.rados.0.smithi118.stdout:3078: done (4 left) 2024-03-20T08:12:08.740 INFO:tasks.rados.rados.0.smithi118.stdout:3079: done (3 left) 2024-03-20T08:12:08.740 INFO:tasks.rados.rados.0.smithi118.stdout:3080: done (2 left) 2024-03-20T08:12:08.740 INFO:tasks.rados.rados.0.smithi118.stdout:3084: snap_create 2024-03-20T08:12:08.740 INFO:tasks.rados.rados.0.smithi118.stdout:3083: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:12:08.740 INFO:tasks.rados.rados.0.smithi118.stdout:3083: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:12:08.740 INFO:tasks.rados.rados.0.smithi118.stdout:3083: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:12:08.740 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34077 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:12:08.740 INFO:tasks.rados.rados.0.smithi118.stdout:3082: expect (ObjNum 798 snap 275 seq_num 798) 2024-03-20T08:12:08.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:08.615+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:08.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:08.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:08.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:08.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:08.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:08.915 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: osdmap e689: 8 total, 8 up, 8 in 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:08.916 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:08.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:08.622+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:08.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:08.622+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:08.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:08.629+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:08.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:08.616+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:08.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:08.615+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:08.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:08.619+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:08.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:08.626+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:08.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:08.626+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:08.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:08.626+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:08.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:08.635+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:08.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:08.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:08.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:08.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:08.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:08.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:08.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:08.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: osdmap e689: 8 total, 8 up, 8 in 2024-03-20T08:12:08.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:08.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:08.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:08.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:08.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:08.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:08.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:08.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:08.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:08.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:08.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:08.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:08.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:08.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: osdmap e689: 8 total, 8 up, 8 in 2024-03-20T08:12:08.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:08.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:08.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:08.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:08.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:08.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:08.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:08.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:08.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:08.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:08.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:08.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:08.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:08.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:08.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:08.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:08.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:08.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:09.376 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:09.189+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:09.376 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:09.189+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:09.376 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:09.189+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:09.376 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:09.100+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:09.376 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:09.100+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:09.631 INFO:tasks.rados.rados.0.smithi118.stdout:3082: done (2 left) 2024-03-20T08:12:09.631 INFO:tasks.rados.rados.0.smithi118.stdout:3083: done (1 left) 2024-03-20T08:12:09.631 INFO:tasks.rados.rados.0.smithi118.stdout:3084: done (0 left) 2024-03-20T08:12:09.631 INFO:tasks.rados.rados.0.smithi118.stdout:3085: read oid 13 snap -1 2024-03-20T08:12:09.631 INFO:tasks.rados.rados.0.smithi118.stdout:3085: expect (ObjNum 894 snap 308 seq_num 894) 2024-03-20T08:12:09.631 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 314 2024-03-20T08:12:09.631 INFO:tasks.rados.rados.0.smithi118.stdout:3086: seq_num 923 ranges {6045696=663552,6709248=401408} 2024-03-20T08:12:09.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:09.482+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:09.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:09.482+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:09.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:09.482+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:09.632 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:09.631+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:09.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: pgmap v716: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 22 op/s 2024-03-20T08:12:09.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: osdmap e690: 8 total, 8 up, 8 in 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:09.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:09.667 INFO:tasks.rados.rados.0.smithi118.stdout:3086: writing smithi11881747-12 from 6045696 to 6709248 tid 1 2024-03-20T08:12:09.669 INFO:tasks.rados.rados.0.smithi118.stdout:3086: writing smithi11881747-12 from 6709248 to 7110656 tid 2 2024-03-20T08:12:09.670 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 314 2024-03-20T08:12:09.670 INFO:tasks.rados.rados.0.smithi118.stdout:3087: seq_num 924 ranges {737280=679936,1417216=401408} 2024-03-20T08:12:09.677 INFO:tasks.rados.rados.0.smithi118.stdout:3087: writing smithi11881747-24 from 737280 to 1417216 tid 1 2024-03-20T08:12:09.680 INFO:tasks.rados.rados.0.smithi118.stdout:3087: writing smithi11881747-24 from 1417216 to 1818624 tid 2 2024-03-20T08:12:09.680 INFO:tasks.rados.rados.0.smithi118.stdout:3088: delete oid 15 current snap is 314 2024-03-20T08:12:09.680 INFO:tasks.rados.rados.0.smithi118.stdout:3085: expect (ObjNum 894 snap 308 seq_num 894) 2024-03-20T08:12:09.712 INFO:tasks.rados.rados.0.smithi118.stdout:3086: finishing write tid 1 to smithi11881747-12 2024-03-20T08:12:09.712 INFO:tasks.rados.rados.0.smithi118.stdout:3086: finishing write tid 2 to smithi11881747-12 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3086: finishing write tid 3 to smithi11881747-12 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3085: done (3 left) 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3088: done (2 left) 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3089: read oid 32 snap -1 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3089: expect (ObjNum 921 snap 311 seq_num 921) 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3090: rmattr oid 41 current snap is 314 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3086: finishing write tid 4 to smithi11881747-12 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3086: oid 12 updating version 0 to 44232 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3086: oid 12 version 44232 is already newer than 44231 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3086: oid 12 version 44232 is already newer than 44230 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 44232 (ObjNum 923 snap 314 seq_num 923) dirty exists 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3086: left oid 12 (ObjNum 923 snap 314 seq_num 923) 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3087: finishing write tid 1 to smithi11881747-24 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3087: finishing write tid 2 to smithi11881747-24 2024-03-20T08:12:09.713 INFO:tasks.rados.rados.0.smithi118.stdout:3087: finishing write tid 3 to smithi11881747-24 2024-03-20T08:12:09.714 INFO:tasks.rados.rados.0.smithi118.stdout:3086: done (3 left) 2024-03-20T08:12:09.714 INFO:tasks.rados.rados.0.smithi118.stdout:3091: snap_create 2024-03-20T08:12:09.714 INFO:tasks.rados.rados.0.smithi118.stdout:3087: finishing write tid 4 to smithi11881747-24 2024-03-20T08:12:09.714 INFO:tasks.rados.rados.0.smithi118.stdout:3087: oid 24 updating version 0 to 65246 2024-03-20T08:12:09.714 INFO:tasks.rados.rados.0.smithi118.stdout:3087: oid 24 updating version 65246 to 65248 2024-03-20T08:12:09.714 INFO:tasks.rados.rados.0.smithi118.stdout:3087: oid 24 version 65248 is already newer than 65247 2024-03-20T08:12:09.714 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65248 (ObjNum 924 snap 314 seq_num 924) dirty exists 2024-03-20T08:12:09.714 INFO:tasks.rados.rados.0.smithi118.stdout:3087: left oid 24 (ObjNum 924 snap 314 seq_num 924) 2024-03-20T08:12:09.716 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 45608 (ObjNum 883 snap 306 seq_num 883) dirty exists 2024-03-20T08:12:09.717 INFO:tasks.rados.rados.0.smithi118.stdout:3089: expect (ObjNum 921 snap 311 seq_num 921) 2024-03-20T08:12:09.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:09.631+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:09.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:09.675+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:09.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:09.675+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:09.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:09.631+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:09.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:09.631+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:09.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:09.675+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:09.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:09.675+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:09.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:09.634+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:09.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:09.636+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:09.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:09.670+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:09.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:09.671+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:09.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:09.671+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:09.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:09.677+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:09.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: pgmap v716: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 22 op/s 2024-03-20T08:12:09.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:09.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: osdmap e690: 8 total, 8 up, 8 in 2024-03-20T08:12:09.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:09.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: pgmap v716: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.6 MiB/s rd, 5.8 MiB/s wr, 22 op/s 2024-03-20T08:12:09.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: osdmap e690: 8 total, 8 up, 8 in 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:09.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:09.925 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:09.631+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:09.926 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:09.716+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:09.926 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:09.716+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:10.641 INFO:tasks.rados.rados.0.smithi118.stdout:3087: done (3 left) 2024-03-20T08:12:10.641 INFO:tasks.rados.rados.0.smithi118.stdout:3089: done (2 left) 2024-03-20T08:12:10.641 INFO:tasks.rados.rados.0.smithi118.stdout:3090: done (1 left) 2024-03-20T08:12:10.642 INFO:tasks.rados.rados.0.smithi118.stdout:3091: done (0 left) 2024-03-20T08:12:10.642 INFO:tasks.rados.rados.0.smithi118.stdout:3092: read oid 49 snap -1 2024-03-20T08:12:10.642 INFO:tasks.rados.rados.0.smithi118.stdout:3092: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:12:10.642 INFO:tasks.rados.rados.0.smithi118.stdout:3093: snap_remove snap 302 2024-03-20T08:12:10.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:10.640+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:10.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:10.790+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:10.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:10.791+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:10.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:10.791+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:10.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:10.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: osdmap e691: 8 total, 8 up, 8 in 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:10.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: Upgrade: Finalizing container_image settings 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-20T08:12:10.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-20T08:12:10.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: Upgrade: Complete! 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:10.662+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:10.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:10.640+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:10.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:10.641+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:10.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:10.641+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:10.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:10.641+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:10.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:10.641+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:10.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:10.641+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:10.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: osdmap e691: 8 total, 8 up, 8 in 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:10.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: osdmap e691: 8 total, 8 up, 8 in 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:10.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 165=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.928 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.930 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: Upgrade: Finalizing container_image settings 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-20T08:12:10.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-20T08:12:10.932 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: Upgrade: Finalizing container_image settings 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.933 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-20T08:12:10.934 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-20T08:12:10.935 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-20T08:12:10.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-20T08:12:10.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-20T08:12:10.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-20T08:12:10.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-20T08:12:10.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: Upgrade: Complete! 2024-03-20T08:12:10.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.937 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.938 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: Upgrade: Complete! 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:10.939 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:12:10.940 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:12:10.940 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:12:10.940 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:11.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:11.036+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:11.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:11.036+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:11.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:11.007+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:11.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:11.008+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:11.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:11.008+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:11.643 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:11.318+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:11.643 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:11.318+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:11.643 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:11.318+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:11.643 INFO:tasks.rados.rados.0.smithi118.stdout:3093: done (1 left) 2024-03-20T08:12:11.644 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 315 2024-03-20T08:12:11.644 INFO:tasks.rados.rados.0.smithi118.stdout:3094: seq_num 926 ranges {8200192=712704,8912896=401408} 2024-03-20T08:12:11.692 INFO:tasks.rados.rados.0.smithi118.stdout:3094: writing smithi11881747-16 from 8200192 to 8912896 tid 1 2024-03-20T08:12:11.693 INFO:tasks.rados.rados.0.smithi118.stdout:3094: writing smithi11881747-16 from 8912896 to 9314304 tid 2 2024-03-20T08:12:11.694 INFO:tasks.rados.rados.0.smithi118.stdout:3092: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:12:11.703 INFO:tasks.rados.rados.0.smithi118.stdout:3094: finishing write tid 1 to smithi11881747-16 2024-03-20T08:12:11.703 INFO:tasks.rados.rados.0.smithi118.stdout:3094: finishing write tid 2 to smithi11881747-16 2024-03-20T08:12:11.703 INFO:tasks.rados.rados.0.smithi118.stdout:3094: finishing write tid 3 to smithi11881747-16 2024-03-20T08:12:11.703 INFO:tasks.rados.rados.0.smithi118.stdout:3094: finishing write tid 4 to smithi11881747-16 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3094: oid 16 updating version 0 to 45611 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3094: oid 16 version 45611 is already newer than 45609 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3094: oid 16 version 45611 is already newer than 45610 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 45611 (ObjNum 926 snap 315 seq_num 926) dirty exists 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3094: left oid 16 (ObjNum 926 snap 315 seq_num 926) 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3095: read oid 17 snap -1 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3095: expect (ObjNum 917 snap 310 seq_num 917) 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3092: done (2 left) 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3094: done (1 left) 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3096: read oid 9 snap -1 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3096: expect deleted 2024-03-20T08:12:11.704 INFO:tasks.rados.rados.0.smithi118.stdout:3097: snap_remove snap 308 2024-03-20T08:12:11.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:11.704+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:11.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:11.705+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:11.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:11.827+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:11.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:11.643+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:11.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:11.643+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:11.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:11.640+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:11.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:11.640+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:11.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:11.641+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:11.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:11.641+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:11.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: pgmap v719: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 16 op/s 2024-03-20T08:12:11.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:11.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: osdmap e692: 8 total, 8 up, 8 in 2024-03-20T08:12:11.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:11.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:11.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:11.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:11.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:11.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:11.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:11.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:11.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:11.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:11.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:11.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:11.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: pgmap v719: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 16 op/s 2024-03-20T08:12:11.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: osdmap e692: 8 total, 8 up, 8 in 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:11.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:11.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:12.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:11.642+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:12.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:11.705+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:12.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:11.710+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:12.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:11.695+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:12.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:11.695+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:12.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:11.696+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:12.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:11.828+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:12.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:11.828+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: pgmap v719: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 4.1 MiB/s wr, 16 op/s 2024-03-20T08:12:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: osdmap e692: 8 total, 8 up, 8 in 2024-03-20T08:12:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:12.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:12.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:12.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:12.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:12.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:12.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:12.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:12.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:12:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:12:12.654 INFO:tasks.rados.rados.0.smithi118.stdout:3097: done (2 left) 2024-03-20T08:12:12.654 INFO:tasks.rados.rados.0.smithi118.stdout:3098: read oid 26 snap 303 2024-03-20T08:12:12.654 INFO:tasks.rados.rados.0.smithi118.stdout:3098: expect (ObjNum 860 snap 297 seq_num 860) 2024-03-20T08:12:12.654 INFO:tasks.rados.rados.0.smithi118.stdout:3099: read oid 41 snap 304 2024-03-20T08:12:12.654 INFO:tasks.rados.rados.0.smithi118.stdout:3099: expect (ObjNum 321766048 snap 21898 seq_num 322303312) 2024-03-20T08:12:12.654 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 315 2024-03-20T08:12:12.654 INFO:tasks.rados.rados.0.smithi118.stdout:3100: seq_num 927 ranges {0=729088,729088=401408} 2024-03-20T08:12:12.657 INFO:tasks.rados.rados.0.smithi118.stdout:3100: writing smithi11881747-49 from 0 to 729088 tid 1 2024-03-20T08:12:12.660 INFO:tasks.rados.rados.0.smithi118.stdout:3100: writing smithi11881747-49 from 729088 to 1130496 tid 2 2024-03-20T08:12:12.660 INFO:tasks.rados.rados.0.smithi118.stdout:3101: snap_remove snap 287 2024-03-20T08:12:12.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:12.663+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:12.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:12.650+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:12.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:12.650+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:12.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:12.651+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:12.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:12.664+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:12.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:12.655+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:12.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:12.656+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:12.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:12.656+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:12.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:12.656+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:12.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:12.657+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:12.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:12.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:12.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:12.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:12.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: osdmap e693: 8 total, 8 up, 8 in 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:12.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: osdmap e693: 8 total, 8 up, 8 in 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:12.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:12.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:13.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:12.654+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:13.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:12.659+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:13.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:12.676+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:13.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:12.676+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:13.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:12.656+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:13.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:12.656+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:13.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: osdmap e693: 8 total, 8 up, 8 in 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:13.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:13.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:13.657 INFO:tasks.rados.rados.0.smithi118.stdout:3101: done (5 left) 2024-03-20T08:12:13.657 INFO:tasks.rados.rados.0.smithi118.stdout:3102: copy_from oid 27 from oid 39 current snap is 315 2024-03-20T08:12:13.657 INFO:tasks.rados.rados.0.smithi118.stdout:3095: expect (ObjNum 917 snap 310 seq_num 917) 2024-03-20T08:12:13.697 INFO:tasks.rados.rados.0.smithi118.stdout:3099: expect (ObjNum 321766048 snap 21898 seq_num 322303312) 2024-03-20T08:12:13.700 INFO:tasks.rados.rados.0.smithi118.stdout:3100: finishing write tid 1 to smithi11881747-49 2024-03-20T08:12:13.700 INFO:tasks.rados.rados.0.smithi118.stdout:3095: done (5 left) 2024-03-20T08:12:13.700 INFO:tasks.rados.rados.0.smithi118.stdout:3096: done (4 left) 2024-03-20T08:12:13.701 INFO:tasks.rados.rados.0.smithi118.stdout:3099: done (3 left) 2024-03-20T08:12:13.701 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 15 current snap is 315 2024-03-20T08:12:13.701 INFO:tasks.rados.rados.0.smithi118.stdout:3100: finishing write tid 2 to smithi11881747-49 2024-03-20T08:12:13.701 INFO:tasks.rados.rados.0.smithi118.stdout:3103: seq_num 928 ranges {0=745472,745472=401408} 2024-03-20T08:12:13.704 INFO:tasks.rados.rados.0.smithi118.stdout:3103: writing smithi11881747-15 from 0 to 745472 tid 1 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3103: writing smithi11881747-15 from 745472 to 1146880 tid 2 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3104: rollback oid 35 current snap is 315 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3100: finishing write tid 3 to smithi11881747-49 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3100: finishing write tid 4 to smithi11881747-49 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3100: oid 49 updating version 0 to 59818 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3100: oid 49 updating version 59818 to 59819 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3100: oid 49 version 59819 is already newer than 59817 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 59819 (ObjNum 927 snap 315 seq_num 927) dirty exists 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3100: left oid 49 (ObjNum 927 snap 315 seq_num 927) 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 35 to 266 2024-03-20T08:12:13.707 INFO:tasks.rados.rados.0.smithi118.stdout:3098: expect (ObjNum 860 snap 297 seq_num 860) 2024-03-20T08:12:13.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:13.711+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:13.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:13.658+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:13.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:13.658+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:13.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:13.752+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:13.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:13.752+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:13.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:13.752+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:13.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:13.583+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:13.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:13.583+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:13.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:13.583+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:13.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:13.659+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:13.891 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:13.659+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:13.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: pgmap v722: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 8.3 MiB/s rd, 0 B/s wr, 8 op/s 2024-03-20T08:12:13.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:13.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:13.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: osdmap e694: 8 total, 8 up, 8 in 2024-03-20T08:12:13.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:13.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:13.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:13.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:13.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:13.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:13.656+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:13.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:13.656+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:13.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:13.735+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:13.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:13.735+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:13.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:13.736+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:13.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:13.736+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: pgmap v722: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 8.3 MiB/s rd, 0 B/s wr, 8 op/s 2024-03-20T08:12:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: osdmap e694: 8 total, 8 up, 8 in 2024-03-20T08:12:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:13.924 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:12:13] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:12:13.925 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:13.709+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:13.925 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:13.660+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:13.925 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:13.660+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: pgmap v722: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 8.3 MiB/s rd, 0 B/s wr, 8 op/s 2024-03-20T08:12:13.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: osdmap e694: 8 total, 8 up, 8 in 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:13.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:13.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:13.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:13.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:12:14.072 INFO:tasks.rados.rados.0.smithi118.stdout:3102: finishing copy_from to smithi11881747-27 2024-03-20T08:12:14.072 INFO:tasks.rados.rados.0.smithi118.stdout:3102: got expected ENOENT (src dne) 2024-03-20T08:12:14.072 INFO:tasks.rados.rados.0.smithi118.stdout:3102: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:12:14.072 INFO:tasks.rados.rados.0.smithi118.stdout:3104: finishing rollback tid 1 to smithi11881747-35 2024-03-20T08:12:14.072 INFO:tasks.rados.rados.0.smithi118.stdout:3104: finishing rollback tid 2 to smithi11881747-35 2024-03-20T08:12:14.072 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 45614 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3103: finishing write tid 1 to smithi11881747-15 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3103: finishing write tid 2 to smithi11881747-15 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3098: done (4 left) 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3100: done (3 left) 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3102: done (2 left) 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3104: done (1 left) 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3105: rollback oid 25 current snap is 315 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 25 to 307 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3103: finishing write tid 3 to smithi11881747-15 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3103: finishing write tid 4 to smithi11881747-15 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3103: oid 15 updating version 0 to 7104 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3103: oid 15 updating version 7104 to 7105 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3103: oid 15 version 7105 is already newer than 7103 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 7105 (ObjNum 928 snap 315 seq_num 928) dirty exists 2024-03-20T08:12:14.073 INFO:tasks.rados.rados.0.smithi118.stdout:3103: left oid 15 (ObjNum 928 snap 315 seq_num 928) 2024-03-20T08:12:14.074 INFO:tasks.rados.rados.0.smithi118.stdout:3103: done (1 left) 2024-03-20T08:12:14.074 INFO:tasks.rados.rados.0.smithi118.stdout:3106: snap_create 2024-03-20T08:12:14.075 INFO:tasks.rados.rados.0.smithi118.stdout:3105: finishing rollback tid 0 to smithi11881747-25 2024-03-20T08:12:14.075 INFO:tasks.rados.rados.0.smithi118.stdout:3105: finishing rollback tid 1 to smithi11881747-25 2024-03-20T08:12:14.076 INFO:tasks.rados.rados.0.smithi118.stdout:3105: finishing rollback tid 2 to smithi11881747-25 2024-03-20T08:12:14.076 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 45604 (ObjNum 161 snap 0 seq_num 2) dirty exists 2024-03-20T08:12:14.663 INFO:tasks.rados.rados.0.smithi118.stdout:3105: done (1 left) 2024-03-20T08:12:14.663 INFO:tasks.rados.rados.0.smithi118.stdout:3106: done (0 left) 2024-03-20T08:12:14.663 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 30 current snap is 316 2024-03-20T08:12:14.663 INFO:tasks.rados.rados.0.smithi118.stdout:3107: seq_num 929 ranges {8929280=761856,9691136=401408} 2024-03-20T08:12:14.715 INFO:tasks.rados.rados.0.smithi118.stdout:3107: writing smithi11881747-30 from 8929280 to 9691136 tid 1 2024-03-20T08:12:14.718 INFO:tasks.rados.rados.0.smithi118.stdout:3107: writing smithi11881747-30 from 9691136 to 10092544 tid 2 2024-03-20T08:12:14.718 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 316 2024-03-20T08:12:14.718 INFO:tasks.rados.rados.0.smithi118.stdout:3108: seq_num 930 ranges {9756672=786432,10543104=401408} 2024-03-20T08:12:14.775 INFO:tasks.rados.rados.0.smithi118.stdout:3108: writing smithi11881747-38 from 9756672 to 10543104 tid 1 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3108: writing smithi11881747-38 from 10543104 to 10944512 tid 2 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3109: snap_create 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3107: finishing write tid 1 to smithi11881747-30 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3107: finishing write tid 2 to smithi11881747-30 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3107: finishing write tid 3 to smithi11881747-30 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3107: finishing write tid 4 to smithi11881747-30 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3107: oid 30 updating version 0 to 34029 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3107: oid 30 updating version 34029 to 34030 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3107: oid 30 version 34030 is already newer than 34028 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 34030 (ObjNum 929 snap 316 seq_num 929) dirty exists 2024-03-20T08:12:14.778 INFO:tasks.rados.rados.0.smithi118.stdout:3107: left oid 30 (ObjNum 929 snap 316 seq_num 929) 2024-03-20T08:12:14.790 INFO:tasks.rados.rados.0.smithi118.stdout:3108: finishing write tid 1 to smithi11881747-38 2024-03-20T08:12:14.790 INFO:tasks.rados.rados.0.smithi118.stdout:3108: finishing write tid 2 to smithi11881747-38 2024-03-20T08:12:14.790 INFO:tasks.rados.rados.0.smithi118.stdout:3108: finishing write tid 3 to smithi11881747-38 2024-03-20T08:12:14.796 INFO:tasks.rados.rados.0.smithi118.stdout:3108: finishing write tid 4 to smithi11881747-38 2024-03-20T08:12:14.796 INFO:tasks.rados.rados.0.smithi118.stdout:3108: oid 38 updating version 0 to 61719 2024-03-20T08:12:14.796 INFO:tasks.rados.rados.0.smithi118.stdout:3108: oid 38 version 61719 is already newer than 61718 2024-03-20T08:12:14.796 INFO:tasks.rados.rados.0.smithi118.stdout:3108: oid 38 version 61719 is already newer than 61717 2024-03-20T08:12:14.796 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 61719 (ObjNum 930 snap 316 seq_num 930) dirty exists 2024-03-20T08:12:14.796 INFO:tasks.rados.rados.0.smithi118.stdout:3108: left oid 38 (ObjNum 930 snap 316 seq_num 930) 2024-03-20T08:12:14.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:14.780+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:14.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:14.780+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:14.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:14.780+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:14.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:14.720+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:14.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:14.660+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:14.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:14.661+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:14.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:14.661+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:14.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:14.661+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:14.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:14.661+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: osdmap e695: 8 total, 8 up, 8 in 2024-03-20T08:12:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:14.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:14.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:14.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:14.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:14.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:14.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:14.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: osdmap e695: 8 total, 8 up, 8 in 2024-03-20T08:12:14.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:14.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:14.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:15.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:14.663+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:15.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:14.781+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:15.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:14.781+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:15.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:14.665+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:15.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:14.665+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:15.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:14.665+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:15.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:14.721+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:15.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:14.722+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:15.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:14.722+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: osdmap e695: 8 total, 8 up, 8 in 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:15.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:15.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:15.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:15.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:15.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:15.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:15.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:15.149+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:15.664 INFO:tasks.rados.rados.0.smithi118.stdout:3107: done (2 left) 2024-03-20T08:12:15.664 INFO:tasks.rados.rados.0.smithi118.stdout:3108: done (1 left) 2024-03-20T08:12:15.664 INFO:tasks.rados.rados.0.smithi118.stdout:3109: done (0 left) 2024-03-20T08:12:15.664 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 317 2024-03-20T08:12:15.664 INFO:tasks.rados.rados.0.smithi118.stdout:3110: seq_num 931 ranges {425984=802816,1228800=401408} 2024-03-20T08:12:15.670 INFO:tasks.rados.rados.0.smithi118.stdout:3110: writing smithi11881747-31 from 425984 to 1228800 tid 1 2024-03-20T08:12:15.673 INFO:tasks.rados.rados.0.smithi118.stdout:3110: writing smithi11881747-31 from 1228800 to 1630208 tid 2 2024-03-20T08:12:15.673 INFO:tasks.rados.rados.0.smithi118.stdout:3111: rmattr oid 35 current snap is 317 2024-03-20T08:12:15.673 INFO:tasks.rados.rados.0.smithi118.stdout:3111: done (1 left) 2024-03-20T08:12:15.673 INFO:tasks.rados.rados.0.smithi118.stdout:3112: read oid 29 snap 272 2024-03-20T08:12:15.673 INFO:tasks.rados.rados.0.smithi118.stdout:3112: expect (ObjNum 638 snap 206 seq_num 638) 2024-03-20T08:12:15.673 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 317 2024-03-20T08:12:15.674 INFO:tasks.rados.rados.0.smithi118.stdout:3113: seq_num 932 ranges {1130496=417792,1548288=663552,2211840=139264} 2024-03-20T08:12:15.683 INFO:tasks.rados.rados.0.smithi118.stdout:3113: writing smithi11881747-49 from 1130496 to 1548288 tid 1 2024-03-20T08:12:15.688 INFO:tasks.rados.rados.0.smithi118.stdout:3113: writing smithi11881747-49 from 1548288 to 2211840 tid 2 2024-03-20T08:12:15.689 INFO:tasks.rados.rados.0.smithi118.stdout:3113: writing smithi11881747-49 from 2211840 to 2351104 tid 3 2024-03-20T08:12:15.689 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 317 2024-03-20T08:12:15.689 INFO:tasks.rados.rados.0.smithi118.stdout:3114: seq_num 933 ranges {10731520=434176,11165696=548864,11714560=253952} 2024-03-20T08:12:15.751 INFO:tasks.rados.rados.0.smithi118.stdout:3114: writing smithi11881747-6 from 10731520 to 11165696 tid 1 2024-03-20T08:12:15.754 INFO:tasks.rados.rados.0.smithi118.stdout:3114: writing smithi11881747-6 from 11165696 to 11714560 tid 2 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3114: writing smithi11881747-6 from 11714560 to 11968512 tid 3 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3115: copy_from oid 36 from oid 18 current snap is 317 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: finishing write tid 1 to smithi11881747-31 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: finishing write tid 2 to smithi11881747-31 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: finishing write tid 3 to smithi11881747-31 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: finishing write tid 4 to smithi11881747-31 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: oid 31 updating version 0 to 44244 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: oid 31 version 44244 is already newer than 44243 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: oid 31 updating version 44244 to 44245 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 44245 (ObjNum 931 snap 317 seq_num 931) dirty exists 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: left oid 31 (ObjNum 931 snap 317 seq_num 931) 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3113: finishing write tid 1 to smithi11881747-49 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3110: done (4 left) 2024-03-20T08:12:15.756 INFO:tasks.rados.rados.0.smithi118.stdout:3116: copy_from oid 23 from oid 14 current snap is 317 2024-03-20T08:12:15.757 INFO:tasks.rados.rados.0.smithi118.stdout:3112: expect (ObjNum 638 snap 206 seq_num 638) 2024-03-20T08:12:15.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:15.758+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:15.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:15.758+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:15.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:15.661+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:15.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:15.662+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:15.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:15.662+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:15.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:15.663+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:15.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:15.663+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:15.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:15.663+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:15.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:15.690+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: pgmap v725: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.1 MiB/s rd, 532 KiB/s wr, 8 op/s 2024-03-20T08:12:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:15.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: osdmap e696: 8 total, 8 up, 8 in 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:15.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: pgmap v725: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.1 MiB/s rd, 532 KiB/s wr, 8 op/s 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: osdmap e696: 8 total, 8 up, 8 in 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:15.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:15.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:15.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:15.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:15.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:15.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: finishing write tid 2 to smithi11881747-49 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: finishing write tid 3 to smithi11881747-49 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: finishing write tid 4 to smithi11881747-49 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: finishing write tid 5 to smithi11881747-49 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: oid 49 updating version 0 to 59822 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: oid 49 version 59822 is already newer than 59820 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: oid 49 updating version 59822 to 59823 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: oid 49 version 59823 is already newer than 59821 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 59823 (ObjNum 932 snap 317 seq_num 932) dirty exists 2024-03-20T08:12:16.082 INFO:tasks.rados.rados.0.smithi118.stdout:3113: left oid 49 (ObjNum 932 snap 317 seq_num 932) 2024-03-20T08:12:16.083 INFO:tasks.rados.rados.0.smithi118.stdout:3112: done (4 left) 2024-03-20T08:12:16.083 INFO:tasks.rados.rados.0.smithi118.stdout:3113: done (3 left) 2024-03-20T08:12:16.083 INFO:tasks.rados.rados.0.smithi118.stdout:3117: copy_from oid 41 from oid 4 current snap is 317 2024-03-20T08:12:16.083 INFO:tasks.rados.rados.0.smithi118.stdout:3114: finishing write tid 1 to smithi11881747-6 2024-03-20T08:12:16.083 INFO:tasks.rados.rados.0.smithi118.stdout:3114: finishing write tid 2 to smithi11881747-6 2024-03-20T08:12:16.083 INFO:tasks.rados.rados.0.smithi118.stdout:3114: finishing write tid 3 to smithi11881747-6 2024-03-20T08:12:16.084 INFO:tasks.rados.rados.0.smithi118.stdout:3114: finishing write tid 4 to smithi11881747-6 2024-03-20T08:12:16.084 INFO:tasks.rados.rados.0.smithi118.stdout:3116: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:12:16.084 INFO:tasks.rados.rados.0.smithi118.stdout:3116: finishing copy_from to smithi11881747-23 2024-03-20T08:12:16.084 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 34033 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:12:16.084 INFO:tasks.rados.rados.0.smithi118.stdout:3114: finishing write tid 5 to smithi11881747-6 2024-03-20T08:12:16.084 INFO:tasks.rados.rados.0.smithi118.stdout:3114: oid 6 updating version 0 to 45616 2024-03-20T08:12:16.085 INFO:tasks.rados.rados.0.smithi118.stdout:3114: oid 6 version 45616 is already newer than 45615 2024-03-20T08:12:16.085 INFO:tasks.rados.rados.0.smithi118.stdout:3114: oid 6 updating version 45616 to 45618 2024-03-20T08:12:16.085 INFO:tasks.rados.rados.0.smithi118.stdout:3114: oid 6 version 45618 is already newer than 45617 2024-03-20T08:12:16.085 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 45618 (ObjNum 933 snap 317 seq_num 933) dirty exists 2024-03-20T08:12:16.085 INFO:tasks.rados.rados.0.smithi118.stdout:3114: left oid 6 (ObjNum 933 snap 317 seq_num 933) 2024-03-20T08:12:16.085 INFO:tasks.rados.rados.0.smithi118.stdout:3114: done (3 left) 2024-03-20T08:12:16.086 INFO:tasks.rados.rados.0.smithi118.stdout:3116: done (2 left) 2024-03-20T08:12:16.086 INFO:tasks.rados.rados.0.smithi118.stdout:3118: read oid 13 snap -1 2024-03-20T08:12:16.086 INFO:tasks.rados.rados.0.smithi118.stdout:3118: expect (ObjNum 894 snap 308 seq_num 894) 2024-03-20T08:12:16.086 INFO:tasks.rados.rados.0.smithi118.stdout:3119: read oid 37 snap 304 2024-03-20T08:12:16.086 INFO:tasks.rados.rados.0.smithi118.stdout:3119: expect (ObjNum 794 snap 275 seq_num 794) 2024-03-20T08:12:16.086 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 317 2024-03-20T08:12:16.086 INFO:tasks.rados.rados.0.smithi118.stdout:3115: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:12:16.086 INFO:tasks.rados.rados.0.smithi118.stdout:3120: seq_num 934 ranges {245760=450560,696320=434176,1130496=368640} 2024-03-20T08:12:16.087 INFO:tasks.rados.rados.0.smithi118.stdout:3120: writing smithi11881747-19 from 245760 to 696320 tid 1 2024-03-20T08:12:16.088 INFO:tasks.rados.rados.0.smithi118.stdout:3120: writing smithi11881747-19 from 696320 to 1130496 tid 2 2024-03-20T08:12:16.091 INFO:tasks.rados.rados.0.smithi118.stdout:3120: writing smithi11881747-19 from 1130496 to 1499136 tid 3 2024-03-20T08:12:16.091 INFO:tasks.rados.rados.0.smithi118.stdout:3121: read oid 42 snap -1 2024-03-20T08:12:16.091 INFO:tasks.rados.rados.0.smithi118.stdout:3121: expect (ObjNum 866 snap 301 seq_num 866) 2024-03-20T08:12:16.091 INFO:tasks.rados.rados.0.smithi118.stdout:3122: rollback oid 2 current snap is 317 2024-03-20T08:12:16.091 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 266 2024-03-20T08:12:16.091 INFO:tasks.rados.rados.0.smithi118.stdout:3115: finishing copy_from to smithi11881747-36 2024-03-20T08:12:16.091 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 33996 (ObjNum 814 snap 279 seq_num 814) dirty exists 2024-03-20T08:12:16.091 INFO:tasks.rados.rados.0.smithi118.stdout:3118: expect (ObjNum 894 snap 308 seq_num 894) 2024-03-20T08:12:16.123 INFO:tasks.rados.rados.0.smithi118.stdout:3117: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:12:16.124 INFO:tasks.rados.rados.0.smithi118.stdout:3117: finishing copy_from to smithi11881747-41 2024-03-20T08:12:16.124 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 34027 (ObjNum 914 snap 309 seq_num 914) dirty exists 2024-03-20T08:12:16.124 INFO:tasks.rados.rados.0.smithi118.stdout:3119: expect (ObjNum 794 snap 275 seq_num 794) 2024-03-20T08:12:16.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:15.663+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:16.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:15.755+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:16.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:15.758+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:16.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:15.683+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:16.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:15.683+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:16.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:15.677+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:16.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:15.677+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:16.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:15.678+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:16.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: pgmap v725: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 6.1 MiB/s rd, 532 KiB/s wr, 8 op/s 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: osdmap e696: 8 total, 8 up, 8 in 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:16.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:16.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:16.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:16.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:16.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:16.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:16.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:16.271 INFO:tasks.rados.rados.0.smithi118.stdout:3121: expect (ObjNum 866 snap 301 seq_num 866) 2024-03-20T08:12:16.274 INFO:tasks.rados.rados.0.smithi118.stdout:3122: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:12:16.274 INFO:tasks.rados.rados.0.smithi118.stdout:3122: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:12:16.274 INFO:tasks.rados.rados.0.smithi118.stdout:3122: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45621 (ObjNum 685 snap 225 seq_num 685) dirty exists 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3120: finishing write tid 1 to smithi11881747-19 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3120: finishing write tid 2 to smithi11881747-19 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3120: finishing write tid 3 to smithi11881747-19 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3120: finishing write tid 4 to smithi11881747-19 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3115: done (6 left) 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3117: done (5 left) 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3118: done (4 left) 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3119: done (3 left) 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3121: done (2 left) 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3122: done (1 left) 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3123: read oid 37 snap -1 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3123: expect (ObjNum 920 snap 311 seq_num 920) 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:3124: rollback oid 24 current snap is 317 2024-03-20T08:12:16.275 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 312 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:3120: finishing write tid 5 to smithi11881747-19 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:3120: oid 19 updating version 0 to 7108 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:3120: oid 19 updating version 7108 to 7109 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:3120: oid 19 version 7109 is already newer than 7106 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:3120: oid 19 version 7109 is already newer than 7107 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 7109 (ObjNum 934 snap 317 seq_num 934) dirty exists 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:3120: left oid 19 (ObjNum 934 snap 317 seq_num 934) 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:3120: done (2 left) 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 317 2024-03-20T08:12:16.276 INFO:tasks.rados.rados.0.smithi118.stdout:3125: seq_num 935 ranges {1228800=466944,1695744=720896,2416640=81920} 2024-03-20T08:12:16.283 INFO:tasks.rados.rados.0.smithi118.stdout:3125: writing smithi11881747-8 from 1228800 to 1695744 tid 1 2024-03-20T08:12:16.288 INFO:tasks.rados.rados.0.smithi118.stdout:3125: writing smithi11881747-8 from 1695744 to 2416640 tid 2 2024-03-20T08:12:16.288 INFO:tasks.rados.rados.0.smithi118.stdout:3125: writing smithi11881747-8 from 2416640 to 2498560 tid 3 2024-03-20T08:12:16.288 INFO:tasks.rados.rados.0.smithi118.stdout:3126: snap_remove snap 272 2024-03-20T08:12:16.674 INFO:tasks.rados.rados.0.smithi118.stdout:3126: done (3 left) 2024-03-20T08:12:16.674 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 317 2024-03-20T08:12:16.674 INFO:tasks.rados.rados.0.smithi118.stdout:3127: seq_num 936 ranges {1032192=483328,1515520=655360,2170880=147456} 2024-03-20T08:12:16.682 INFO:tasks.rados.rados.0.smithi118.stdout:3127: writing smithi11881747-32 from 1032192 to 1515520 tid 1 2024-03-20T08:12:16.685 INFO:tasks.rados.rados.0.smithi118.stdout:3127: writing smithi11881747-32 from 1515520 to 2170880 tid 2 2024-03-20T08:12:16.686 INFO:tasks.rados.rados.0.smithi118.stdout:3127: writing smithi11881747-32 from 2170880 to 2318336 tid 3 2024-03-20T08:12:16.687 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 317 2024-03-20T08:12:16.687 INFO:tasks.rados.rados.0.smithi118.stdout:3128: seq_num 937 ranges {3153920=499712,3653632=532480,4186112=270336} 2024-03-20T08:12:16.707 INFO:tasks.rados.rados.0.smithi118.stdout:3128: writing smithi11881747-36 from 3153920 to 3653632 tid 1 2024-03-20T08:12:16.710 INFO:tasks.rados.rados.0.smithi118.stdout:3128: writing smithi11881747-36 from 3653632 to 4186112 tid 2 2024-03-20T08:12:16.712 INFO:tasks.rados.rados.0.smithi118.stdout:3128: writing smithi11881747-36 from 4186112 to 4456448 tid 3 2024-03-20T08:12:16.712 INFO:tasks.rados.rados.0.smithi118.stdout:3129: setattr oid 12 current snap is 317 2024-03-20T08:12:16.712 INFO:tasks.rados.rados.0.smithi118.stdout:3123: expect (ObjNum 920 snap 311 seq_num 920) 2024-03-20T08:12:16.792 INFO:tasks.rados.rados.0.smithi118.stdout:3124: finishing rollback tid 0 to smithi11881747-24 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3124: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3124: finishing rollback tid 2 to smithi11881747-24 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65252 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: finishing write tid 1 to smithi11881747-8 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: finishing write tid 2 to smithi11881747-8 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: finishing write tid 3 to smithi11881747-8 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: finishing write tid 4 to smithi11881747-8 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: finishing write tid 5 to smithi11881747-8 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: oid 8 updating version 0 to 34079 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: oid 8 updating version 34079 to 34080 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: oid 8 updating version 34080 to 34081 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: oid 8 version 34081 is already newer than 34078 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34081 (ObjNum 935 snap 317 seq_num 935) dirty exists 2024-03-20T08:12:16.793 INFO:tasks.rados.rados.0.smithi118.stdout:3125: left oid 8 (ObjNum 935 snap 317 seq_num 935) 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: finishing write tid 1 to smithi11881747-32 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: finishing write tid 2 to smithi11881747-32 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: finishing write tid 3 to smithi11881747-32 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: finishing write tid 4 to smithi11881747-32 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: finishing write tid 5 to smithi11881747-32 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: oid 32 updating version 0 to 34083 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: oid 32 version 34083 is already newer than 34082 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: oid 32 updating version 34083 to 34085 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: oid 32 version 34085 is already newer than 34084 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34085 (ObjNum 936 snap 317 seq_num 936) dirty exists 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3127: left oid 32 (ObjNum 936 snap 317 seq_num 936) 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3128: finishing write tid 1 to smithi11881747-36 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3128: finishing write tid 2 to smithi11881747-36 2024-03-20T08:12:16.794 INFO:tasks.rados.rados.0.smithi118.stdout:3128: finishing write tid 3 to smithi11881747-36 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3128: finishing write tid 4 to smithi11881747-36 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3128: finishing write tid 5 to smithi11881747-36 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3128: oid 36 updating version 0 to 45599 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3128: oid 36 version 45599 is already newer than 45598 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3128: oid 36 updating version 45599 to 45600 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3128: oid 36 version 45600 is already newer than 45597 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45600 (ObjNum 937 snap 317 seq_num 937) dirty exists 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3128: left oid 36 (ObjNum 937 snap 317 seq_num 937) 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3123: done (5 left) 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3124: done (4 left) 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3125: done (3 left) 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3127: done (2 left) 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3128: done (1 left) 2024-03-20T08:12:16.795 INFO:tasks.rados.rados.0.smithi118.stdout:3130: snap_remove snap 312 2024-03-20T08:12:16.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:16.690+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:16.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:16.675+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:16.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:16.689+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:16.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:16.689+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:16.924 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:16.689+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:16.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:16.673+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:16.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:16.673+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:16.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:16.673+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:16.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:16.673+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:16.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:16.673+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:16.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:16.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:16.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:16.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:16.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: osdmap e697: 8 total, 8 up, 8 in 2024-03-20T08:12:16.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:16.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:16.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:16.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:16.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:16.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:16.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:16.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:16.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:16.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:16.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:16.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:16.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:16.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:16.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:16.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: osdmap e697: 8 total, 8 up, 8 in 2024-03-20T08:12:16.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:16.931 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:17.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:16.798+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:17.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:16.799+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:17.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:16.799+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:17.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:16.688+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:17.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:16.688+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:17.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:16.688+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:17.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:16.798+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:17.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:16.798+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:17.140 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:16.674+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: osdmap e697: 8 total, 8 up, 8 in 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 175=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 169=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:17.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:17.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:17.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:17.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:17.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:17.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:17.679 INFO:tasks.rados.rados.0.smithi118.stdout:3130: done (1 left) 2024-03-20T08:12:17.679 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 317 2024-03-20T08:12:17.679 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 44233 (ObjNum 923 snap 314 seq_num 923) dirty exists 2024-03-20T08:12:17.679 INFO:tasks.rados.rados.0.smithi118.stdout:3131: seq_num 938 ranges {655360=516096,1171456=417792,1589248=385024} 2024-03-20T08:12:17.685 INFO:tasks.rados.rados.0.smithi118.stdout:3131: writing smithi11881747-14 from 655360 to 1171456 tid 1 2024-03-20T08:12:17.688 INFO:tasks.rados.rados.0.smithi118.stdout:3131: writing smithi11881747-14 from 1171456 to 1589248 tid 2 2024-03-20T08:12:17.694 INFO:tasks.rados.rados.0.smithi118.stdout:3131: writing smithi11881747-14 from 1589248 to 1974272 tid 3 2024-03-20T08:12:17.694 INFO:tasks.rados.rados.0.smithi118.stdout:3129: done (1 left) 2024-03-20T08:12:17.694 INFO:tasks.rados.rados.0.smithi118.stdout:3132: read oid 4 snap -1 2024-03-20T08:12:17.694 INFO:tasks.rados.rados.0.smithi118.stdout:3132: expect (ObjNum 914 snap 309 seq_num 914) 2024-03-20T08:12:17.694 INFO:tasks.rados.rados.0.smithi118.stdout:3133: setattr oid 2 current snap is 317 2024-03-20T08:12:17.694 INFO:tasks.rados.rados.0.smithi118.stdout:3134: setattr oid 49 current snap is 317 2024-03-20T08:12:17.694 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 317 2024-03-20T08:12:17.694 INFO:tasks.rados.rados.0.smithi118.stdout:3135: seq_num 939 ranges {106496=532480,638976=704512,1343488=98304} 2024-03-20T08:12:17.698 INFO:tasks.rados.rados.0.smithi118.stdout:3135: writing smithi11881747-42 from 106496 to 638976 tid 1 2024-03-20T08:12:17.703 INFO:tasks.rados.rados.0.smithi118.stdout:3135: writing smithi11881747-42 from 638976 to 1343488 tid 2 2024-03-20T08:12:17.704 INFO:tasks.rados.rados.0.smithi118.stdout:3135: writing smithi11881747-42 from 1343488 to 1441792 tid 3 2024-03-20T08:12:17.704 INFO:tasks.rados.rados.0.smithi118.stdout:3136: delete oid 46 current snap is 317 2024-03-20T08:12:17.704 INFO:tasks.rados.rados.0.smithi118.stdout:3132: expect (ObjNum 914 snap 309 seq_num 914) 2024-03-20T08:12:17.735 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 59824 (ObjNum 932 snap 317 seq_num 932) dirty exists 2024-03-20T08:12:17.735 INFO:tasks.rados.rados.0.smithi118.stdout:3132: done (5 left) 2024-03-20T08:12:17.735 INFO:tasks.rados.rados.0.smithi118.stdout:3134: done (4 left) 2024-03-20T08:12:17.735 INFO:tasks.rados.rados.0.smithi118.stdout:3136: done (3 left) 2024-03-20T08:12:17.735 INFO:tasks.rados.rados.0.smithi118.stdout:3137: rollback oid 30 current snap is 317 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 266 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45622 (ObjNum 685 snap 225 seq_num 685) dirty exists 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: finishing write tid 1 to smithi11881747-42 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3131: finishing write tid 1 to smithi11881747-14 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3131: finishing write tid 2 to smithi11881747-14 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: finishing write tid 2 to smithi11881747-42 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3131: finishing write tid 3 to smithi11881747-14 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3131: finishing write tid 4 to smithi11881747-14 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: finishing write tid 3 to smithi11881747-42 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: finishing write tid 4 to smithi11881747-42 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: finishing write tid 5 to smithi11881747-42 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: oid 42 updating version 0 to 45608 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: oid 42 version 45608 is already newer than 45607 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: oid 42 version 45608 is already newer than 45605 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:3135: oid 42 version 45608 is already newer than 45606 2024-03-20T08:12:17.736 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 45608 (ObjNum 939 snap 317 seq_num 939) dirty exists 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3135: left oid 42 (ObjNum 939 snap 317 seq_num 939) 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3131: finishing write tid 5 to smithi11881747-14 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3131: oid 14 updating version 0 to 34086 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3131: oid 14 updating version 34086 to 34088 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3131: oid 14 version 34088 is already newer than 34087 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3131: oid 14 updating version 34088 to 34089 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 34089 (ObjNum 938 snap 317 seq_num 938) dirty exists 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3131: left oid 14 (ObjNum 938 snap 317 seq_num 938) 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3131: done (3 left) 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3133: done (2 left) 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3135: done (1 left) 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 317 2024-03-20T08:12:17.737 INFO:tasks.rados.rados.0.smithi118.stdout:3138: seq_num 941 ranges {2318336=565248,2883584=475136,3358720=327680} 2024-03-20T08:12:17.750 INFO:tasks.rados.rados.0.smithi118.stdout:3138: writing smithi11881747-32 from 2318336 to 2883584 tid 1 2024-03-20T08:12:17.753 INFO:tasks.rados.rados.0.smithi118.stdout:3138: writing smithi11881747-32 from 2883584 to 3358720 tid 2 2024-03-20T08:12:17.755 INFO:tasks.rados.rados.0.smithi118.stdout:3138: writing smithi11881747-32 from 3358720 to 3686400 tid 3 2024-03-20T08:12:17.756 INFO:tasks.rados.rados.0.smithi118.stdout:3139: read oid 46 snap -1 2024-03-20T08:12:17.756 INFO:tasks.rados.rados.0.smithi118.stdout:3139: expect deleted 2024-03-20T08:12:17.756 INFO:tasks.rados.rados.0.smithi118.stdout:3140: snap_remove snap 307 2024-03-20T08:12:17.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:17.678+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:17.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:17.679+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:17.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:17.683+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:17.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:17.683+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:17.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:17.683+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:17.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:17.683+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:17.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: pgmap v728: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 5.1 MiB/s rd, 2.1 MiB/s wr, 12 op/s 2024-03-20T08:12:17.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:17.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: osdmap e698: 8 total, 8 up, 8 in 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:17.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: pgmap v728: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 5.1 MiB/s rd, 2.1 MiB/s wr, 12 op/s 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: osdmap e698: 8 total, 8 up, 8 in 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:17.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:17.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:17.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:17.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:17.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:17.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:17.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:17.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:17.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:17.927 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:17.694+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:17.927 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:17.679+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:17.927 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:17.679+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:18.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:17.678+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:18.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:17.697+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:18.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:17.698+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:18.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:17.692+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:18.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:17.693+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:18.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:17.693+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:18.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: pgmap v728: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.7 GiB used, 707 GiB / 715 GiB avail; 5.1 MiB/s rd, 2.1 MiB/s wr, 12 op/s 2024-03-20T08:12:18.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:18.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: osdmap e698: 8 total, 8 up, 8 in 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 159=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:18.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:18.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:18.147+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:18.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:18.147+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:18.684 INFO:tasks.rados.rados.0.smithi118.stdout:3140: done (3 left) 2024-03-20T08:12:18.684 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 11 current snap is 317 2024-03-20T08:12:18.684 INFO:tasks.rados.rados.0.smithi118.stdout:3137: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:12:18.684 INFO:tasks.rados.rados.0.smithi118.stdout:3141: seq_num 942 ranges {0=581632,581632=761856,1343488=40960} 2024-03-20T08:12:18.687 INFO:tasks.rados.rados.0.smithi118.stdout:3141: writing smithi11881747-11 from 0 to 581632 tid 1 2024-03-20T08:12:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:3141: writing smithi11881747-11 from 581632 to 1343488 tid 2 2024-03-20T08:12:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:3141: writing smithi11881747-11 from 1343488 to 1384448 tid 3 2024-03-20T08:12:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 317 2024-03-20T08:12:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:3137: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:12:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:3137: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:12:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 34033 (ObjNum 650 snap 213 seq_num 650) dirty exists 2024-03-20T08:12:18.692 INFO:tasks.rados.rados.0.smithi118.stdout:3142: seq_num 943 ranges {1531904=598016,2129920=688128,2818048=114688} 2024-03-20T08:12:18.703 INFO:tasks.rados.rados.0.smithi118.stdout:3142: writing smithi11881747-34 from 1531904 to 2129920 tid 1 2024-03-20T08:12:18.707 INFO:tasks.rados.rados.0.smithi118.stdout:3142: writing smithi11881747-34 from 2129920 to 2818048 tid 2 2024-03-20T08:12:18.708 INFO:tasks.rados.rados.0.smithi118.stdout:3142: writing smithi11881747-34 from 2818048 to 2932736 tid 3 2024-03-20T08:12:18.708 INFO:tasks.rados.rados.0.smithi118.stdout:3137: done (4 left) 2024-03-20T08:12:18.708 INFO:tasks.rados.rados.0.smithi118.stdout:3143: snap_remove snap 303 2024-03-20T08:12:19.115 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:18.683+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:19.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:18.694+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:19.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:18.694+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:19.116 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:18.694+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:19.117 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:18.922+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:19.117 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:18.923+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:19.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: osdmap e699: 8 total, 8 up, 8 in 2024-03-20T08:12:19.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:19.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:19.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:19.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:19.117 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:19.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:19.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:19.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:19.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:19.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:19.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:19.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:19.118 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:19.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:18.684+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:19.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:18.683+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:19.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:18.691+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:19.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:18.691+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:19.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:18.692+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:19.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: osdmap e699: 8 total, 8 up, 8 in 2024-03-20T08:12:19.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:19.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:19.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: osdmap e699: 8 total, 8 up, 8 in 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:19.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:19.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:19.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:19.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:19.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 15e=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:19.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:19.115+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:19.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:19.115+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:19.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:19.116+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:19.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.364+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:19.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.364+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:19.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.364+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:19.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.364+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:19.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.364+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:19.691 INFO:tasks.rados.rados.0.smithi118.stdout:3143: done (4 left) 2024-03-20T08:12:19.691 INFO:tasks.rados.rados.0.smithi118.stdout:3144: read oid 1 snap -1 2024-03-20T08:12:19.691 INFO:tasks.rados.rados.0.smithi118.stdout:3144: expect (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 317 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: finishing write tid 1 to smithi11881747-32 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: finishing write tid 2 to smithi11881747-32 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: finishing write tid 3 to smithi11881747-32 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: finishing write tid 4 to smithi11881747-32 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: finishing write tid 5 to smithi11881747-32 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: oid 32 updating version 0 to 34091 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: oid 32 updating version 34091 to 34092 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: oid 32 version 34092 is already newer than 34090 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: oid 32 updating version 34092 to 34093 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34093 (ObjNum 941 snap 317 seq_num 941) dirty exists 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3138: left oid 32 (ObjNum 941 snap 317 seq_num 941) 2024-03-20T08:12:19.692 INFO:tasks.rados.rados.0.smithi118.stdout:3141: finishing write tid 1 to smithi11881747-11 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: finishing write tid 2 to smithi11881747-11 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: finishing write tid 3 to smithi11881747-11 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: finishing write tid 4 to smithi11881747-11 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: finishing write tid 5 to smithi11881747-11 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: oid 11 updating version 0 to 45610 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: oid 11 version 45610 is already newer than 45609 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: oid 11 updating version 45610 to 45612 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: oid 11 version 45612 is already newer than 45611 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 45612 (ObjNum 942 snap 317 seq_num 942) dirty exists 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3141: left oid 11 (ObjNum 942 snap 317 seq_num 942) 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3142: finishing write tid 1 to smithi11881747-34 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3142: finishing write tid 2 to smithi11881747-34 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3142: finishing write tid 3 to smithi11881747-34 2024-03-20T08:12:19.693 INFO:tasks.rados.rados.0.smithi118.stdout:3142: finishing write tid 4 to smithi11881747-34 2024-03-20T08:12:19.694 INFO:tasks.rados.rados.0.smithi118.stdout:3142: finishing write tid 5 to smithi11881747-34 2024-03-20T08:12:19.694 INFO:tasks.rados.rados.0.smithi118.stdout:3142: oid 34 updating version 0 to 45603 2024-03-20T08:12:19.694 INFO:tasks.rados.rados.0.smithi118.stdout:3142: oid 34 version 45603 is already newer than 45602 2024-03-20T08:12:19.694 INFO:tasks.rados.rados.0.smithi118.stdout:3142: oid 34 version 45603 is already newer than 45601 2024-03-20T08:12:19.694 INFO:tasks.rados.rados.0.smithi118.stdout:3142: oid 34 updating version 45603 to 45604 2024-03-20T08:12:19.694 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45604 (ObjNum 943 snap 317 seq_num 943) dirty exists 2024-03-20T08:12:19.694 INFO:tasks.rados.rados.0.smithi118.stdout:3142: left oid 34 (ObjNum 943 snap 317 seq_num 943) 2024-03-20T08:12:19.694 INFO:tasks.rados.rados.0.smithi118.stdout:3145: seq_num 944 ranges {917504=614400,1531904=573440,2105344=229376} 2024-03-20T08:12:19.698 INFO:tasks.rados.rados.0.smithi118.stdout:3145: writing smithi11881747-4 from 917504 to 1531904 tid 1 2024-03-20T08:12:19.701 INFO:tasks.rados.rados.0.smithi118.stdout:3145: writing smithi11881747-4 from 1531904 to 2105344 tid 2 2024-03-20T08:12:19.703 INFO:tasks.rados.rados.0.smithi118.stdout:3145: writing smithi11881747-4 from 2105344 to 2334720 tid 3 2024-03-20T08:12:19.703 INFO:tasks.rados.rados.0.smithi118.stdout:3138: done (5 left) 2024-03-20T08:12:19.703 INFO:tasks.rados.rados.0.smithi118.stdout:3139: done (4 left) 2024-03-20T08:12:19.704 INFO:tasks.rados.rados.0.smithi118.stdout:3141: done (3 left) 2024-03-20T08:12:19.704 INFO:tasks.rados.rados.0.smithi118.stdout:3142: done (2 left) 2024-03-20T08:12:19.704 INFO:tasks.rados.rados.0.smithi118.stdout:3146: copy_from oid 42 from oid 44 current snap is 317 2024-03-20T08:12:19.704 INFO:tasks.rados.rados.0.smithi118.stdout:3147: read oid 22 snap -1 2024-03-20T08:12:19.704 INFO:tasks.rados.rados.0.smithi118.stdout:3147: expect (ObjNum 868 snap 301 seq_num 868) 2024-03-20T08:12:19.704 INFO:tasks.rados.rados.0.smithi118.stdout:3148: snap_create 2024-03-20T08:12:19.707 INFO:tasks.rados.rados.0.smithi118.stdout:3144: expect (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:12:19.795 INFO:tasks.rados.rados.0.smithi118.stdout:3145: finishing write tid 1 to smithi11881747-4 2024-03-20T08:12:19.795 INFO:tasks.rados.rados.0.smithi118.stdout:3145: finishing write tid 2 to smithi11881747-4 2024-03-20T08:12:19.795 INFO:tasks.rados.rados.0.smithi118.stdout:3147: expect (ObjNum 868 snap 301 seq_num 868) 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3145: finishing write tid 3 to smithi11881747-4 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3145: finishing write tid 4 to smithi11881747-4 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3145: finishing write tid 5 to smithi11881747-4 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3145: oid 4 updating version 0 to 34035 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3145: oid 4 updating version 34035 to 34037 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3145: oid 4 version 34037 is already newer than 34034 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3145: oid 4 version 34037 is already newer than 34036 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 34037 (ObjNum 944 snap 317 seq_num 944) dirty exists 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3145: left oid 4 (ObjNum 944 snap 317 seq_num 944) 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3146: finishing copy_from racing read to smithi11881747-42 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:3146: finishing copy_from to smithi11881747-42 2024-03-20T08:12:19.801 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65244 (ObjNum 854 snap 296 seq_num 854) dirty exists 2024-03-20T08:12:20.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:19.692+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:20.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:19.705+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:20.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:19.705+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:20.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:19.699+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:20.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:19.699+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:20.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:19.699+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:20.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:19.697+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:20.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:19.699+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:20.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: pgmap v731: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 13 MiB/s rd, 2.8 MiB/s wr, 19 op/s 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: osdmap e700: 8 total, 8 up, 8 in 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:20.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:20.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:19.707+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:20.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:19.707+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:20.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:19.692+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:20.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:19.693+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:20.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.690+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:20.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.690+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:20.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.690+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:20.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:19.691+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:20.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:19.740+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:20.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:20.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:20.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: pgmap v731: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 13 MiB/s rd, 2.8 MiB/s wr, 19 op/s 2024-03-20T08:12:20.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:20.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: osdmap e700: 8 total, 8 up, 8 in 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: pgmap v731: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 13 MiB/s rd, 2.8 MiB/s wr, 19 op/s 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:20.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:20.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: osdmap e700: 8 total, 8 up, 8 in 2024-03-20T08:12:20.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:20.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:20.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:20.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:20.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:20.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:20.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:20.697 INFO:tasks.rados.rados.0.smithi118.stdout:3144: done (4 left) 2024-03-20T08:12:20.698 INFO:tasks.rados.rados.0.smithi118.stdout:3145: done (3 left) 2024-03-20T08:12:20.698 INFO:tasks.rados.rados.0.smithi118.stdout:3146: done (2 left) 2024-03-20T08:12:20.698 INFO:tasks.rados.rados.0.smithi118.stdout:3147: done (1 left) 2024-03-20T08:12:20.698 INFO:tasks.rados.rados.0.smithi118.stdout:3148: done (0 left) 2024-03-20T08:12:20.698 INFO:tasks.rados.rados.0.smithi118.stdout:3149: copy_from oid 49 from oid 13 current snap is 318 2024-03-20T08:12:20.698 INFO:tasks.rados.rados.0.smithi118.stdout:3150: copy_from oid 46 from oid 17 current snap is 318 2024-03-20T08:12:20.698 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 318 2024-03-20T08:12:20.698 INFO:tasks.rados.rados.0.smithi118.stdout:3151: seq_num 945 ranges {0=630784,630784=458752,1089536=344064} 2024-03-20T08:12:20.701 INFO:tasks.rados.rados.0.smithi118.stdout:3151: writing smithi11881747-25 from 0 to 630784 tid 1 2024-03-20T08:12:20.704 INFO:tasks.rados.rados.0.smithi118.stdout:3151: writing smithi11881747-25 from 630784 to 1089536 tid 2 2024-03-20T08:12:20.706 INFO:tasks.rados.rados.0.smithi118.stdout:3151: writing smithi11881747-25 from 1089536 to 1433600 tid 3 2024-03-20T08:12:20.706 INFO:tasks.rados.rados.0.smithi118.stdout:3152: read oid 11 snap -1 2024-03-20T08:12:20.706 INFO:tasks.rados.rados.0.smithi118.stdout:3152: expect (ObjNum 942 snap 317 seq_num 942) 2024-03-20T08:12:20.707 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 318 2024-03-20T08:12:20.707 INFO:tasks.rados.rados.0.smithi118.stdout:3153: seq_num 946 ranges {7110656=647168,7757824=745472,8503296=57344} 2024-03-20T08:12:20.748 INFO:tasks.rados.rados.0.smithi118.stdout:3153: writing smithi11881747-12 from 7110656 to 7757824 tid 1 2024-03-20T08:12:20.753 INFO:tasks.rados.rados.0.smithi118.stdout:3153: writing smithi11881747-12 from 7757824 to 8503296 tid 2 2024-03-20T08:12:20.754 INFO:tasks.rados.rados.0.smithi118.stdout:3153: writing smithi11881747-12 from 8503296 to 8560640 tid 3 2024-03-20T08:12:20.754 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 318 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3150: finishing copy_from racing read to smithi11881747-46 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3149: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3150: finishing copy_from to smithi11881747-46 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 45590 (ObjNum 917 snap 310 seq_num 917) dirty exists 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3151: finishing write tid 1 to smithi11881747-25 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3149: finishing copy_from to smithi11881747-49 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 34095 (ObjNum 894 snap 308 seq_num 894) dirty exists 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3151: finishing write tid 2 to smithi11881747-25 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3151: finishing write tid 3 to smithi11881747-25 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3151: finishing write tid 4 to smithi11881747-25 2024-03-20T08:12:20.755 INFO:tasks.rados.rados.0.smithi118.stdout:3154: seq_num 947 ranges {3686400=663552,4349952=630784,4980736=172032} 2024-03-20T08:12:20.777 INFO:tasks.rados.rados.0.smithi118.stdout:3154: writing smithi11881747-32 from 3686400 to 4349952 tid 1 2024-03-20T08:12:20.781 INFO:tasks.rados.rados.0.smithi118.stdout:3154: writing smithi11881747-32 from 4349952 to 4980736 tid 2 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3154: writing smithi11881747-32 from 4980736 to 5152768 tid 3 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3149: done (5 left) 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3150: done (4 left) 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3155: rollback oid 9 current snap is 318 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 9 to 304 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3151: finishing write tid 5 to smithi11881747-25 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3151: oid 25 updating version 0 to 65245 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3151: oid 25 updating version 65245 to 65246 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3151: oid 25 updating version 65246 to 65248 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3151: oid 25 version 65248 is already newer than 65247 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 65248 (ObjNum 945 snap 318 seq_num 945) dirty exists 2024-03-20T08:12:20.783 INFO:tasks.rados.rados.0.smithi118.stdout:3151: left oid 25 (ObjNum 945 snap 318 seq_num 945) 2024-03-20T08:12:20.784 INFO:tasks.rados.rados.0.smithi118.stdout:3152: expect (ObjNum 942 snap 317 seq_num 942) 2024-03-20T08:12:20.829 INFO:tasks.rados.rados.0.smithi118.stdout:3153: finishing write tid 1 to smithi11881747-12 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: finishing write tid 2 to smithi11881747-12 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: finishing write tid 3 to smithi11881747-12 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: finishing write tid 4 to smithi11881747-12 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3151: done (4 left) 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3152: done (3 left) 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3156: delete oid 50 current snap is 318 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: finishing write tid 5 to smithi11881747-12 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: oid 12 updating version 0 to 44234 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: oid 12 updating version 44234 to 44237 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: oid 12 version 44237 is already newer than 44235 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: oid 12 version 44237 is already newer than 44236 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 44237 (ObjNum 946 snap 318 seq_num 946) dirty exists 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3153: left oid 12 (ObjNum 946 snap 318 seq_num 946) 2024-03-20T08:12:20.830 INFO:tasks.rados.rados.0.smithi118.stdout:3155: finishing rollback tid 1 to smithi11881747-9 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3155: finishing rollback tid 2 to smithi11881747-9 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 65250 (ObjNum 792 snap 274 seq_num 792) dirty exists 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3154: finishing write tid 1 to smithi11881747-32 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3154: finishing write tid 2 to smithi11881747-32 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3154: finishing write tid 3 to smithi11881747-32 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3154: finishing write tid 4 to smithi11881747-32 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3154: finishing write tid 5 to smithi11881747-32 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3154: oid 32 updating version 0 to 34094 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3154: oid 32 updating version 34094 to 34096 2024-03-20T08:12:20.831 INFO:tasks.rados.rados.0.smithi118.stdout:3154: oid 32 version 34096 is already newer than 34095 2024-03-20T08:12:20.832 INFO:tasks.rados.rados.0.smithi118.stdout:3154: oid 32 updating version 34096 to 34097 2024-03-20T08:12:20.832 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34097 (ObjNum 947 snap 318 seq_num 947) dirty exists 2024-03-20T08:12:20.832 INFO:tasks.rados.rados.0.smithi118.stdout:3154: left oid 32 (ObjNum 947 snap 318 seq_num 947) 2024-03-20T08:12:20.834 INFO:tasks.rados.rados.0.smithi118.stdout:3153: done (3 left) 2024-03-20T08:12:20.834 INFO:tasks.rados.rados.0.smithi118.stdout:3154: done (2 left) 2024-03-20T08:12:20.834 INFO:tasks.rados.rados.0.smithi118.stdout:3155: done (1 left) 2024-03-20T08:12:20.834 INFO:tasks.rados.rados.0.smithi118.stdout:3156: done (0 left) 2024-03-20T08:12:20.834 INFO:tasks.rados.rados.0.smithi118.stdout:3157: read oid 44 snap -1 2024-03-20T08:12:20.834 INFO:tasks.rados.rados.0.smithi118.stdout:3157: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:12:20.834 INFO:tasks.rados.rados.0.smithi118.stdout:3158: snap_remove snap 309 2024-03-20T08:12:21.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:20.695+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:21.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:20.701+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:21.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:20.701+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:21.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:20.701+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:21.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:20.754+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:21.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:20.754+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:21.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:20.707+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:21.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:20.707+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:21.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:20.707+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:21.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:21.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:21.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:21.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:21.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:21.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:21.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: osdmap e701: 8 total, 8 up, 8 in 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:21.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:21.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:21.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:21.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:21.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:21.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:20.704+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:21.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:20.695+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:21.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:20.701+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:21.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:20.701+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:21.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:20.701+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:21.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: osdmap e701: 8 total, 8 up, 8 in 2024-03-20T08:12:21.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:21.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:21.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:21.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:20.695+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:21.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:20.695+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:21.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:20.696+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:21.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:20.696+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:21.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:20.696+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:21.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: osdmap e701: 8 total, 8 up, 8 in 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:21.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:21.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:21.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:21.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:21.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:21.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:21.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:21.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:21.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:21.182 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:21.182 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:21.182 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:21.702 INFO:tasks.rados.rados.0.smithi118.stdout:3158: done (1 left) 2024-03-20T08:12:21.702 INFO:tasks.rados.rados.0.smithi118.stdout:3159: read oid 42 snap -1 2024-03-20T08:12:21.702 INFO:tasks.rados.rados.0.smithi118.stdout:3159: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:12:21.702 INFO:tasks.rados.rados.0.smithi118.stdout:3160: rollback oid 46 current snap is 318 2024-03-20T08:12:21.702 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 316 2024-03-20T08:12:21.702 INFO:tasks.rados.rados.0.smithi118.stdout:3157: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:12:22.092 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:21.701+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:22.092 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:21.705+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:22.093 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:21.709+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:22.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: pgmap v734: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 2.7 MiB/s wr, 32 op/s 2024-03-20T08:12:22.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:22.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:22.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: osdmap e702: 8 total, 8 up, 8 in 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:22.094 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:22.095 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:22.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:21.705+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:22.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:21.715+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:22.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:21.700+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:22.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:21.701+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:22.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:21.701+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:22.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:21.702+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:22.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:21.702+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:22.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:21.702+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:22.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:21.711+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:22.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: pgmap v734: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 2.7 MiB/s wr, 32 op/s 2024-03-20T08:12:22.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:22.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:22.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:22.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: osdmap e702: 8 total, 8 up, 8 in 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:22.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:22.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: pgmap v734: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 2.7 MiB/s wr, 32 op/s 2024-03-20T08:12:22.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:22.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:22.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:22.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:22.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:22.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:22.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: osdmap e702: 8 total, 8 up, 8 in 2024-03-20T08:12:22.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:22.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:22.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:22.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:22.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:22.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:22.180 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:22.180 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:22.180 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:22.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:22.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:22.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:22.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:12:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:12:22.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:22.092+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:22.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:22.092+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:22.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:22.092+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:22.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:22.092+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:22.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:22.092+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:22.488 INFO:tasks.rados.rados.0.smithi118.stdout:3160: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:12:22.488 INFO:tasks.rados.rados.0.smithi118.stdout:3160: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:12:22.488 INFO:tasks.rados.rados.0.smithi118.stdout:3160: finishing rollback tid 2 to smithi11881747-46 2024-03-20T08:12:22.488 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 45626 (ObjNum 797 snap 275 seq_num 797) dirty exists 2024-03-20T08:12:22.488 INFO:tasks.rados.rados.0.smithi118.stdout:3159: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:12:23.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: osdmap e703: 8 total, 8 up, 8 in 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:23.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:23.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:23.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:23.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: osdmap e703: 8 total, 8 up, 8 in 2024-03-20T08:12:23.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:23.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:23.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: osdmap e703: 8 total, 8 up, 8 in 2024-03-20T08:12:23.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:23.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:23.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:23.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:23.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 176=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:23.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:23.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:23.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:23.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:23.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:23.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:23.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:23.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:23.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:23.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:23.321 INFO:tasks.rados.rados.0.smithi118.stdout:3157: done (2 left) 2024-03-20T08:12:23.321 INFO:tasks.rados.rados.0.smithi118.stdout:3159: done (1 left) 2024-03-20T08:12:23.321 INFO:tasks.rados.rados.0.smithi118.stdout:3160: done (0 left) 2024-03-20T08:12:23.321 INFO:tasks.rados.rados.0.smithi118.stdout:3161: read oid 31 snap -1 2024-03-20T08:12:23.321 INFO:tasks.rados.rados.0.smithi118.stdout:3161: expect (ObjNum 931 snap 317 seq_num 931) 2024-03-20T08:12:23.321 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 318 2024-03-20T08:12:23.321 INFO:tasks.rados.rados.0.smithi118.stdout:3162: seq_num 949 ranges {4874240=704512,5578752=794624,6373376=8192} 2024-03-20T08:12:23.350 INFO:tasks.rados.rados.0.smithi118.stdout:3162: writing smithi11881747-46 from 4874240 to 5578752 tid 1 2024-03-20T08:12:23.355 INFO:tasks.rados.rados.0.smithi118.stdout:3162: writing smithi11881747-46 from 5578752 to 6373376 tid 2 2024-03-20T08:12:23.356 INFO:tasks.rados.rados.0.smithi118.stdout:3162: writing smithi11881747-46 from 6373376 to 6381568 tid 3 2024-03-20T08:12:23.356 INFO:tasks.rados.rados.0.smithi118.stdout:3163: read oid 19 snap -1 2024-03-20T08:12:23.356 INFO:tasks.rados.rados.0.smithi118.stdout:3163: expect (ObjNum 934 snap 317 seq_num 934) 2024-03-20T08:12:23.356 INFO:tasks.rados.rados.0.smithi118.stdout:3164: rollback oid 8 current snap is 318 2024-03-20T08:12:23.356 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 8 to 316 2024-03-20T08:12:23.356 INFO:tasks.rados.rados.0.smithi118.stdout:3161: expect (ObjNum 931 snap 317 seq_num 931) 2024-03-20T08:12:23.407 INFO:tasks.rados.rados.0.smithi118.stdout:3162: finishing write tid 1 to smithi11881747-46 2024-03-20T08:12:23.407 INFO:tasks.rados.rados.0.smithi118.stdout:3162: finishing write tid 2 to smithi11881747-46 2024-03-20T08:12:23.408 INFO:tasks.rados.rados.0.smithi118.stdout:3162: finishing write tid 3 to smithi11881747-46 2024-03-20T08:12:23.408 INFO:tasks.rados.rados.0.smithi118.stdout:3161: done (3 left) 2024-03-20T08:12:23.408 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 318 2024-03-20T08:12:23.408 INFO:tasks.rados.rados.0.smithi118.stdout:3162: finishing write tid 4 to smithi11881747-46 2024-03-20T08:12:23.408 INFO:tasks.rados.rados.0.smithi118.stdout:3163: expect (ObjNum 934 snap 317 seq_num 934) 2024-03-20T08:12:23.451 INFO:tasks.rados.rados.0.smithi118.stdout:3162: finishing write tid 5 to smithi11881747-46 2024-03-20T08:12:23.451 INFO:tasks.rados.rados.0.smithi118.stdout:3162: oid 46 updating version 0 to 45628 2024-03-20T08:12:23.451 INFO:tasks.rados.rados.0.smithi118.stdout:3162: oid 46 version 45628 is already newer than 45627 2024-03-20T08:12:23.451 INFO:tasks.rados.rados.0.smithi118.stdout:3162: oid 46 updating version 45628 to 45629 2024-03-20T08:12:23.452 INFO:tasks.rados.rados.0.smithi118.stdout:3162: oid 46 updating version 45629 to 45630 2024-03-20T08:12:23.452 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 45630 (ObjNum 949 snap 318 seq_num 949) dirty exists 2024-03-20T08:12:23.452 INFO:tasks.rados.rados.0.smithi118.stdout:3162: left oid 46 (ObjNum 949 snap 318 seq_num 949) 2024-03-20T08:12:23.452 INFO:tasks.rados.rados.0.smithi118.stdout:3164: finishing rollback tid 0 to smithi11881747-8 2024-03-20T08:12:23.452 INFO:tasks.rados.rados.0.smithi118.stdout:3164: finishing rollback tid 1 to smithi11881747-8 2024-03-20T08:12:23.452 INFO:tasks.rados.rados.0.smithi118.stdout:3164: finishing rollback tid 2 to smithi11881747-8 2024-03-20T08:12:23.452 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34100 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:12:23.452 INFO:tasks.rados.rados.0.smithi118.stdout:3165: seq_num 950 ranges {9314304=720896,10035200=679936,10715136=122880} 2024-03-20T08:12:23.505 INFO:tasks.rados.rados.0.smithi118.stdout:3165: writing smithi11881747-16 from 9314304 to 10035200 tid 1 2024-03-20T08:12:23.509 INFO:tasks.rados.rados.0.smithi118.stdout:3165: writing smithi11881747-16 from 10035200 to 10715136 tid 2 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3165: writing smithi11881747-16 from 10715136 to 10838016 tid 3 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3162: done (3 left) 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3163: done (2 left) 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3164: done (1 left) 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3166: read oid 50 snap -1 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3166: expect deleted 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3167: read oid 4 snap 213 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3167: expect (ObjNum 605 snap 197 seq_num 605) 2024-03-20T08:12:23.511 INFO:tasks.rados.rados.0.smithi118.stdout:3168: read oid 9 snap -1 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3168: expect (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3169: read oid 32 snap -1 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3169: expect (ObjNum 947 snap 318 seq_num 947) 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3170: rollback oid 28 current snap is 318 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 28 to 317 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3171: read oid 39 snap -1 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3171: expect deleted 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3172: read oid 8 snap -1 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3172: expect (ObjNum 646 snap 212 seq_num 646) 2024-03-20T08:12:23.512 INFO:tasks.rados.rados.0.smithi118.stdout:3173: snap_remove snap 314 2024-03-20T08:12:23.711 INFO:tasks.rados.rados.0.smithi118.stdout:3173: done (8 left) 2024-03-20T08:12:23.712 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 318 2024-03-20T08:12:23.712 INFO:tasks.rados.rados.0.smithi118.stdout:3174: seq_num 951 ranges {1630208=737280,2367488=614400,2981888=188416} 2024-03-20T08:12:23.724 INFO:tasks.rados.rados.0.smithi118.stdout:3174: writing smithi11881747-31 from 1630208 to 2367488 tid 1 2024-03-20T08:12:23.727 INFO:tasks.rados.rados.0.smithi118.stdout:3174: writing smithi11881747-31 from 2367488 to 2981888 tid 2 2024-03-20T08:12:23.729 INFO:tasks.rados.rados.0.smithi118.stdout:3174: writing smithi11881747-31 from 2981888 to 3170304 tid 3 2024-03-20T08:12:23.729 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 318 2024-03-20T08:12:23.729 INFO:tasks.rados.rados.0.smithi118.stdout:3175: seq_num 952 ranges {6381568=753664,7135232=499712,7634944=303104} 2024-03-20T08:12:23.768 INFO:tasks.rados.rados.0.smithi118.stdout:3175: writing smithi11881747-46 from 6381568 to 7135232 tid 1 2024-03-20T08:12:23.772 INFO:tasks.rados.rados.0.smithi118.stdout:3175: writing smithi11881747-46 from 7135232 to 7634944 tid 2 2024-03-20T08:12:23.774 INFO:tasks.rados.rados.0.smithi118.stdout:3175: writing smithi11881747-46 from 7634944 to 7938048 tid 3 2024-03-20T08:12:23.775 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 318 2024-03-20T08:12:23.775 INFO:tasks.rados.rados.0.smithi118.stdout:3176: seq_num 953 ranges {917504=770048,1687552=786432,2473984=16384} 2024-03-20T08:12:23.784 INFO:tasks.rados.rados.0.smithi118.stdout:3176: writing smithi11881747-41 from 917504 to 1687552 tid 1 2024-03-20T08:12:23.790 INFO:tasks.rados.rados.0.smithi118.stdout:3176: writing smithi11881747-41 from 1687552 to 2473984 tid 2 2024-03-20T08:12:23.790 INFO:tasks.rados.rados.0.smithi118.stdout:3176: writing smithi11881747-41 from 2473984 to 2490368 tid 3 2024-03-20T08:12:23.790 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 318 2024-03-20T08:12:23.790 INFO:tasks.rados.rados.0.smithi118.stdout:3177: seq_num 954 ranges {1433600=786432,2220032=671744,2891776=131072} 2024-03-20T08:12:23.802 INFO:tasks.rados.rados.0.smithi118.stdout:3177: writing smithi11881747-25 from 1433600 to 2220032 tid 1 2024-03-20T08:12:23.807 INFO:tasks.rados.rados.0.smithi118.stdout:3177: writing smithi11881747-25 from 2220032 to 2891776 tid 2 2024-03-20T08:12:23.808 INFO:tasks.rados.rados.0.smithi118.stdout:3177: writing smithi11881747-25 from 2891776 to 3022848 tid 3 2024-03-20T08:12:23.808 INFO:tasks.rados.rados.0.smithi118.stdout:3166: done (11 left) 2024-03-20T08:12:23.808 INFO:tasks.rados.rados.0.smithi118.stdout:3171: done (10 left) 2024-03-20T08:12:23.809 INFO:tasks.rados.rados.0.smithi118.stdout:3178: read oid 24 snap -1 2024-03-20T08:12:23.809 INFO:tasks.rados.rados.0.smithi118.stdout:3178: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:12:23.809 INFO:tasks.rados.rados.0.smithi118.stdout:3179: snap_remove snap 315 2024-03-20T08:12:23.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-mon[93121]: pgmap v737: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 34 op/s 2024-03-20T08:12:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:12:23] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:12:23.923 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:23.711+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:23.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-mon[95692]: pgmap v737: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 34 op/s 2024-03-20T08:12:23.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:23.792+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:23.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:23.792+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:23.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:23.793+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:23.924 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:23.711+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:23.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:23.788+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:23.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:23.790+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:23.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:23.791+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:23.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:23.791+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:23.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:23.791+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:24.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-mon[87080]: pgmap v737: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 4.1 MiB/s rd, 6.8 MiB/s wr, 34 op/s 2024-03-20T08:12:24.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:23.710+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:24.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:23.772+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:24.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:23.772+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:24.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:23.774+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:24.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:23.728+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:24.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:23.731+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:24.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:23.731+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:24.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:23.732+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:24.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:23.732+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:24.717 INFO:tasks.rados.rados.0.smithi118.stdout:3179: done (11 left) 2024-03-20T08:12:24.717 INFO:tasks.rados.rados.0.smithi118.stdout:3180: delete oid 45 current snap is 318 2024-03-20T08:12:24.717 INFO:tasks.rados.rados.0.smithi118.stdout:3165: finishing write tid 1 to smithi11881747-16 2024-03-20T08:12:24.717 INFO:tasks.rados.rados.0.smithi118.stdout:3165: finishing write tid 2 to smithi11881747-16 2024-03-20T08:12:24.717 INFO:tasks.rados.rados.0.smithi118.stdout:3167: expect (ObjNum 605 snap 197 seq_num 605) 2024-03-20T08:12:24.736 INFO:tasks.rados.rados.0.smithi118.stdout:3165: finishing write tid 3 to smithi11881747-16 2024-03-20T08:12:24.736 INFO:tasks.rados.rados.0.smithi118.stdout:3165: finishing write tid 4 to smithi11881747-16 2024-03-20T08:12:24.736 INFO:tasks.rados.rados.0.smithi118.stdout:3168: expect (ObjNum 792 snap 274 seq_num 792) 2024-03-20T08:12:24.815 INFO:tasks.rados.rados.0.smithi118.stdout:3170: finishing rollback tid 0 to smithi11881747-28 2024-03-20T08:12:24.815 INFO:tasks.rados.rados.0.smithi118.stdout:3165: finishing write tid 5 to smithi11881747-16 2024-03-20T08:12:24.815 INFO:tasks.rados.rados.0.smithi118.stdout:3165: oid 16 updating version 0 to 45613 2024-03-20T08:12:24.815 INFO:tasks.rados.rados.0.smithi118.stdout:3165: oid 16 version 45613 is already newer than 45612 2024-03-20T08:12:24.815 INFO:tasks.rados.rados.0.smithi118.stdout:3165: oid 16 updating version 45613 to 45615 2024-03-20T08:12:24.815 INFO:tasks.rados.rados.0.smithi118.stdout:3165: oid 16 version 45615 is already newer than 45614 2024-03-20T08:12:24.816 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 45615 (ObjNum 950 snap 318 seq_num 950) dirty exists 2024-03-20T08:12:24.816 INFO:tasks.rados.rados.0.smithi118.stdout:3165: left oid 16 (ObjNum 950 snap 318 seq_num 950) 2024-03-20T08:12:24.816 INFO:tasks.rados.rados.0.smithi118.stdout:3170: finishing rollback tid 1 to smithi11881747-28 2024-03-20T08:12:24.816 INFO:tasks.rados.rados.0.smithi118.stdout:3170: finishing rollback tid 2 to smithi11881747-28 2024-03-20T08:12:24.816 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 45618 (ObjNum 900 snap 309 seq_num 900) dirty exists 2024-03-20T08:12:24.816 INFO:tasks.rados.rados.0.smithi118.stdout:3169: expect (ObjNum 947 snap 318 seq_num 947) 2024-03-20T08:12:24.834 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -e sha1=e7e80a6ac9849f0f23432d0d9832f096730308ae -- bash -c 'ceph orch ps' 2024-03-20T08:12:25.046 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:24.715+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: osdmap e704: 8 total, 8 up, 8 in 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:25.047 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:25.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:25.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:25.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:25.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:25.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:25.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:25.048 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:24 smithi196 ceph-mon[87080]: from='client.34491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:12:25.051 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:12:25.081 INFO:tasks.rados.rados.0.smithi118.stdout:3172: expect (ObjNum 646 snap 212 seq_num 646) 2024-03-20T08:12:25.116 INFO:tasks.rados.rados.0.smithi118.stdout:3174: finishing write tid 1 to smithi11881747-31 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: finishing write tid 2 to smithi11881747-31 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: finishing write tid 3 to smithi11881747-31 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: finishing write tid 4 to smithi11881747-31 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: finishing write tid 5 to smithi11881747-31 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: oid 31 updating version 0 to 44247 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: oid 31 updating version 44247 to 44248 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: oid 31 updating version 44248 to 44249 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: oid 31 version 44249 is already newer than 44246 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 44249 (ObjNum 951 snap 318 seq_num 951) dirty exists 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3174: left oid 31 (ObjNum 951 snap 318 seq_num 951) 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3175: finishing write tid 1 to smithi11881747-46 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3175: finishing write tid 2 to smithi11881747-46 2024-03-20T08:12:25.117 INFO:tasks.rados.rados.0.smithi118.stdout:3175: finishing write tid 3 to smithi11881747-46 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3175: finishing write tid 4 to smithi11881747-46 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3175: finishing write tid 5 to smithi11881747-46 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3175: oid 46 updating version 0 to 45633 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3175: oid 46 version 45633 is already newer than 45632 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3175: oid 46 updating version 45633 to 45634 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3175: oid 46 version 45634 is already newer than 45631 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 45634 (ObjNum 952 snap 318 seq_num 952) dirty exists 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3175: left oid 46 (ObjNum 952 snap 318 seq_num 952) 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3176: finishing write tid 1 to smithi11881747-41 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3176: finishing write tid 2 to smithi11881747-41 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3176: finishing write tid 3 to smithi11881747-41 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3176: finishing write tid 4 to smithi11881747-41 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3176: finishing write tid 5 to smithi11881747-41 2024-03-20T08:12:25.118 INFO:tasks.rados.rados.0.smithi118.stdout:3176: oid 41 updating version 0 to 45622 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3176: oid 41 version 45622 is already newer than 45621 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3176: oid 41 version 45622 is already newer than 45620 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3176: oid 41 version 45622 is already newer than 45619 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 45622 (ObjNum 953 snap 318 seq_num 953) dirty exists 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3176: left oid 41 (ObjNum 953 snap 318 seq_num 953) 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3177: finishing write tid 1 to smithi11881747-25 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3177: finishing write tid 2 to smithi11881747-25 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3177: finishing write tid 3 to smithi11881747-25 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3165: done (11 left) 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3167: done (10 left) 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3168: done (9 left) 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3169: done (8 left) 2024-03-20T08:12:25.119 INFO:tasks.rados.rados.0.smithi118.stdout:3170: done (7 left) 2024-03-20T08:12:25.120 INFO:tasks.rados.rados.0.smithi118.stdout:3172: done (6 left) 2024-03-20T08:12:25.120 INFO:tasks.rados.rados.0.smithi118.stdout:3174: done (5 left) 2024-03-20T08:12:25.120 INFO:tasks.rados.rados.0.smithi118.stdout:3175: done (4 left) 2024-03-20T08:12:25.120 INFO:tasks.rados.rados.0.smithi118.stdout:3176: done (3 left) 2024-03-20T08:12:25.120 INFO:tasks.rados.rados.0.smithi118.stdout:3180: done (2 left) 2024-03-20T08:12:25.120 INFO:tasks.rados.rados.0.smithi118.stdout:3181: delete oid 3 current snap is 318 2024-03-20T08:12:25.120 INFO:tasks.rados.rados.0.smithi118.stdout:3177: finishing write tid 4 to smithi11881747-25 2024-03-20T08:12:25.120 INFO:tasks.rados.rados.0.smithi118.stdout:3178: expect (ObjNum 760 snap 261 seq_num 760) 2024-03-20T08:12:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:3177: finishing write tid 5 to smithi11881747-25 2024-03-20T08:12:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:3177: oid 25 updating version 0 to 65254 2024-03-20T08:12:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:3177: oid 25 version 65254 is already newer than 65253 2024-03-20T08:12:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:3177: oid 25 version 65254 is already newer than 65251 2024-03-20T08:12:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:3177: oid 25 version 65254 is already newer than 65252 2024-03-20T08:12:25.137 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 65254 (ObjNum 954 snap 318 seq_num 954) dirty exists 2024-03-20T08:12:25.138 INFO:tasks.rados.rados.0.smithi118.stdout:3177: left oid 25 (ObjNum 954 snap 318 seq_num 954) 2024-03-20T08:12:25.138 INFO:tasks.rados.rados.0.smithi118.stdout:3177: done (2 left) 2024-03-20T08:12:25.138 INFO:tasks.rados.rados.0.smithi118.stdout:3178: done (1 left) 2024-03-20T08:12:25.138 INFO:tasks.rados.rados.0.smithi118.stdout:3181: done (0 left) 2024-03-20T08:12:25.138 INFO:tasks.rados.rados.0.smithi118.stdout:3182: copy_from oid 7 from oid 33 current snap is 318 2024-03-20T08:12:25.138 INFO:tasks.rados.rados.0.smithi118.stdout:3183: setattr oid 8 current snap is 318 2024-03-20T08:12:25.138 INFO:tasks.rados.rados.0.smithi118.stdout:3184: delete oid 48 current snap is 318 2024-03-20T08:12:25.144 INFO:tasks.rados.rados.0.smithi118.stdout:3184: done (2 left) 2024-03-20T08:12:25.144 INFO:tasks.rados.rados.0.smithi118.stdout:3185: snap_create 2024-03-20T08:12:25.170 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34101 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:12:25.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:24.725+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:25.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:24.725+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:25.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:24.714+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:25.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:24.715+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:25.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:24.715+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:25.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:24.716+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:25.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:24.724+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:25.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:24.717+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:25.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:24.717+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: osdmap e704: 8 total, 8 up, 8 in 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:25.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[95692]: from='client.34491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:12:25.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: osdmap e704: 8 total, 8 up, 8 in 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:25.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:25.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:24 smithi118 ceph-mon[93121]: from='client.34491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:12:25.181 INFO:tasks.rados.rados.0.smithi118.stdout:3182: finishing copy_from racing read to smithi11881747-7 2024-03-20T08:12:25.182 INFO:tasks.rados.rados.0.smithi118.stdout:3182: finishing copy_from to smithi11881747-7 2024-03-20T08:12:25.182 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 34034 (ObjNum 913 snap 309 seq_num 913) dirty exists 2024-03-20T08:12:25.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:25.170+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:25.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:25.171+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:25.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:25.170+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:25.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:25.170+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:25.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:25.171+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:25.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:25.046+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:25.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:25.046+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:25.721 INFO:tasks.rados.rados.0.smithi118.stdout:3182: done (2 left) 2024-03-20T08:12:25.722 INFO:tasks.rados.rados.0.smithi118.stdout:3183: done (1 left) 2024-03-20T08:12:25.722 INFO:tasks.rados.rados.0.smithi118.stdout:3185: done (0 left) 2024-03-20T08:12:25.722 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 319 2024-03-20T08:12:25.722 INFO:tasks.rados.rados.0.smithi118.stdout:3186: seq_num 958 ranges {0=450560,450560=655360,1105920=548864} 2024-03-20T08:12:25.724 INFO:tasks.rados.rados.0.smithi118.stdout:3186: writing smithi11881747-45 from 0 to 450560 tid 1 2024-03-20T08:12:25.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: writing smithi11881747-45 from 450560 to 1105920 tid 2 2024-03-20T08:12:25.732 INFO:tasks.rados.rados.0.smithi118.stdout:3186: writing smithi11881747-45 from 1105920 to 1654784 tid 3 2024-03-20T08:12:25.732 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 319 2024-03-20T08:12:25.732 INFO:tasks.rados.rados.0.smithi118.stdout:3187: seq_num 959 ranges {2326528=466944,2793472=540672,3334144=557056,3891200=106496} 2024-03-20T08:12:25.748 INFO:tasks.rados.rados.0.smithi118.stdout:3187: writing smithi11881747-1 from 2326528 to 2793472 tid 1 2024-03-20T08:12:25.752 INFO:tasks.rados.rados.0.smithi118.stdout:3187: writing smithi11881747-1 from 2793472 to 3334144 tid 2 2024-03-20T08:12:25.756 INFO:tasks.rados.rados.0.smithi118.stdout:3187: writing smithi11881747-1 from 3334144 to 3891200 tid 3 2024-03-20T08:12:25.756 INFO:tasks.rados.rados.0.smithi118.stdout:3187: writing smithi11881747-1 from 3891200 to 3997696 tid 4 2024-03-20T08:12:25.756 INFO:tasks.rados.rados.0.smithi118.stdout:3188: snap_remove snap 310 2024-03-20T08:12:25.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:25.731+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:25.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:25.720+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:25.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:25.720+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:25.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:25.721+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:25.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:25.722+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:25.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:25.727+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:25.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:25.727+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:25.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:25.728+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:25.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:25.728+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:25.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:25.732+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:25.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: pgmap v739: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 26 op/s 2024-03-20T08:12:25.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:25.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:25.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:25.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:25.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:25.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: osdmap e705: 8 total, 8 up, 8 in 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: pgmap v739: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 26 op/s 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:25.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: osdmap e705: 8 total, 8 up, 8 in 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:26.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:25.722+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:26.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:25.784+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:26.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:25.784+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:26.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:25.784+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:26.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:25.753+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:26.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:25.755+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:26.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:25.756+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:26.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:25.756+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:26.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:25.756+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: pgmap v739: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 26 op/s 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 166=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: osdmap e705: 8 total, 8 up, 8 in 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:26.727 INFO:tasks.rados.rados.0.smithi118.stdout:3188: done (2 left) 2024-03-20T08:12:26.727 INFO:tasks.rados.rados.0.smithi118.stdout:3189: read oid 9 snap 294 2024-03-20T08:12:26.727 INFO:tasks.rados.rados.0.smithi118.stdout:3189: expect (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:12:26.727 INFO:tasks.rados.rados.0.smithi118.stdout:3190: rmattr oid 31 current snap is 319 2024-03-20T08:12:26.727 INFO:tasks.rados.rados.0.smithi118.stdout:3190: done (3 left) 2024-03-20T08:12:26.727 INFO:tasks.rados.rados.0.smithi118.stdout:3191: read oid 22 snap 318 2024-03-20T08:12:26.727 INFO:tasks.rados.rados.0.smithi118.stdout:3191: expect (ObjNum 868 snap 301 seq_num 868) 2024-03-20T08:12:26.727 INFO:tasks.rados.rados.0.smithi118.stdout:3192: delete oid 10 current snap is 319 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: finishing write tid 1 to smithi11881747-45 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: finishing write tid 2 to smithi11881747-45 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: finishing write tid 3 to smithi11881747-45 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: finishing write tid 4 to smithi11881747-45 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: finishing write tid 5 to smithi11881747-45 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: oid 45 updating version 0 to 34081 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: oid 45 version 34081 is already newer than 34078 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: oid 45 version 34081 is already newer than 34079 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: oid 45 version 34081 is already newer than 34080 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 34081 (ObjNum 958 snap 319 seq_num 958) dirty exists 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3186: left oid 45 (ObjNum 958 snap 319 seq_num 958) 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3187: finishing write tid 1 to smithi11881747-1 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3187: finishing write tid 2 to smithi11881747-1 2024-03-20T08:12:26.728 INFO:tasks.rados.rados.0.smithi118.stdout:3187: finishing write tid 3 to smithi11881747-1 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: finishing write tid 4 to smithi11881747-1 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: finishing write tid 5 to smithi11881747-1 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: finishing write tid 6 to smithi11881747-1 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: oid 1 updating version 0 to 44254 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: oid 1 version 44254 is already newer than 44253 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: oid 1 version 44254 is already newer than 44252 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: oid 1 version 44254 is already newer than 44251 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: oid 1 version 44254 is already newer than 44250 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 44254 (ObjNum 959 snap 319 seq_num 959) dirty exists 2024-03-20T08:12:26.729 INFO:tasks.rados.rados.0.smithi118.stdout:3187: left oid 1 (ObjNum 959 snap 319 seq_num 959) 2024-03-20T08:12:26.732 INFO:tasks.rados.rados.0.smithi118.stdout:3191: expect (ObjNum 868 snap 301 seq_num 868) 2024-03-20T08:12:26.738 INFO:tasks.rados.rados.0.smithi118.stdout:3189: expect (ObjNum 809 snap 279 seq_num 809) 2024-03-20T08:12:26.843 INFO:tasks.rados.rados.0.smithi118.stdout:3186: done (4 left) 2024-03-20T08:12:26.844 INFO:tasks.rados.rados.0.smithi118.stdout:3187: done (3 left) 2024-03-20T08:12:26.844 INFO:tasks.rados.rados.0.smithi118.stdout:3189: done (2 left) 2024-03-20T08:12:26.844 INFO:tasks.rados.rados.0.smithi118.stdout:3191: done (1 left) 2024-03-20T08:12:26.844 INFO:tasks.rados.rados.0.smithi118.stdout:3192: done (0 left) 2024-03-20T08:12:26.844 INFO:tasks.rados.rados.0.smithi118.stdout:3193: copy_from oid 29 from oid 23 current snap is 319 2024-03-20T08:12:26.844 INFO:tasks.rados.rados.0.smithi118.stdout:3194: delete oid 4 current snap is 319 2024-03-20T08:12:26.845 INFO:tasks.rados.rados.0.smithi118.stdout:3194: done (1 left) 2024-03-20T08:12:26.845 INFO:tasks.rados.rados.0.smithi118.stdout:3195: snap_create 2024-03-20T08:12:26.849 INFO:tasks.rados.rados.0.smithi118.stdout:3193: finishing copy_from racing read to smithi11881747-29 2024-03-20T08:12:26.850 INFO:tasks.rados.rados.0.smithi118.stdout:3193: finishing copy_from to smithi11881747-29 2024-03-20T08:12:26.850 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 34033 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:12:26.916 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:26.726+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:26.917 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:26.726+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:26.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:26.731+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:26.917 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:26.731+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:26.918 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:26.731+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:26.918 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:26.732+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:26.918 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:26.731+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:26.919 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:26.732+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:26.919 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:26.732+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:26.920 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:26.920 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:26.920 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:26.920 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:26.921 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: osdmap e706: 8 total, 8 up, 8 in 2024-03-20T08:12:26.921 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:26.921 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:26.921 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:26.921 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:26.921 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:26.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:26.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:26.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:26.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:26.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:26.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:26.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: osdmap e706: 8 total, 8 up, 8 in 2024-03-20T08:12:26.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:26.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:26.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:26.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:26.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:26.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:26.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:26.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:26.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:26.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:26.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:26.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:26.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:26.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:26.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:26.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:26.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:26.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:26.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:27.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:26.727+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:27.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:26.731+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:27.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:26.734+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:27.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:26.731+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:27.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:26.732+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:27.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:26.726+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:27.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:26.726+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:27.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:26.726+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:27.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:27.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:27.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:27.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:27.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: osdmap e706: 8 total, 8 up, 8 in 2024-03-20T08:12:27.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:27.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:27.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:27.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:27.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:27.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:27.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:27.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:27.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:27.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:27.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:27.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:27.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:27.157 INFO:teuthology.orchestra.run.smithi118.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-20T08:12:27.157 INFO:teuthology.orchestra.run.smithi118.stdout:alertmanager.a smithi118 *:9093,9094 running (53s) 49s ago 13m 15.6M - 0.25.0 c8568f914cd2 68a6e855b9aa 2024-03-20T08:12:27.157 INFO:teuthology.orchestra.run.smithi118.stdout:grafana.a smithi196 *:3000 running (25s) 21s ago 12m 54.4M - 9.4.12 a0ea5e3c46d9 b2e2f67e1516 2024-03-20T08:12:27.157 INFO:teuthology.orchestra.run.smithi118.stdout:mds.a smithi118 running (2m) 49s ago 14m 14.5M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 b9e9c53270c7 2024-03-20T08:12:27.158 INFO:teuthology.orchestra.run.smithi118.stdout:mds.b smithi196 running (2m) 21s ago 14m 14.3M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 237cfe0be476 2024-03-20T08:12:27.158 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.x smithi196 *:8443,9283,8765 running (8m) 21s ago 17m 453M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 043bae454c20 2024-03-20T08:12:27.158 INFO:teuthology.orchestra.run.smithi118.stdout:mgr.y smithi118 *:8443,9283,8765 running (9m) 49s ago 19m 517M - 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4dac06b91452 2024-03-20T08:12:27.158 INFO:teuthology.orchestra.run.smithi118.stdout:mon.a smithi118 running (8m) 49s ago 19m 71.8M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 51c68c4d30af 2024-03-20T08:12:27.158 INFO:teuthology.orchestra.run.smithi118.stdout:mon.b smithi196 running (7m) 21s ago 18m 59.6M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 bbde40d464ea 2024-03-20T08:12:27.158 INFO:teuthology.orchestra.run.smithi118.stdout:mon.c smithi118 running (8m) 49s ago 17m 61.8M 2048M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 1ea7cd20eda3 2024-03-20T08:12:27.159 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.a smithi118 *:9100 running (103s) 49s ago 12m 8070k - 1.5.0 0da6a335fe13 768c22c1ab59 2024-03-20T08:12:27.159 INFO:teuthology.orchestra.run.smithi118.stdout:node-exporter.b smithi196 *:9100 running (98s) 21s ago 12m 9021k - 1.5.0 0da6a335fe13 d2c82f232b48 2024-03-20T08:12:27.159 INFO:teuthology.orchestra.run.smithi118.stdout:osd.0 smithi118 running (7m) 49s ago 17m 519M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 56e63ece2c91 2024-03-20T08:12:27.159 INFO:teuthology.orchestra.run.smithi118.stdout:osd.1 smithi118 running (6m) 49s ago 16m 535M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 3eae2fe0915a 2024-03-20T08:12:27.159 INFO:teuthology.orchestra.run.smithi118.stdout:osd.2 smithi118 running (5m) 49s ago 16m 380M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 21c627633721 2024-03-20T08:12:27.159 INFO:teuthology.orchestra.run.smithi118.stdout:osd.3 smithi118 running (5m) 49s ago 16m 162M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 c7c15dc3876f 2024-03-20T08:12:27.159 INFO:teuthology.orchestra.run.smithi118.stdout:osd.4 smithi196 running (4m) 21s ago 15m 424M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 4b7be73bd9c3 2024-03-20T08:12:27.160 INFO:teuthology.orchestra.run.smithi118.stdout:osd.5 smithi196 running (4m) 21s ago 15m 445M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 72b569d10541 2024-03-20T08:12:27.160 INFO:teuthology.orchestra.run.smithi118.stdout:osd.6 smithi196 running (3m) 21s ago 14m 220M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 dca658056148 2024-03-20T08:12:27.160 INFO:teuthology.orchestra.run.smithi118.stdout:osd.7 smithi196 running (2m) 21s ago 14m 129M 2480M 19.0.0-2153-ge7e80a6a 9ceba81fb2c8 52392403da44 2024-03-20T08:12:27.160 INFO:teuthology.orchestra.run.smithi118.stdout:prometheus.a smithi196 *:9095 running (76s) 21s ago 13m 44.9M - 2.43.0 a07b618ecd1d bf82125604d2 2024-03-20T08:12:27.731 INFO:tasks.rados.rados.0.smithi118.stdout:3193: done (1 left) 2024-03-20T08:12:27.731 INFO:tasks.rados.rados.0.smithi118.stdout:3195: done (0 left) 2024-03-20T08:12:27.731 INFO:tasks.rados.rados.0.smithi118.stdout:3196: copy_from oid 14 from oid 19 current snap is 320 2024-03-20T08:12:27.731 INFO:tasks.rados.rados.0.smithi118.stdout:3197: rmattr oid 49 current snap is 320 2024-03-20T08:12:27.731 INFO:tasks.rados.rados.0.smithi118.stdout:3198: delete oid 31 current snap is 320 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: pgmap v742: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 18 MiB/s rd, 2.9 MiB/s wr, 17 op/s 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: osdmap e707: 8 total, 8 up, 8 in 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:27.732 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: pgmap v742: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 18 MiB/s rd, 2.9 MiB/s wr, 17 op/s 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: osdmap e707: 8 total, 8 up, 8 in 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:27.733 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:27.734 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[93121]: from='client.34497 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:12:27.735 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:27.735 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:27.735 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:27.735 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-mon[95692]: from='client.34497 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:12:27.737 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 59825 (ObjNum 894 snap 308 seq_num 894) dirty exists 2024-03-20T08:12:27.742 INFO:tasks.rados.rados.0.smithi118.stdout:3197: done (2 left) 2024-03-20T08:12:27.742 INFO:tasks.rados.rados.0.smithi118.stdout:3198: done (1 left) 2024-03-20T08:12:27.742 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 320 2024-03-20T08:12:27.742 INFO:tasks.rados.rados.0.smithi118.stdout:3199: seq_num 963 ranges {0=532480,532480=475136,1007616=532480,1540096=196608} 2024-03-20T08:12:27.744 INFO:tasks.rados.rados.0.smithi118.stdout:3199: writing smithi11881747-39 from 0 to 532480 tid 1 2024-03-20T08:12:27.748 INFO:tasks.rados.rados.0.smithi118.stdout:3199: writing smithi11881747-39 from 532480 to 1007616 tid 2 2024-03-20T08:12:27.751 INFO:tasks.rados.rados.0.smithi118.stdout:3199: writing smithi11881747-39 from 1007616 to 1540096 tid 3 2024-03-20T08:12:27.753 INFO:tasks.rados.rados.0.smithi118.stdout:3199: writing smithi11881747-39 from 1540096 to 1736704 tid 4 2024-03-20T08:12:27.753 INFO:tasks.rados.rados.0.smithi118.stdout:3200: delete oid 5 current snap is 320 2024-03-20T08:12:27.753 INFO:tasks.rados.rados.0.smithi118.stdout:3199: finishing write tid 1 to smithi11881747-39 2024-03-20T08:12:27.753 INFO:tasks.rados.rados.0.smithi118.stdout:3196: finishing copy_from racing read to smithi11881747-14 2024-03-20T08:12:27.753 INFO:tasks.rados.rados.0.smithi118.stdout:3196: finishing copy_from to smithi11881747-14 2024-03-20T08:12:27.753 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 7109 (ObjNum 934 snap 317 seq_num 934) dirty exists 2024-03-20T08:12:27.753 INFO:tasks.rados.rados.0.smithi118.stdout:3199: finishing write tid 2 to smithi11881747-39 2024-03-20T08:12:27.756 INFO:tasks.rados.rados.0.smithi118.stdout:3196: done (2 left) 2024-03-20T08:12:27.756 INFO:tasks.rados.rados.0.smithi118.stdout:3200: done (1 left) 2024-03-20T08:12:27.756 INFO:tasks.rados.rados.0.smithi118.stdout:3201: rmattr oid 47 current snap is 320 2024-03-20T08:12:27.756 INFO:tasks.rados.rados.0.smithi118.stdout:3202: read oid 5 snap 226 2024-03-20T08:12:27.756 INFO:tasks.rados.rados.0.smithi118.stdout:3202: expect (ObjNum 628 snap 204 seq_num 628) 2024-03-20T08:12:27.756 INFO:tasks.rados.rados.0.smithi118.stdout:3203: delete oid 14 current snap is 320 2024-03-20T08:12:27.757 INFO:tasks.rados.rados.0.smithi118.stdout:3199: finishing write tid 3 to smithi11881747-39 2024-03-20T08:12:27.757 INFO:tasks.rados.rados.0.smithi118.stdout:3199: finishing write tid 4 to smithi11881747-39 2024-03-20T08:12:27.761 INFO:tasks.rados.rados.0.smithi118.stdout:3199: finishing write tid 5 to smithi11881747-39 2024-03-20T08:12:27.761 INFO:tasks.rados.rados.0.smithi118.stdout:3203: done (3 left) 2024-03-20T08:12:27.761 INFO:tasks.rados.rados.0.smithi118.stdout:3204: copy_from oid 41 from oid 17 current snap is 320 2024-03-20T08:12:27.762 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 320 2024-03-20T08:12:27.762 INFO:tasks.rados.rados.0.smithi118.stdout:3205: seq_num 966 ranges {2326528=581632,2908160=581632,3489792=622592} 2024-03-20T08:12:27.778 INFO:tasks.rados.rados.0.smithi118.stdout:3205: writing smithi11881747-9 from 2326528 to 2908160 tid 1 2024-03-20T08:12:27.781 INFO:tasks.rados.rados.0.smithi118.stdout:3205: writing smithi11881747-9 from 2908160 to 3489792 tid 2 2024-03-20T08:12:27.785 INFO:tasks.rados.rados.0.smithi118.stdout:3205: writing smithi11881747-9 from 3489792 to 4112384 tid 3 2024-03-20T08:12:27.785 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 320 2024-03-20T08:12:27.785 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 45623 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:12:27.785 INFO:tasks.rados.rados.0.smithi118.stdout:3206: seq_num 967 ranges {139264=598016,737280=466944,1204224=516096,1720320=221184} 2024-03-20T08:12:27.789 INFO:tasks.rados.rados.0.smithi118.stdout:3206: writing smithi11881747-22 from 139264 to 737280 tid 1 2024-03-20T08:12:27.792 INFO:tasks.rados.rados.0.smithi118.stdout:3206: writing smithi11881747-22 from 737280 to 1204224 tid 2 2024-03-20T08:12:27.795 INFO:tasks.rados.rados.0.smithi118.stdout:3206: writing smithi11881747-22 from 1204224 to 1720320 tid 3 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3206: writing smithi11881747-22 from 1720320 to 1941504 tid 4 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3201: done (5 left) 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3207: rollback oid 25 current snap is 320 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 25 to 316 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3199: finishing write tid 6 to smithi11881747-39 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3199: oid 39 updating version 0 to 59828 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3199: oid 39 updating version 59828 to 59830 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3199: oid 39 version 59830 is already newer than 59829 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3199: oid 39 version 59830 is already newer than 59826 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3199: oid 39 version 59830 is already newer than 59827 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 59830 (ObjNum 963 snap 320 seq_num 963) dirty exists 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3199: left oid 39 (ObjNum 963 snap 320 seq_num 963) 2024-03-20T08:12:27.797 INFO:tasks.rados.rados.0.smithi118.stdout:3202: expect (ObjNum 628 snap 204 seq_num 628) 2024-03-20T08:12:27.799 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -e sha1=e7e80a6ac9849f0f23432d0d9832f096730308ae -- bash -c 'ceph versions' 2024-03-20T08:12:27.841 INFO:tasks.rados.rados.0.smithi118.stdout:3204: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:12:27.841 INFO:tasks.rados.rados.0.smithi118.stdout:3204: finishing copy_from to smithi11881747-41 2024-03-20T08:12:27.841 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 45590 (ObjNum 917 snap 310 seq_num 917) dirty exists 2024-03-20T08:12:27.841 INFO:tasks.rados.rados.0.smithi118.stdout:3205: finishing write tid 1 to smithi11881747-9 2024-03-20T08:12:27.841 INFO:tasks.rados.rados.0.smithi118.stdout:3205: finishing write tid 2 to smithi11881747-9 2024-03-20T08:12:27.841 INFO:tasks.rados.rados.0.smithi118.stdout:3199: done (5 left) 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3202: done (4 left) 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3204: done (3 left) 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3208: rmattr oid 2 current snap is 320 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: finishing write tid 3 to smithi11881747-9 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: finishing write tid 4 to smithi11881747-9 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: finishing write tid 5 to smithi11881747-9 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: oid 9 updating version 0 to 65258 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: oid 9 version 65258 is already newer than 65256 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: oid 9 version 65258 is already newer than 65257 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: oid 9 updating version 65258 to 65259 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 65259 (ObjNum 966 snap 320 seq_num 966) dirty exists 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: left oid 9 (ObjNum 966 snap 320 seq_num 966) 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3206: finishing write tid 1 to smithi11881747-22 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3207: finishing rollback tid 0 to smithi11881747-25 2024-03-20T08:12:27.842 INFO:tasks.rados.rados.0.smithi118.stdout:3205: done (3 left) 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3209: setattr oid 12 current snap is 320 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3207: finishing rollback tid 1 to smithi11881747-25 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3207: finishing rollback tid 2 to smithi11881747-25 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 65262 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: finishing write tid 2 to smithi11881747-22 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: finishing write tid 3 to smithi11881747-22 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: finishing write tid 4 to smithi11881747-22 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: finishing write tid 5 to smithi11881747-22 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: finishing write tid 6 to smithi11881747-22 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: oid 22 updating version 0 to 61724 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: oid 22 updating version 61724 to 61725 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: oid 22 version 61725 is already newer than 61723 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: oid 22 updating version 61725 to 61727 2024-03-20T08:12:27.843 INFO:tasks.rados.rados.0.smithi118.stdout:3206: oid 22 version 61727 is already newer than 61726 2024-03-20T08:12:27.844 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 61727 (ObjNum 967 snap 320 seq_num 967) dirty exists 2024-03-20T08:12:27.844 INFO:tasks.rados.rados.0.smithi118.stdout:3206: left oid 22 (ObjNum 967 snap 320 seq_num 967) 2024-03-20T08:12:27.844 INFO:tasks.rados.rados.0.smithi118.stdout:3206: done (3 left) 2024-03-20T08:12:27.844 INFO:tasks.rados.rados.0.smithi118.stdout:3207: done (2 left) 2024-03-20T08:12:27.844 INFO:tasks.rados.rados.0.smithi118.stdout:3210: snap_create 2024-03-20T08:12:27.845 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45635 (ObjNum 685 snap 225 seq_num 685) dirty exists 2024-03-20T08:12:27.848 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 44238 (ObjNum 946 snap 318 seq_num 946) dirty exists 2024-03-20T08:12:28.048 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:12:28.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:27.732+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:28.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:27.738+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:28.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:27.738+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:28.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:27.738+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:28.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:27.740+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:28.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:27.741+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:28.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:27.767+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:28.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:27.768+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:28.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:27.768+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: pgmap v742: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 18 MiB/s rd, 2.9 MiB/s wr, 17 op/s 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: osdmap e707: 8 total, 8 up, 8 in 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:28.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:28.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:28.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:28.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:28.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:28.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:28.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:28.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:27 smithi196 ceph-mon[87080]: from='client.34497 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-20T08:12:28.163 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:27.734+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:28.164 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:27.731+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:28.164 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:27.731+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:28.164 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:27.731+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:28.164 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:27.737+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:28.164 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:27.760+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:28.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:27.760+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:28.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:27.760+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:28.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:27.760+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:28.165 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:27.762+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:28.737 INFO:tasks.rados.rados.0.smithi118.stdout:3208: done (2 left) 2024-03-20T08:12:28.738 INFO:tasks.rados.rados.0.smithi118.stdout:3209: done (1 left) 2024-03-20T08:12:28.738 INFO:tasks.rados.rados.0.smithi118.stdout:3210: done (0 left) 2024-03-20T08:12:28.738 INFO:tasks.rados.rados.0.smithi118.stdout:3211: delete oid 42 current snap is 321 2024-03-20T08:12:28.743 INFO:tasks.rados.rados.0.smithi118.stdout:3211: done (0 left) 2024-03-20T08:12:28.744 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 321 2024-03-20T08:12:28.744 INFO:tasks.rados.rados.0.smithi118.stdout:3212: seq_num 969 ranges {5988352=638976,6627328=630784,7258112=532480,7790592=40960} 2024-03-20T08:12:28.780 INFO:tasks.rados.rados.0.smithi118.stdout:3212: writing smithi11881747-26 from 5988352 to 6627328 tid 1 2024-03-20T08:12:28.783 INFO:tasks.rados.rados.0.smithi118.stdout:3212: writing smithi11881747-26 from 6627328 to 7258112 tid 2 2024-03-20T08:12:28.788 INFO:tasks.rados.rados.0.smithi118.stdout:3212: writing smithi11881747-26 from 7258112 to 7790592 tid 3 2024-03-20T08:12:28.788 INFO:tasks.rados.rados.0.smithi118.stdout:3212: writing smithi11881747-26 from 7790592 to 7831552 tid 4 2024-03-20T08:12:28.788 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 321 2024-03-20T08:12:28.788 INFO:tasks.rados.rados.0.smithi118.stdout:3213: seq_num 970 ranges {0=655360,655360=516096,1171456=688128} 2024-03-20T08:12:28.791 INFO:tasks.rados.rados.0.smithi118.stdout:3213: writing smithi11881747-42 from 0 to 655360 tid 1 2024-03-20T08:12:28.794 INFO:tasks.rados.rados.0.smithi118.stdout:3213: writing smithi11881747-42 from 655360 to 1171456 tid 2 2024-03-20T08:12:28.799 INFO:tasks.rados.rados.0.smithi118.stdout:3213: writing smithi11881747-42 from 1171456 to 1859584 tid 3 2024-03-20T08:12:28.799 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 321 2024-03-20T08:12:28.799 INFO:tasks.rados.rados.0.smithi118.stdout:3212: finishing write tid 1 to smithi11881747-26 2024-03-20T08:12:28.800 INFO:tasks.rados.rados.0.smithi118.stdout:3213: finishing write tid 1 to smithi11881747-42 2024-03-20T08:12:28.800 INFO:tasks.rados.rados.0.smithi118.stdout:3213: finishing write tid 2 to smithi11881747-42 2024-03-20T08:12:28.800 INFO:tasks.rados.rados.0.smithi118.stdout:3212: finishing write tid 2 to smithi11881747-26 2024-03-20T08:12:28.800 INFO:tasks.rados.rados.0.smithi118.stdout:3214: seq_num 971 ranges {1941504=671744,2613248=802816,3416064=401408} 2024-03-20T08:12:28.814 INFO:tasks.rados.rados.0.smithi118.stdout:3214: writing smithi11881747-22 from 1941504 to 2613248 tid 1 2024-03-20T08:12:28.818 INFO:tasks.rados.rados.0.smithi118.stdout:3214: writing smithi11881747-22 from 2613248 to 3416064 tid 2 2024-03-20T08:12:28.821 INFO:tasks.rados.rados.0.smithi118.stdout:3214: writing smithi11881747-22 from 3416064 to 3817472 tid 3 2024-03-20T08:12:28.821 INFO:tasks.rados.rados.0.smithi118.stdout:3215: delete oid 44 current snap is 321 2024-03-20T08:12:28.821 INFO:tasks.rados.rados.0.smithi118.stdout:3212: finishing write tid 3 to smithi11881747-26 2024-03-20T08:12:28.821 INFO:tasks.rados.rados.0.smithi118.stdout:3212: finishing write tid 4 to smithi11881747-26 2024-03-20T08:12:28.821 INFO:tasks.rados.rados.0.smithi118.stdout:3212: finishing write tid 5 to smithi11881747-26 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3213: finishing write tid 3 to smithi11881747-42 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3213: finishing write tid 4 to smithi11881747-42 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3213: finishing write tid 5 to smithi11881747-42 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3213: oid 42 updating version 0 to 65267 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3213: oid 42 version 65267 is already newer than 65266 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3213: oid 42 version 65267 is already newer than 65264 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3213: oid 42 version 65267 is already newer than 65265 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65267 (ObjNum 970 snap 321 seq_num 970) dirty exists 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3213: left oid 42 (ObjNum 970 snap 321 seq_num 970) 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3212: finishing write tid 6 to smithi11881747-26 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3212: oid 26 updating version 0 to 65258 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3212: oid 26 version 65258 is already newer than 65256 2024-03-20T08:12:28.822 INFO:tasks.rados.rados.0.smithi118.stdout:3212: oid 26 version 65258 is already newer than 65257 2024-03-20T08:12:28.823 INFO:tasks.rados.rados.0.smithi118.stdout:3212: oid 26 updating version 65258 to 65259 2024-03-20T08:12:28.823 INFO:tasks.rados.rados.0.smithi118.stdout:3212: oid 26 updating version 65259 to 65260 2024-03-20T08:12:28.823 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65260 (ObjNum 969 snap 321 seq_num 969) dirty exists 2024-03-20T08:12:28.823 INFO:tasks.rados.rados.0.smithi118.stdout:3212: left oid 26 (ObjNum 969 snap 321 seq_num 969) 2024-03-20T08:12:28.824 INFO:tasks.rados.rados.0.smithi118.stdout:3214: finishing write tid 1 to smithi11881747-22 2024-03-20T08:12:28.827 INFO:tasks.rados.rados.0.smithi118.stdout:3212: done (3 left) 2024-03-20T08:12:28.827 INFO:tasks.rados.rados.0.smithi118.stdout:3213: done (2 left) 2024-03-20T08:12:28.827 INFO:tasks.rados.rados.0.smithi118.stdout:3215: done (1 left) 2024-03-20T08:12:28.828 INFO:tasks.rados.rados.0.smithi118.stdout:3216: read oid 18 snap -1 2024-03-20T08:12:28.828 INFO:tasks.rados.rados.0.smithi118.stdout:3216: expect (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:12:28.828 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 321 2024-03-20T08:12:28.828 INFO:tasks.rados.rados.0.smithi118.stdout:3217: seq_num 973 ranges {7938048=704512,8642560=573440,9216000=507904,9723904=122880} 2024-03-20T08:12:28.874 INFO:tasks.rados.rados.0.smithi118.stdout:3217: writing smithi11881747-46 from 7938048 to 8642560 tid 1 2024-03-20T08:12:28.877 INFO:tasks.rados.rados.0.smithi118.stdout:3217: writing smithi11881747-46 from 8642560 to 9216000 tid 2 2024-03-20T08:12:28.880 INFO:tasks.rados.rados.0.smithi118.stdout:3217: writing smithi11881747-46 from 9216000 to 9723904 tid 3 2024-03-20T08:12:28.881 INFO:tasks.rados.rados.0.smithi118.stdout:3217: writing smithi11881747-46 from 9723904 to 9846784 tid 4 2024-03-20T08:12:28.882 INFO:tasks.rados.rados.0.smithi118.stdout:3218: read oid 29 snap -1 2024-03-20T08:12:28.882 INFO:tasks.rados.rados.0.smithi118.stdout:3218: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:12:28.882 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 321 2024-03-20T08:12:28.882 INFO:tasks.rados.rados.0.smithi118.stdout:3219: seq_num 974 ranges {737280=720896,1458176=507904,1966080=671744,2637824=24576} 2024-03-20T08:12:28.889 INFO:tasks.rados.rados.0.smithi118.stdout:3219: writing smithi11881747-24 from 737280 to 1458176 tid 1 2024-03-20T08:12:28.892 INFO:tasks.rados.rados.0.smithi118.stdout:3219: writing smithi11881747-24 from 1458176 to 1966080 tid 2 2024-03-20T08:12:28.896 INFO:tasks.rados.rados.0.smithi118.stdout:3219: writing smithi11881747-24 from 1966080 to 2637824 tid 3 2024-03-20T08:12:28.896 INFO:tasks.rados.rados.0.smithi118.stdout:3219: writing smithi11881747-24 from 2637824 to 2662400 tid 4 2024-03-20T08:12:28.896 INFO:tasks.rados.rados.0.smithi118.stdout:3220: read oid 44 snap 317 2024-03-20T08:12:28.896 INFO:tasks.rados.rados.0.smithi118.stdout:3220: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:12:28.897 INFO:tasks.rados.rados.0.smithi118.stdout:3221: read oid 30 snap -1 2024-03-20T08:12:28.897 INFO:tasks.rados.rados.0.smithi118.stdout:3221: expect (ObjNum 650 snap 213 seq_num 650) 2024-03-20T08:12:28.897 INFO:tasks.rados.rados.0.smithi118.stdout:3222: snap_remove snap 313 2024-03-20T08:12:29.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:28.736+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:29.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:28.767+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:29.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:28.767+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:29.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:28.766+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:29.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:28.766+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:29.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:28.744+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:29.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:28.744+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:29.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:28.744+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:29.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: osdmap e708: 8 total, 8 up, 8 in 2024-03-20T08:12:29.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:29.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:29.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:29.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:29.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:29.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:29.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:29.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:29.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:29.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:29.145 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:29.146 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:29.146 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:29.147 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:29.147 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:29.147 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:29.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:28.741+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:29.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:28.744+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:29.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:28.737+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:29.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:28.737+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:29.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:28.735+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:29.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:28.736+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:29.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:28.736+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:29.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:28.736+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: osdmap e708: 8 total, 8 up, 8 in 2024-03-20T08:12:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:29.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:28.744+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: osdmap e708: 8 total, 8 up, 8 in 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:29.741 INFO:tasks.rados.rados.0.smithi118.stdout:3222: done (7 left) 2024-03-20T08:12:29.742 INFO:tasks.rados.rados.0.smithi118.stdout:3223: snap_remove snap 304 2024-03-20T08:12:29.816 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:29.741+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:29.817 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:29.784+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:29.817 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:29.785+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:29.817 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:29.785+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:29.817 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:29.785+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:29.817 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:29.785+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:29.817 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:29.744+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: pgmap v745: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 26 MiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: osdmap e709: 8 total, 8 up, 8 in 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:29.818 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: pgmap v745: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 26 MiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: osdmap e709: 8 total, 8 up, 8 in 2024-03-20T08:12:29.819 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:29.820 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:29.821 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:29.821 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:29.821 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:30.126 INFO:teuthology.orchestra.run.smithi118.stdout:{ 2024-03-20T08:12:30.126 INFO:teuthology.orchestra.run.smithi118.stdout: "mon": { 2024-03-20T08:12:30.126 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 3 2024-03-20T08:12:30.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:12:30.126 INFO:teuthology.orchestra.run.smithi118.stdout: "mgr": { 2024-03-20T08:12:30.126 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:12:30.126 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:12:30.126 INFO:teuthology.orchestra.run.smithi118.stdout: "osd": { 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 8 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout: "mds": { 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 2 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout: }, 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout: "overall": { 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)": 15 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout: } 2024-03-20T08:12:30.127 INFO:teuthology.orchestra.run.smithi118.stdout:} 2024-03-20T08:12:30.127 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:29.865+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:30.127 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:29.866+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:30.127 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:29.866+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:30.138 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:29.742+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:30.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:29.866+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:30.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:29.868+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:30.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:29.868+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:30.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:29.866+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:30.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:29.867+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:30.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:29.867+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:30.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:29.868+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:30.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:29.868+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: pgmap v745: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.2 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 26 MiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-03-20T08:12:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: osdmap e709: 8 total, 8 up, 8 in 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:30.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:30.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:3223: done (7 left) 2024-03-20T08:12:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:3224: copy_from oid 10 from oid 15 current snap is 321 2024-03-20T08:12:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:3214: finishing write tid 2 to smithi11881747-22 2024-03-20T08:12:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:3214: finishing write tid 3 to smithi11881747-22 2024-03-20T08:12:30.743 INFO:tasks.rados.rados.0.smithi118.stdout:3214: finishing write tid 4 to smithi11881747-22 2024-03-20T08:12:30.744 INFO:tasks.rados.rados.0.smithi118.stdout:3216: expect (ObjNum 814 snap 279 seq_num 814) 2024-03-20T08:12:30.769 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -e sha1=e7e80a6ac9849f0f23432d0d9832f096730308ae -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-20T08:12:30.855 INFO:tasks.rados.rados.0.smithi118.stdout:3214: finishing write tid 5 to smithi11881747-22 2024-03-20T08:12:30.855 INFO:tasks.rados.rados.0.smithi118.stdout:3214: oid 22 updating version 0 to 61731 2024-03-20T08:12:30.855 INFO:tasks.rados.rados.0.smithi118.stdout:3214: oid 22 version 61731 is already newer than 61730 2024-03-20T08:12:30.855 INFO:tasks.rados.rados.0.smithi118.stdout:3214: oid 22 updating version 61731 to 61732 2024-03-20T08:12:30.855 INFO:tasks.rados.rados.0.smithi118.stdout:3214: oid 22 version 61732 is already newer than 61729 2024-03-20T08:12:30.855 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 61732 (ObjNum 971 snap 321 seq_num 971) dirty exists 2024-03-20T08:12:30.855 INFO:tasks.rados.rados.0.smithi118.stdout:3214: left oid 22 (ObjNum 971 snap 321 seq_num 971) 2024-03-20T08:12:30.855 INFO:tasks.rados.rados.0.smithi118.stdout:3217: finishing write tid 1 to smithi11881747-46 2024-03-20T08:12:30.856 INFO:tasks.rados.rados.0.smithi118.stdout:3217: finishing write tid 2 to smithi11881747-46 2024-03-20T08:12:30.856 INFO:tasks.rados.rados.0.smithi118.stdout:3214: done (7 left) 2024-03-20T08:12:30.856 INFO:tasks.rados.rados.0.smithi118.stdout:3216: done (6 left) 2024-03-20T08:12:30.856 INFO:tasks.rados.rados.0.smithi118.stdout:3225: setattr oid 47 current snap is 321 2024-03-20T08:12:30.856 INFO:tasks.rados.rados.0.smithi118.stdout:3217: finishing write tid 3 to smithi11881747-46 2024-03-20T08:12:30.856 INFO:tasks.rados.rados.0.smithi118.stdout:3218: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: finishing write tid 4 to smithi11881747-46 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: finishing write tid 5 to smithi11881747-46 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: finishing write tid 6 to smithi11881747-46 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: oid 46 updating version 0 to 45640 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: oid 46 version 45640 is already newer than 45637 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: oid 46 version 45640 is already newer than 45638 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: oid 46 version 45640 is already newer than 45636 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: oid 46 version 45640 is already newer than 45639 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 45640 (ObjNum 973 snap 321 seq_num 973) dirty exists 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3217: left oid 46 (ObjNum 973 snap 321 seq_num 973) 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3219: finishing write tid 1 to smithi11881747-24 2024-03-20T08:12:30.887 INFO:tasks.rados.rados.0.smithi118.stdout:3221: expect (ObjNum 650 snap 213 seq_num 650) 2024-03-20T08:12:30.922 INFO:tasks.rados.rados.0.smithi118.stdout:3219: finishing write tid 2 to smithi11881747-24 2024-03-20T08:12:30.922 INFO:tasks.rados.rados.0.smithi118.stdout:3219: finishing write tid 3 to smithi11881747-24 2024-03-20T08:12:30.922 INFO:tasks.rados.rados.0.smithi118.stdout:3217: done (6 left) 2024-03-20T08:12:30.922 INFO:tasks.rados.rados.0.smithi118.stdout:3218: done (5 left) 2024-03-20T08:12:30.922 INFO:tasks.rados.rados.0.smithi118.stdout:3221: done (4 left) 2024-03-20T08:12:30.922 INFO:tasks.rados.rados.0.smithi118.stdout:3226: snap_remove snap 318 2024-03-20T08:12:31.028 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:12:31.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:30.742+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:31.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:30.757+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:31.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:30.757+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:31.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:30.757+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:31.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:30.865+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:31.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:30.866+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:31.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:30.749+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:31.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:30.749+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: osdmap e710: 8 total, 8 up, 8 in 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:31.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:30 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/2509560929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:31.150 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:30.742+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:31.150 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:30.744+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:31.151 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:30.754+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:31.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:30.866+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:31.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:30.867+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:31.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:30.867+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:31.151 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:30.867+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:31.152 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:30.743+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:31.152 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:30.743+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:31.152 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:31.152 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: osdmap e710: 8 total, 8 up, 8 in 2024-03-20T08:12:31.152 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:31.152 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:31.152 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:31.152 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: osdmap e710: 8 total, 8 up, 8 in 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:31.153 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:31.154 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:31.155 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:31.155 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:31.155 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/2509560929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:31.155 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:31.155 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:31.155 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:31.155 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:30 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/2509560929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:31.753 INFO:tasks.rados.rados.0.smithi118.stdout:3226: done (4 left) 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 321 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: finishing write tid 4 to smithi11881747-24 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: finishing write tid 5 to smithi11881747-24 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: finishing write tid 6 to smithi11881747-24 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: oid 24 updating version 0 to 65264 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: oid 24 updating version 65264 to 65265 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: oid 24 updating version 65265 to 65266 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: oid 24 updating version 65266 to 65267 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: oid 24 updating version 65267 to 65268 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65268 (ObjNum 974 snap 321 seq_num 974) dirty exists 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3219: left oid 24 (ObjNum 974 snap 321 seq_num 974) 2024-03-20T08:12:31.754 INFO:tasks.rados.rados.0.smithi118.stdout:3220: expect (ObjNum 854 snap 296 seq_num 854) 2024-03-20T08:12:31.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:31.753+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:31.911 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:31.753+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:31.912 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:31.754+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:31.912 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:31.754+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:31.912 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:31.754+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:31.912 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: pgmap v748: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-03-20T08:12:31.912 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:31.912 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: osdmap e711: 8 total, 8 up, 8 in 2024-03-20T08:12:31.912 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:31.912 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:31.913 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:31.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: pgmap v748: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-03-20T08:12:31.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:31.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: osdmap e711: 8 total, 8 up, 8 in 2024-03-20T08:12:31.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:31.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:31.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:31.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:31.914 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:31.915 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:31.915 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:31.755+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:32.125 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:31.752+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:32.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: pgmap v748: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 6.1 MiB/s wr, 31 op/s 2024-03-20T08:12:32.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:32.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: osdmap e711: 8 total, 8 up, 8 in 2024-03-20T08:12:32.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:32.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:32.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:32.126 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:32.127 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:32.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:32.123+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:32.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:32.123+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:32.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:12:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:12:32.574 INFO:tasks.rados.rados.0.smithi118.stdout:3224: finishing copy_from racing read to smithi11881747-10 2024-03-20T08:12:32.574 INFO:tasks.rados.rados.0.smithi118.stdout:3224: finishing copy_from to smithi11881747-10 2024-03-20T08:12:32.574 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 7105 (ObjNum 928 snap 315 seq_num 928) dirty exists 2024-03-20T08:12:32.574 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 45624 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:12:32.574 INFO:tasks.rados.rados.0.smithi118.stdout:3227: seq_num 975 ranges {9846784=737280,10584064=786432,11370496=417792} 2024-03-20T08:12:32.630 INFO:tasks.rados.rados.0.smithi118.stdout:3227: writing smithi11881747-46 from 9846784 to 10584064 tid 1 2024-03-20T08:12:32.636 INFO:tasks.rados.rados.0.smithi118.stdout:3227: writing smithi11881747-46 from 10584064 to 11370496 tid 2 2024-03-20T08:12:32.638 INFO:tasks.rados.rados.0.smithi118.stdout:3227: writing smithi11881747-46 from 11370496 to 11788288 tid 3 2024-03-20T08:12:32.639 INFO:tasks.rados.rados.0.smithi118.stdout:3219: done (4 left) 2024-03-20T08:12:32.639 INFO:tasks.rados.rados.0.smithi118.stdout:3220: done (3 left) 2024-03-20T08:12:32.639 INFO:tasks.rados.rados.0.smithi118.stdout:3224: done (2 left) 2024-03-20T08:12:32.639 INFO:tasks.rados.rados.0.smithi118.stdout:3225: done (1 left) 2024-03-20T08:12:32.639 INFO:tasks.rados.rados.0.smithi118.stdout:3228: rollback oid 28 current snap is 321 2024-03-20T08:12:32.639 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 28 to 319 2024-03-20T08:12:32.639 INFO:tasks.rados.rados.0.smithi118.stdout:3227: finishing write tid 1 to smithi11881747-46 2024-03-20T08:12:32.639 INFO:tasks.rados.rados.0.smithi118.stdout:3229: snap_remove snap 320 2024-03-20T08:12:32.760 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:32.530+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:32.760 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:32.530+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:32.760 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:32.530+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:32.760 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:32.586+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:32.765 INFO:tasks.rados.rados.0.smithi118.stdout:3229: done (2 left) 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3230: delete oid 24 current snap is 321 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3228: finishing rollback tid 0 to smithi11881747-28 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3227: finishing write tid 2 to smithi11881747-46 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3228: finishing rollback tid 1 to smithi11881747-28 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3227: finishing write tid 3 to smithi11881747-46 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3228: finishing rollback tid 2 to smithi11881747-28 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 45627 (ObjNum 900 snap 309 seq_num 900) dirty exists 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3227: finishing write tid 4 to smithi11881747-46 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3227: finishing write tid 5 to smithi11881747-46 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3227: oid 46 updating version 0 to 45642 2024-03-20T08:12:32.766 INFO:tasks.rados.rados.0.smithi118.stdout:3227: oid 46 version 45642 is already newer than 45641 2024-03-20T08:12:32.767 INFO:tasks.rados.rados.0.smithi118.stdout:3227: oid 46 updating version 45642 to 45643 2024-03-20T08:12:32.767 INFO:tasks.rados.rados.0.smithi118.stdout:3227: oid 46 updating version 45643 to 45644 2024-03-20T08:12:32.767 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 45644 (ObjNum 975 snap 321 seq_num 975) dirty exists 2024-03-20T08:12:32.767 INFO:tasks.rados.rados.0.smithi118.stdout:3227: left oid 46 (ObjNum 975 snap 321 seq_num 975) 2024-03-20T08:12:32.772 INFO:tasks.rados.rados.0.smithi118.stdout:3227: done (2 left) 2024-03-20T08:12:32.773 INFO:tasks.rados.rados.0.smithi118.stdout:3228: done (1 left) 2024-03-20T08:12:32.773 INFO:tasks.rados.rados.0.smithi118.stdout:3230: done (0 left) 2024-03-20T08:12:32.773 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 321 2024-03-20T08:12:32.773 INFO:tasks.rados.rados.0.smithi118.stdout:3231: seq_num 977 ranges {966656=770048,1736704=557056,2293760=491520,2785280=155648} 2024-03-20T08:12:32.781 INFO:tasks.rados.rados.0.smithi118.stdout:3231: writing smithi11881747-41 from 966656 to 1736704 tid 1 2024-03-20T08:12:32.784 INFO:tasks.rados.rados.0.smithi118.stdout:3231: writing smithi11881747-41 from 1736704 to 2293760 tid 2 2024-03-20T08:12:32.787 INFO:tasks.rados.rados.0.smithi118.stdout:3231: writing smithi11881747-41 from 2293760 to 2785280 tid 3 2024-03-20T08:12:32.789 INFO:tasks.rados.rados.0.smithi118.stdout:3231: writing smithi11881747-41 from 2785280 to 2940928 tid 4 2024-03-20T08:12:32.789 INFO:tasks.rados.rados.0.smithi118.stdout:3232: delete oid 48 current snap is 321 2024-03-20T08:12:32.789 INFO:tasks.rados.rados.0.smithi118.stdout:3231: finishing write tid 1 to smithi11881747-41 2024-03-20T08:12:32.790 INFO:tasks.rados.rados.0.smithi118.stdout:3231: finishing write tid 2 to smithi11881747-41 2024-03-20T08:12:32.793 INFO:tasks.rados.rados.0.smithi118.stdout:3231: finishing write tid 3 to smithi11881747-41 2024-03-20T08:12:32.793 INFO:tasks.rados.rados.0.smithi118.stdout:3231: finishing write tid 4 to smithi11881747-41 2024-03-20T08:12:32.794 INFO:tasks.rados.rados.0.smithi118.stdout:3231: finishing write tid 5 to smithi11881747-41 2024-03-20T08:12:32.795 INFO:tasks.rados.rados.0.smithi118.stdout:3231: finishing write tid 6 to smithi11881747-41 2024-03-20T08:12:32.795 INFO:tasks.rados.rados.0.smithi118.stdout:3231: oid 41 updating version 0 to 45631 2024-03-20T08:12:32.795 INFO:tasks.rados.rados.0.smithi118.stdout:3231: oid 41 version 45631 is already newer than 45629 2024-03-20T08:12:32.795 INFO:tasks.rados.rados.0.smithi118.stdout:3231: oid 41 version 45631 is already newer than 45628 2024-03-20T08:12:32.795 INFO:tasks.rados.rados.0.smithi118.stdout:3231: oid 41 version 45631 is already newer than 45630 2024-03-20T08:12:32.795 INFO:tasks.rados.rados.0.smithi118.stdout:3231: oid 41 updating version 45631 to 45632 2024-03-20T08:12:32.795 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 45632 (ObjNum 977 snap 321 seq_num 977) dirty exists 2024-03-20T08:12:32.795 INFO:tasks.rados.rados.0.smithi118.stdout:3231: left oid 41 (ObjNum 977 snap 321 seq_num 977) 2024-03-20T08:12:32.797 INFO:tasks.rados.rados.0.smithi118.stdout:3231: done (1 left) 2024-03-20T08:12:32.797 INFO:tasks.rados.rados.0.smithi118.stdout:3232: done (0 left) 2024-03-20T08:12:32.797 INFO:tasks.rados.rados.0.smithi118.stdout:3233: copy_from oid 29 from oid 35 current snap is 321 2024-03-20T08:12:32.797 INFO:tasks.rados.rados.0.smithi118.stdout:3234: copy_from oid 11 from oid 26 current snap is 321 2024-03-20T08:12:32.797 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 321 2024-03-20T08:12:32.797 INFO:tasks.rados.rados.0.smithi118.stdout:3235: seq_num 979 ranges {0=802816,802816=729088,1531904=458752,1990656=16384} 2024-03-20T08:12:32.801 INFO:tasks.rados.rados.0.smithi118.stdout:3235: writing smithi11881747-27 from 0 to 802816 tid 1 2024-03-20T08:12:32.805 INFO:tasks.rados.rados.0.smithi118.stdout:3235: writing smithi11881747-27 from 802816 to 1531904 tid 2 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:3235: writing smithi11881747-27 from 1531904 to 1990656 tid 3 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:3235: writing smithi11881747-27 from 1990656 to 2007040 tid 4 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:3236: rollback oid 15 current snap is 321 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 15 to 294 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:3235: finishing write tid 1 to smithi11881747-27 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:3233: finishing copy_from racing read to smithi11881747-29 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:3233: finishing copy_from to smithi11881747-29 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 45614 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:3233: done (3 left) 2024-03-20T08:12:32.809 INFO:tasks.rados.rados.0.smithi118.stdout:3237: delete oid 21 current snap is 321 2024-03-20T08:12:32.815 INFO:tasks.rados.rados.0.smithi118.stdout:3235: finishing write tid 2 to smithi11881747-27 2024-03-20T08:12:32.816 INFO:tasks.rados.rados.0.smithi118.stdout:3235: finishing write tid 3 to smithi11881747-27 2024-03-20T08:12:32.816 INFO:tasks.rados.rados.0.smithi118.stdout:3235: finishing write tid 4 to smithi11881747-27 2024-03-20T08:12:32.816 INFO:tasks.rados.rados.0.smithi118.stdout:3237: done (3 left) 2024-03-20T08:12:32.816 INFO:tasks.rados.rados.0.smithi118.stdout:3238: copy_from oid 12 from oid 42 current snap is 321 2024-03-20T08:12:32.817 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 321 2024-03-20T08:12:32.817 INFO:tasks.rados.rados.0.smithi118.stdout:3239: seq_num 981 ranges {966656=434176,1400832=540672,1941504=466944,2408448=524288,2932736=73728} 2024-03-20T08:12:32.824 INFO:tasks.rados.rados.0.smithi118.stdout:3239: writing smithi11881747-17 from 966656 to 1400832 tid 1 2024-03-20T08:12:32.828 INFO:tasks.rados.rados.0.smithi118.stdout:3239: writing smithi11881747-17 from 1400832 to 1941504 tid 2 2024-03-20T08:12:32.831 INFO:tasks.rados.rados.0.smithi118.stdout:3239: writing smithi11881747-17 from 1941504 to 2408448 tid 3 2024-03-20T08:12:32.834 INFO:tasks.rados.rados.0.smithi118.stdout:3239: writing smithi11881747-17 from 2408448 to 2932736 tid 4 2024-03-20T08:12:32.837 INFO:tasks.rados.rados.0.smithi118.stdout:3239: writing smithi11881747-17 from 2932736 to 3006464 tid 5 2024-03-20T08:12:32.837 INFO:tasks.rados.rados.0.smithi118.stdout:3240: rollback oid 35 current snap is 321 2024-03-20T08:12:32.837 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 35 to 294 2024-03-20T08:12:32.837 INFO:tasks.rados.rados.0.smithi118.stdout:3235: finishing write tid 5 to smithi11881747-27 2024-03-20T08:12:32.837 INFO:tasks.rados.rados.0.smithi118.stdout:3236: finishing rollback tid 0 to smithi11881747-15 2024-03-20T08:12:32.837 INFO:tasks.rados.rados.0.smithi118.stdout:3235: finishing write tid 6 to smithi11881747-27 2024-03-20T08:12:32.838 INFO:tasks.rados.rados.0.smithi118.stdout:3235: oid 27 updating version 0 to 59834 2024-03-20T08:12:32.838 INFO:tasks.rados.rados.0.smithi118.stdout:3235: oid 27 version 59834 is already newer than 59832 2024-03-20T08:12:32.838 INFO:tasks.rados.rados.0.smithi118.stdout:3235: oid 27 version 59834 is already newer than 59833 2024-03-20T08:12:32.838 INFO:tasks.rados.rados.0.smithi118.stdout:3235: oid 27 version 59834 is already newer than 59831 2024-03-20T08:12:32.838 INFO:tasks.rados.rados.0.smithi118.stdout:3235: oid 27 updating version 59834 to 59835 2024-03-20T08:12:32.838 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 59835 (ObjNum 979 snap 321 seq_num 979) dirty exists 2024-03-20T08:12:32.838 INFO:tasks.rados.rados.0.smithi118.stdout:3235: left oid 27 (ObjNum 979 snap 321 seq_num 979) 2024-03-20T08:12:32.842 INFO:tasks.rados.rados.0.smithi118.stdout:3236: finishing rollback tid 1 to smithi11881747-15 2024-03-20T08:12:32.842 INFO:tasks.rados.rados.0.smithi118.stdout:3236: finishing rollback tid 2 to smithi11881747-15 2024-03-20T08:12:32.842 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 7112 (ObjNum 813 snap 279 seq_num 813) dirty exists 2024-03-20T08:12:32.842 INFO:tasks.rados.rados.0.smithi118.stdout:3239: finishing write tid 1 to smithi11881747-17 2024-03-20T08:12:32.842 INFO:tasks.rados.rados.0.smithi118.stdout:3238: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:12:32.842 INFO:tasks.rados.rados.0.smithi118.stdout:3238: finishing copy_from to smithi11881747-12 2024-03-20T08:12:32.843 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65267 (ObjNum 970 snap 321 seq_num 970) dirty exists 2024-03-20T08:12:32.843 INFO:tasks.rados.rados.0.smithi118.stdout:3235: done (5 left) 2024-03-20T08:12:32.843 INFO:tasks.rados.rados.0.smithi118.stdout:3236: done (4 left) 2024-03-20T08:12:32.843 INFO:tasks.rados.rados.0.smithi118.stdout:3238: done (3 left) 2024-03-20T08:12:32.843 INFO:tasks.rados.rados.0.smithi118.stdout:3241: rollback oid 24 current snap is 321 2024-03-20T08:12:32.843 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 294 2024-03-20T08:12:32.844 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 321 2024-03-20T08:12:32.844 INFO:tasks.rados.rados.0.smithi118.stdout:3242: seq_num 982 ranges {0=450560,450560=425984,876544=679936,1556480=466944,2023424=32768} 2024-03-20T08:12:32.845 INFO:tasks.rados.rados.0.smithi118.stdout:3242: writing smithi11881747-3 from 0 to 450560 tid 1 2024-03-20T08:12:32.845 INFO:tasks.rados.rados.0.smithi118.stdout:3242: writing smithi11881747-3 from 450560 to 876544 tid 2 2024-03-20T08:12:32.846 INFO:tasks.rados.rados.0.smithi118.stdout:3242: writing smithi11881747-3 from 876544 to 1556480 tid 3 2024-03-20T08:12:32.851 INFO:tasks.rados.rados.0.smithi118.stdout:3242: writing smithi11881747-3 from 1556480 to 2023424 tid 4 2024-03-20T08:12:32.851 INFO:tasks.rados.rados.0.smithi118.stdout:3242: writing smithi11881747-3 from 2023424 to 2056192 tid 5 2024-03-20T08:12:32.851 INFO:tasks.rados.rados.0.smithi118.stdout:3243: delete oid 49 current snap is 321 2024-03-20T08:12:32.851 INFO:tasks.rados.rados.0.smithi118.stdout:3234: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:12:32.852 INFO:tasks.rados.rados.0.smithi118.stdout:3239: finishing write tid 2 to smithi11881747-17 2024-03-20T08:12:32.852 INFO:tasks.rados.rados.0.smithi118.stdout:3239: finishing write tid 3 to smithi11881747-17 2024-03-20T08:12:32.852 INFO:tasks.rados.rados.0.smithi118.stdout:3239: finishing write tid 4 to smithi11881747-17 2024-03-20T08:12:32.852 INFO:tasks.rados.rados.0.smithi118.stdout:3240: finishing rollback tid 0 to smithi11881747-35 2024-03-20T08:12:32.852 INFO:tasks.rados.rados.0.smithi118.stdout:3239: finishing write tid 5 to smithi11881747-17 2024-03-20T08:12:32.852 INFO:tasks.rados.rados.0.smithi118.stdout:3240: finishing rollback tid 1 to smithi11881747-35 2024-03-20T08:12:32.852 INFO:tasks.rados.rados.0.smithi118.stdout:3240: finishing rollback tid 2 to smithi11881747-35 2024-03-20T08:12:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 45647 (ObjNum 760 snap 261 seq_num 760) dirty exists 2024-03-20T08:12:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:3234: finishing copy_from to smithi11881747-11 2024-03-20T08:12:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 65260 (ObjNum 969 snap 321 seq_num 969) dirty exists 2024-03-20T08:12:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:3239: finishing write tid 6 to smithi11881747-17 2024-03-20T08:12:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:3241: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:12:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:3241: finishing rollback tid 2 to smithi11881747-24 2024-03-20T08:12:32.853 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65271 (ObjNum 842 snap 291 seq_num 842) dirty exists 2024-03-20T08:12:32.854 INFO:tasks.rados.rados.0.smithi118.stdout:3239: finishing write tid 7 to smithi11881747-17 2024-03-20T08:12:32.854 INFO:tasks.rados.rados.0.smithi118.stdout:3239: oid 17 updating version 0 to 45594 2024-03-20T08:12:32.854 INFO:tasks.rados.rados.0.smithi118.stdout:3239: oid 17 version 45594 is already newer than 45593 2024-03-20T08:12:32.854 INFO:tasks.rados.rados.0.smithi118.stdout:3239: oid 17 version 45594 is already newer than 45592 2024-03-20T08:12:32.854 INFO:tasks.rados.rados.0.smithi118.stdout:3239: oid 17 updating version 45594 to 45596 2024-03-20T08:12:32.854 INFO:tasks.rados.rados.0.smithi118.stdout:3239: oid 17 version 45596 is already newer than 45591 2024-03-20T08:12:32.854 INFO:tasks.rados.rados.0.smithi118.stdout:3239: oid 17 version 45596 is already newer than 45595 2024-03-20T08:12:32.854 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 45596 (ObjNum 981 snap 321 seq_num 981) dirty exists 2024-03-20T08:12:32.855 INFO:tasks.rados.rados.0.smithi118.stdout:3239: left oid 17 (ObjNum 981 snap 321 seq_num 981) 2024-03-20T08:12:32.855 INFO:tasks.rados.rados.0.smithi118.stdout:3242: finishing write tid 1 to smithi11881747-3 2024-03-20T08:12:32.856 INFO:tasks.rados.rados.0.smithi118.stdout:3234: done (5 left) 2024-03-20T08:12:32.856 INFO:tasks.rados.rados.0.smithi118.stdout:3239: done (4 left) 2024-03-20T08:12:32.856 INFO:tasks.rados.rados.0.smithi118.stdout:3240: done (3 left) 2024-03-20T08:12:32.857 INFO:tasks.rados.rados.0.smithi118.stdout:3241: done (2 left) 2024-03-20T08:12:32.857 INFO:tasks.rados.rados.0.smithi118.stdout:3243: done (1 left) 2024-03-20T08:12:32.857 INFO:tasks.rados.rados.0.smithi118.stdout:3244: read oid 16 snap -1 2024-03-20T08:12:32.857 INFO:tasks.rados.rados.0.smithi118.stdout:3244: expect (ObjNum 950 snap 318 seq_num 950) 2024-03-20T08:12:32.857 INFO:tasks.rados.rados.0.smithi118.stdout:3245: delete oid 31 current snap is 321 2024-03-20T08:12:32.857 INFO:tasks.rados.rados.0.smithi118.stdout:3245: done (2 left) 2024-03-20T08:12:32.857 INFO:tasks.rados.rados.0.smithi118.stdout:3246: copy_from oid 7 from oid 28 current snap is 321 2024-03-20T08:12:32.857 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 321 2024-03-20T08:12:32.858 INFO:tasks.rados.rados.0.smithi118.stdout:3247: seq_num 985 ranges {2932736=499712,3432448=483328,3915776=450560,4366336=581632,4947968=90112} 2024-03-20T08:12:32.877 INFO:tasks.rados.rados.0.smithi118.stdout:3247: writing smithi11881747-34 from 2932736 to 3432448 tid 1 2024-03-20T08:12:32.880 INFO:tasks.rados.rados.0.smithi118.stdout:3247: writing smithi11881747-34 from 3432448 to 3915776 tid 2 2024-03-20T08:12:32.883 INFO:tasks.rados.rados.0.smithi118.stdout:3247: writing smithi11881747-34 from 3915776 to 4366336 tid 3 2024-03-20T08:12:32.887 INFO:tasks.rados.rados.0.smithi118.stdout:3247: writing smithi11881747-34 from 4366336 to 4947968 tid 4 2024-03-20T08:12:32.888 INFO:tasks.rados.rados.0.smithi118.stdout:3247: writing smithi11881747-34 from 4947968 to 5038080 tid 5 2024-03-20T08:12:32.888 INFO:tasks.rados.rados.0.smithi118.stdout:3248: read oid 11 snap -1 2024-03-20T08:12:32.888 INFO:tasks.rados.rados.0.smithi118.stdout:3248: expect (ObjNum 969 snap 321 seq_num 969) 2024-03-20T08:12:32.888 INFO:tasks.rados.rados.0.smithi118.stdout:3249: copy_from oid 35 from oid 15 current snap is 321 2024-03-20T08:12:32.889 INFO:tasks.rados.rados.0.smithi118.stdout:3242: finishing write tid 2 to smithi11881747-3 2024-03-20T08:12:32.889 INFO:tasks.rados.rados.0.smithi118.stdout:3242: finishing write tid 3 to smithi11881747-3 2024-03-20T08:12:32.889 INFO:tasks.rados.rados.0.smithi118.stdout:3242: finishing write tid 4 to smithi11881747-3 2024-03-20T08:12:32.889 INFO:tasks.rados.rados.0.smithi118.stdout:3250: copy_from oid 17 from oid 46 current snap is 321 2024-03-20T08:12:32.889 INFO:tasks.rados.rados.0.smithi118.stdout:3242: finishing write tid 5 to smithi11881747-3 2024-03-20T08:12:32.889 INFO:tasks.rados.rados.0.smithi118.stdout:3242: finishing write tid 6 to smithi11881747-3 2024-03-20T08:12:32.889 INFO:tasks.rados.rados.0.smithi118.stdout:3247: finishing write tid 1 to smithi11881747-34 2024-03-20T08:12:32.889 INFO:tasks.rados.rados.0.smithi118.stdout:3251: setattr oid 8 current snap is 321 2024-03-20T08:12:32.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:32.761+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:32.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:32.585+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:32.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:32.585+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:32.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:32.585+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:32.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:32.767+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:32.767+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:32.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:32.771+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:32.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:32.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:32.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-mon[87080]: osdmap e712: 8 total, 8 up, 8 in 2024-03-20T08:12:32.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:32.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:32.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:32.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:32.892 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:32.770+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:32.892 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:32.770+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:32.893 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:32.586+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:32.893 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:32.586+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:32.893 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:32.586+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:32.893 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:32.800+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:32.893 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:32.800+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:32.893 INFO:tasks.rados.rados.0.smithi118.stdout:3242: finishing write tid 7 to smithi11881747-3 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3242: oid 3 updating version 0 to 34040 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3242: oid 3 version 34040 is already newer than 34037 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3242: oid 3 version 34040 is already newer than 34038 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3242: oid 3 version 34040 is already newer than 34039 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3242: oid 3 updating version 34040 to 34041 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3242: oid 3 version 34041 is already newer than 34036 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34041 (ObjNum 982 snap 321 seq_num 982) dirty exists 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3242: left oid 3 (ObjNum 982 snap 321 seq_num 982) 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3242: done (7 left) 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3252: read oid 18 snap 213 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3252: expect deleted 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3253: setattr oid 50 current snap is 321 2024-03-20T08:12:32.894 INFO:tasks.rados.rados.0.smithi118.stdout:3252: done (8 left) 2024-03-20T08:12:32.895 INFO:tasks.rados.rados.0.smithi118.stdout:3254: rollback oid 43 current snap is 321 2024-03-20T08:12:32.895 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 43 to 316 2024-03-20T08:12:32.895 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 321 2024-03-20T08:12:32.895 INFO:tasks.rados.rados.0.smithi118.stdout:3255: seq_num 986 ranges {2940928=516096,3457024=770048,4227072=655360,4882432=180224} 2024-03-20T08:12:32.911 INFO:tasks.rados.rados.0.smithi118.stdout:3255: writing smithi11881747-41 from 2940928 to 3457024 tid 1 2024-03-20T08:12:32.916 INFO:tasks.rados.rados.0.smithi118.stdout:3255: writing smithi11881747-41 from 3457024 to 4227072 tid 2 2024-03-20T08:12:32.922 INFO:tasks.rados.rados.0.smithi118.stdout:3255: writing smithi11881747-41 from 4227072 to 4882432 tid 3 2024-03-20T08:12:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:3255: writing smithi11881747-41 from 4882432 to 5062656 tid 4 2024-03-20T08:12:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:3256: copy_from oid 24 from oid 14 current snap is 321 2024-03-20T08:12:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:3247: finishing write tid 2 to smithi11881747-34 2024-03-20T08:12:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:3254: finishing rollback tid 0 to smithi11881747-43 2024-03-20T08:12:32.924 INFO:tasks.rados.rados.0.smithi118.stdout:3254: finishing rollback tid 1 to smithi11881747-43 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:3254: finishing rollback tid 2 to smithi11881747-43 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 44259 (ObjNum 904 snap 309 seq_num 904) dirty exists 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34103 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:3249: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:3247: finishing write tid 3 to smithi11881747-34 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:3247: finishing write tid 4 to smithi11881747-34 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:3247: finishing write tid 5 to smithi11881747-34 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:3249: finishing copy_from to smithi11881747-35 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 7112 (ObjNum 813 snap 279 seq_num 813) dirty exists 2024-03-20T08:12:32.925 INFO:tasks.rados.rados.0.smithi118.stdout:3244: expect (ObjNum 950 snap 318 seq_num 950) 2024-03-20T08:12:33.135 INFO:teuthology.orchestra.run.smithi118.stdout:true 2024-03-20T08:12:33.135 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:32.771+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:33.135 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:32.771+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:33.136 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:32.784+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:33.136 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:32.787+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:33.136 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:32.788+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:33.136 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:32.789+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:33.136 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:32.765+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:33.136 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:32.765+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:33.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:33.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:33.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[93121]: osdmap e712: 8 total, 8 up, 8 in 2024-03-20T08:12:33.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:33.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:33.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:33.137 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:33.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:33.138 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:33.138 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[95692]: osdmap e712: 8 total, 8 up, 8 in 2024-03-20T08:12:33.138 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:33.138 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:33.138 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:33.138 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:33.138 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:32.765+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:33.780 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:12:33] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:12:33.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:33.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:33.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:33.780 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: pgmap v751: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 24 op/s 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: osdmap e713: 8 total, 8 up, 8 in 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:33.781 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: pgmap v751: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 24 op/s 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: osdmap e713: 8 total, 8 up, 8 in 2024-03-20T08:12:33.782 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:33.783 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3757482320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:33.784 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:33.785 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:33.785 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:33.785 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:33.785 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:33.785 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:33 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/3757482320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:33.798 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -e sha1=e7e80a6ac9849f0f23432d0d9832f096730308ae -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-20T08:12:34.014 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:12:34.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:34.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: pgmap v751: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 14 MiB/s rd, 4.3 MiB/s wr, 24 op/s 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: osdmap e713: 8 total, 8 up, 8 in 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17b=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 171=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:33 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/3757482320' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 65272 (ObjNum 49 snap 0 seq_num 1953066355) dirty exists 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:3244: done (10 left) 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:3249: done (9 left) 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:3251: done (8 left) 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:3253: done (7 left) 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:3254: done (6 left) 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 321 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:3256: finishing copy_from to smithi11881747-24 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:3256: got expected ENOENT (src dne) 2024-03-20T08:12:34.215 INFO:tasks.rados.rados.0.smithi118.stdout:3256: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: finishing write tid 6 to smithi11881747-34 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: finishing write tid 7 to smithi11881747-34 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: oid 34 updating version 0 to 45609 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: oid 34 version 45609 is already newer than 45607 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: oid 34 updating version 45609 to 45610 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: oid 34 version 45610 is already newer than 45606 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: oid 34 version 45610 is already newer than 45608 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: oid 34 version 45610 is already newer than 45605 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45610 (ObjNum 985 snap 321 seq_num 985) dirty exists 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3247: left oid 34 (ObjNum 985 snap 321 seq_num 985) 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3255: finishing write tid 1 to smithi11881747-41 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3255: finishing write tid 2 to smithi11881747-41 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3255: finishing write tid 3 to smithi11881747-41 2024-03-20T08:12:34.216 INFO:tasks.rados.rados.0.smithi118.stdout:3257: seq_num 987 ranges {9994240=532480,10526720=647168,11173888=417792,11591680=409600,12001280=131072} 2024-03-20T08:12:34.273 INFO:tasks.rados.rados.0.smithi118.stdout:3257: writing smithi11881747-33 from 9994240 to 10526720 tid 1 2024-03-20T08:12:34.277 INFO:tasks.rados.rados.0.smithi118.stdout:3257: writing smithi11881747-33 from 10526720 to 11173888 tid 2 2024-03-20T08:12:34.279 INFO:tasks.rados.rados.0.smithi118.stdout:3257: writing smithi11881747-33 from 11173888 to 11591680 tid 3 2024-03-20T08:12:34.282 INFO:tasks.rados.rados.0.smithi118.stdout:3257: writing smithi11881747-33 from 11591680 to 12001280 tid 4 2024-03-20T08:12:34.284 INFO:tasks.rados.rados.0.smithi118.stdout:3257: writing smithi11881747-33 from 12001280 to 12132352 tid 5 2024-03-20T08:12:34.284 INFO:tasks.rados.rados.0.smithi118.stdout:3247: done (6 left) 2024-03-20T08:12:34.284 INFO:tasks.rados.rados.0.smithi118.stdout:3256: done (5 left) 2024-03-20T08:12:34.284 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 321 2024-03-20T08:12:34.284 INFO:tasks.rados.rados.0.smithi118.stdout:3258: seq_num 988 ranges {7831552=557056,8388608=532480,8921088=622592,9543680=450560} 2024-03-20T08:12:34.331 INFO:tasks.rados.rados.0.smithi118.stdout:3258: writing smithi11881747-26 from 7831552 to 8388608 tid 1 2024-03-20T08:12:34.334 INFO:tasks.rados.rados.0.smithi118.stdout:3258: writing smithi11881747-26 from 8388608 to 8921088 tid 2 2024-03-20T08:12:34.339 INFO:tasks.rados.rados.0.smithi118.stdout:3258: writing smithi11881747-26 from 8921088 to 9543680 tid 3 2024-03-20T08:12:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:3258: writing smithi11881747-26 from 9543680 to 9994240 tid 4 2024-03-20T08:12:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:3259: read oid 23 snap -1 2024-03-20T08:12:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:3259: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:12:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:3260: delete oid 24 current snap is 321 2024-03-20T08:12:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:3250: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:12:34.342 INFO:tasks.rados.rados.0.smithi118.stdout:3255: finishing write tid 4 to smithi11881747-41 2024-03-20T08:12:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:3255: finishing write tid 5 to smithi11881747-41 2024-03-20T08:12:34.343 INFO:tasks.rados.rados.0.smithi118.stdout:3248: expect (ObjNum 969 snap 321 seq_num 969) 2024-03-20T08:12:34.838 INFO:tasks.rados.rados.0.smithi118.stdout:3255: finishing write tid 6 to smithi11881747-41 2024-03-20T08:12:34.838 INFO:tasks.rados.rados.0.smithi118.stdout:3255: oid 41 updating version 0 to 45634 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3255: oid 41 version 45634 is already newer than 45633 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3255: oid 41 updating version 45634 to 45636 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3255: oid 41 version 45636 is already newer than 45635 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3255: oid 41 updating version 45636 to 45637 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 45637 (ObjNum 986 snap 321 seq_num 986) dirty exists 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3255: left oid 41 (ObjNum 986 snap 321 seq_num 986) 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3250: finishing copy_from to smithi11881747-17 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 45644 (ObjNum 975 snap 321 seq_num 975) dirty exists 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3246: finishing copy_from racing read to smithi11881747-7 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3246: finishing copy_from to smithi11881747-7 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 45627 (ObjNum 900 snap 309 seq_num 900) dirty exists 2024-03-20T08:12:34.839 INFO:tasks.rados.rados.0.smithi118.stdout:3257: finishing write tid 1 to smithi11881747-33 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3257: finishing write tid 2 to smithi11881747-33 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3257: finishing write tid 3 to smithi11881747-33 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3257: finishing write tid 4 to smithi11881747-33 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3257: finishing write tid 5 to smithi11881747-33 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3246: done (7 left) 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3248: done (6 left) 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3250: done (5 left) 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3255: done (4 left) 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3260: done (3 left) 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 321 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3257: finishing write tid 6 to smithi11881747-33 2024-03-20T08:12:34.840 INFO:tasks.rados.rados.0.smithi118.stdout:3257: finishing write tid 7 to smithi11881747-33 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3257: oid 33 updating version 0 to 34047 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3257: oid 33 version 34047 is already newer than 34042 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3257: oid 33 version 34047 is already newer than 34044 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3257: oid 33 version 34047 is already newer than 34045 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3257: oid 33 version 34047 is already newer than 34043 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3257: oid 33 version 34047 is already newer than 34046 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34047 (ObjNum 987 snap 321 seq_num 987) dirty exists 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3257: left oid 33 (ObjNum 987 snap 321 seq_num 987) 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3258: finishing write tid 1 to smithi11881747-26 2024-03-20T08:12:34.841 INFO:tasks.rados.rados.0.smithi118.stdout:3261: seq_num 990 ranges {0=589824,589824=753664,1343488=638976,1982464=212992} 2024-03-20T08:12:34.842 INFO:tasks.rados.rados.0.smithi118.stdout:3261: writing smithi11881747-44 from 0 to 589824 tid 1 2024-03-20T08:12:34.845 INFO:tasks.rados.rados.0.smithi118.stdout:3261: writing smithi11881747-44 from 589824 to 1343488 tid 2 2024-03-20T08:12:34.848 INFO:tasks.rados.rados.0.smithi118.stdout:3261: writing smithi11881747-44 from 1343488 to 1982464 tid 3 2024-03-20T08:12:34.850 INFO:tasks.rados.rados.0.smithi118.stdout:3261: writing smithi11881747-44 from 1982464 to 2195456 tid 4 2024-03-20T08:12:34.850 INFO:tasks.rados.rados.0.smithi118.stdout:3257: done (3 left) 2024-03-20T08:12:34.850 INFO:tasks.rados.rados.0.smithi118.stdout:3262: delete oid 31 current snap is 321 2024-03-20T08:12:34.850 INFO:tasks.rados.rados.0.smithi118.stdout:3258: finishing write tid 2 to smithi11881747-26 2024-03-20T08:12:34.850 INFO:tasks.rados.rados.0.smithi118.stdout:3259: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:12:34.873 INFO:tasks.rados.rados.0.smithi118.stdout:3258: finishing write tid 3 to smithi11881747-26 2024-03-20T08:12:34.873 INFO:tasks.rados.rados.0.smithi118.stdout:3258: finishing write tid 4 to smithi11881747-26 2024-03-20T08:12:34.873 INFO:tasks.rados.rados.0.smithi118.stdout:3259: done (3 left) 2024-03-20T08:12:34.873 INFO:tasks.rados.rados.0.smithi118.stdout:3262: done (2 left) 2024-03-20T08:12:34.873 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 321 2024-03-20T08:12:34.873 INFO:tasks.rados.rados.0.smithi118.stdout:3258: finishing write tid 5 to smithi11881747-26 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3258: finishing write tid 6 to smithi11881747-26 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3258: oid 26 updating version 0 to 65274 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3258: oid 26 updating version 65274 to 65277 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3258: oid 26 version 65277 is already newer than 65276 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3258: oid 26 version 65277 is already newer than 65273 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3258: oid 26 version 65277 is already newer than 65275 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65277 (ObjNum 988 snap 321 seq_num 988) dirty exists 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3258: left oid 26 (ObjNum 988 snap 321 seq_num 988) 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3261: finishing write tid 1 to smithi11881747-44 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3261: finishing write tid 2 to smithi11881747-44 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3261: finishing write tid 3 to smithi11881747-44 2024-03-20T08:12:34.874 INFO:tasks.rados.rados.0.smithi118.stdout:3261: finishing write tid 4 to smithi11881747-44 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3261: finishing write tid 5 to smithi11881747-44 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3261: finishing write tid 6 to smithi11881747-44 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3261: oid 44 updating version 0 to 65279 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3261: oid 44 updating version 65279 to 65281 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3261: oid 44 updating version 65281 to 65283 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3261: oid 44 version 65283 is already newer than 65282 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3261: oid 44 version 65283 is already newer than 65280 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 65283 (ObjNum 990 snap 321 seq_num 990) dirty exists 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3261: left oid 44 (ObjNum 990 snap 321 seq_num 990) 2024-03-20T08:12:34.875 INFO:tasks.rados.rados.0.smithi118.stdout:3263: seq_num 992 ranges {3686400=622592,4308992=524288,4833280=606208,5439488=475136} 2024-03-20T08:12:34.897 INFO:tasks.rados.rados.0.smithi118.stdout:3263: writing smithi11881747-2 from 3686400 to 4308992 tid 1 2024-03-20T08:12:34.899 INFO:tasks.rados.rados.0.smithi118.stdout:3263: writing smithi11881747-2 from 4308992 to 4833280 tid 2 2024-03-20T08:12:34.903 INFO:tasks.rados.rados.0.smithi118.stdout:3263: writing smithi11881747-2 from 4833280 to 5439488 tid 3 2024-03-20T08:12:34.906 INFO:tasks.rados.rados.0.smithi118.stdout:3263: writing smithi11881747-2 from 5439488 to 5914624 tid 4 2024-03-20T08:12:34.906 INFO:tasks.rados.rados.0.smithi118.stdout:3258: done (2 left) 2024-03-20T08:12:34.906 INFO:tasks.rados.rados.0.smithi118.stdout:3261: done (1 left) 2024-03-20T08:12:34.906 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 321 2024-03-20T08:12:34.906 INFO:tasks.rados.rados.0.smithi118.stdout:3263: finishing write tid 1 to smithi11881747-2 2024-03-20T08:12:34.906 INFO:tasks.rados.rados.0.smithi118.stdout:3263: finishing write tid 2 to smithi11881747-2 2024-03-20T08:12:34.906 INFO:tasks.rados.rados.0.smithi118.stdout:3264: seq_num 993 ranges {0=638976,638976=409600,1048576=409600,1458176=688128,2146304=98304} 2024-03-20T08:12:34.909 INFO:tasks.rados.rados.0.smithi118.stdout:3264: writing smithi11881747-14 from 0 to 638976 tid 1 2024-03-20T08:12:34.912 INFO:tasks.rados.rados.0.smithi118.stdout:3264: writing smithi11881747-14 from 638976 to 1048576 tid 2 2024-03-20T08:12:34.914 INFO:tasks.rados.rados.0.smithi118.stdout:3264: writing smithi11881747-14 from 1048576 to 1458176 tid 3 2024-03-20T08:12:34.919 INFO:tasks.rados.rados.0.smithi118.stdout:3264: writing smithi11881747-14 from 1458176 to 2146304 tid 4 2024-03-20T08:12:34.920 INFO:tasks.rados.rados.0.smithi118.stdout:3264: writing smithi11881747-14 from 2146304 to 2244608 tid 5 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3265: rollback oid 27 current snap is 321 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 266 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: finishing write tid 3 to smithi11881747-2 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: finishing write tid 4 to smithi11881747-2 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: finishing write tid 5 to smithi11881747-2 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: finishing write tid 6 to smithi11881747-2 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: oid 2 updating version 0 to 45652 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: oid 2 version 45652 is already newer than 45648 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: oid 2 version 45652 is already newer than 45650 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: oid 2 version 45652 is already newer than 45649 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: oid 2 version 45652 is already newer than 45651 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45652 (ObjNum 992 snap 321 seq_num 992) dirty exists 2024-03-20T08:12:34.921 INFO:tasks.rados.rados.0.smithi118.stdout:3263: left oid 2 (ObjNum 992 snap 321 seq_num 992) 2024-03-20T08:12:34.922 INFO:tasks.rados.rados.0.smithi118.stdout:3264: finishing write tid 1 to smithi11881747-14 2024-03-20T08:12:34.922 INFO:tasks.rados.rados.0.smithi118.stdout:3263: done (2 left) 2024-03-20T08:12:34.922 INFO:tasks.rados.rados.0.smithi118.stdout:3266: delete oid 33 current snap is 321 2024-03-20T08:12:34.924 INFO:tasks.rados.rados.0.smithi118.stdout:3265: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:12:34.925 INFO:tasks.rados.rados.0.smithi118.stdout:3264: finishing write tid 2 to smithi11881747-14 2024-03-20T08:12:34.926 INFO:tasks.rados.rados.0.smithi118.stdout:3265: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:12:34.926 INFO:tasks.rados.rados.0.smithi118.stdout:3265: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:12:34.926 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 59839 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:12:34.927 INFO:tasks.rados.rados.0.smithi118.stdout:3265: done (2 left) 2024-03-20T08:12:34.927 INFO:tasks.rados.rados.0.smithi118.stdout:3266: done (1 left) 2024-03-20T08:12:34.927 INFO:tasks.rados.rados.0.smithi118.stdout:3267: delete oid 36 current snap is 321 2024-03-20T08:12:34.933 INFO:tasks.rados.rados.0.smithi118.stdout:3267: done (1 left) 2024-03-20T08:12:34.933 INFO:tasks.rados.rados.0.smithi118.stdout:3268: read oid 24 snap 213 2024-03-20T08:12:34.933 INFO:tasks.rados.rados.0.smithi118.stdout:3268: expect (ObjNum 617 snap 202 seq_num 617) 2024-03-20T08:12:34.933 INFO:tasks.rados.rados.0.smithi118.stdout:3269: snap_remove snap 317 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3269: done (2 left) 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3270: delete oid 30 current snap is 321 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3264: finishing write tid 3 to smithi11881747-14 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3264: finishing write tid 4 to smithi11881747-14 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3264: finishing write tid 5 to smithi11881747-14 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3264: finishing write tid 6 to smithi11881747-14 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3264: finishing write tid 7 to smithi11881747-14 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3264: oid 14 updating version 0 to 34106 2024-03-20T08:12:35.771 INFO:tasks.rados.rados.0.smithi118.stdout:3264: oid 14 version 34106 is already newer than 34104 2024-03-20T08:12:35.772 INFO:tasks.rados.rados.0.smithi118.stdout:3264: oid 14 updating version 34106 to 34108 2024-03-20T08:12:35.772 INFO:tasks.rados.rados.0.smithi118.stdout:3264: oid 14 updating version 34108 to 34109 2024-03-20T08:12:35.772 INFO:tasks.rados.rados.0.smithi118.stdout:3264: oid 14 version 34109 is already newer than 34105 2024-03-20T08:12:35.772 INFO:tasks.rados.rados.0.smithi118.stdout:3264: oid 14 version 34109 is already newer than 34107 2024-03-20T08:12:35.772 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 34109 (ObjNum 993 snap 321 seq_num 993) dirty exists 2024-03-20T08:12:35.772 INFO:tasks.rados.rados.0.smithi118.stdout:3264: left oid 14 (ObjNum 993 snap 321 seq_num 993) 2024-03-20T08:12:35.772 INFO:tasks.rados.rados.0.smithi118.stdout:3268: expect (ObjNum 617 snap 202 seq_num 617) 2024-03-20T08:12:35.772 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-mon[93121]: pgmap v753: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-03-20T08:12:35.772 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-mon[95692]: pgmap v753: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-03-20T08:12:35.790 INFO:tasks.rados.rados.0.smithi118.stdout:3264: done (2 left) 2024-03-20T08:12:35.791 INFO:tasks.rados.rados.0.smithi118.stdout:3268: done (1 left) 2024-03-20T08:12:35.791 INFO:tasks.rados.rados.0.smithi118.stdout:3270: done (0 left) 2024-03-20T08:12:35.791 INFO:tasks.rados.rados.0.smithi118.stdout:3271: delete oid 48 current snap is 321 2024-03-20T08:12:35.799 INFO:tasks.rados.rados.0.smithi118.stdout:3271: done (0 left) 2024-03-20T08:12:35.799 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 321 2024-03-20T08:12:35.799 INFO:tasks.rados.rados.0.smithi118.stdout:3272: seq_num 998 ranges {8200192=720896,8921088=679936,9601024=598016,10199040=327680} 2024-03-20T08:12:35.847 INFO:tasks.rados.rados.0.smithi118.stdout:3272: writing smithi11881747-7 from 8200192 to 8921088 tid 1 2024-03-20T08:12:35.851 INFO:tasks.rados.rados.0.smithi118.stdout:3272: writing smithi11881747-7 from 8921088 to 9601024 tid 2 2024-03-20T08:12:35.856 INFO:tasks.rados.rados.0.smithi118.stdout:3272: writing smithi11881747-7 from 9601024 to 10199040 tid 3 2024-03-20T08:12:35.859 INFO:tasks.rados.rados.0.smithi118.stdout:3272: writing smithi11881747-7 from 10199040 to 10526720 tid 4 2024-03-20T08:12:35.859 INFO:tasks.rados.rados.0.smithi118.stdout:3273: rollback oid 19 current snap is 321 2024-03-20T08:12:35.859 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 19 to 266 2024-03-20T08:12:35.859 INFO:tasks.rados.rados.0.smithi118.stdout:3272: finishing write tid 1 to smithi11881747-7 2024-03-20T08:12:35.859 INFO:tasks.rados.rados.0.smithi118.stdout:3274: rollback oid 24 current snap is 321 2024-03-20T08:12:35.859 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 24 to 226 2024-03-20T08:12:35.859 INFO:tasks.rados.rados.0.smithi118.stdout:3275: snap_create 2024-03-20T08:12:35.863 INFO:tasks.rados.rados.0.smithi118.stdout:3272: finishing write tid 2 to smithi11881747-7 2024-03-20T08:12:35.866 INFO:tasks.rados.rados.0.smithi118.stdout:3273: finishing rollback tid 0 to smithi11881747-19 2024-03-20T08:12:35.866 INFO:tasks.rados.rados.0.smithi118.stdout:3273: finishing rollback tid 1 to smithi11881747-19 2024-03-20T08:12:35.866 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 7114 (ObjNum 289 snap 0 seq_num 1273582497) dirty dne 2024-03-20T08:12:35.867 INFO:tasks.rados.rados.0.smithi118.stdout:3274: finishing rollback tid 1 to smithi11881747-24 2024-03-20T08:12:35.867 INFO:tasks.rados.rados.0.smithi118.stdout:3274: finishing rollback tid 2 to smithi11881747-24 2024-03-20T08:12:35.867 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65285 (ObjNum 687 snap 225 seq_num 687) dirty exists 2024-03-20T08:12:35.872 INFO:tasks.rados.rados.0.smithi118.stdout:3272: finishing write tid 3 to smithi11881747-7 2024-03-20T08:12:35.872 INFO:tasks.rados.rados.0.smithi118.stdout:3272: finishing write tid 4 to smithi11881747-7 2024-03-20T08:12:35.872 INFO:tasks.rados.rados.0.smithi118.stdout:3272: finishing write tid 5 to smithi11881747-7 2024-03-20T08:12:35.877 INFO:tasks.rados.rados.0.smithi118.stdout:3272: finishing write tid 6 to smithi11881747-7 2024-03-20T08:12:35.877 INFO:tasks.rados.rados.0.smithi118.stdout:3272: oid 7 updating version 0 to 65268 2024-03-20T08:12:35.877 INFO:tasks.rados.rados.0.smithi118.stdout:3272: oid 7 updating version 65268 to 65271 2024-03-20T08:12:35.877 INFO:tasks.rados.rados.0.smithi118.stdout:3272: oid 7 updating version 65271 to 65272 2024-03-20T08:12:35.877 INFO:tasks.rados.rados.0.smithi118.stdout:3272: oid 7 version 65272 is already newer than 65270 2024-03-20T08:12:35.877 INFO:tasks.rados.rados.0.smithi118.stdout:3272: oid 7 version 65272 is already newer than 65269 2024-03-20T08:12:35.877 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 65272 (ObjNum 998 snap 321 seq_num 998) dirty exists 2024-03-20T08:12:35.877 INFO:tasks.rados.rados.0.smithi118.stdout:3272: left oid 7 (ObjNum 998 snap 321 seq_num 998) 2024-03-20T08:12:36.074 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:35.798+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:36.075 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:35.798+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:36.075 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:35.798+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:36.075 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:35.864+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:36.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:35.769+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:36.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:35.770+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:36.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:35.770+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:36.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:35.770+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:36.076 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:35.771+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:36.077 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:35.774+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:36.077 INFO:teuthology.orchestra.run.smithi118.stdout: "ceph version 19.0.0-2153-ge7e80a6a (e7e80a6ac9849f0f23432d0d9832f096730308ae) squid (dev)" 2024-03-20T08:12:36.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-mon[87080]: pgmap v753: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-03-20T08:12:36.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:35.771+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:36.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:35.798+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:36.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:35.798+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:36.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:35.780+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:36.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:35.780+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:36.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:35.780+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:36.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:35.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:36.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:35.777+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:36.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:35.778+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:36.776 INFO:tasks.rados.rados.0.smithi118.stdout:3272: done (3 left) 2024-03-20T08:12:36.776 INFO:tasks.rados.rados.0.smithi118.stdout:3273: done (2 left) 2024-03-20T08:12:36.777 INFO:tasks.rados.rados.0.smithi118.stdout:3274: done (1 left) 2024-03-20T08:12:36.777 INFO:tasks.rados.rados.0.smithi118.stdout:3275: done (0 left) 2024-03-20T08:12:36.777 INFO:tasks.rados.rados.0.smithi118.stdout:3276: read oid 49 snap -1 2024-03-20T08:12:36.777 INFO:tasks.rados.rados.0.smithi118.stdout:3276: expect deleted 2024-03-20T08:12:36.777 INFO:tasks.rados.rados.0.smithi118.stdout:3277: snap_remove snap 311 2024-03-20T08:12:36.922 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-20T08:12:36.922 INFO:teuthology.task.print:**** done end upgrade, wait... 2024-03-20T08:12:36.923 DEBUG:teuthology.parallel:result is None 2024-03-20T08:12:37.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:36.775+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:37.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:36.775+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:37.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:37.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:37.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: osdmap e714: 8 total, 8 up, 8 in 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:36 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/1557376159' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:37.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:36.776+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:37.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:36.776+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:37.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: osdmap e714: 8 total, 8 up, 8 in 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:37.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:36.776+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/1557376159' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: osdmap e714: 8 total, 8 up, 8 in 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 178=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:36 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/1557376159' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-20T08:12:37.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.285+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:37.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.286+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:37.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.286+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:37.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.286+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:37.777 INFO:tasks.rados.rados.0.smithi118.stdout:3277: done (1 left) 2024-03-20T08:12:37.777 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 322 2024-03-20T08:12:37.777 INFO:tasks.rados.rados.0.smithi118.stdout:3278: seq_num 999 ranges {1859584=737280,2596864=565248,3162112=761856,3923968=278528} 2024-03-20T08:12:37.791 INFO:tasks.rados.rados.0.smithi118.stdout:3278: writing smithi11881747-12 from 1859584 to 2596864 tid 1 2024-03-20T08:12:37.794 INFO:tasks.rados.rados.0.smithi118.stdout:3278: writing smithi11881747-12 from 2596864 to 3162112 tid 2 2024-03-20T08:12:37.799 INFO:tasks.rados.rados.0.smithi118.stdout:3278: writing smithi11881747-12 from 3162112 to 3923968 tid 3 2024-03-20T08:12:37.802 INFO:tasks.rados.rados.0.smithi118.stdout:3278: writing smithi11881747-12 from 3923968 to 4202496 tid 4 2024-03-20T08:12:37.802 INFO:tasks.rados.rados.0.smithi118.stdout:3276: done (1 left) 2024-03-20T08:12:37.802 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 322 2024-03-20T08:12:37.802 INFO:tasks.rados.rados.0.smithi118.stdout:3279: seq_num 1000 ranges {5062656=753664,5816320=442368,6258688=565248,6823936=458752,7282688=139264} 2024-03-20T08:12:37.834 INFO:tasks.rados.rados.0.smithi118.stdout:3279: writing smithi11881747-41 from 5062656 to 5816320 tid 1 2024-03-20T08:12:37.836 INFO:tasks.rados.rados.0.smithi118.stdout:3279: writing smithi11881747-41 from 5816320 to 6258688 tid 2 2024-03-20T08:12:37.839 INFO:tasks.rados.rados.0.smithi118.stdout:3279: writing smithi11881747-41 from 6258688 to 6823936 tid 3 2024-03-20T08:12:37.843 INFO:tasks.rados.rados.0.smithi118.stdout:3279: writing smithi11881747-41 from 6823936 to 7282688 tid 4 2024-03-20T08:12:37.844 INFO:tasks.rados.rados.0.smithi118.stdout:3279: writing smithi11881747-41 from 7282688 to 7421952 tid 5 2024-03-20T08:12:37.844 INFO:tasks.rados.rados.0.smithi118.stdout:3280: snap_remove snap 316 2024-03-20T08:12:38.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:37.800+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:38.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:37.802+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:38.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:37.802+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-mon[87080]: pgmap v755: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 7.8 MiB/s wr, 29 op/s 2024-03-20T08:12:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-mon[87080]: osdmap e715: 8 total, 8 up, 8 in 2024-03-20T08:12:38.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:37.732+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:38.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:37.734+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:38.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:37.801+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:38.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:37.802+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:38.141 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:37.733+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:38.141 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:37.776+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:38.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:37.731+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:38.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:37.732+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:38.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:37.732+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:38.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:37.838+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:38.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:37.838+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:38.142 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:37.838+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:38.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:37.777+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:38.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:37.684+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:38.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:37.684+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:38.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:37.776+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:38.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:37.777+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:38.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:37.777+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:38.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[93121]: pgmap v755: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 7.8 MiB/s wr, 29 op/s 2024-03-20T08:12:38.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[93121]: osdmap e715: 8 total, 8 up, 8 in 2024-03-20T08:12:38.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[95692]: pgmap v755: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 7.8 MiB/s wr, 29 op/s 2024-03-20T08:12:38.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:38.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:38.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:38.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:38.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-mon[95692]: osdmap e715: 8 total, 8 up, 8 in 2024-03-20T08:12:38.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.794+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:38.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.795+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:38.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.795+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:38.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.795+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:38.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:37.804+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:38.788 INFO:tasks.rados.rados.0.smithi118.stdout:3280: done (2 left) 2024-03-20T08:12:38.788 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 322 2024-03-20T08:12:38.788 INFO:tasks.rados.rados.0.smithi118.stdout:3278: finishing write tid 1 to smithi11881747-12 2024-03-20T08:12:38.788 INFO:tasks.rados.rados.0.smithi118.stdout:3278: finishing write tid 2 to smithi11881747-12 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: finishing write tid 3 to smithi11881747-12 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: finishing write tid 4 to smithi11881747-12 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: finishing write tid 5 to smithi11881747-12 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: finishing write tid 6 to smithi11881747-12 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: oid 12 updating version 0 to 65275 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: oid 12 version 65275 is already newer than 65274 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: oid 12 updating version 65275 to 65276 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: oid 12 version 65276 is already newer than 65273 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: oid 12 updating version 65276 to 65277 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65277 (ObjNum 999 snap 322 seq_num 999) dirty exists 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3278: left oid 12 (ObjNum 999 snap 322 seq_num 999) 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3279: finishing write tid 1 to smithi11881747-41 2024-03-20T08:12:38.789 INFO:tasks.rados.rados.0.smithi118.stdout:3281: seq_num 1001 ranges {3153920=770048,3923968=729088,4653056=770048,5423104=106496} 2024-03-20T08:12:38.812 INFO:tasks.rados.rados.0.smithi118.stdout:3281: writing smithi11881747-18 from 3153920 to 3923968 tid 1 2024-03-20T08:12:38.817 INFO:tasks.rados.rados.0.smithi118.stdout:3281: writing smithi11881747-18 from 3923968 to 4653056 tid 2 2024-03-20T08:12:38.821 INFO:tasks.rados.rados.0.smithi118.stdout:3281: writing smithi11881747-18 from 4653056 to 5423104 tid 3 2024-03-20T08:12:38.823 INFO:tasks.rados.rados.0.smithi118.stdout:3281: writing smithi11881747-18 from 5423104 to 5529600 tid 4 2024-03-20T08:12:38.823 INFO:tasks.rados.rados.0.smithi118.stdout:3278: done (2 left) 2024-03-20T08:12:38.823 INFO:tasks.rados.rados.0.smithi118.stdout:3282: rollback oid 2 current snap is 322 2024-03-20T08:12:38.823 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 213 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: finishing write tid 2 to smithi11881747-41 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: finishing write tid 3 to smithi11881747-41 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: finishing write tid 4 to smithi11881747-41 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: finishing write tid 5 to smithi11881747-41 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: finishing write tid 6 to smithi11881747-41 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: finishing write tid 7 to smithi11881747-41 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: oid 41 updating version 0 to 45642 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: oid 41 version 45642 is already newer than 45641 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: oid 41 version 45642 is already newer than 45639 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: oid 41 updating version 45642 to 45643 2024-03-20T08:12:38.824 INFO:tasks.rados.rados.0.smithi118.stdout:3279: oid 41 version 45643 is already newer than 45638 2024-03-20T08:12:38.825 INFO:tasks.rados.rados.0.smithi118.stdout:3279: oid 41 version 45643 is already newer than 45640 2024-03-20T08:12:38.825 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 45643 (ObjNum 1000 snap 322 seq_num 1000) dirty exists 2024-03-20T08:12:38.825 INFO:tasks.rados.rados.0.smithi118.stdout:3279: left oid 41 (ObjNum 1000 snap 322 seq_num 1000) 2024-03-20T08:12:38.825 INFO:tasks.rados.rados.0.smithi118.stdout:3281: finishing write tid 1 to smithi11881747-18 2024-03-20T08:12:38.825 INFO:tasks.rados.rados.0.smithi118.stdout:3281: finishing write tid 2 to smithi11881747-18 2024-03-20T08:12:38.825 INFO:tasks.rados.rados.0.smithi118.stdout:3279: done (2 left) 2024-03-20T08:12:38.825 INFO:tasks.rados.rados.0.smithi118.stdout:3283: snap_remove snap 226 2024-03-20T08:12:39.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:38.788+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:39.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:38.790+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:39.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:38.790+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:39.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:38.826+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:39.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:38.830+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:39.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:38.819+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:39.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:38.819+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:39.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:38.819+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: osdmap e716: 8 total, 8 up, 8 in 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:39.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:38.895+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:39.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:38.895+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:39.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:38.818+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:39.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:38.818+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:39.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:38.731+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:39.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:38.787+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:39.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:38.893+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:39.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:38.894+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:39.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:38.894+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:39.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:38.894+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:39.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: osdmap e716: 8 total, 8 up, 8 in 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:39.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: osdmap e716: 8 total, 8 up, 8 in 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 170=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:39.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:39.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:39.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:39.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:39.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:39.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:39.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:39.792 INFO:tasks.rados.rados.0.smithi118.stdout:3283: done (2 left) 2024-03-20T08:12:39.792 INFO:tasks.rados.rados.0.smithi118.stdout:3284: rmattr oid 14 current snap is 322 2024-03-20T08:12:39.792 INFO:tasks.rados.rados.0.smithi118.stdout:3284: done (2 left) 2024-03-20T08:12:39.792 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 322 2024-03-20T08:12:39.792 INFO:tasks.rados.rados.0.smithi118.stdout:3285: seq_num 1002 ranges {843776=786432,1630208=614400,2244608=581632,2826240=409600} 2024-03-20T08:12:39.802 INFO:tasks.rados.rados.0.smithi118.stdout:3285: writing smithi11881747-13 from 843776 to 1630208 tid 1 2024-03-20T08:12:39.806 INFO:tasks.rados.rados.0.smithi118.stdout:3285: writing smithi11881747-13 from 1630208 to 2244608 tid 2 2024-03-20T08:12:39.810 INFO:tasks.rados.rados.0.smithi118.stdout:3285: writing smithi11881747-13 from 2244608 to 2826240 tid 3 2024-03-20T08:12:39.813 INFO:tasks.rados.rados.0.smithi118.stdout:3285: writing smithi11881747-13 from 2826240 to 3235840 tid 4 2024-03-20T08:12:39.813 INFO:tasks.rados.rados.0.smithi118.stdout:3286: snap_remove snap 321 2024-03-20T08:12:40.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:39.792+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:40.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:39.810+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:40.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:39.810+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:40.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:39.810+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:40.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:40.022+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:40.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:40.023+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: pgmap v758: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 35 op/s 2024-03-20T08:12:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: osdmap e717: 8 total, 8 up, 8 in 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:40.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:39.792+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:40.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:39.806+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:40.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:39.806+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:40.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:39.809+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:40.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:39.793+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: pgmap v758: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 35 op/s 2024-03-20T08:12:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: osdmap e717: 8 total, 8 up, 8 in 2024-03-20T08:12:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: pgmap v758: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 35 op/s 2024-03-20T08:12:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: osdmap e717: 8 total, 8 up, 8 in 2024-03-20T08:12:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:40.798 INFO:tasks.rados.rados.0.smithi118.stdout:3286: done (3 left) 2024-03-20T08:12:40.798 INFO:tasks.rados.rados.0.smithi118.stdout:3287: copy_from oid 26 from oid 27 current snap is 322 2024-03-20T08:12:40.798 INFO:tasks.rados.rados.0.smithi118.stdout:3281: finishing write tid 3 to smithi11881747-18 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: finishing write tid 4 to smithi11881747-18 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: finishing write tid 5 to smithi11881747-18 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: finishing write tid 6 to smithi11881747-18 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: oid 18 updating version 0 to 59844 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: oid 18 version 59844 is already newer than 59843 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: oid 18 version 59844 is already newer than 59842 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: oid 18 version 59844 is already newer than 59840 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: oid 18 version 59844 is already newer than 59841 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 59844 (ObjNum 1001 snap 322 seq_num 1001) dirty exists 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3281: left oid 18 (ObjNum 1001 snap 322 seq_num 1001) 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3282: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3282: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45654 (ObjNum 49 snap 0 seq_num 323689472) dirty dne 2024-03-20T08:12:40.799 INFO:tasks.rados.rados.0.smithi118.stdout:3285: finishing write tid 1 to smithi11881747-13 2024-03-20T08:12:40.800 INFO:tasks.rados.rados.0.smithi118.stdout:3285: finishing write tid 2 to smithi11881747-13 2024-03-20T08:12:40.800 INFO:tasks.rados.rados.0.smithi118.stdout:3285: finishing write tid 3 to smithi11881747-13 2024-03-20T08:12:40.800 INFO:tasks.rados.rados.0.smithi118.stdout:3285: finishing write tid 4 to smithi11881747-13 2024-03-20T08:12:40.800 INFO:tasks.rados.rados.0.smithi118.stdout:3285: finishing write tid 5 to smithi11881747-13 2024-03-20T08:12:40.800 INFO:tasks.rados.rados.0.smithi118.stdout:3285: finishing write tid 6 to smithi11881747-13 2024-03-20T08:12:40.800 INFO:tasks.rados.rados.0.smithi118.stdout:3285: oid 13 updating version 0 to 65268 2024-03-20T08:12:40.800 INFO:tasks.rados.rados.0.smithi118.stdout:3285: oid 13 updating version 65268 to 65270 2024-03-20T08:12:40.800 INFO:tasks.rados.rados.0.smithi118.stdout:3285: oid 13 version 65270 is already newer than 65269 2024-03-20T08:12:40.801 INFO:tasks.rados.rados.0.smithi118.stdout:3285: oid 13 updating version 65270 to 65272 2024-03-20T08:12:40.801 INFO:tasks.rados.rados.0.smithi118.stdout:3285: oid 13 version 65272 is already newer than 65271 2024-03-20T08:12:40.801 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 65272 (ObjNum 1002 snap 322 seq_num 1002) dirty exists 2024-03-20T08:12:40.801 INFO:tasks.rados.rados.0.smithi118.stdout:3285: left oid 13 (ObjNum 1002 snap 322 seq_num 1002) 2024-03-20T08:12:40.801 INFO:tasks.rados.rados.0.smithi118.stdout:3281: done (3 left) 2024-03-20T08:12:40.802 INFO:tasks.rados.rados.0.smithi118.stdout:3282: done (2 left) 2024-03-20T08:12:40.802 INFO:tasks.rados.rados.0.smithi118.stdout:3285: done (1 left) 2024-03-20T08:12:40.802 INFO:tasks.rados.rados.0.smithi118.stdout:3288: copy_from oid 35 from oid 5 current snap is 322 2024-03-20T08:12:40.802 INFO:tasks.rados.rados.0.smithi118.stdout:3289: setattr oid 12 current snap is 322 2024-03-20T08:12:40.802 INFO:tasks.rados.rados.0.smithi118.stdout:3290: delete oid 1 current snap is 322 2024-03-20T08:12:40.806 INFO:tasks.rados.rados.0.smithi118.stdout:3288: finishing copy_from to smithi11881747-35 2024-03-20T08:12:40.806 INFO:tasks.rados.rados.0.smithi118.stdout:3288: got expected ENOENT (src dne) 2024-03-20T08:12:40.806 INFO:tasks.rados.rados.0.smithi118.stdout:3288: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:12:40.808 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65278 (ObjNum 999 snap 322 seq_num 999) dirty exists 2024-03-20T08:12:40.808 INFO:tasks.rados.rados.0.smithi118.stdout:3288: done (3 left) 2024-03-20T08:12:40.808 INFO:tasks.rados.rados.0.smithi118.stdout:3289: done (2 left) 2024-03-20T08:12:40.808 INFO:tasks.rados.rados.0.smithi118.stdout:3290: done (1 left) 2024-03-20T08:12:40.808 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 322 2024-03-20T08:12:40.808 INFO:tasks.rados.rados.0.smithi118.stdout:3291: seq_num 1004 ranges {1654784=16384} 2024-03-20T08:12:40.817 INFO:tasks.rados.rados.0.smithi118.stdout:3291: writing smithi11881747-45 from 1654784 to 1671168 tid 1 2024-03-20T08:12:40.817 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 322 2024-03-20T08:12:40.817 INFO:tasks.rados.rados.0.smithi118.stdout:3292: seq_num 1005 ranges {10526720=32768} 2024-03-20T08:12:40.875 INFO:tasks.rados.rados.0.smithi118.stdout:3292: writing smithi11881747-7 from 10526720 to 10559488 tid 1 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3293: setattr oid 17 current snap is 322 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3291: finishing write tid 1 to smithi11881747-45 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3291: finishing write tid 2 to smithi11881747-45 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3287: finishing copy_from racing read to smithi11881747-26 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3291: finishing write tid 3 to smithi11881747-45 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3291: oid 45 updating version 0 to 34082 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3291: oid 45 updating version 34082 to 34083 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 34083 (ObjNum 1004 snap 322 seq_num 1004) dirty exists 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3291: left oid 45 (ObjNum 1004 snap 322 seq_num 1004) 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:3287: finishing copy_from to smithi11881747-26 2024-03-20T08:12:40.876 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 59839 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:12:40.877 INFO:tasks.rados.rados.0.smithi118.stdout:3287: done (3 left) 2024-03-20T08:12:40.877 INFO:tasks.rados.rados.0.smithi118.stdout:3291: done (2 left) 2024-03-20T08:12:40.877 INFO:tasks.rados.rados.0.smithi118.stdout:3294: rollback oid 14 current snap is 322 2024-03-20T08:12:40.877 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 294 2024-03-20T08:12:40.877 INFO:tasks.rados.rados.0.smithi118.stdout:3295: setattr oid 23 current snap is 322 2024-03-20T08:12:40.877 INFO:tasks.rados.rados.0.smithi118.stdout:3296: snap_create 2024-03-20T08:12:40.879 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 45645 (ObjNum 975 snap 321 seq_num 975) dirty exists 2024-03-20T08:12:40.881 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 45644 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:12:40.883 INFO:tasks.rados.rados.0.smithi118.stdout:3292: finishing write tid 1 to smithi11881747-7 2024-03-20T08:12:40.884 INFO:tasks.rados.rados.0.smithi118.stdout:3292: finishing write tid 2 to smithi11881747-7 2024-03-20T08:12:40.884 INFO:tasks.rados.rados.0.smithi118.stdout:3294: finishing rollback tid 0 to smithi11881747-14 2024-03-20T08:12:40.887 INFO:tasks.rados.rados.0.smithi118.stdout:3294: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:12:40.888 INFO:tasks.rados.rados.0.smithi118.stdout:3294: finishing rollback tid 2 to smithi11881747-14 2024-03-20T08:12:40.888 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 34112 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:12:40.889 INFO:tasks.rados.rados.0.smithi118.stdout:3292: finishing write tid 3 to smithi11881747-7 2024-03-20T08:12:40.889 INFO:tasks.rados.rados.0.smithi118.stdout:3292: oid 7 updating version 0 to 65279 2024-03-20T08:12:40.889 INFO:tasks.rados.rados.0.smithi118.stdout:3292: oid 7 updating version 65279 to 65280 2024-03-20T08:12:40.889 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 65280 (ObjNum 1005 snap 322 seq_num 1005) dirty exists 2024-03-20T08:12:40.889 INFO:tasks.rados.rados.0.smithi118.stdout:3292: left oid 7 (ObjNum 1005 snap 322 seq_num 1005) 2024-03-20T08:12:41.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:40.799+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:41.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:40.806+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:41.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:40.808+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:41.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:40.802+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:41.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:40.803+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:41.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:40.803+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:41.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:40.801+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:41.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:40.801+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:41.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:40.801+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: osdmap e718: 8 total, 8 up, 8 in 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:41.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:41.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:40.806+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:41.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:40.807+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:41.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:40.797+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:41.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:40.797+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:41.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:40.797+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:41.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:40.797+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:41.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:40.797+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: osdmap e718: 8 total, 8 up, 8 in 2024-03-20T08:12:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:41.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: osdmap e718: 8 total, 8 up, 8 in 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:41.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:41.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:41.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:41.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:41.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:40.805+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:41.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:40.805+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:41.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:40.810+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:41.802 INFO:tasks.rados.rados.0.smithi118.stdout:3292: done (4 left) 2024-03-20T08:12:41.802 INFO:tasks.rados.rados.0.smithi118.stdout:3293: done (3 left) 2024-03-20T08:12:41.802 INFO:tasks.rados.rados.0.smithi118.stdout:3294: done (2 left) 2024-03-20T08:12:41.802 INFO:tasks.rados.rados.0.smithi118.stdout:3295: done (1 left) 2024-03-20T08:12:41.802 INFO:tasks.rados.rados.0.smithi118.stdout:3296: done (0 left) 2024-03-20T08:12:41.802 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 323 2024-03-20T08:12:41.802 INFO:tasks.rados.rados.0.smithi118.stdout:3297: seq_num 1006 ranges {1671168=49152} 2024-03-20T08:12:41.813 INFO:tasks.rados.rados.0.smithi118.stdout:3297: writing smithi11881747-45 from 1671168 to 1720320 tid 1 2024-03-20T08:12:41.813 INFO:tasks.rados.rados.0.smithi118.stdout:3298: read oid 41 snap 322 2024-03-20T08:12:41.813 INFO:tasks.rados.rados.0.smithi118.stdout:3298: expect (ObjNum 1000 snap 322 seq_num 1000) 2024-03-20T08:12:41.813 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 323 2024-03-20T08:12:41.813 INFO:tasks.rados.rados.0.smithi118.stdout:3299: seq_num 1007 ranges {3145728=65536} 2024-03-20T08:12:41.830 INFO:tasks.rados.rados.0.smithi118.stdout:3299: writing smithi11881747-35 from 3145728 to 3211264 tid 1 2024-03-20T08:12:41.830 INFO:tasks.rados.rados.0.smithi118.stdout:3300: delete oid 40 current snap is 323 2024-03-20T08:12:41.830 INFO:tasks.rados.rados.0.smithi118.stdout:3297: finishing write tid 1 to smithi11881747-45 2024-03-20T08:12:41.831 INFO:tasks.rados.rados.0.smithi118.stdout:3297: finishing write tid 2 to smithi11881747-45 2024-03-20T08:12:41.831 INFO:tasks.rados.rados.0.smithi118.stdout:3297: finishing write tid 3 to smithi11881747-45 2024-03-20T08:12:41.831 INFO:tasks.rados.rados.0.smithi118.stdout:3297: oid 45 updating version 0 to 34085 2024-03-20T08:12:41.831 INFO:tasks.rados.rados.0.smithi118.stdout:3297: oid 45 version 34085 is already newer than 34084 2024-03-20T08:12:41.831 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 34085 (ObjNum 1006 snap 323 seq_num 1006) dirty exists 2024-03-20T08:12:41.831 INFO:tasks.rados.rados.0.smithi118.stdout:3297: left oid 45 (ObjNum 1006 snap 323 seq_num 1006) 2024-03-20T08:12:41.837 INFO:tasks.rados.rados.0.smithi118.stdout:3299: finishing write tid 1 to smithi11881747-35 2024-03-20T08:12:41.837 INFO:tasks.rados.rados.0.smithi118.stdout:3299: finishing write tid 2 to smithi11881747-35 2024-03-20T08:12:41.837 INFO:tasks.rados.rados.0.smithi118.stdout:3297: done (3 left) 2024-03-20T08:12:41.838 INFO:tasks.rados.rados.0.smithi118.stdout:3300: done (2 left) 2024-03-20T08:12:41.838 INFO:tasks.rados.rados.0.smithi118.stdout:3301: copy_from oid 10 from oid 50 current snap is 323 2024-03-20T08:12:41.838 INFO:tasks.rados.rados.0.smithi118.stdout:3302: setattr oid 26 current snap is 323 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3299: finishing write tid 3 to smithi11881747-35 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3299: oid 35 updating version 0 to 45655 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3299: oid 35 updating version 45655 to 45656 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 45656 (ObjNum 1007 snap 323 seq_num 1007) dirty exists 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3299: left oid 35 (ObjNum 1007 snap 323 seq_num 1007) 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3299: done (3 left) 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3303: rollback oid 42 current snap is 323 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 294 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3304: rmattr oid 24 current snap is 323 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3305: rollback oid 27 current snap is 323 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 266 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3306: copy_from oid 5 from oid 47 current snap is 323 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:3307: rollback oid 2 current snap is 323 2024-03-20T08:12:41.840 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 213 2024-03-20T08:12:41.841 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 323 2024-03-20T08:12:41.841 INFO:tasks.rados.rados.0.smithi118.stdout:3308: seq_num 1009 ranges {5529600=106496} 2024-03-20T08:12:41.871 INFO:tasks.rados.rados.0.smithi118.stdout:3308: writing smithi11881747-18 from 5529600 to 5636096 tid 1 2024-03-20T08:12:41.871 INFO:tasks.rados.rados.0.smithi118.stdout:3307: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:12:41.871 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 45654 (ObjNum 65 snap 0 seq_num 1733253224) dirty dne 2024-03-20T08:12:41.871 INFO:tasks.rados.rados.0.smithi118.stdout:3298: expect (ObjNum 1000 snap 322 seq_num 1000) 2024-03-20T08:12:42.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:41.803+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:42.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:41.834+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:42.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:41.837+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:42.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:41.804+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:42.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:41.804+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:42.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:41.804+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:42.141 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:41.842+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:42.141 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:41.842+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:42.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: pgmap v761: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 25 op/s 2024-03-20T08:12:42.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:42.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:42.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:42.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:42.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: osdmap e719: 8 total, 8 up, 8 in 2024-03-20T08:12:42.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:42.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:42.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:42.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:42.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:42.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:42.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:42.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:42.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:42.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:42.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:42.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:41.819+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:42.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:41.819+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:42.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:41.819+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:41.801+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:41.802+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:41.802+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:41.802+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:42.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: pgmap v761: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 25 op/s 2024-03-20T08:12:42.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: osdmap e719: 8 total, 8 up, 8 in 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: pgmap v761: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 25 op/s 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17a=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: osdmap e719: 8 total, 8 up, 8 in 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:42.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:42.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:42.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:42.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:42.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:42.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:42.177 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:41.819+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:42.177 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:41.851+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:42.225 INFO:tasks.rados.rados.0.smithi118.stdout:3303: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:12:42.225 INFO:tasks.rados.rados.0.smithi118.stdout:3303: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:12:42.225 INFO:tasks.rados.rados.0.smithi118.stdout:3303: finishing rollback tid 2 to smithi11881747-42 2024-03-20T08:12:42.225 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65275 (ObjNum 825 snap 283 seq_num 825) dirty exists 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3305: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3309: read oid 28 snap 319 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3309: expect (ObjNum 900 snap 309 seq_num 900) 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3298: done (9 left) 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3303: done (8 left) 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3307: done (7 left) 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3310: read oid 48 snap -1 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3310: expect deleted 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:3311: delete oid 14 current snap is 323 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65288 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:12:42.226 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65290 (ObjNum 687 snap 225 seq_num 687) dirty exists 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3305: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3306: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3305: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 59847 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3301: finishing copy_from racing read to smithi11881747-10 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3306: finishing copy_from to smithi11881747-5 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 45624 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3301: finishing copy_from to smithi11881747-10 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 65272 (ObjNum 33 snap 0 seq_num 322624368) dirty exists 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3308: finishing write tid 1 to smithi11881747-18 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3308: finishing write tid 2 to smithi11881747-18 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3308: finishing write tid 3 to smithi11881747-18 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3308: oid 18 updating version 0 to 59848 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:3308: oid 18 updating version 59848 to 59849 2024-03-20T08:12:42.227 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 59849 (ObjNum 1009 snap 323 seq_num 1009) dirty exists 2024-03-20T08:12:42.228 INFO:tasks.rados.rados.0.smithi118.stdout:3308: left oid 18 (ObjNum 1009 snap 323 seq_num 1009) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3301: done (8 left) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3302: done (7 left) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3304: done (6 left) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3305: done (5 left) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3306: done (4 left) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3308: done (3 left) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3310: done (2 left) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3311: done (1 left) 2024-03-20T08:12:42.230 INFO:tasks.rados.rados.0.smithi118.stdout:3312: delete oid 33 current snap is 323 2024-03-20T08:12:42.238 INFO:tasks.rados.rados.0.smithi118.stdout:3312: done (1 left) 2024-03-20T08:12:42.238 INFO:tasks.rados.rados.0.smithi118.stdout:3313: read oid 26 snap -1 2024-03-20T08:12:42.239 INFO:tasks.rados.rados.0.smithi118.stdout:3313: expect (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:12:42.239 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 323 2024-03-20T08:12:42.239 INFO:tasks.rados.rados.0.smithi118.stdout:3314: seq_num 1012 ranges {2056192=155648} 2024-03-20T08:12:42.254 INFO:tasks.rados.rados.0.smithi118.stdout:3314: writing smithi11881747-3 from 2056192 to 2211840 tid 1 2024-03-20T08:12:42.254 INFO:tasks.rados.rados.0.smithi118.stdout:3315: snap_create 2024-03-20T08:12:42.255 INFO:tasks.rados.rados.0.smithi118.stdout:3309: expect (ObjNum 900 snap 309 seq_num 900) 2024-03-20T08:12:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:12:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:12:43.077 INFO:tasks.rados.rados.0.smithi118.stdout:3313: expect (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:12:43.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:42.807+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:43.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:42.807+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:43.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:42.807+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: osdmap e720: 8 total, 8 up, 8 in 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:43.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:43.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:42.808+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:43.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:42.808+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:43.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:42.809+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:43.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:42.809+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: osdmap e720: 8 total, 8 up, 8 in 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:43.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: osdmap e720: 8 total, 8 up, 8 in 2024-03-20T08:12:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:43.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:43.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:43.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:43.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:43.296 INFO:tasks.rados.rados.0.smithi118.stdout:3314: finishing write tid 1 to smithi11881747-3 2024-03-20T08:12:43.296 INFO:tasks.rados.rados.0.smithi118.stdout:3314: finishing write tid 2 to smithi11881747-3 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3314: finishing write tid 3 to smithi11881747-3 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3314: oid 3 updating version 0 to 34050 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3314: oid 3 version 34050 is already newer than 34049 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34050 (ObjNum 1012 snap 323 seq_num 1012) dirty exists 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3314: left oid 3 (ObjNum 1012 snap 323 seq_num 1012) 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3309: done (3 left) 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3313: done (2 left) 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3314: done (1 left) 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3315: done (0 left) 2024-03-20T08:12:43.297 INFO:tasks.rados.rados.0.smithi118.stdout:3316: delete oid 25 current snap is 324 2024-03-20T08:12:43.299 INFO:tasks.rados.rados.0.smithi118.stdout:3316: done (0 left) 2024-03-20T08:12:43.300 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 324 2024-03-20T08:12:43.300 INFO:tasks.rados.rados.0.smithi118.stdout:3317: seq_num 1014 ranges {1720320=188416} 2024-03-20T08:12:43.310 INFO:tasks.rados.rados.0.smithi118.stdout:3317: writing smithi11881747-45 from 1720320 to 1908736 tid 1 2024-03-20T08:12:43.311 INFO:tasks.rados.rados.0.smithi118.stdout:3318: read oid 17 snap -1 2024-03-20T08:12:43.311 INFO:tasks.rados.rados.0.smithi118.stdout:3318: expect (ObjNum 975 snap 321 seq_num 975) 2024-03-20T08:12:43.311 INFO:tasks.rados.rados.0.smithi118.stdout:3319: snap_create 2024-03-20T08:12:43.323 INFO:tasks.rados.rados.0.smithi118.stdout:3317: finishing write tid 1 to smithi11881747-45 2024-03-20T08:12:43.323 INFO:tasks.rados.rados.0.smithi118.stdout:3317: finishing write tid 2 to smithi11881747-45 2024-03-20T08:12:43.328 INFO:tasks.rados.rados.0.smithi118.stdout:3317: finishing write tid 3 to smithi11881747-45 2024-03-20T08:12:43.328 INFO:tasks.rados.rados.0.smithi118.stdout:3317: oid 45 updating version 0 to 34088 2024-03-20T08:12:43.328 INFO:tasks.rados.rados.0.smithi118.stdout:3317: oid 45 version 34088 is already newer than 34087 2024-03-20T08:12:43.328 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 34088 (ObjNum 1014 snap 324 seq_num 1014) dirty exists 2024-03-20T08:12:43.328 INFO:tasks.rados.rados.0.smithi118.stdout:3317: left oid 45 (ObjNum 1014 snap 324 seq_num 1014) 2024-03-20T08:12:43.371 INFO:tasks.rados.rados.0.smithi118.stdout:3318: expect (ObjNum 975 snap 321 seq_num 975) 2024-03-20T08:12:43.441 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:43.165+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:43.441 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:43.165+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:43.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:43.300+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:43.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:43.301+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:43.441 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:43.301+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:43.584 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:43.293+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:43.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.314+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:43.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.314+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:43.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.314+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:43.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.314+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:43.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.324+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:43.805 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:43.441+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:43.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:43.814+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:12:43] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:12:43.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.812+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:43.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.812+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:43.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.813+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:43.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:43.813+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:43.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: pgmap v764: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 541 KiB/s rd, 2.2 MiB/s wr, 9 op/s 2024-03-20T08:12:43.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: osdmap e721: 8 total, 8 up, 8 in 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: pgmap v764: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 541 KiB/s rd, 2.2 MiB/s wr, 9 op/s 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: osdmap e721: 8 total, 8 up, 8 in 2024-03-20T08:12:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:43.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:44.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:43.814+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: pgmap v764: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 541 KiB/s rd, 2.2 MiB/s wr, 9 op/s 2024-03-20T08:12:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: osdmap e721: 8 total, 8 up, 8 in 2024-03-20T08:12:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:44.295 INFO:tasks.rados.rados.0.smithi118.stdout:3317: done (2 left) 2024-03-20T08:12:44.295 INFO:tasks.rados.rados.0.smithi118.stdout:3318: done (1 left) 2024-03-20T08:12:44.295 INFO:tasks.rados.rados.0.smithi118.stdout:3319: done (0 left) 2024-03-20T08:12:44.296 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 325 2024-03-20T08:12:44.296 INFO:tasks.rados.rados.0.smithi118.stdout:3320: seq_num 1015 ranges {3170304=204800} 2024-03-20T08:12:44.314 INFO:tasks.rados.rados.0.smithi118.stdout:3320: writing smithi11881747-42 from 3170304 to 3375104 tid 1 2024-03-20T08:12:44.314 INFO:tasks.rados.rados.0.smithi118.stdout:3321: read oid 13 snap -1 2024-03-20T08:12:44.314 INFO:tasks.rados.rados.0.smithi118.stdout:3321: expect (ObjNum 1002 snap 322 seq_num 1002) 2024-03-20T08:12:44.314 INFO:tasks.rados.rados.0.smithi118.stdout:3322: rmattr oid 11 current snap is 325 2024-03-20T08:12:44.314 INFO:tasks.rados.rados.0.smithi118.stdout:3323: copy_from oid 4 from oid 23 current snap is 325 2024-03-20T08:12:44.314 INFO:tasks.rados.rados.0.smithi118.stdout:3324: read oid 44 snap -1 2024-03-20T08:12:44.315 INFO:tasks.rados.rados.0.smithi118.stdout:3324: expect (ObjNum 990 snap 321 seq_num 990) 2024-03-20T08:12:44.315 INFO:tasks.rados.rados.0.smithi118.stdout:3325: setattr oid 48 current snap is 325 2024-03-20T08:12:44.317 INFO:tasks.rados.rados.0.smithi118.stdout:3326: read oid 22 snap -1 2024-03-20T08:12:44.318 INFO:tasks.rados.rados.0.smithi118.stdout:3326: expect (ObjNum 971 snap 321 seq_num 971) 2024-03-20T08:12:44.318 INFO:tasks.rados.rados.0.smithi118.stdout:3327: snap_remove snap 324 2024-03-20T08:12:44.642 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:44.332+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:44.642 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:44.333+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:44.643 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:44.318+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:44.643 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:44.321+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:44.643 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:44.321+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:44.643 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:44.322+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:44.643 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:44.322+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:44.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:44.319+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:44.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:44.317+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:44.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:44.317+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:44.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:44.335+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:3327: done (7 left) 2024-03-20T08:12:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:3328: read oid 24 snap -1 2024-03-20T08:12:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:3328: expect (ObjNum 687 snap 225 seq_num 687) 2024-03-20T08:12:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:3329: snap_create 2024-03-20T08:12:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:3320: finishing write tid 1 to smithi11881747-42 2024-03-20T08:12:44.820 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 61737 (ObjNum 129 snap 0 seq_num 321896808) dirty exists 2024-03-20T08:12:44.821 INFO:tasks.rados.rados.0.smithi118.stdout:3324: expect (ObjNum 990 snap 321 seq_num 990) 2024-03-20T08:12:44.914 INFO:tasks.rados.rados.0.smithi118.stdout:3323: finishing copy_from racing read to smithi11881747-4 2024-03-20T08:12:44.914 INFO:tasks.rados.rados.0.smithi118.stdout:3321: expect (ObjNum 1002 snap 322 seq_num 1002) 2024-03-20T08:12:45.087 INFO:tasks.rados.rados.0.smithi118.stdout:3320: finishing write tid 2 to smithi11881747-42 2024-03-20T08:12:45.087 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 65279 (ObjNum 969 snap 321 seq_num 969) dirty exists 2024-03-20T08:12:45.087 INFO:tasks.rados.rados.0.smithi118.stdout:3320: finishing write tid 3 to smithi11881747-42 2024-03-20T08:12:45.087 INFO:tasks.rados.rados.0.smithi118.stdout:3320: oid 42 updating version 0 to 65277 2024-03-20T08:12:45.087 INFO:tasks.rados.rados.0.smithi118.stdout:3320: oid 42 updating version 65277 to 65278 2024-03-20T08:12:45.087 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65278 (ObjNum 1015 snap 325 seq_num 1015) dirty exists 2024-03-20T08:12:45.087 INFO:tasks.rados.rados.0.smithi118.stdout:3320: left oid 42 (ObjNum 1015 snap 325 seq_num 1015) 2024-03-20T08:12:45.087 INFO:tasks.rados.rados.0.smithi118.stdout:3323: finishing copy_from to smithi11881747-4 2024-03-20T08:12:45.088 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 45644 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:12:45.088 INFO:tasks.rados.rados.0.smithi118.stdout:3326: expect (ObjNum 971 snap 321 seq_num 971) 2024-03-20T08:12:45.132 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:44.826+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:45.132 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:44.826+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:45.133 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:44.823+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:45.133 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:44.823+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:45.133 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:44.826+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:45.133 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:44.818+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:45.133 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:44.818+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:45.133 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:44.818+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:45.134 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: osdmap e722: 8 total, 8 up, 8 in 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:45.135 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:45.136 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: osdmap e722: 8 total, 8 up, 8 in 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:45.137 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: osdmap e722: 8 total, 8 up, 8 in 2024-03-20T08:12:45.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:45.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:45.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:45.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:45.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:45.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:45.138 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:45.138 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:45.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:45.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:45.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:45.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:45.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:45.139 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:45.139 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:44.817+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:45.139 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:44.819+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:45.139 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:44.819+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:45.235 INFO:tasks.rados.rados.0.smithi118.stdout:3328: expect (ObjNum 687 snap 225 seq_num 687) 2024-03-20T08:12:45.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:45.132+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:45.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:45.133+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:45.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:45.133+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:45.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.132+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:45.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.132+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:45.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.132+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:45.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.133+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:45.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.133+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:45.820 INFO:tasks.rados.rados.0.smithi118.stdout:3320: done (8 left) 2024-03-20T08:12:45.820 INFO:tasks.rados.rados.0.smithi118.stdout:3321: done (7 left) 2024-03-20T08:12:45.820 INFO:tasks.rados.rados.0.smithi118.stdout:3322: done (6 left) 2024-03-20T08:12:45.820 INFO:tasks.rados.rados.0.smithi118.stdout:3323: done (5 left) 2024-03-20T08:12:45.820 INFO:tasks.rados.rados.0.smithi118.stdout:3324: done (4 left) 2024-03-20T08:12:45.820 INFO:tasks.rados.rados.0.smithi118.stdout:3325: done (3 left) 2024-03-20T08:12:45.820 INFO:tasks.rados.rados.0.smithi118.stdout:3326: done (2 left) 2024-03-20T08:12:45.821 INFO:tasks.rados.rados.0.smithi118.stdout:3328: done (1 left) 2024-03-20T08:12:45.821 INFO:tasks.rados.rados.0.smithi118.stdout:3329: done (0 left) 2024-03-20T08:12:45.821 INFO:tasks.rados.rados.0.smithi118.stdout:3330: copy_from oid 23 from oid 22 current snap is 326 2024-03-20T08:12:45.821 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 326 2024-03-20T08:12:45.821 INFO:tasks.rados.rados.0.smithi118.stdout:3331: seq_num 1016 ranges {0=221184} 2024-03-20T08:12:45.822 INFO:tasks.rados.rados.0.smithi118.stdout:3331: writing smithi11881747-33 from 0 to 221184 tid 1 2024-03-20T08:12:45.822 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 326 2024-03-20T08:12:45.822 INFO:tasks.rados.rados.0.smithi118.stdout:3332: seq_num 1017 ranges {3235840=237568} 2024-03-20T08:12:45.840 INFO:tasks.rados.rados.0.smithi118.stdout:3332: writing smithi11881747-13 from 3235840 to 3473408 tid 1 2024-03-20T08:12:45.840 INFO:tasks.rados.rados.0.smithi118.stdout:3333: rmattr oid 16 current snap is 326 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:3333: done (3 left) 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:3334: snap_create 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:3331: finishing write tid 1 to smithi11881747-33 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:3331: finishing write tid 2 to smithi11881747-33 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:3331: finishing write tid 3 to smithi11881747-33 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:3331: oid 33 updating version 0 to 34051 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:3331: oid 33 updating version 34051 to 34052 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34052 (ObjNum 1016 snap 326 seq_num 1016) dirty exists 2024-03-20T08:12:45.841 INFO:tasks.rados.rados.0.smithi118.stdout:3331: left oid 33 (ObjNum 1016 snap 326 seq_num 1016) 2024-03-20T08:12:45.845 INFO:tasks.rados.rados.0.smithi118.stdout:3330: finishing copy_from racing read to smithi11881747-23 2024-03-20T08:12:45.845 INFO:tasks.rados.rados.0.smithi118.stdout:3332: finishing write tid 1 to smithi11881747-13 2024-03-20T08:12:45.846 INFO:tasks.rados.rados.0.smithi118.stdout:3332: finishing write tid 2 to smithi11881747-13 2024-03-20T08:12:45.847 INFO:tasks.rados.rados.0.smithi118.stdout:3330: finishing copy_from to smithi11881747-23 2024-03-20T08:12:45.847 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 61732 (ObjNum 971 snap 321 seq_num 971) dirty exists 2024-03-20T08:12:45.848 INFO:tasks.rados.rados.0.smithi118.stdout:3332: finishing write tid 3 to smithi11881747-13 2024-03-20T08:12:45.848 INFO:tasks.rados.rados.0.smithi118.stdout:3332: oid 13 updating version 0 to 65281 2024-03-20T08:12:45.848 INFO:tasks.rados.rados.0.smithi118.stdout:3332: oid 13 version 65281 is already newer than 65280 2024-03-20T08:12:45.848 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 65281 (ObjNum 1017 snap 326 seq_num 1017) dirty exists 2024-03-20T08:12:45.848 INFO:tasks.rados.rados.0.smithi118.stdout:3332: left oid 13 (ObjNum 1017 snap 326 seq_num 1017) 2024-03-20T08:12:46.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:45.829+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:46.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:45.819+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:46.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:45.819+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:46.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:45.829+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:46.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:45.829+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:46.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:45.844+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:46.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:45.844+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:46.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:45.844+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: pgmap v767: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:46.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: osdmap e723: 8 total, 8 up, 8 in 2024-03-20T08:12:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:46.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:45.819+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:46.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:45.819+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:46.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:45.819+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:46.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.823+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.824+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.824+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:45.834+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:46.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:46.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:46.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:46.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:46.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: pgmap v767: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: osdmap e723: 8 total, 8 up, 8 in 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: pgmap v767: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: osdmap e723: 8 total, 8 up, 8 in 2024-03-20T08:12:46.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:46.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:46.177 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:45.828+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:46.177 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:45.820+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:46.830 INFO:tasks.rados.rados.0.smithi118.stdout:3330: done (3 left) 2024-03-20T08:12:46.831 INFO:tasks.rados.rados.0.smithi118.stdout:3331: done (2 left) 2024-03-20T08:12:46.831 INFO:tasks.rados.rados.0.smithi118.stdout:3332: done (1 left) 2024-03-20T08:12:46.831 INFO:tasks.rados.rados.0.smithi118.stdout:3334: done (0 left) 2024-03-20T08:12:46.831 INFO:tasks.rados.rados.0.smithi118.stdout:3335: delete oid 7 current snap is 327 2024-03-20T08:12:46.837 INFO:tasks.rados.rados.0.smithi118.stdout:3335: done (0 left) 2024-03-20T08:12:46.837 INFO:tasks.rados.rados.0.smithi118.stdout:3336: snap_remove snap 323 2024-03-20T08:12:47.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:46.836+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:47.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:46.829+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:47.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:46.829+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:47.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:46.829+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:47.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:46.834+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:47.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:46.834+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:47.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:46.834+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:47.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:46.834+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:47.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:46.834+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: osdmap e724: 8 total, 8 up, 8 in 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:47.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:46.834+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:47.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:46.834+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:47.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:46.834+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:47.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:46.829+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:47.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:46.830+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:46.828+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:46.829+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:46.831+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:46.831+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:46.831+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: osdmap e724: 8 total, 8 up, 8 in 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:47.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: osdmap e724: 8 total, 8 up, 8 in 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:47.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3336: done (0 left) 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3337: read oid 20 snap -1 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3337: expect (ObjNum 880 snap 305 seq_num 880) 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3338: read oid 12 snap -1 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3338: expect (ObjNum 999 snap 322 seq_num 999) 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3339: read oid 6 snap -1 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3339: expect (ObjNum 933 snap 317 seq_num 933) 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3340: read oid 17 snap -1 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3340: expect (ObjNum 975 snap 321 seq_num 975) 2024-03-20T08:12:47.837 INFO:tasks.rados.rados.0.smithi118.stdout:3341: snap_remove snap 294 2024-03-20T08:12:48.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:47.837+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:48.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:47.932+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:48.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:47.932+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:48.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:47.932+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:48.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:47.837+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:48.141 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:47.837+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:48.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:47.877+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:48.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:47.880+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:48.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: pgmap v770: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 27 MiB/s rd, 2.1 MiB/s wr, 39 op/s 2024-03-20T08:12:48.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:48.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:48.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:48.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:48.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:48.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: osdmap e725: 8 total, 8 up, 8 in 2024-03-20T08:12:48.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:48.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:48.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:48.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:48.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:48.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:48.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:48.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:48.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:48.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:48.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:47.842+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:48.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:47.933+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:48.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:47.933+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:48.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:47.933+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:48.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:47.842+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:48.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:47.835+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:48.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:47.836+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:48.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:47.836+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:48.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:47.837+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:48.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: pgmap v770: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 27 MiB/s rd, 2.1 MiB/s wr, 39 op/s 2024-03-20T08:12:48.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:48.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:48.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:48.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:48.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:48.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: osdmap e725: 8 total, 8 up, 8 in 2024-03-20T08:12:48.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: pgmap v770: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 27 MiB/s rd, 2.1 MiB/s wr, 39 op/s 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:48.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: osdmap e725: 8 total, 8 up, 8 in 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:48.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:48.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:48.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:48.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:48.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:48.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:48.843 INFO:tasks.rados.rados.0.smithi118.stdout:3341: done (4 left) 2024-03-20T08:12:48.843 INFO:tasks.rados.rados.0.smithi118.stdout:3342: read oid 50 snap 213 2024-03-20T08:12:48.843 INFO:tasks.rados.rados.0.smithi118.stdout:3342: expect deleted 2024-03-20T08:12:48.843 INFO:tasks.rados.rados.0.smithi118.stdout:3343: read oid 21 snap -1 2024-03-20T08:12:48.843 INFO:tasks.rados.rados.0.smithi118.stdout:3343: expect deleted 2024-03-20T08:12:48.843 INFO:tasks.rados.rados.0.smithi118.stdout:3344: snap_remove snap 326 2024-03-20T08:12:49.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:48.843+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:49.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:48.848+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:49.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:48.848+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:49.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:48.848+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:49.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:48.934+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:49.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:48.934+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:49.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:48.932+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:49.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:48.933+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:49.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:48.934+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: osdmap e726: 8 total, 8 up, 8 in 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:49.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:49.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:49.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:49.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:49.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:49.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:49.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:48 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:49.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:48.841+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:49.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:48.841+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:49.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:48.932+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:49.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:48.843+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:49.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:48.843+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:49.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:49.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: osdmap e726: 8 total, 8 up, 8 in 2024-03-20T08:12:49.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:49.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:49.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:49.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:49.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:49.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:49.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:49.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:49.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:49.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:48.932+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:49.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:48.932+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:49.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:48.932+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:49.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:48.933+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:49.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:48.933+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:49.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:49.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:49.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:49.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:49.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: osdmap e726: 8 total, 8 up, 8 in 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:49.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:49.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:49.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:49.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:48 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:49.843 INFO:tasks.rados.rados.0.smithi118.stdout:3344: done (6 left) 2024-03-20T08:12:49.843 INFO:tasks.rados.rados.0.smithi118.stdout:3345: rmattr oid 47 current snap is 327 2024-03-20T08:12:49.843 INFO:tasks.rados.rados.0.smithi118.stdout:3337: expect (ObjNum 880 snap 305 seq_num 880) 2024-03-20T08:12:49.855 INFO:tasks.rados.rados.0.smithi118.stdout:3338: expect (ObjNum 999 snap 322 seq_num 999) 2024-03-20T08:12:50.063 INFO:tasks.rados.rados.0.smithi118.stdout:3340: expect (ObjNum 975 snap 321 seq_num 975) 2024-03-20T08:12:50.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:49.845+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:50.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:49.851+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:50.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:49.851+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:50.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:49.851+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:50.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: pgmap v773: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 31 op/s 2024-03-20T08:12:50.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:50.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: osdmap e727: 8 total, 8 up, 8 in 2024-03-20T08:12:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:50.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:49.842+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:50.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:49.842+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:50.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:49.843+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:50.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:49.845+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:50.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:49.846+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:50.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:49.847+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:50.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:49.847+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:50.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:49.849+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: pgmap v773: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 31 op/s 2024-03-20T08:12:50.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: osdmap e727: 8 total, 8 up, 8 in 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: pgmap v773: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 31 op/s 2024-03-20T08:12:50.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: osdmap e727: 8 total, 8 up, 8 in 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:50.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:50.633 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:50.234+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:50.633 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:50.234+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:50.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:50.633+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:50.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:50.633+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:50.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:50.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:50.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:50.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:50.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:50.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: osdmap e728: 8 total, 8 up, 8 in 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:50.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:50.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:50.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:50.632+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:50.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:50.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:50.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:50.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:50.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: osdmap e728: 8 total, 8 up, 8 in 2024-03-20T08:12:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:50.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:50.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:50.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:50.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: osdmap e728: 8 total, 8 up, 8 in 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:50.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:50.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:50.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:50.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:50.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:50.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:50.967 INFO:tasks.rados.rados.0.smithi118.stdout:3339: expect (ObjNum 933 snap 317 seq_num 933) 2024-03-20T08:12:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3337: done (6 left) 2024-03-20T08:12:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3338: done (5 left) 2024-03-20T08:12:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3339: done (4 left) 2024-03-20T08:12:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3340: done (3 left) 2024-03-20T08:12:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3342: done (2 left) 2024-03-20T08:12:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3343: done (1 left) 2024-03-20T08:12:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3346: snap_create 2024-03-20T08:12:52.042 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 61733 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:12:52.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:52.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:52.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:51 smithi196 ceph-mon[87080]: pgmap v776: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 22 MiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-03-20T08:12:52.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:52.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:52.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:52.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:52.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:52.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[95692]: pgmap v776: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 22 MiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-03-20T08:12:52.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:52.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:52.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:52.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:52.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[93121]: pgmap v776: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 22 MiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-03-20T08:12:52.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:52.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:52.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:12:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:12:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3345: done (1 left) 2024-03-20T08:12:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3346: done (0 left) 2024-03-20T08:12:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3347: delete oid 43 current snap is 328 2024-03-20T08:12:52.867 INFO:tasks.rados.rados.0.smithi118.stdout:3347: done (0 left) 2024-03-20T08:12:52.867 INFO:tasks.rados.rados.0.smithi118.stdout:3348: snap_create 2024-03-20T08:12:53.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:52.867+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:53.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:52.864+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:53.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:52.866+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:53.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:52.860+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:53.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:52.860+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:53.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:52.860+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:53.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:52.867+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:53.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:52.867+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:53.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:52.867+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:53.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:52.860+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:53.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:52.860+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:53.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:52.860+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:53.862 INFO:tasks.rados.rados.0.smithi118.stdout:3348: done (0 left) 2024-03-20T08:12:53.862 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 329 2024-03-20T08:12:53.862 INFO:tasks.rados.rados.0.smithi118.stdout:3349: seq_num 1020 ranges {221184=286720} 2024-03-20T08:12:53.862 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:53.524+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:53.862 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:53.525+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:53.863 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:12:53] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:12:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.523+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.524+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.524+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.525+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:53.863 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.525+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:53.865 INFO:tasks.rados.rados.0.smithi118.stdout:3349: writing smithi11881747-33 from 221184 to 507904 tid 1 2024-03-20T08:12:53.865 INFO:tasks.rados.rados.0.smithi118.stdout:3350: copy_from oid 10 from oid 40 current snap is 329 2024-03-20T08:12:53.865 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 329 2024-03-20T08:12:53.865 INFO:tasks.rados.rados.0.smithi118.stdout:3351: seq_num 1021 ranges {5636096=303104} 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3351: writing smithi11881747-18 from 5636096 to 5939200 tid 1 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3352: setattr oid 34 current snap is 329 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3350: finishing copy_from to smithi11881747-10 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3350: got expected ENOENT (src dne) 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3350: finishing copy_from racing read to smithi11881747-10 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3349: finishing write tid 1 to smithi11881747-33 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3349: finishing write tid 2 to smithi11881747-33 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3349: finishing write tid 3 to smithi11881747-33 2024-03-20T08:12:53.899 INFO:tasks.rados.rados.0.smithi118.stdout:3349: oid 33 updating version 0 to 34053 2024-03-20T08:12:53.900 INFO:tasks.rados.rados.0.smithi118.stdout:3349: oid 33 updating version 34053 to 34054 2024-03-20T08:12:53.900 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34054 (ObjNum 1020 snap 329 seq_num 1020) dirty exists 2024-03-20T08:12:53.900 INFO:tasks.rados.rados.0.smithi118.stdout:3349: left oid 33 (ObjNum 1020 snap 329 seq_num 1020) 2024-03-20T08:12:53.900 INFO:tasks.rados.rados.0.smithi118.stdout:3349: done (3 left) 2024-03-20T08:12:53.900 INFO:tasks.rados.rados.0.smithi118.stdout:3350: done (2 left) 2024-03-20T08:12:53.900 INFO:tasks.rados.rados.0.smithi118.stdout:3353: snap_remove snap 319 2024-03-20T08:12:54.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:53.904+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:54.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:53.903+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:54.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:53.903+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:54.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:53.903+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:54.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:53.871+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:54.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:53.871+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:54.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:53.862+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:54.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:53.862+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:54.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: pgmap v777: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 716 KiB/s wr, 8 op/s 2024-03-20T08:12:54.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: osdmap e729: 8 total, 8 up, 8 in 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:54.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:53.862+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:53.862+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:53.862+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:53.862+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:54.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:53.873+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.869+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.869+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.869+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:53.879+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: pgmap v777: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 716 KiB/s wr, 8 op/s 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: osdmap e729: 8 total, 8 up, 8 in 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:54.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: pgmap v777: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 716 KiB/s wr, 8 op/s 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: osdmap e729: 8 total, 8 up, 8 in 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:54.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:54.877 INFO:tasks.rados.rados.0.smithi118.stdout:3353: done (2 left) 2024-03-20T08:12:54.877 INFO:tasks.rados.rados.0.smithi118.stdout:3354: snap_remove snap 322 2024-03-20T08:12:55.137 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:54.873+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:55.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:55.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:55.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:55.137 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: osdmap e730: 8 total, 8 up, 8 in 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:55.138 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:55.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:55.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:55.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:55.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:55.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:54.872+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:55.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:54.873+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:55.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:54.874+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:55.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:54.874+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:55.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:54.874+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:55.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:54.874+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:55.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:54.875+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:55.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:55.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:55.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:55.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: osdmap e730: 8 total, 8 up, 8 in 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:55.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: osdmap e730: 8 total, 8 up, 8 in 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:55.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:55.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:55.137+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:55.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:55.137+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:55.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:55.137+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:55.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:55.177+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:55.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:55.177+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:55.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:55.233+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:55.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:55.233+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:55.877 INFO:tasks.rados.rados.0.smithi118.stdout:3354: done (2 left) 2024-03-20T08:12:55.877 INFO:tasks.rados.rados.0.smithi118.stdout:3355: snap_remove snap 325 2024-03-20T08:12:56.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:55.877+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: pgmap v780: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 0 B/s wr, 5 op/s 2024-03-20T08:12:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:56.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:56.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:56.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:56.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:56.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: osdmap e731: 8 total, 8 up, 8 in 2024-03-20T08:12:56.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:56.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:56.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:56.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:56.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:55.878+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:56.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:55.877+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:56.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:55.877+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:56.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:55.878+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: pgmap v780: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 0 B/s wr, 5 op/s 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: osdmap e731: 8 total, 8 up, 8 in 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:56.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: pgmap v780: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 0 B/s wr, 5 op/s 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: osdmap e731: 8 total, 8 up, 8 in 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:56.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:56.878 INFO:tasks.rados.rados.0.smithi118.stdout:3355: done (2 left) 2024-03-20T08:12:56.878 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 329 2024-03-20T08:12:56.878 INFO:tasks.rados.rados.0.smithi118.stdout:3356: seq_num 1022 ranges {0=319488} 2024-03-20T08:12:56.881 INFO:tasks.rados.rados.0.smithi118.stdout:3356: writing smithi11881747-49 from 0 to 319488 tid 1 2024-03-20T08:12:56.881 INFO:tasks.rados.rados.0.smithi118.stdout:3357: snap_create 2024-03-20T08:12:56.881 INFO:tasks.rados.rados.0.smithi118.stdout:3351: finishing write tid 1 to smithi11881747-18 2024-03-20T08:12:56.881 INFO:tasks.rados.rados.0.smithi118.stdout:3351: finishing write tid 2 to smithi11881747-18 2024-03-20T08:12:56.882 INFO:tasks.rados.rados.0.smithi118.stdout:3351: finishing write tid 3 to smithi11881747-18 2024-03-20T08:12:56.882 INFO:tasks.rados.rados.0.smithi118.stdout:3351: oid 18 updating version 0 to 59851 2024-03-20T08:12:56.883 INFO:tasks.rados.rados.0.smithi118.stdout:3351: oid 18 version 59851 is already newer than 59850 2024-03-20T08:12:56.883 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 59851 (ObjNum 1021 snap 329 seq_num 1021) dirty exists 2024-03-20T08:12:56.883 INFO:tasks.rados.rados.0.smithi118.stdout:3351: left oid 18 (ObjNum 1021 snap 329 seq_num 1021) 2024-03-20T08:12:56.883 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45612 (ObjNum 985 snap 321 seq_num 985) dirty exists 2024-03-20T08:12:56.886 INFO:tasks.rados.rados.0.smithi118.stdout:3356: finishing write tid 1 to smithi11881747-49 2024-03-20T08:12:56.888 INFO:tasks.rados.rados.0.smithi118.stdout:3356: finishing write tid 2 to smithi11881747-49 2024-03-20T08:12:56.889 INFO:tasks.rados.rados.0.smithi118.stdout:3356: finishing write tid 3 to smithi11881747-49 2024-03-20T08:12:56.889 INFO:tasks.rados.rados.0.smithi118.stdout:3356: oid 49 updating version 0 to 59852 2024-03-20T08:12:56.889 INFO:tasks.rados.rados.0.smithi118.stdout:3356: oid 49 updating version 59852 to 59853 2024-03-20T08:12:56.889 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 59853 (ObjNum 1022 snap 329 seq_num 1022) dirty exists 2024-03-20T08:12:56.890 INFO:tasks.rados.rados.0.smithi118.stdout:3356: left oid 49 (ObjNum 1022 snap 329 seq_num 1022) 2024-03-20T08:12:57.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:56.877+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:57.069 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:56.877+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:57.069 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:56.733+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:57.070 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:56.887+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:57.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:57.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:57.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[93121]: osdmap e732: 8 total, 8 up, 8 in 2024-03-20T08:12:57.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:57.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:57.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:57.070 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:57.071 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:56.733+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:57.071 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:56.733+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:57.071 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:56.733+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:57.071 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:56.733+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:57.071 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.068+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:57.071 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:57.071 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:57.071 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[95692]: osdmap e732: 8 total, 8 up, 8 in 2024-03-20T08:12:57.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:57.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:57.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:57.072 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:57.072 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:56.887+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:57.072 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:56.887+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:57.072 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:56.879+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:57.073 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:57.073 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:57.073 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-mon[87080]: osdmap e732: 8 total, 8 up, 8 in 2024-03-20T08:12:57.073 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:57.073 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:57.074 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:57.074 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:57.074 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:56.735+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:57.074 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:56.737+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:57.074 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:56.737+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:56.735+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:56.735+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:56.736+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:56.888+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:56.888+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:56.888+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:56.737+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:56.737+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:56.880+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:57.075 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:56.880+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:57.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:57.069+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:57.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:57.069+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:57.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.068+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:57.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.068+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:57.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.068+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:57.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.068+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:3351: done (3 left) 2024-03-20T08:12:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:3352: done (2 left) 2024-03-20T08:12:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:3356: done (1 left) 2024-03-20T08:12:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:3357: done (0 left) 2024-03-20T08:12:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:3358: rmattr oid 43 current snap is 330 2024-03-20T08:12:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:3358: done (0 left) 2024-03-20T08:12:57.885 INFO:tasks.rados.rados.0.smithi118.stdout:3359: read oid 26 snap 329 2024-03-20T08:12:57.886 INFO:tasks.rados.rados.0.smithi118.stdout:3359: expect (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:12:57.886 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 330 2024-03-20T08:12:57.886 INFO:tasks.rados.rados.0.smithi118.stdout:3360: seq_num 1023 ranges {4202496=335872} 2024-03-20T08:12:57.911 INFO:tasks.rados.rados.0.smithi118.stdout:3360: writing smithi11881747-12 from 4202496 to 4538368 tid 1 2024-03-20T08:12:57.911 INFO:tasks.rados.rados.0.smithi118.stdout:3361: read oid 50 snap -1 2024-03-20T08:12:57.911 INFO:tasks.rados.rados.0.smithi118.stdout:3361: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:12:57.911 INFO:tasks.rados.rados.0.smithi118.stdout:3362: read oid 35 snap -1 2024-03-20T08:12:57.911 INFO:tasks.rados.rados.0.smithi118.stdout:3362: expect (ObjNum 1007 snap 323 seq_num 1007) 2024-03-20T08:12:57.911 INFO:tasks.rados.rados.0.smithi118.stdout:3363: delete oid 22 current snap is 330 2024-03-20T08:12:57.914 INFO:tasks.rados.rados.0.smithi118.stdout:3359: expect (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:12:58.135 INFO:tasks.rados.rados.0.smithi118.stdout:3361: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:12:58.136 INFO:tasks.rados.rados.0.smithi118.stdout:3360: finishing write tid 1 to smithi11881747-12 2024-03-20T08:12:58.136 INFO:tasks.rados.rados.0.smithi118.stdout:3360: finishing write tid 2 to smithi11881747-12 2024-03-20T08:12:58.136 INFO:tasks.rados.rados.0.smithi118.stdout:3360: finishing write tid 3 to smithi11881747-12 2024-03-20T08:12:58.136 INFO:tasks.rados.rados.0.smithi118.stdout:3360: oid 12 updating version 0 to 65282 2024-03-20T08:12:58.136 INFO:tasks.rados.rados.0.smithi118.stdout:3360: oid 12 updating version 65282 to 65283 2024-03-20T08:12:58.136 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65283 (ObjNum 1023 snap 330 seq_num 1023) dirty exists 2024-03-20T08:12:58.136 INFO:tasks.rados.rados.0.smithi118.stdout:3360: left oid 12 (ObjNum 1023 snap 330 seq_num 1023) 2024-03-20T08:12:58.137 INFO:tasks.rados.rados.0.smithi118.stdout:3362: expect (ObjNum 1007 snap 323 seq_num 1007) 2024-03-20T08:12:58.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:57.884+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:58.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:57.915+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:12:58.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:57.915+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:58.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:57.922+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:58.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:57.888+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:58.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:57.897+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:58.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:57.897+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:58.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:57.897+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:58.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:57.897+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: pgmap v783: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 74 KiB/s wr, 10 op/s 2024-03-20T08:12:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: osdmap e733: 8 total, 8 up, 8 in 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:58.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:58.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:58.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:58.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:58.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:57.915+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:12:58.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:57.915+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:58.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:57.915+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.883+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.883+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.883+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:57.884+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:58.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:58.124+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:58.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:57.884+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: pgmap v783: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 74 KiB/s wr, 10 op/s 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:58.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: pgmap v783: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 3.0 MiB/s rd, 74 KiB/s wr, 10 op/s 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:58.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: osdmap e733: 8 total, 8 up, 8 in 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: osdmap e733: 8 total, 8 up, 8 in 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:58.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:58.331 INFO:tasks.rados.rados.0.smithi118.stdout:3359: done (4 left) 2024-03-20T08:12:58.331 INFO:tasks.rados.rados.0.smithi118.stdout:3360: done (3 left) 2024-03-20T08:12:58.331 INFO:tasks.rados.rados.0.smithi118.stdout:3361: done (2 left) 2024-03-20T08:12:58.331 INFO:tasks.rados.rados.0.smithi118.stdout:3362: done (1 left) 2024-03-20T08:12:58.331 INFO:tasks.rados.rados.0.smithi118.stdout:3363: done (0 left) 2024-03-20T08:12:58.331 INFO:tasks.rados.rados.0.smithi118.stdout:3364: read oid 12 snap -1 2024-03-20T08:12:58.331 INFO:tasks.rados.rados.0.smithi118.stdout:3364: expect (ObjNum 1023 snap 330 seq_num 1023) 2024-03-20T08:12:58.331 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 330 2024-03-20T08:12:58.332 INFO:tasks.rados.rados.0.smithi118.stdout:3365: seq_num 1025 ranges {10944512=368640} 2024-03-20T08:12:58.395 INFO:tasks.rados.rados.0.smithi118.stdout:3365: writing smithi11881747-38 from 10944512 to 11313152 tid 1 2024-03-20T08:12:58.395 INFO:tasks.rados.rados.0.smithi118.stdout:3366: delete oid 15 current snap is 330 2024-03-20T08:12:58.395 INFO:tasks.rados.rados.0.smithi118.stdout:3364: expect (ObjNum 1023 snap 330 seq_num 1023) 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3365: finishing write tid 1 to smithi11881747-38 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3365: finishing write tid 2 to smithi11881747-38 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3365: finishing write tid 3 to smithi11881747-38 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3365: oid 38 updating version 0 to 61742 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3365: oid 38 version 61742 is already newer than 61741 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 61742 (ObjNum 1025 snap 330 seq_num 1025) dirty exists 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3365: left oid 38 (ObjNum 1025 snap 330 seq_num 1025) 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3364: done (2 left) 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3365: done (1 left) 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3366: done (0 left) 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 330 2024-03-20T08:12:58.616 INFO:tasks.rados.rados.0.smithi118.stdout:3367: seq_num 1027 ranges {3473408=409600} 2024-03-20T08:12:58.637 INFO:tasks.rados.rados.0.smithi118.stdout:3367: writing smithi11881747-13 from 3473408 to 3883008 tid 1 2024-03-20T08:12:58.637 INFO:tasks.rados.rados.0.smithi118.stdout:3368: snap_create 2024-03-20T08:12:58.641 INFO:tasks.rados.rados.0.smithi118.stdout:3367: finishing write tid 1 to smithi11881747-13 2024-03-20T08:12:58.641 INFO:tasks.rados.rados.0.smithi118.stdout:3367: finishing write tid 2 to smithi11881747-13 2024-03-20T08:12:58.642 INFO:tasks.rados.rados.0.smithi118.stdout:3367: finishing write tid 3 to smithi11881747-13 2024-03-20T08:12:58.642 INFO:tasks.rados.rados.0.smithi118.stdout:3367: oid 13 updating version 0 to 65282 2024-03-20T08:12:58.642 INFO:tasks.rados.rados.0.smithi118.stdout:3367: oid 13 updating version 65282 to 65283 2024-03-20T08:12:58.642 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 65283 (ObjNum 1027 snap 330 seq_num 1027) dirty exists 2024-03-20T08:12:58.643 INFO:tasks.rados.rados.0.smithi118.stdout:3367: left oid 13 (ObjNum 1027 snap 330 seq_num 1027) 2024-03-20T08:12:58.896 INFO:tasks.rados.rados.0.smithi118.stdout:3367: done (1 left) 2024-03-20T08:12:58.896 INFO:tasks.rados.rados.0.smithi118.stdout:3368: done (0 left) 2024-03-20T08:12:58.897 INFO:tasks.rados.rados.0.smithi118.stdout:3369: read oid 2 snap 328 2024-03-20T08:12:58.897 INFO:tasks.rados.rados.0.smithi118.stdout:3369: expect deleted 2024-03-20T08:12:58.897 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 331 2024-03-20T08:12:58.897 INFO:tasks.rados.rados.0.smithi118.stdout:3370: seq_num 1028 ranges {4538368=425984} 2024-03-20T08:12:58.923 INFO:tasks.rados.rados.0.smithi118.stdout:3370: writing smithi11881747-12 from 4538368 to 4964352 tid 1 2024-03-20T08:12:58.923 INFO:tasks.rados.rados.0.smithi118.stdout:3371: read oid 18 snap -1 2024-03-20T08:12:58.923 INFO:tasks.rados.rados.0.smithi118.stdout:3371: expect (ObjNum 1021 snap 329 seq_num 1021) 2024-03-20T08:12:58.923 INFO:tasks.rados.rados.0.smithi118.stdout:3372: read oid 14 snap -1 2024-03-20T08:12:58.923 INFO:tasks.rados.rados.0.smithi118.stdout:3372: expect deleted 2024-03-20T08:12:58.923 INFO:tasks.rados.rados.0.smithi118.stdout:3373: delete oid 50 current snap is 331 2024-03-20T08:12:58.934 INFO:tasks.rados.rados.0.smithi118.stdout:3369: done (4 left) 2024-03-20T08:12:58.934 INFO:tasks.rados.rados.0.smithi118.stdout:3372: done (3 left) 2024-03-20T08:12:58.934 INFO:tasks.rados.rados.0.smithi118.stdout:3373: done (2 left) 2024-03-20T08:12:58.934 INFO:tasks.rados.rados.0.smithi118.stdout:3374: read oid 7 snap -1 2024-03-20T08:12:58.934 INFO:tasks.rados.rados.0.smithi118.stdout:3374: expect deleted 2024-03-20T08:12:58.934 INFO:tasks.rados.rados.0.smithi118.stdout:3375: copy_from oid 30 from oid 6 current snap is 331 2024-03-20T08:12:58.935 INFO:tasks.rados.rados.0.smithi118.stdout:3376: copy_from oid 24 from oid 5 current snap is 331 2024-03-20T08:12:58.935 INFO:tasks.rados.rados.0.smithi118.stdout:3377: delete oid 31 current snap is 331 2024-03-20T08:12:58.935 INFO:tasks.rados.rados.0.smithi118.stdout:3370: finishing write tid 1 to smithi11881747-12 2024-03-20T08:12:58.935 INFO:tasks.rados.rados.0.smithi118.stdout:3370: finishing write tid 2 to smithi11881747-12 2024-03-20T08:12:58.939 INFO:tasks.rados.rados.0.smithi118.stdout:3377: done (5 left) 2024-03-20T08:12:58.939 INFO:tasks.rados.rados.0.smithi118.stdout:3378: copy_from oid 13 from oid 22 current snap is 331 2024-03-20T08:12:58.939 INFO:tasks.rados.rados.0.smithi118.stdout:3379: read oid 39 snap -1 2024-03-20T08:12:58.939 INFO:tasks.rados.rados.0.smithi118.stdout:3379: expect (ObjNum 963 snap 320 seq_num 963) 2024-03-20T08:12:58.939 INFO:tasks.rados.rados.0.smithi118.stdout:3380: snap_remove snap 330 2024-03-20T08:12:59.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:12:58.928+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:59.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:58.932+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:59.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:58.932+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:59.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:58.896+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:59.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:58.896+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:58.893+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:58.894+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:58.894+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:58.896+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:12:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:58.896+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: osdmap e734: 8 total, 8 up, 8 in 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: osdmap e734: 8 total, 8 up, 8 in 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:59.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:59.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:59.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:58.895+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:59.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:58.929+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:59.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:58.929+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:59.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:58.900+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:59.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:12:58.900+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:58.927+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:12:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:58.927+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:59.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:58.933+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 177=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: osdmap e734: 8 total, 8 up, 8 in 2024-03-20T08:12:59.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:12:59.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:12:59.901 INFO:tasks.rados.rados.0.smithi118.stdout:3380: done (7 left) 2024-03-20T08:12:59.901 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 331 2024-03-20T08:12:59.901 INFO:tasks.rados.rados.0.smithi118.stdout:3381: seq_num 1031 ranges {0=475136} 2024-03-20T08:12:59.904 INFO:tasks.rados.rados.0.smithi118.stdout:3381: writing smithi11881747-21 from 0 to 475136 tid 1 2024-03-20T08:12:59.904 INFO:tasks.rados.rados.0.smithi118.stdout:3382: copy_from oid 45 from oid 50 current snap is 331 2024-03-20T08:12:59.904 INFO:tasks.rados.rados.0.smithi118.stdout:3378: finishing copy_from to smithi11881747-13 2024-03-20T08:12:59.904 INFO:tasks.rados.rados.0.smithi118.stdout:3378: got expected ENOENT (src dne) 2024-03-20T08:12:59.905 INFO:tasks.rados.rados.0.smithi118.stdout:3370: finishing write tid 3 to smithi11881747-12 2024-03-20T08:12:59.905 INFO:tasks.rados.rados.0.smithi118.stdout:3370: oid 12 updating version 0 to 65285 2024-03-20T08:12:59.905 INFO:tasks.rados.rados.0.smithi118.stdout:3370: oid 12 version 65285 is already newer than 65284 2024-03-20T08:12:59.905 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65285 (ObjNum 1028 snap 331 seq_num 1028) dirty exists 2024-03-20T08:12:59.905 INFO:tasks.rados.rados.0.smithi118.stdout:3370: left oid 12 (ObjNum 1028 snap 331 seq_num 1028) 2024-03-20T08:12:59.905 INFO:tasks.rados.rados.0.smithi118.stdout:3378: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:12:59.905 INFO:tasks.rados.rados.0.smithi118.stdout:3371: expect (ObjNum 1021 snap 329 seq_num 1021) 2024-03-20T08:13:00.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:12:59.901+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:00.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:59.908+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:00.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:59.908+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:00.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:12:59.908+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:59.899+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:59.899+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:59.900+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:12:59.901+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:00.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: pgmap v786: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 144 KiB/s wr, 6 op/s 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: osdmap e735: 8 total, 8 up, 8 in 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:00.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: pgmap v786: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 144 KiB/s wr, 6 op/s 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:00.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: osdmap e735: 8 total, 8 up, 8 in 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:12:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:00.290 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:12:59.900+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:00.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:59.908+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:00.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:59.910+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:00.291 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:12:59.910+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:00.291 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:59.910+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:00.291 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:12:59.910+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: pgmap v786: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.3 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 144 KiB/s wr, 6 op/s 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: osdmap e735: 8 total, 8 up, 8 in 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:00.292 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:00.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:00.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:00.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:00.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:00.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:00.293 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:12:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:00.322 INFO:tasks.rados.rados.0.smithi118.stdout:3376: finishing copy_from racing read to smithi11881747-24 2024-03-20T08:13:00.322 INFO:tasks.rados.rados.0.smithi118.stdout:3379: expect (ObjNum 963 snap 320 seq_num 963) 2024-03-20T08:13:00.367 INFO:tasks.rados.rados.0.smithi118.stdout:3376: finishing copy_from to smithi11881747-24 2024-03-20T08:13:00.367 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 45624 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:13:00.367 INFO:tasks.rados.rados.0.smithi118.stdout:3375: finishing copy_from racing read to smithi11881747-30 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3375: finishing copy_from to smithi11881747-30 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 45618 (ObjNum 933 snap 317 seq_num 933) dirty exists 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3370: done (8 left) 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3371: done (7 left) 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3374: done (6 left) 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3375: done (5 left) 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3376: done (4 left) 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3378: done (3 left) 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3379: done (2 left) 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 47 current snap is 331 2024-03-20T08:13:00.368 INFO:tasks.rados.rados.0.smithi118.stdout:3383: seq_num 1032 ranges {270336=491520} 2024-03-20T08:13:00.371 INFO:tasks.rados.rados.0.smithi118.stdout:3383: writing smithi11881747-47 from 270336 to 761856 tid 1 2024-03-20T08:13:00.371 INFO:tasks.rados.rados.0.smithi118.stdout:3384: snap_create 2024-03-20T08:13:00.371 INFO:tasks.rados.rados.0.smithi118.stdout:3382: finishing copy_from to smithi11881747-45 2024-03-20T08:13:00.371 INFO:tasks.rados.rados.0.smithi118.stdout:3382: got expected ENOENT (src dne) 2024-03-20T08:13:00.372 INFO:tasks.rados.rados.0.smithi118.stdout:3382: finishing copy_from racing read to smithi11881747-45 2024-03-20T08:13:00.372 INFO:tasks.rados.rados.0.smithi118.stdout:3381: finishing write tid 1 to smithi11881747-21 2024-03-20T08:13:00.372 INFO:tasks.rados.rados.0.smithi118.stdout:3381: finishing write tid 2 to smithi11881747-21 2024-03-20T08:13:00.372 INFO:tasks.rados.rados.0.smithi118.stdout:3381: finishing write tid 3 to smithi11881747-21 2024-03-20T08:13:00.372 INFO:tasks.rados.rados.0.smithi118.stdout:3381: oid 21 updating version 0 to 61743 2024-03-20T08:13:00.372 INFO:tasks.rados.rados.0.smithi118.stdout:3381: oid 21 updating version 61743 to 61744 2024-03-20T08:13:00.372 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 61744 (ObjNum 1031 snap 331 seq_num 1031) dirty exists 2024-03-20T08:13:00.372 INFO:tasks.rados.rados.0.smithi118.stdout:3381: left oid 21 (ObjNum 1031 snap 331 seq_num 1031) 2024-03-20T08:13:00.375 INFO:tasks.rados.rados.0.smithi118.stdout:3383: finishing write tid 1 to smithi11881747-47 2024-03-20T08:13:00.376 INFO:tasks.rados.rados.0.smithi118.stdout:3383: finishing write tid 2 to smithi11881747-47 2024-03-20T08:13:00.377 INFO:tasks.rados.rados.0.smithi118.stdout:3383: finishing write tid 3 to smithi11881747-47 2024-03-20T08:13:00.377 INFO:tasks.rados.rados.0.smithi118.stdout:3383: oid 47 updating version 0 to 61735 2024-03-20T08:13:00.377 INFO:tasks.rados.rados.0.smithi118.stdout:3383: oid 47 version 61735 is already newer than 61734 2024-03-20T08:13:00.377 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 61735 (ObjNum 1032 snap 331 seq_num 1032) dirty exists 2024-03-20T08:13:00.377 INFO:tasks.rados.rados.0.smithi118.stdout:3383: left oid 47 (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:13:00.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:00.290+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:00.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:00.290+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:00.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:00.290+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:00.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:00.289+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:00.907 INFO:tasks.rados.rados.0.smithi118.stdout:3381: done (3 left) 2024-03-20T08:13:00.907 INFO:tasks.rados.rados.0.smithi118.stdout:3382: done (2 left) 2024-03-20T08:13:00.907 INFO:tasks.rados.rados.0.smithi118.stdout:3383: done (1 left) 2024-03-20T08:13:00.907 INFO:tasks.rados.rados.0.smithi118.stdout:3384: done (0 left) 2024-03-20T08:13:00.907 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 332 2024-03-20T08:13:00.907 INFO:tasks.rados.rados.0.smithi118.stdout:3385: seq_num 1033 ranges {5152768=507904} 2024-03-20T08:13:00.937 INFO:tasks.rados.rados.0.smithi118.stdout:3385: writing smithi11881747-32 from 5152768 to 5660672 tid 1 2024-03-20T08:13:00.937 INFO:tasks.rados.rados.0.smithi118.stdout:3386: snap_create 2024-03-20T08:13:00.946 INFO:tasks.rados.rados.0.smithi118.stdout:3385: finishing write tid 1 to smithi11881747-32 2024-03-20T08:13:00.947 INFO:tasks.rados.rados.0.smithi118.stdout:3385: finishing write tid 2 to smithi11881747-32 2024-03-20T08:13:00.950 INFO:tasks.rados.rados.0.smithi118.stdout:3385: finishing write tid 3 to smithi11881747-32 2024-03-20T08:13:00.950 INFO:tasks.rados.rados.0.smithi118.stdout:3385: oid 32 updating version 0 to 34114 2024-03-20T08:13:00.950 INFO:tasks.rados.rados.0.smithi118.stdout:3385: oid 32 updating version 34114 to 34115 2024-03-20T08:13:00.950 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34115 (ObjNum 1033 snap 332 seq_num 1033) dirty exists 2024-03-20T08:13:00.950 INFO:tasks.rados.rados.0.smithi118.stdout:3385: left oid 32 (ObjNum 1033 snap 332 seq_num 1033) 2024-03-20T08:13:01.169 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:00.904+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:01.170 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:00.905+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:01.170 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:00.906+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:01.170 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:00.906+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:01.170 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:00.950+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: osdmap e736: 8 total, 8 up, 8 in 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:01.171 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:01.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: osdmap e736: 8 total, 8 up, 8 in 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:01.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:01.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:00.944+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:01.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:00.944+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:01.175 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:00.944+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:01.175 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:00.905+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:01.176 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:00.905+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:01.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:01.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:01.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:01.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: osdmap e736: 8 total, 8 up, 8 in 2024-03-20T08:13:01.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:01.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:01.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:01.176 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:01.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:01.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:01.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:01.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:01.177 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:01.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:01.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:01.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:01.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:01.178 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:01.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.168+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:01.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.168+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:01.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.168+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:01.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.168+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:01.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.168+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:01.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:01.170+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:01.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:01.171+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:01.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:01.171+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:01.912 INFO:tasks.rados.rados.0.smithi118.stdout:3385: done (1 left) 2024-03-20T08:13:01.912 INFO:tasks.rados.rados.0.smithi118.stdout:3386: done (0 left) 2024-03-20T08:13:01.912 INFO:tasks.rados.rados.0.smithi118.stdout:3387: read oid 10 snap -1 2024-03-20T08:13:01.912 INFO:tasks.rados.rados.0.smithi118.stdout:3387: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:13:01.912 INFO:tasks.rados.rados.0.smithi118.stdout:3388: setattr oid 12 current snap is 333 2024-03-20T08:13:01.913 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 333 2024-03-20T08:13:01.913 INFO:tasks.rados.rados.0.smithi118.stdout:3389: seq_num 1034 ranges {0=524288} 2024-03-20T08:13:01.917 INFO:tasks.rados.rados.0.smithi118.stdout:3389: writing smithi11881747-1 from 0 to 524288 tid 1 2024-03-20T08:13:01.917 INFO:tasks.rados.rados.0.smithi118.stdout:3390: read oid 17 snap 328 2024-03-20T08:13:01.917 INFO:tasks.rados.rados.0.smithi118.stdout:3390: expect (ObjNum 975 snap 321 seq_num 975) 2024-03-20T08:13:01.917 INFO:tasks.rados.rados.0.smithi118.stdout:3391: snap_create 2024-03-20T08:13:01.917 INFO:tasks.rados.rados.0.smithi118.stdout:3387: expect (ObjNum 0 snap 0 seq_num 0) 2024-03-20T08:13:01.924 INFO:tasks.rados.rados.0.smithi118.stdout:3389: finishing write tid 1 to smithi11881747-1 2024-03-20T08:13:01.925 INFO:tasks.rados.rados.0.smithi118.stdout:3389: finishing write tid 2 to smithi11881747-1 2024-03-20T08:13:01.926 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65286 (ObjNum 1028 snap 331 seq_num 1028) dirty exists 2024-03-20T08:13:01.930 INFO:tasks.rados.rados.0.smithi118.stdout:3389: finishing write tid 3 to smithi11881747-1 2024-03-20T08:13:01.930 INFO:tasks.rados.rados.0.smithi118.stdout:3389: oid 1 updating version 0 to 65274 2024-03-20T08:13:01.930 INFO:tasks.rados.rados.0.smithi118.stdout:3389: oid 1 updating version 65274 to 65275 2024-03-20T08:13:01.930 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 65275 (ObjNum 1034 snap 333 seq_num 1034) dirty exists 2024-03-20T08:13:01.930 INFO:tasks.rados.rados.0.smithi118.stdout:3389: left oid 1 (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:13:01.977 INFO:tasks.rados.rados.0.smithi118.stdout:3390: expect (ObjNum 975 snap 321 seq_num 975) 2024-03-20T08:13:02.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:01.909+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:02.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:01.909+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:02.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:01.910+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:02.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:01.910+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:02.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.915+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:02.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.915+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:02.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.916+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:02.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:01.922+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:02.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:01.933+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:02.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: pgmap v789: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.1 MiB/s wr, 8 op/s 2024-03-20T08:13:02.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:02.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:02.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:02.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:02.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: osdmap e737: 8 total, 8 up, 8 in 2024-03-20T08:13:02.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:02.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: pgmap v789: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.1 MiB/s wr, 8 op/s 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: osdmap e737: 8 total, 8 up, 8 in 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:02.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:02.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:02.313 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:01.926+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:02.314 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:01.916+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:02.314 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:01.916+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:02.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:01.924+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:02.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:01.924+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:02.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:01.926+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:02.315 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:01.911+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:02.315 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:01.911+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:02.315 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:01.911+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:02.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: pgmap v789: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.1 MiB/s wr, 8 op/s 2024-03-20T08:13:02.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:02.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:02.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:02.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:02.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: osdmap e737: 8 total, 8 up, 8 in 2024-03-20T08:13:02.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:02.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:02.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:02.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:02.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:13:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:13:02.917 INFO:tasks.rados.rados.0.smithi118.stdout:3387: done (4 left) 2024-03-20T08:13:02.918 INFO:tasks.rados.rados.0.smithi118.stdout:3388: done (3 left) 2024-03-20T08:13:02.918 INFO:tasks.rados.rados.0.smithi118.stdout:3389: done (2 left) 2024-03-20T08:13:02.918 INFO:tasks.rados.rados.0.smithi118.stdout:3390: done (1 left) 2024-03-20T08:13:02.918 INFO:tasks.rados.rados.0.smithi118.stdout:3391: done (0 left) 2024-03-20T08:13:02.918 INFO:tasks.rados.rados.0.smithi118.stdout:3392: copy_from oid 40 from oid 14 current snap is 334 2024-03-20T08:13:02.918 INFO:tasks.rados.rados.0.smithi118.stdout:3393: copy_from oid 34 from oid 4 current snap is 334 2024-03-20T08:13:02.918 INFO:tasks.rados.rados.0.smithi118.stdout:3394: snap_create 2024-03-20T08:13:02.922 INFO:tasks.rados.rados.0.smithi118.stdout:3392: finishing copy_from to smithi11881747-40 2024-03-20T08:13:02.922 INFO:tasks.rados.rados.0.smithi118.stdout:3392: got expected ENOENT (src dne) 2024-03-20T08:13:02.923 INFO:tasks.rados.rados.0.smithi118.stdout:3392: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:13:02.932 INFO:tasks.rados.rados.0.smithi118.stdout:3393: finishing copy_from racing read to smithi11881747-34 2024-03-20T08:13:02.932 INFO:tasks.rados.rados.0.smithi118.stdout:3393: finishing copy_from to smithi11881747-34 2024-03-20T08:13:02.932 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45644 (ObjNum 599 snap 196 seq_num 599) dirty exists 2024-03-20T08:13:03.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:02.916+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:03.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:02.916+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:03.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:02.921+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:03.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:02.921+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:03.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:02.921+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:03.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:02.921+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:03.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:02.921+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:03.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:03.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:03.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:03.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:03.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: osdmap e738: 8 total, 8 up, 8 in 2024-03-20T08:13:03.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:03.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:03.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:03.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: osdmap e738: 8 total, 8 up, 8 in 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:03.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:03.179 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:02.923+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:03.179 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:02.920+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:03.179 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:02.920+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:03.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:03.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:03.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:03.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:03.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:03.180 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:03.180 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:03.180 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:03.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:02.917+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:03.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:02.924+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:03.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:02.924+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:03.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:02.924+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:03.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:03.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:03.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:03.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:03.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:03.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:03.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:03.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: osdmap e738: 8 total, 8 up, 8 in 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:03.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:03.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:03.462+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:03.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:03.462+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:03.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:03.461+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:03.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:03.461+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:03.917 INFO:tasks.rados.rados.0.smithi118.stdout:3392: done (2 left) 2024-03-20T08:13:03.918 INFO:tasks.rados.rados.0.smithi118.stdout:3393: done (1 left) 2024-03-20T08:13:03.918 INFO:tasks.rados.rados.0.smithi118.stdout:3394: done (0 left) 2024-03-20T08:13:03.918 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:13:03] "GET /metrics HTTP/1.1" 200 36413 "" "Prometheus/2.43.0" 2024-03-20T08:13:03.918 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 335 2024-03-20T08:13:03.919 INFO:tasks.rados.rados.0.smithi118.stdout:3395: seq_num 1035 ranges {0=540672} 2024-03-20T08:13:03.921 INFO:tasks.rados.rados.0.smithi118.stdout:3395: writing smithi11881747-7 from 0 to 540672 tid 1 2024-03-20T08:13:03.921 INFO:tasks.rados.rados.0.smithi118.stdout:3396: copy_from oid 48 from oid 38 current snap is 335 2024-03-20T08:13:03.921 INFO:tasks.rados.rados.0.smithi118.stdout:3397: copy_from oid 29 from oid 8 current snap is 335 2024-03-20T08:13:03.921 INFO:tasks.rados.rados.0.smithi118.stdout:3398: read oid 14 snap 213 2024-03-20T08:13:03.921 INFO:tasks.rados.rados.0.smithi118.stdout:3398: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:13:03.921 INFO:tasks.rados.rados.0.smithi118.stdout:3399: setattr oid 15 current snap is 335 2024-03-20T08:13:03.923 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 335 2024-03-20T08:13:03.923 INFO:tasks.rados.rados.0.smithi118.stdout:3400: seq_num 1036 ranges {0=557056} 2024-03-20T08:13:03.927 INFO:tasks.rados.rados.0.smithi118.stdout:3400: writing smithi11881747-10 from 0 to 557056 tid 1 2024-03-20T08:13:03.927 INFO:tasks.rados.rados.0.smithi118.stdout:3401: snap_remove snap 333 2024-03-20T08:13:04.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:03.941+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:04.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:03.917+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:04.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:03.917+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:04.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:03.917+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:04.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:03.931+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:04.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:03.926+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:04.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:03.927+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:04.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:03.927+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:04.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:03.938+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: pgmap v792: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-03-20T08:13:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: osdmap e739: 8 total, 8 up, 8 in 2024-03-20T08:13:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:04.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:04.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:04.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:04.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: pgmap v792: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-03-20T08:13:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: osdmap e739: 8 total, 8 up, 8 in 2024-03-20T08:13:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:04.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:03.918+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:04.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:03.919+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:04.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:03.919+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:04.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:03.933+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:04.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:03.933+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:04.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:03.933+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:04.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:03.936+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:04.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:03.936+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:04.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:03.937+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: pgmap v792: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 10 op/s 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: osdmap e739: 8 total, 8 up, 8 in 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:04.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:04.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:04.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:04.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:04.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:04.928 INFO:tasks.rados.rados.0.smithi118.stdout:3401: done (6 left) 2024-03-20T08:13:04.928 INFO:tasks.rados.rados.0.smithi118.stdout:3402: setattr oid 24 current snap is 335 2024-03-20T08:13:04.928 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 7116 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:13:04.928 INFO:tasks.rados.rados.0.smithi118.stdout:3398: expect (ObjNum 599 snap 196 seq_num 599) 2024-03-20T08:13:04.968 INFO:tasks.rados.rados.0.smithi118.stdout:3400: finishing write tid 1 to smithi11881747-10 2024-03-20T08:13:04.968 INFO:tasks.rados.rados.0.smithi118.stdout:3400: finishing write tid 2 to smithi11881747-10 2024-03-20T08:13:04.968 INFO:tasks.rados.rados.0.smithi118.stdout:3400: finishing write tid 3 to smithi11881747-10 2024-03-20T08:13:04.968 INFO:tasks.rados.rados.0.smithi118.stdout:3400: oid 10 updating version 0 to 65277 2024-03-20T08:13:04.968 INFO:tasks.rados.rados.0.smithi118.stdout:3400: oid 10 version 65277 is already newer than 65276 2024-03-20T08:13:04.968 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 65277 (ObjNum 1036 snap 335 seq_num 1036) dirty exists 2024-03-20T08:13:04.968 INFO:tasks.rados.rados.0.smithi118.stdout:3400: left oid 10 (ObjNum 1036 snap 335 seq_num 1036) 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3397: finishing copy_from racing read to smithi11881747-29 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3395: finishing write tid 1 to smithi11881747-7 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3395: finishing write tid 2 to smithi11881747-7 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3397: finishing copy_from to smithi11881747-29 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 34103 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3395: finishing write tid 3 to smithi11881747-7 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3395: oid 7 updating version 0 to 65288 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3395: oid 7 version 65288 is already newer than 65287 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 65288 (ObjNum 1035 snap 335 seq_num 1035) dirty exists 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3395: left oid 7 (ObjNum 1035 snap 335 seq_num 1035) 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3396: finishing copy_from racing read to smithi11881747-48 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3396: finishing copy_from to smithi11881747-48 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 61742 (ObjNum 1025 snap 330 seq_num 1025) dirty exists 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3395: done (6 left) 2024-03-20T08:13:04.969 INFO:tasks.rados.rados.0.smithi118.stdout:3396: done (5 left) 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3397: done (4 left) 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3398: done (3 left) 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3399: done (2 left) 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3400: done (1 left) 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3403: read oid 44 snap -1 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3403: expect (ObjNum 990 snap 321 seq_num 990) 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3404: read oid 50 snap -1 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3404: expect deleted 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3405: copy_from oid 42 from oid 11 current snap is 335 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3406: rmattr oid 19 current snap is 335 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3406: done (4 left) 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 335 2024-03-20T08:13:04.970 INFO:tasks.rados.rados.0.smithi118.stdout:3407: seq_num 1037 ranges {4964352=573440} 2024-03-20T08:13:04.998 INFO:tasks.rados.rados.0.smithi118.stdout:3407: writing smithi11881747-12 from 4964352 to 5537792 tid 1 2024-03-20T08:13:04.998 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 335 2024-03-20T08:13:04.998 INFO:tasks.rados.rados.0.smithi118.stdout:3408: seq_num 1038 ranges {8200192=589824} 2024-03-20T08:13:05.045 INFO:tasks.rados.rados.0.smithi118.stdout:3408: writing smithi11881747-28 from 8200192 to 8790016 tid 1 2024-03-20T08:13:05.045 INFO:tasks.rados.rados.0.smithi118.stdout:3409: snap_remove snap 329 2024-03-20T08:13:05.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:04.929+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:05.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:05.003+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:05.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:05.003+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:05.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:04.928+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:05.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:04.929+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:05.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:04.971+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:05.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:04.971+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:05.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:04.989+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:05.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:05.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:05.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:05.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:05.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: osdmap e740: 8 total, 8 up, 8 in 2024-03-20T08:13:05.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:05.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:05.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:05.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:05.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:05.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:05.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:05.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:05.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:05.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:05.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:05.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:05.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:04.974+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:05.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:04.970+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:05.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:04.971+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:05.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:05.049+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:05.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:05.049+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:04.927+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:04.927+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:04.927+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:04.927+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:05.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:04.927+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: osdmap e740: 8 total, 8 up, 8 in 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:05.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:05.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: osdmap e740: 8 total, 8 up, 8 in 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:05.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 186=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:05.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:05.932 INFO:tasks.rados.rados.0.smithi118.stdout:3409: done (6 left) 2024-03-20T08:13:05.933 INFO:tasks.rados.rados.0.smithi118.stdout:3410: snap_create 2024-03-20T08:13:05.933 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65296 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:13:05.933 INFO:tasks.rados.rados.0.smithi118.stdout:3403: expect (ObjNum 990 snap 321 seq_num 990) 2024-03-20T08:13:05.991 INFO:tasks.rados.rados.0.smithi118.stdout:3405: finishing copy_from racing read to smithi11881747-42 2024-03-20T08:13:05.991 INFO:tasks.rados.rados.0.smithi118.stdout:3405: finishing copy_from to smithi11881747-42 2024-03-20T08:13:05.991 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65279 (ObjNum 969 snap 321 seq_num 969) dirty exists 2024-03-20T08:13:05.991 INFO:tasks.rados.rados.0.smithi118.stdout:3407: finishing write tid 1 to smithi11881747-12 2024-03-20T08:13:05.991 INFO:tasks.rados.rados.0.smithi118.stdout:3407: finishing write tid 2 to smithi11881747-12 2024-03-20T08:13:05.991 INFO:tasks.rados.rados.0.smithi118.stdout:3407: finishing write tid 3 to smithi11881747-12 2024-03-20T08:13:05.991 INFO:tasks.rados.rados.0.smithi118.stdout:3407: oid 12 updating version 0 to 65289 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:3407: oid 12 updating version 65289 to 65290 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65290 (ObjNum 1037 snap 335 seq_num 1037) dirty exists 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:3407: left oid 12 (ObjNum 1037 snap 335 seq_num 1037) 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:3408: finishing write tid 1 to smithi11881747-28 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:3408: finishing write tid 2 to smithi11881747-28 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:3408: finishing write tid 3 to smithi11881747-28 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:3408: oid 28 updating version 0 to 61736 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:3408: oid 28 updating version 61736 to 61737 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 61737 (ObjNum 1038 snap 335 seq_num 1038) dirty exists 2024-03-20T08:13:05.992 INFO:tasks.rados.rados.0.smithi118.stdout:3408: left oid 28 (ObjNum 1038 snap 335 seq_num 1038) 2024-03-20T08:13:06.239 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:05.933+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: pgmap v795: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 3.3 MiB/s rd, 736 KiB/s wr, 11 op/s 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: osdmap e741: 8 total, 8 up, 8 in 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:06.240 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:06.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:06.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:06.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:06.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:06.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:06.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:06.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:06.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:05.934+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:06.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:05.932+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:06.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:05.933+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:06.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:05.933+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:06.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.090+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:06.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.090+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:06.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.090+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:06.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.090+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: pgmap v795: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 3.3 MiB/s rd, 736 KiB/s wr, 11 op/s 2024-03-20T08:13:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:06.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: osdmap e741: 8 total, 8 up, 8 in 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:06.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: pgmap v795: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 3.3 MiB/s rd, 736 KiB/s wr, 11 op/s 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: osdmap e741: 8 total, 8 up, 8 in 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:06.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:06.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:06.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:06.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:06.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:06.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:06.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:06.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:06.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:06.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:06.239+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:06.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:06.239+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:06.937 INFO:tasks.rados.rados.0.smithi118.stdout:3402: done (6 left) 2024-03-20T08:13:06.938 INFO:tasks.rados.rados.0.smithi118.stdout:3403: done (5 left) 2024-03-20T08:13:06.938 INFO:tasks.rados.rados.0.smithi118.stdout:3404: done (4 left) 2024-03-20T08:13:06.938 INFO:tasks.rados.rados.0.smithi118.stdout:3405: done (3 left) 2024-03-20T08:13:06.938 INFO:tasks.rados.rados.0.smithi118.stdout:3407: done (2 left) 2024-03-20T08:13:06.938 INFO:tasks.rados.rados.0.smithi118.stdout:3408: done (1 left) 2024-03-20T08:13:06.938 INFO:tasks.rados.rados.0.smithi118.stdout:3410: done (0 left) 2024-03-20T08:13:06.938 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 336 2024-03-20T08:13:06.938 INFO:tasks.rados.rados.0.smithi118.stdout:3411: seq_num 1039 ranges {11788288=606208} 2024-03-20T08:13:07.005 INFO:tasks.rados.rados.0.smithi118.stdout:3411: writing smithi11881747-17 from 11788288 to 12394496 tid 1 2024-03-20T08:13:07.005 INFO:tasks.rados.rados.0.smithi118.stdout:3412: read oid 7 snap -1 2024-03-20T08:13:07.005 INFO:tasks.rados.rados.0.smithi118.stdout:3412: expect (ObjNum 1035 snap 335 seq_num 1035) 2024-03-20T08:13:07.005 INFO:tasks.rados.rados.0.smithi118.stdout:3413: snap_remove snap 266 2024-03-20T08:13:07.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:06.939+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:07.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:06.941+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:07.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:06.941+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: osdmap e742: 8 total, 8 up, 8 in 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:07.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:07.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:06.838+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:06.838+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:06.838+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:06.961+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:06.961+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:06.963+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:06.897+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:06.898+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:06.898+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:06.962+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:07.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:06.962+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:07.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:06.937+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:07.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:06.937+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:07.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:06.962+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:07.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:06.962+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:07.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:06.963+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: osdmap e742: 8 total, 8 up, 8 in 2024-03-20T08:13:07.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:07.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:07.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:07.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:07.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:07.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:07.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:07.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.962+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:07.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.962+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:07.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.963+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:07.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.963+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:07.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:06.963+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: osdmap e742: 8 total, 8 up, 8 in 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:07.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:07.939 INFO:tasks.rados.rados.0.smithi118.stdout:3413: done (2 left) 2024-03-20T08:13:07.939 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 336 2024-03-20T08:13:07.939 INFO:tasks.rados.rados.0.smithi118.stdout:3414: seq_num 1040 ranges {10838016=622592} 2024-03-20T08:13:08.001 INFO:tasks.rados.rados.0.smithi118.stdout:3414: writing smithi11881747-16 from 10838016 to 11460608 tid 1 2024-03-20T08:13:08.001 INFO:tasks.rados.rados.0.smithi118.stdout:3415: rmattr oid 29 current snap is 336 2024-03-20T08:13:08.002 INFO:tasks.rados.rados.0.smithi118.stdout:3412: expect (ObjNum 1035 snap 335 seq_num 1035) 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3411: finishing write tid 1 to smithi11881747-17 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3411: finishing write tid 2 to smithi11881747-17 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3411: finishing write tid 3 to smithi11881747-17 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3411: oid 17 updating version 0 to 45646 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3411: oid 17 updating version 45646 to 45647 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 45647 (ObjNum 1039 snap 336 seq_num 1039) dirty exists 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3411: left oid 17 (ObjNum 1039 snap 336 seq_num 1039) 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3414: finishing write tid 1 to smithi11881747-16 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3411: done (3 left) 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3412: done (2 left) 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 336 2024-03-20T08:13:08.018 INFO:tasks.rados.rados.0.smithi118.stdout:3414: finishing write tid 2 to smithi11881747-16 2024-03-20T08:13:08.019 INFO:tasks.rados.rados.0.smithi118.stdout:3416: seq_num 1041 ranges {0=638976} 2024-03-20T08:13:08.021 INFO:tasks.rados.rados.0.smithi118.stdout:3416: writing smithi11881747-31 from 0 to 638976 tid 1 2024-03-20T08:13:08.021 INFO:tasks.rados.rados.0.smithi118.stdout:3417: delete oid 36 current snap is 336 2024-03-20T08:13:08.021 INFO:tasks.rados.rados.0.smithi118.stdout:3414: finishing write tid 3 to smithi11881747-16 2024-03-20T08:13:08.021 INFO:tasks.rados.rados.0.smithi118.stdout:3414: oid 16 updating version 0 to 61738 2024-03-20T08:13:08.021 INFO:tasks.rados.rados.0.smithi118.stdout:3414: oid 16 updating version 61738 to 61739 2024-03-20T08:13:08.021 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 61739 (ObjNum 1040 snap 336 seq_num 1040) dirty exists 2024-03-20T08:13:08.021 INFO:tasks.rados.rados.0.smithi118.stdout:3414: left oid 16 (ObjNum 1040 snap 336 seq_num 1040) 2024-03-20T08:13:08.021 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 61740 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:13:08.026 INFO:tasks.rados.rados.0.smithi118.stdout:3414: done (3 left) 2024-03-20T08:13:08.026 INFO:tasks.rados.rados.0.smithi118.stdout:3415: done (2 left) 2024-03-20T08:13:08.026 INFO:tasks.rados.rados.0.smithi118.stdout:3417: done (1 left) 2024-03-20T08:13:08.026 INFO:tasks.rados.rados.0.smithi118.stdout:3418: copy_from oid 35 from oid 7 current snap is 336 2024-03-20T08:13:08.027 INFO:tasks.rados.rados.0.smithi118.stdout:3419: rmattr oid 43 current snap is 336 2024-03-20T08:13:08.027 INFO:tasks.rados.rados.0.smithi118.stdout:3419: done (2 left) 2024-03-20T08:13:08.027 INFO:tasks.rados.rados.0.smithi118.stdout:3420: snap_remove snap 331 2024-03-20T08:13:08.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:07.938+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:08.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:07.962+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:08.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:07.964+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:08.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:07.964+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:08.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:07.965+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:08.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:07.965+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:08.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:07.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:08.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:07.963+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:08.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:07.963+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: pgmap v798: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 17 op/s 2024-03-20T08:13:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: osdmap e743: 8 total, 8 up, 8 in 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:08.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:08.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:08.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:08.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:08.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:07.939+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:08.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:07.962+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:08.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:07.938+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:08.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:07.938+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:08.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:07.939+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:08.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:07.962+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:08.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:07.964+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:08.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:07.964+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:08.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:07.964+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:08.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:08.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:08.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: pgmap v798: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 17 op/s 2024-03-20T08:13:08.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:08.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: osdmap e743: 8 total, 8 up, 8 in 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:08.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:08.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:08.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:08.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:08.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:08.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:08.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:08.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: pgmap v798: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 2.6 MiB/s wr, 17 op/s 2024-03-20T08:13:08.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: osdmap e743: 8 total, 8 up, 8 in 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17d=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:08.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:08.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:08.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:08.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3420: done (2 left) 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3421: setattr oid 26 current snap is 336 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3416: finishing write tid 1 to smithi11881747-31 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3416: finishing write tid 2 to smithi11881747-31 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3416: finishing write tid 3 to smithi11881747-31 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3416: oid 31 updating version 0 to 65278 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3416: oid 31 updating version 65278 to 65279 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 65279 (ObjNum 1041 snap 336 seq_num 1041) dirty exists 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3416: left oid 31 (ObjNum 1041 snap 336 seq_num 1041) 2024-03-20T08:13:08.950 INFO:tasks.rados.rados.0.smithi118.stdout:3418: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:13:08.951 INFO:tasks.rados.rados.0.smithi118.stdout:3418: finishing copy_from to smithi11881747-35 2024-03-20T08:13:08.951 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 65288 (ObjNum 1035 snap 335 seq_num 1035) dirty exists 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:3416: done (2 left) 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:3418: done (1 left) 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:3422: copy_from oid 30 from oid 9 current snap is 336 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:3423: read oid 44 snap -1 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:3423: expect (ObjNum 990 snap 321 seq_num 990) 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:3424: read oid 49 snap 327 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:3424: expect deleted 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 3 current snap is 336 2024-03-20T08:13:08.952 INFO:tasks.rados.rados.0.smithi118.stdout:3425: seq_num 1043 ranges {2211840=671744} 2024-03-20T08:13:08.967 INFO:tasks.rados.rados.0.smithi118.stdout:3425: writing smithi11881747-3 from 2211840 to 2883584 tid 1 2024-03-20T08:13:08.967 INFO:tasks.rados.rados.0.smithi118.stdout:3426: snap_remove snap 328 2024-03-20T08:13:09.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:08.954+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:09.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:08.954+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:09.197 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:08.970+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:09.198 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:08.958+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:09.198 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:08.958+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:09.198 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:08.949+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:09.198 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:08.949+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: osdmap e744: 8 total, 8 up, 8 in 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:09.199 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:09.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:09.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:09.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:09.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:09.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:09.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:09.200 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:09.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:08.948+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:09.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:08.948+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:09.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:08.958+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:09.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:08.950+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:09.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:08.950+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:09.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:08.971+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:09.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:08.971+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:09.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:08.971+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:09.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:08.983+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:09.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:08.983+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:09.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:09.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:09.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: osdmap e744: 8 total, 8 up, 8 in 2024-03-20T08:13:09.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:09.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:09.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:09.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: osdmap e744: 8 total, 8 up, 8 in 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 179=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:09.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:09.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:09.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:09.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:09.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:09.197+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:09.953 INFO:tasks.rados.rados.0.smithi118.stdout:3426: done (5 left) 2024-03-20T08:13:09.953 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 336 2024-03-20T08:13:09.954 INFO:tasks.rados.rados.0.smithi118.stdout:3427: seq_num 1044 ranges {655360=688128} 2024-03-20T08:13:09.961 INFO:tasks.rados.rados.0.smithi118.stdout:3427: writing smithi11881747-34 from 655360 to 1343488 tid 1 2024-03-20T08:13:09.961 INFO:tasks.rados.rados.0.smithi118.stdout:3428: snap_create 2024-03-20T08:13:09.961 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65298 (ObjNum 413 snap 136 seq_num 413) dirty exists 2024-03-20T08:13:09.961 INFO:tasks.rados.rados.0.smithi118.stdout:3423: expect (ObjNum 990 snap 321 seq_num 990) 2024-03-20T08:13:10.023 INFO:tasks.rados.rados.0.smithi118.stdout:3422: finishing copy_from racing read to smithi11881747-30 2024-03-20T08:13:10.023 INFO:tasks.rados.rados.0.smithi118.stdout:3422: finishing copy_from to smithi11881747-30 2024-03-20T08:13:10.023 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 65259 (ObjNum 966 snap 320 seq_num 966) dirty exists 2024-03-20T08:13:10.023 INFO:tasks.rados.rados.0.smithi118.stdout:3425: finishing write tid 1 to smithi11881747-3 2024-03-20T08:13:10.023 INFO:tasks.rados.rados.0.smithi118.stdout:3425: finishing write tid 2 to smithi11881747-3 2024-03-20T08:13:10.023 INFO:tasks.rados.rados.0.smithi118.stdout:3425: finishing write tid 3 to smithi11881747-3 2024-03-20T08:13:10.023 INFO:tasks.rados.rados.0.smithi118.stdout:3425: oid 3 updating version 0 to 34056 2024-03-20T08:13:10.023 INFO:tasks.rados.rados.0.smithi118.stdout:3425: oid 3 version 34056 is already newer than 34055 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34056 (ObjNum 1043 snap 336 seq_num 1043) dirty exists 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:3425: left oid 3 (ObjNum 1043 snap 336 seq_num 1043) 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:3427: finishing write tid 1 to smithi11881747-34 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:3427: finishing write tid 2 to smithi11881747-34 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:3427: finishing write tid 3 to smithi11881747-34 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:3427: oid 34 updating version 0 to 45645 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:3427: oid 34 updating version 45645 to 45646 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45646 (ObjNum 1044 snap 336 seq_num 1044) dirty exists 2024-03-20T08:13:10.024 INFO:tasks.rados.rados.0.smithi118.stdout:3427: left oid 34 (ObjNum 1044 snap 336 seq_num 1044) 2024-03-20T08:13:10.321 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:09.954+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:10.321 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:09.955+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:10.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:09.955+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:10.322 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:09.955+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:10.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: pgmap v801: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-03-20T08:13:10.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:10.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:10.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:10.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:10.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: osdmap e745: 8 total, 8 up, 8 in 2024-03-20T08:13:10.322 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:10.323 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:10.326 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:09.967+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:10.326 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:09.964+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:10.327 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:09.974+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:10.327 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:09.974+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:10.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:09.953+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:10.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:09.953+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:10.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:09.954+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:10.327 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:09.954+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:10.327 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: pgmap v801: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: osdmap e745: 8 total, 8 up, 8 in 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:10.328 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: pgmap v801: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 5.7 MiB/s wr, 14 op/s 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: osdmap e745: 8 total, 8 up, 8 in 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:10.329 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:10.330 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:10.331 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:10.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:10.321+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:10.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:10.321+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:10.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:10.327+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:10.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:10.327+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:10.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:10.327+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:10.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:10.326+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:10.958 INFO:tasks.rados.rados.0.smithi118.stdout:3421: done (6 left) 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3422: done (5 left) 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3423: done (4 left) 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3424: done (3 left) 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3425: done (2 left) 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3427: done (1 left) 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3428: done (0 left) 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3429: read oid 3 snap -1 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3429: expect (ObjNum 1043 snap 336 seq_num 1043) 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 22 current snap is 337 2024-03-20T08:13:10.959 INFO:tasks.rados.rados.0.smithi118.stdout:3430: seq_num 1045 ranges {0=704512} 2024-03-20T08:13:10.963 INFO:tasks.rados.rados.0.smithi118.stdout:3430: writing smithi11881747-22 from 0 to 704512 tid 1 2024-03-20T08:13:10.963 INFO:tasks.rados.rados.0.smithi118.stdout:3431: read oid 37 snap -1 2024-03-20T08:13:10.963 INFO:tasks.rados.rados.0.smithi118.stdout:3431: expect (ObjNum 920 snap 311 seq_num 920) 2024-03-20T08:13:10.963 INFO:tasks.rados.rados.0.smithi118.stdout:3432: read oid 40 snap -1 2024-03-20T08:13:10.963 INFO:tasks.rados.rados.0.smithi118.stdout:3432: expect deleted 2024-03-20T08:13:10.963 INFO:tasks.rados.rados.0.smithi118.stdout:3433: snap_remove snap 335 2024-03-20T08:13:11.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:10.957+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:11.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:10.965+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:11.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:10.966+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: osdmap e746: 8 total, 8 up, 8 in 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:11.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:11.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:11.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:11.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:11.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:10 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:13:11.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:10.958+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:11.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:10.958+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:11.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:10.961+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:11.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:10.962+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:11.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:10.961+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:11.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:10.962+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:11.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:10.963+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:11.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:10.965+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:11.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:10.965+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: osdmap e746: 8 total, 8 up, 8 in 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:11.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: osdmap e746: 8 total, 8 up, 8 in 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:11.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:11.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:10 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:13:11.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:11.398+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:11.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:11.398+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:11.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:11.426+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:11.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:11.426+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:11.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:11.426+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:11.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:11.427+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:11.962 INFO:tasks.rados.rados.0.smithi118.stdout:3433: done (4 left) 2024-03-20T08:13:11.962 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 337 2024-03-20T08:13:11.963 INFO:tasks.rados.rados.0.smithi118.stdout:3434: seq_num 1046 ranges {0=729088} 2024-03-20T08:13:11.966 INFO:tasks.rados.rados.0.smithi118.stdout:3434: writing smithi11881747-2 from 0 to 729088 tid 1 2024-03-20T08:13:11.966 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 337 2024-03-20T08:13:11.966 INFO:tasks.rados.rados.0.smithi118.stdout:3431: expect (ObjNum 920 snap 311 seq_num 920) 2024-03-20T08:13:12.029 INFO:tasks.rados.rados.0.smithi118.stdout:3429: expect (ObjNum 1043 snap 336 seq_num 1043) 2024-03-20T08:13:12.174 INFO:tasks.rados.rados.0.smithi118.stdout:3430: finishing write tid 1 to smithi11881747-22 2024-03-20T08:13:12.174 INFO:tasks.rados.rados.0.smithi118.stdout:3430: finishing write tid 2 to smithi11881747-22 2024-03-20T08:13:12.174 INFO:tasks.rados.rados.0.smithi118.stdout:3430: finishing write tid 3 to smithi11881747-22 2024-03-20T08:13:12.174 INFO:tasks.rados.rados.0.smithi118.stdout:3430: oid 22 updating version 0 to 61748 2024-03-20T08:13:12.174 INFO:tasks.rados.rados.0.smithi118.stdout:3430: oid 22 version 61748 is already newer than 61747 2024-03-20T08:13:12.174 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 61748 (ObjNum 1045 snap 337 seq_num 1045) dirty exists 2024-03-20T08:13:12.174 INFO:tasks.rados.rados.0.smithi118.stdout:3430: left oid 22 (ObjNum 1045 snap 337 seq_num 1045) 2024-03-20T08:13:12.175 INFO:tasks.rados.rados.0.smithi118.stdout:3435: seq_num 1047 ranges {557056=745472} 2024-03-20T08:13:12.181 INFO:tasks.rados.rados.0.smithi118.stdout:3435: writing smithi11881747-10 from 557056 to 1302528 tid 1 2024-03-20T08:13:12.181 INFO:tasks.rados.rados.0.smithi118.stdout:3429: done (5 left) 2024-03-20T08:13:12.181 INFO:tasks.rados.rados.0.smithi118.stdout:3430: done (4 left) 2024-03-20T08:13:12.181 INFO:tasks.rados.rados.0.smithi118.stdout:3431: done (3 left) 2024-03-20T08:13:12.181 INFO:tasks.rados.rados.0.smithi118.stdout:3432: done (2 left) 2024-03-20T08:13:12.182 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 337 2024-03-20T08:13:12.182 INFO:tasks.rados.rados.0.smithi118.stdout:3434: finishing write tid 1 to smithi11881747-2 2024-03-20T08:13:12.182 INFO:tasks.rados.rados.0.smithi118.stdout:3436: seq_num 1048 ranges {507904=761856} 2024-03-20T08:13:12.188 INFO:tasks.rados.rados.0.smithi118.stdout:3436: writing smithi11881747-33 from 507904 to 1269760 tid 1 2024-03-20T08:13:12.188 INFO:tasks.rados.rados.0.smithi118.stdout:3437: snap_remove snap 332 2024-03-20T08:13:12.314 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:11.962+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:12.314 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:12.062+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:12.315 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:12.062+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:12.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:11.962+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:12.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:11.963+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:12.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:11.963+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:12.315 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:11.970+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:12.316 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:11.970+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:12.316 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:11.973+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:12.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:12.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:12.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: pgmap v804: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 19 op/s 2024-03-20T08:13:12.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:12.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:12.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:12.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: osdmap e747: 8 total, 8 up, 8 in 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:12.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:12.063+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:12.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:11.973+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:12.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:12.062+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:12.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:12.063+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:12.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:12.063+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:12.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:11.962+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:12.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:11.962+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:12.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:11.962+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:12.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:11.963+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:12.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:12.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:12.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: pgmap v804: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 19 op/s 2024-03-20T08:13:12.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:12.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:12.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:12.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: osdmap e747: 8 total, 8 up, 8 in 2024-03-20T08:13:12.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:12.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: pgmap v804: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 19 op/s 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: osdmap e747: 8 total, 8 up, 8 in 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:12.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:12.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:13:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:13:12.968 INFO:tasks.rados.rados.0.smithi118.stdout:3437: done (3 left) 2024-03-20T08:13:12.968 INFO:tasks.rados.rados.0.smithi118.stdout:3438: snap_remove snap 334 2024-03-20T08:13:13.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:12.969+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:13.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:12.969+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:13.240 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:12.969+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:13.241 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:12.968+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:13.241 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:12.970+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:13.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:13.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:13.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:13.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: osdmap e748: 8 total, 8 up, 8 in 2024-03-20T08:13:13.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:13.242 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:13.243 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:13.243 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:12.968+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:13.243 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:12.968+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:13.243 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: osdmap e748: 8 total, 8 up, 8 in 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:13.244 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: osdmap e748: 8 total, 8 up, 8 in 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:13.245 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:13.246 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:13.246 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:13.246 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:13.246 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:13.246 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:13.246 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:13.246 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:13.247 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:13.585 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:13.241+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:13.586 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:13.240+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:13.586 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:13.240+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:13.240+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:13.240+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:13.240+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:13.241+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:13.241+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:13.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:13.240+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:13.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:13.240+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:13.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:13.240+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:13:13] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:13:13.972 INFO:tasks.rados.rados.0.smithi118.stdout:3438: done (3 left) 2024-03-20T08:13:13.972 INFO:tasks.rados.rados.0.smithi118.stdout:3439: snap_remove snap 327 2024-03-20T08:13:14.362 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:13.972+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:14.362 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:13.972+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:14.362 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:13.972+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:14.362 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: pgmap v807: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 15 op/s 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: osdmap e749: 8 total, 8 up, 8 in 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:14.363 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:14.364 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:14.364 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:14.364 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:14.364 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:13.973+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:14.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:13.974+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:14.365 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:13.974+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:14.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: pgmap v807: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 15 op/s 2024-03-20T08:13:14.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:14.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:14.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:14.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: osdmap e749: 8 total, 8 up, 8 in 2024-03-20T08:13:14.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:14.366 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:14.366 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:14.200+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:14.367 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:14.200+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:14.367 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:14.200+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:14.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: pgmap v807: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 15 op/s 2024-03-20T08:13:14.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:14.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:14.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:14.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: osdmap e749: 8 total, 8 up, 8 in 2024-03-20T08:13:14.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:14.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:14.367 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:14.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:14.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:14.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:14.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:14.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:14.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:14.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:14.368 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:14.642 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:14.362+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:14.642 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:14.362+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:14.642 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:14.362+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:14.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:14.364+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:14.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:14.362+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:14.363+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:14.363+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:14.364+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:14.365+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:3439: done (3 left) 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:3440: read oid 1 snap -1 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:3440: expect (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:3441: read oid 6 snap -1 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:3441: expect (ObjNum 933 snap 317 seq_num 933) 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 38 current snap is 337 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:3434: finishing write tid 2 to smithi11881747-2 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:3434: finishing write tid 3 to smithi11881747-2 2024-03-20T08:13:14.980 INFO:tasks.rados.rados.0.smithi118.stdout:3434: oid 2 updating version 0 to 65290 2024-03-20T08:13:14.981 INFO:tasks.rados.rados.0.smithi118.stdout:3434: oid 2 version 65290 is already newer than 65289 2024-03-20T08:13:14.981 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65290 (ObjNum 1046 snap 337 seq_num 1046) dirty exists 2024-03-20T08:13:14.981 INFO:tasks.rados.rados.0.smithi118.stdout:3434: left oid 2 (ObjNum 1046 snap 337 seq_num 1046) 2024-03-20T08:13:14.981 INFO:tasks.rados.rados.0.smithi118.stdout:3442: seq_num 1049 ranges {11313152=778240} 2024-03-20T08:13:15.048 INFO:tasks.rados.rados.0.smithi118.stdout:3442: writing smithi11881747-38 from 11313152 to 12091392 tid 1 2024-03-20T08:13:15.048 INFO:tasks.rados.rados.0.smithi118.stdout:3434: done (5 left) 2024-03-20T08:13:15.048 INFO:tasks.rados.rados.0.smithi118.stdout:3443: delete oid 28 current snap is 337 2024-03-20T08:13:15.048 INFO:tasks.rados.rados.0.smithi118.stdout:3435: finishing write tid 1 to smithi11881747-10 2024-03-20T08:13:15.048 INFO:tasks.rados.rados.0.smithi118.stdout:3435: finishing write tid 2 to smithi11881747-10 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3435: finishing write tid 3 to smithi11881747-10 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3435: oid 10 updating version 0 to 65281 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3435: oid 10 version 65281 is already newer than 65280 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 65281 (ObjNum 1047 snap 337 seq_num 1047) dirty exists 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3435: left oid 10 (ObjNum 1047 snap 337 seq_num 1047) 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3436: finishing write tid 1 to smithi11881747-33 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3436: finishing write tid 2 to smithi11881747-33 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3436: finishing write tid 3 to smithi11881747-33 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3436: oid 33 updating version 0 to 34058 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3436: oid 33 version 34058 is already newer than 34057 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34058 (ObjNum 1048 snap 337 seq_num 1048) dirty exists 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3436: left oid 33 (ObjNum 1048 snap 337 seq_num 1048) 2024-03-20T08:13:15.049 INFO:tasks.rados.rados.0.smithi118.stdout:3440: expect (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:13:15.065 INFO:tasks.rados.rados.0.smithi118.stdout:3441: expect (ObjNum 933 snap 317 seq_num 933) 2024-03-20T08:13:15.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:14.976+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:15.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:14.986+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:15.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:14.986+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:15.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:14.986+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:15.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:14.988+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:15.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:14.989+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:15.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:14.982+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:15.391 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:15.016+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: osdmap e750: 8 total, 8 up, 8 in 2024-03-20T08:13:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:15.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:15.035+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:15.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:15.035+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:15.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:14.977+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:15.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:14.977+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:15.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:14.979+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:15.051+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:15.051+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:15.051+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:15.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:15.051+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:15.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:15.053+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: osdmap e750: 8 total, 8 up, 8 in 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:15.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: osdmap e750: 8 total, 8 up, 8 in 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:15.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:15.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:15.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:15.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:15.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:16.153 INFO:tasks.rados.rados.0.smithi118.stdout:3442: finishing write tid 1 to smithi11881747-38 2024-03-20T08:13:16.153 INFO:tasks.rados.rados.0.smithi118.stdout:3435: done (5 left) 2024-03-20T08:13:16.153 INFO:tasks.rados.rados.0.smithi118.stdout:3436: done (4 left) 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3440: done (3 left) 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3441: done (2 left) 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3443: done (1 left) 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3444: read oid 26 snap 213 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3444: expect (ObjNum 620 snap 202 seq_num 620) 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3445: delete oid 31 current snap is 337 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3442: finishing write tid 2 to smithi11881747-38 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3442: finishing write tid 3 to smithi11881747-38 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3442: oid 38 updating version 0 to 61750 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3442: oid 38 updating version 61750 to 61751 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 61751 (ObjNum 1049 snap 337 seq_num 1049) dirty exists 2024-03-20T08:13:16.154 INFO:tasks.rados.rados.0.smithi118.stdout:3442: left oid 38 (ObjNum 1049 snap 337 seq_num 1049) 2024-03-20T08:13:16.156 INFO:tasks.rados.rados.0.smithi118.stdout:3442: done (2 left) 2024-03-20T08:13:16.156 INFO:tasks.rados.rados.0.smithi118.stdout:3445: done (1 left) 2024-03-20T08:13:16.156 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 337 2024-03-20T08:13:16.156 INFO:tasks.rados.rados.0.smithi118.stdout:3446: seq_num 1052 ranges {5660672=425984,6086656=401408} 2024-03-20T08:13:16.189 INFO:tasks.rados.rados.0.smithi118.stdout:3446: writing smithi11881747-32 from 5660672 to 6086656 tid 1 2024-03-20T08:13:16.192 INFO:tasks.rados.rados.0.smithi118.stdout:3446: writing smithi11881747-32 from 6086656 to 6488064 tid 2 2024-03-20T08:13:16.192 INFO:tasks.rados.rados.0.smithi118.stdout:3447: read oid 31 snap -1 2024-03-20T08:13:16.192 INFO:tasks.rados.rados.0.smithi118.stdout:3447: expect deleted 2024-03-20T08:13:16.192 INFO:tasks.rados.rados.0.smithi118.stdout:3448: copy_from oid 19 from oid 49 current snap is 337 2024-03-20T08:13:16.192 INFO:tasks.rados.rados.0.smithi118.stdout:3444: expect (ObjNum 620 snap 202 seq_num 620) 2024-03-20T08:13:16.237 INFO:tasks.rados.rados.0.smithi118.stdout:3448: finishing copy_from racing read to smithi11881747-19 2024-03-20T08:13:16.237 INFO:tasks.rados.rados.0.smithi118.stdout:3448: finishing copy_from to smithi11881747-19 2024-03-20T08:13:16.237 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 59853 (ObjNum 1022 snap 329 seq_num 1022) dirty exists 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: finishing write tid 1 to smithi11881747-32 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: finishing write tid 2 to smithi11881747-32 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: finishing write tid 3 to smithi11881747-32 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: finishing write tid 4 to smithi11881747-32 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: oid 32 updating version 0 to 34118 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: oid 32 version 34118 is already newer than 34116 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: oid 32 version 34118 is already newer than 34117 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34118 (ObjNum 1052 snap 337 seq_num 1052) dirty exists 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: left oid 32 (ObjNum 1052 snap 337 seq_num 1052) 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3444: done (3 left) 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3446: done (2 left) 2024-03-20T08:13:16.238 INFO:tasks.rados.rados.0.smithi118.stdout:3447: done (1 left) 2024-03-20T08:13:16.239 INFO:tasks.rados.rados.0.smithi118.stdout:3448: done (0 left) 2024-03-20T08:13:16.239 INFO:tasks.rados.rados.0.smithi118.stdout:3449: snap_remove snap 213 2024-03-20T08:13:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: pgmap v810: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 3.4 MiB/s rd, 842 KiB/s wr, 9 op/s 2024-03-20T08:13:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: osdmap e751: 8 total, 8 up, 8 in 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:16.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: pgmap v810: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 3.4 MiB/s rd, 842 KiB/s wr, 9 op/s 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: osdmap e751: 8 total, 8 up, 8 in 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: pgmap v810: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 3.4 MiB/s rd, 842 KiB/s wr, 9 op/s 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 185=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:16.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: osdmap e751: 8 total, 8 up, 8 in 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:16.983 INFO:tasks.rados.rados.0.smithi118.stdout:3449: done (0 left) 2024-03-20T08:13:16.983 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 337 2024-03-20T08:13:16.983 INFO:tasks.rados.rados.0.smithi118.stdout:3450: seq_num 1053 ranges {0=442368,442368=401408} 2024-03-20T08:13:16.986 INFO:tasks.rados.rados.0.smithi118.stdout:3450: writing smithi11881747-43 from 0 to 442368 tid 1 2024-03-20T08:13:16.988 INFO:tasks.rados.rados.0.smithi118.stdout:3450: writing smithi11881747-43 from 442368 to 843776 tid 2 2024-03-20T08:13:16.989 INFO:tasks.rados.rados.0.smithi118.stdout:3451: snap_create 2024-03-20T08:13:16.995 INFO:tasks.rados.rados.0.smithi118.stdout:3450: finishing write tid 1 to smithi11881747-43 2024-03-20T08:13:16.995 INFO:tasks.rados.rados.0.smithi118.stdout:3450: finishing write tid 2 to smithi11881747-43 2024-03-20T08:13:16.995 INFO:tasks.rados.rados.0.smithi118.stdout:3450: finishing write tid 3 to smithi11881747-43 2024-03-20T08:13:16.997 INFO:tasks.rados.rados.0.smithi118.stdout:3450: finishing write tid 4 to smithi11881747-43 2024-03-20T08:13:16.997 INFO:tasks.rados.rados.0.smithi118.stdout:3450: oid 43 updating version 0 to 65285 2024-03-20T08:13:16.997 INFO:tasks.rados.rados.0.smithi118.stdout:3450: oid 43 version 65285 is already newer than 65284 2024-03-20T08:13:16.997 INFO:tasks.rados.rados.0.smithi118.stdout:3450: oid 43 version 65285 is already newer than 65283 2024-03-20T08:13:16.997 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65285 (ObjNum 1053 snap 337 seq_num 1053) dirty exists 2024-03-20T08:13:16.998 INFO:tasks.rados.rados.0.smithi118.stdout:3450: left oid 43 (ObjNum 1053 snap 337 seq_num 1053) 2024-03-20T08:13:17.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:16.984+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:17.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:16.996+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:17.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:16.996+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:17.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:16.993+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:17.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:16.993+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:17.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:16.993+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:17.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:16.984+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:17.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:16.985+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:17.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:17.471+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:17.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:17.471+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:17.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:17.471+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:17.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:17.472+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:17.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:17.472+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:17.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:17.472+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:17.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.471+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:17.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.471+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:17.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.471+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:17.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.472+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:17.985 INFO:tasks.rados.rados.0.smithi118.stdout:3450: done (1 left) 2024-03-20T08:13:17.985 INFO:tasks.rados.rados.0.smithi118.stdout:3451: done (0 left) 2024-03-20T08:13:17.985 INFO:tasks.rados.rados.0.smithi118.stdout:3452: rmattr oid 21 current snap is 338 2024-03-20T08:13:17.985 INFO:tasks.rados.rados.0.smithi118.stdout:3452: done (0 left) 2024-03-20T08:13:17.985 INFO:tasks.rados.rados.0.smithi118.stdout:3453: snap_remove snap 337 2024-03-20T08:13:18.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:17.984+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:18.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:18.135+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:18.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:18.137+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:18.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:18.135+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:18.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:18.135+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:18.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:18.138+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:18.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:18.137+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:18.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:18.137+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:18.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: pgmap v812: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 893 KiB/s wr, 8 op/s 2024-03-20T08:13:18.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: osdmap e752: 8 total, 8 up, 8 in 2024-03-20T08:13:18.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:18.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:18.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:18.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:18.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:18.136+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:18.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:18.136+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:18.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:17.986+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:18.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:17.987+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:18.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:18.136+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:18.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.984+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:18.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.984+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:18.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.985+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:18.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.985+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:18.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:17.985+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:18.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: pgmap v812: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 893 KiB/s wr, 8 op/s 2024-03-20T08:13:18.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: osdmap e752: 8 total, 8 up, 8 in 2024-03-20T08:13:18.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:18.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:18.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: pgmap v812: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 2.9 MiB/s rd, 893 KiB/s wr, 8 op/s 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: osdmap e752: 8 total, 8 up, 8 in 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:18.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:18.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:18.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:18.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:18.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:18.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:18.996 INFO:tasks.rados.rados.0.smithi118.stdout:3453: done (0 left) 2024-03-20T08:13:18.997 INFO:tasks.rados.rados.0.smithi118.stdout:3454: rmattr oid 45 current snap is 338 2024-03-20T08:13:18.997 INFO:tasks.rados.rados.0.smithi118.stdout:3454: done (0 left) 2024-03-20T08:13:18.997 INFO:tasks.rados.rados.0.smithi118.stdout:3455: setattr oid 23 current snap is 338 2024-03-20T08:13:18.998 INFO:tasks.rados.rados.0.smithi118.stdout:3456: read oid 11 snap 336 2024-03-20T08:13:18.998 INFO:tasks.rados.rados.0.smithi118.stdout:3456: expect (ObjNum 969 snap 321 seq_num 969) 2024-03-20T08:13:18.998 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 338 2024-03-20T08:13:18.998 INFO:tasks.rados.rados.0.smithi118.stdout:3457: seq_num 1054 ranges {0=458752,458752=401408} 2024-03-20T08:13:19.001 INFO:tasks.rados.rados.0.smithi118.stdout:3457: writing smithi11881747-36 from 0 to 458752 tid 1 2024-03-20T08:13:19.004 INFO:tasks.rados.rados.0.smithi118.stdout:3457: writing smithi11881747-36 from 458752 to 860160 tid 2 2024-03-20T08:13:19.004 INFO:tasks.rados.rados.0.smithi118.stdout:3458: read oid 29 snap -1 2024-03-20T08:13:19.004 INFO:tasks.rados.rados.0.smithi118.stdout:3458: expect (ObjNum 646 snap 212 seq_num 646) 2024-03-20T08:13:19.005 INFO:tasks.rados.rados.0.smithi118.stdout:3459: delete oid 49 current snap is 338 2024-03-20T08:13:19.005 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 61742 (ObjNum 971 snap 321 seq_num 971) dirty exists 2024-03-20T08:13:19.008 INFO:tasks.rados.rados.0.smithi118.stdout:3455: done (4 left) 2024-03-20T08:13:19.009 INFO:tasks.rados.rados.0.smithi118.stdout:3459: done (3 left) 2024-03-20T08:13:19.009 INFO:tasks.rados.rados.0.smithi118.stdout:3460: read oid 25 snap -1 2024-03-20T08:13:19.009 INFO:tasks.rados.rados.0.smithi118.stdout:3460: expect deleted 2024-03-20T08:13:19.009 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 338 2024-03-20T08:13:19.009 INFO:tasks.rados.rados.0.smithi118.stdout:3461: seq_num 1056 ranges {5537792=491520,6029312=401408} 2024-03-20T08:13:19.043 INFO:tasks.rados.rados.0.smithi118.stdout:3461: writing smithi11881747-12 from 5537792 to 6029312 tid 1 2024-03-20T08:13:19.045 INFO:tasks.rados.rados.0.smithi118.stdout:3461: writing smithi11881747-12 from 6029312 to 6430720 tid 2 2024-03-20T08:13:19.045 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 338 2024-03-20T08:13:19.046 INFO:tasks.rados.rados.0.smithi118.stdout:3462: seq_num 1057 ranges {11788288=507904,12296192=401408} 2024-03-20T08:13:19.113 INFO:tasks.rados.rados.0.smithi118.stdout:3462: writing smithi11881747-46 from 11788288 to 12296192 tid 1 2024-03-20T08:13:19.115 INFO:tasks.rados.rados.0.smithi118.stdout:3462: writing smithi11881747-46 from 12296192 to 12697600 tid 2 2024-03-20T08:13:19.115 INFO:tasks.rados.rados.0.smithi118.stdout:3463: read oid 24 snap 336 2024-03-20T08:13:19.115 INFO:tasks.rados.rados.0.smithi118.stdout:3463: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:13:19.115 INFO:tasks.rados.rados.0.smithi118.stdout:3464: delete oid 4 current snap is 338 2024-03-20T08:13:19.115 INFO:tasks.rados.rados.0.smithi118.stdout:3458: expect (ObjNum 646 snap 212 seq_num 646) 2024-03-20T08:13:19.157 INFO:tasks.rados.rados.0.smithi118.stdout:3457: finishing write tid 1 to smithi11881747-36 2024-03-20T08:13:19.157 INFO:tasks.rados.rados.0.smithi118.stdout:3458: done (7 left) 2024-03-20T08:13:19.157 INFO:tasks.rados.rados.0.smithi118.stdout:3460: done (6 left) 2024-03-20T08:13:19.157 INFO:tasks.rados.rados.0.smithi118.stdout:3464: done (5 left) 2024-03-20T08:13:19.157 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 338 2024-03-20T08:13:19.157 INFO:tasks.rados.rados.0.smithi118.stdout:3457: finishing write tid 2 to smithi11881747-36 2024-03-20T08:13:19.157 INFO:tasks.rados.rados.0.smithi118.stdout:3457: finishing write tid 3 to smithi11881747-36 2024-03-20T08:13:19.158 INFO:tasks.rados.rados.0.smithi118.stdout:3457: finishing write tid 4 to smithi11881747-36 2024-03-20T08:13:19.158 INFO:tasks.rados.rados.0.smithi118.stdout:3457: oid 36 updating version 0 to 45649 2024-03-20T08:13:19.158 INFO:tasks.rados.rados.0.smithi118.stdout:3457: oid 36 version 45649 is already newer than 45647 2024-03-20T08:13:19.158 INFO:tasks.rados.rados.0.smithi118.stdout:3457: oid 36 version 45649 is already newer than 45648 2024-03-20T08:13:19.158 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45649 (ObjNum 1054 snap 338 seq_num 1054) dirty exists 2024-03-20T08:13:19.158 INFO:tasks.rados.rados.0.smithi118.stdout:3457: left oid 36 (ObjNum 1054 snap 338 seq_num 1054) 2024-03-20T08:13:19.158 INFO:tasks.rados.rados.0.smithi118.stdout:3456: expect (ObjNum 969 snap 321 seq_num 969) 2024-03-20T08:13:19.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:19.048+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:19.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:19.048+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:19.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:18.994+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:19.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:19.048+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:19.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:19.049+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:19.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:19.049+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:19.002+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:19.002+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:19.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:19.002+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: osdmap e753: 8 total, 8 up, 8 in 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:19.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:19.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:19.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:19.002+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:19.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:19.002+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:19.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:19.008+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:19.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:18.996+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:19.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:18.997+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:19.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:18.997+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:19.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:18.998+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: osdmap e753: 8 total, 8 up, 8 in 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: osdmap e753: 8 total, 8 up, 8 in 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:19.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:19.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:19.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:18.996+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:19.427 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:19.008+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:19.648 INFO:tasks.rados.rados.0.smithi118.stdout:3461: finishing write tid 1 to smithi11881747-12 2024-03-20T08:13:19.648 INFO:tasks.rados.rados.0.smithi118.stdout:3465: seq_num 1059 ranges {475136=540672,1015808=401408} 2024-03-20T08:13:19.653 INFO:tasks.rados.rados.0.smithi118.stdout:3465: writing smithi11881747-21 from 475136 to 1015808 tid 1 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3465: writing smithi11881747-21 from 1015808 to 1417216 tid 2 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3456: done (5 left) 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3457: done (4 left) 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3466: rollback oid 40 current snap is 338 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 336 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3461: finishing write tid 2 to smithi11881747-12 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3461: finishing write tid 3 to smithi11881747-12 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3461: finishing write tid 4 to smithi11881747-12 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3461: oid 12 updating version 0 to 65291 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3461: oid 12 updating version 65291 to 65292 2024-03-20T08:13:19.657 INFO:tasks.rados.rados.0.smithi118.stdout:3461: oid 12 updating version 65292 to 65293 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65293 (ObjNum 1056 snap 338 seq_num 1056) dirty exists 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:3461: left oid 12 (ObjNum 1056 snap 338 seq_num 1056) 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:3462: finishing write tid 1 to smithi11881747-46 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:3461: done (4 left) 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:3467: read oid 4 snap -1 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:3467: expect deleted 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 6 current snap is 338 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:3462: finishing write tid 2 to smithi11881747-46 2024-03-20T08:13:19.658 INFO:tasks.rados.rados.0.smithi118.stdout:3463: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:13:19.671 INFO:tasks.rados.rados.0.smithi118.stdout:3462: finishing write tid 3 to smithi11881747-46 2024-03-20T08:13:19.671 INFO:tasks.rados.rados.0.smithi118.stdout:3462: finishing write tid 4 to smithi11881747-46 2024-03-20T08:13:19.671 INFO:tasks.rados.rados.0.smithi118.stdout:3462: oid 46 updating version 0 to 65291 2024-03-20T08:13:19.671 INFO:tasks.rados.rados.0.smithi118.stdout:3462: oid 46 updating version 65291 to 65292 2024-03-20T08:13:19.672 INFO:tasks.rados.rados.0.smithi118.stdout:3462: oid 46 updating version 65292 to 65293 2024-03-20T08:13:19.672 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 65293 (ObjNum 1057 snap 338 seq_num 1057) dirty exists 2024-03-20T08:13:19.672 INFO:tasks.rados.rados.0.smithi118.stdout:3462: left oid 46 (ObjNum 1057 snap 338 seq_num 1057) 2024-03-20T08:13:19.672 INFO:tasks.rados.rados.0.smithi118.stdout:3468: seq_num 1060 ranges {11968512=557056,12525568=401408} 2024-03-20T08:13:19.740 INFO:tasks.rados.rados.0.smithi118.stdout:3468: writing smithi11881747-6 from 11968512 to 12525568 tid 1 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:3468: writing smithi11881747-6 from 12525568 to 12926976 tid 2 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:3462: done (5 left) 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:3463: done (4 left) 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:3469: rmattr oid 9 current snap is 338 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:3469: done (4 left) 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:3470: copy_from oid 38 from oid 2 current snap is 338 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:3466: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 34086 (ObjNum 33 snap 0 seq_num 323086192) dirty dne 2024-03-20T08:13:19.743 INFO:tasks.rados.rados.0.smithi118.stdout:3465: finishing write tid 1 to smithi11881747-21 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3465: finishing write tid 2 to smithi11881747-21 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3466: done (4 left) 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3467: done (3 left) 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3471: copy_from oid 49 from oid 28 current snap is 338 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3465: finishing write tid 3 to smithi11881747-21 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3465: finishing write tid 4 to smithi11881747-21 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3465: oid 21 updating version 0 to 61753 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3465: oid 21 updating version 61753 to 61755 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3465: oid 21 version 61755 is already newer than 61754 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 61755 (ObjNum 1059 snap 338 seq_num 1059) dirty exists 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3465: left oid 21 (ObjNum 1059 snap 338 seq_num 1059) 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3465: done (3 left) 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3472: copy_from oid 37 from oid 39 current snap is 338 2024-03-20T08:13:19.744 INFO:tasks.rados.rados.0.smithi118.stdout:3473: snap_remove snap 336 2024-03-20T08:13:20.002 INFO:tasks.rados.rados.0.smithi118.stdout:3473: done (4 left) 2024-03-20T08:13:20.002 INFO:tasks.rados.rados.0.smithi118.stdout:3474: delete oid 26 current snap is 338 2024-03-20T08:13:20.002 INFO:tasks.rados.rados.0.smithi118.stdout:3471: finishing copy_from to smithi11881747-49 2024-03-20T08:13:20.003 INFO:tasks.rados.rados.0.smithi118.stdout:3471: got expected ENOENT (src dne) 2024-03-20T08:13:20.003 INFO:tasks.rados.rados.0.smithi118.stdout:3471: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:13:20.003 INFO:tasks.rados.rados.0.smithi118.stdout:3468: finishing write tid 1 to smithi11881747-6 2024-03-20T08:13:20.003 INFO:tasks.rados.rados.0.smithi118.stdout:3468: finishing write tid 2 to smithi11881747-6 2024-03-20T08:13:20.003 INFO:tasks.rados.rados.0.smithi118.stdout:3468: finishing write tid 3 to smithi11881747-6 2024-03-20T08:13:20.003 INFO:tasks.rados.rados.0.smithi118.stdout:3468: finishing write tid 4 to smithi11881747-6 2024-03-20T08:13:20.004 INFO:tasks.rados.rados.0.smithi118.stdout:3468: oid 6 updating version 0 to 65296 2024-03-20T08:13:20.004 INFO:tasks.rados.rados.0.smithi118.stdout:3468: oid 6 version 65296 is already newer than 65295 2024-03-20T08:13:20.004 INFO:tasks.rados.rados.0.smithi118.stdout:3468: oid 6 version 65296 is already newer than 65294 2024-03-20T08:13:20.004 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 65296 (ObjNum 1060 snap 338 seq_num 1060) dirty exists 2024-03-20T08:13:20.004 INFO:tasks.rados.rados.0.smithi118.stdout:3468: left oid 6 (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:13:20.004 INFO:tasks.rados.rados.0.smithi118.stdout:3472: finishing copy_from racing read to smithi11881747-37 2024-03-20T08:13:20.004 INFO:tasks.rados.rados.0.smithi118.stdout:3472: finishing copy_from to smithi11881747-37 2024-03-20T08:13:20.005 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 59830 (ObjNum 963 snap 320 seq_num 963) dirty exists 2024-03-20T08:13:20.005 INFO:tasks.rados.rados.0.smithi118.stdout:3470: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:13:20.005 INFO:tasks.rados.rados.0.smithi118.stdout:3470: finishing copy_from to smithi11881747-38 2024-03-20T08:13:20.005 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 65290 (ObjNum 1046 snap 337 seq_num 1046) dirty exists 2024-03-20T08:13:20.008 INFO:tasks.rados.rados.0.smithi118.stdout:3468: done (4 left) 2024-03-20T08:13:20.008 INFO:tasks.rados.rados.0.smithi118.stdout:3470: done (3 left) 2024-03-20T08:13:20.009 INFO:tasks.rados.rados.0.smithi118.stdout:3471: done (2 left) 2024-03-20T08:13:20.009 INFO:tasks.rados.rados.0.smithi118.stdout:3472: done (1 left) 2024-03-20T08:13:20.009 INFO:tasks.rados.rados.0.smithi118.stdout:3474: done (0 left) 2024-03-20T08:13:20.009 INFO:tasks.rados.rados.0.smithi118.stdout:3475: copy_from oid 26 from oid 35 current snap is 338 2024-03-20T08:13:20.009 INFO:tasks.rados.rados.0.smithi118.stdout:3476: snap_create 2024-03-20T08:13:20.021 INFO:tasks.rados.rados.0.smithi118.stdout:3475: finishing copy_from racing read to smithi11881747-26 2024-03-20T08:13:20.021 INFO:tasks.rados.rados.0.smithi118.stdout:3475: finishing copy_from to smithi11881747-26 2024-03-20T08:13:20.021 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65288 (ObjNum 1035 snap 335 seq_num 1035) dirty exists 2024-03-20T08:13:20.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:20.002+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:20.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:20.004+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:20.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:20.004+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:20.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:20.008+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:20.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:20.007+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:20.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:20.007+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:20.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:20.015+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:20.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:20.015+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: pgmap v815: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 885 B/s rd, 651 KiB/s wr, 3 op/s 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: osdmap e754: 8 total, 8 up, 8 in 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:20.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:20.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:20.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:20.007+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:20.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:20.007+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:20.000+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:20.001+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:20.001+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:20.001+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:20.001+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: pgmap v815: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 885 B/s rd, 651 KiB/s wr, 3 op/s 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: osdmap e754: 8 total, 8 up, 8 in 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: pgmap v815: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 885 B/s rd, 651 KiB/s wr, 3 op/s 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: osdmap e754: 8 total, 8 up, 8 in 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:20.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:20.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:20.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:20.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:20.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:20.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:20.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:20.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:20.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:20.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17c=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:20.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:20.601+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:20.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:20.601+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:20.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:20.601+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:21.007 INFO:tasks.rados.rados.0.smithi118.stdout:3475: done (1 left) 2024-03-20T08:13:21.007 INFO:tasks.rados.rados.0.smithi118.stdout:3476: done (0 left) 2024-03-20T08:13:21.008 INFO:tasks.rados.rados.0.smithi118.stdout:3477: snap_create 2024-03-20T08:13:21.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:21.006+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:21.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:21.006+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:21.292 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:21.006+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:21.292 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:21.006+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:21.292 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: osdmap e755: 8 total, 8 up, 8 in 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:21.293 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: osdmap e755: 8 total, 8 up, 8 in 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:21.294 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:21.295 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:21.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:21.006+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:21.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: osdmap e755: 8 total, 8 up, 8 in 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:21.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:21.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:21.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:21.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:21.291+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:21.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:21.291+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:21.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:21.291+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:21.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:21.291+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:22.007 INFO:tasks.rados.rados.0.smithi118.stdout:3477: done (0 left) 2024-03-20T08:13:22.007 INFO:tasks.rados.rados.0.smithi118.stdout:3478: copy_from oid 38 from oid 6 current snap is 340 2024-03-20T08:13:22.008 INFO:tasks.rados.rados.0.smithi118.stdout:3479: setattr oid 26 current snap is 340 2024-03-20T08:13:22.009 INFO:tasks.rados.rados.0.smithi118.stdout:3480: snap_create 2024-03-20T08:13:22.031 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65302 (ObjNum 1035 snap 335 seq_num 1035) dirty exists 2024-03-20T08:13:22.070 INFO:tasks.rados.rados.0.smithi118.stdout:3478: finishing copy_from racing read to smithi11881747-38 2024-03-20T08:13:22.072 INFO:tasks.rados.rados.0.smithi118.stdout:3478: finishing copy_from to smithi11881747-38 2024-03-20T08:13:22.072 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 65296 (ObjNum 1060 snap 338 seq_num 1060) dirty exists 2024-03-20T08:13:22.314 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:22.007+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:22.314 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:22.031+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:22.314 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:22.031+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:22.315 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:22.017+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:22.315 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:22.017+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:22.315 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:22.017+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:22.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:22.024+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:22.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:22.030+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:22.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:22.031+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:22.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: pgmap v818: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: osdmap e756: 8 total, 8 up, 8 in 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:22.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:22.029+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:22.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:22.029+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: pgmap v818: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: osdmap e756: 8 total, 8 up, 8 in 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:22.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:22.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:22.006+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:22.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:22.006+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:22.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:22.007+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:22.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:22.007+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:22.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:22.007+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:22.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: pgmap v818: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-20T08:13:22.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:22.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: osdmap e756: 8 total, 8 up, 8 in 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:22.425 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:22.007+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:22.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:22.007+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:22.426 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:22.036+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:13:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:13:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:13:23.014 INFO:tasks.rados.rados.0.smithi118.stdout:3478: done (2 left) 2024-03-20T08:13:23.014 INFO:tasks.rados.rados.0.smithi118.stdout:3479: done (1 left) 2024-03-20T08:13:23.014 INFO:tasks.rados.rados.0.smithi118.stdout:3480: done (0 left) 2024-03-20T08:13:23.014 INFO:tasks.rados.rados.0.smithi118.stdout:3481: delete oid 34 current snap is 341 2024-03-20T08:13:23.021 INFO:tasks.rados.rados.0.smithi118.stdout:3481: done (0 left) 2024-03-20T08:13:23.022 INFO:tasks.rados.rados.0.smithi118.stdout:3482: rmattr oid 6 current snap is 341 2024-03-20T08:13:23.022 INFO:tasks.rados.rados.0.smithi118.stdout:3482: done (0 left) 2024-03-20T08:13:23.022 INFO:tasks.rados.rados.0.smithi118.stdout:3483: read oid 36 snap -1 2024-03-20T08:13:23.022 INFO:tasks.rados.rados.0.smithi118.stdout:3483: expect (ObjNum 1054 snap 338 seq_num 1054) 2024-03-20T08:13:23.022 INFO:tasks.rados.rados.0.smithi118.stdout:3484: delete oid 50 current snap is 341 2024-03-20T08:13:23.027 INFO:tasks.rados.rados.0.smithi118.stdout:3483: expect (ObjNum 1054 snap 338 seq_num 1054) 2024-03-20T08:13:23.050 INFO:tasks.rados.rados.0.smithi118.stdout:3483: done (1 left) 2024-03-20T08:13:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:3484: done (0 left) 2024-03-20T08:13:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:3485: rollback oid 23 current snap is 341 2024-03-20T08:13:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 23 to 339 2024-03-20T08:13:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:3486: read oid 4 snap 339 2024-03-20T08:13:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:3486: expect deleted 2024-03-20T08:13:23.051 INFO:tasks.rados.rados.0.smithi118.stdout:3487: snap_remove snap 338 2024-03-20T08:13:23.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:23.013+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:23.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:23.029+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:23.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:23.029+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:23.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:23.241+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:23.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:23.241+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:23.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:23.025+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:23.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:23.025+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:23.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:23.029+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: osdmap e757: 8 total, 8 up, 8 in 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:23.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:23.015+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:23.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:23.019+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:23.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:23.022+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:23.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:23.022+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:23.013+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:23.014+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:23.015+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:23.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:23.015+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:23.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:23.240+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: osdmap e757: 8 total, 8 up, 8 in 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:23.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: osdmap e757: 8 total, 8 up, 8 in 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:23.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:23.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:23.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:23.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:23.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:23.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:23.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:23.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:23.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:23.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:23.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:23.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:13:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:13:23] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:13:24.024 INFO:tasks.rados.rados.0.smithi118.stdout:3487: done (2 left) 2024-03-20T08:13:24.024 INFO:tasks.rados.rados.0.smithi118.stdout:3488: snap_remove snap 340 2024-03-20T08:13:24.341 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:24.024+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:24.341 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:24.101+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:24.341 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:24.101+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:24.341 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:24.101+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: pgmap v821: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: osdmap e758: 8 total, 8 up, 8 in 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:24.342 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:24.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:24.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:24.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:24.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:24.343 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: pgmap v821: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: osdmap e758: 8 total, 8 up, 8 in 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:24.191+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:24.192+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:24.192+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:24.192+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:24.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:24.192+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:24.425 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:24.025+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: pgmap v821: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 4.4 MiB/s wr, 12 op/s 2024-03-20T08:13:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: osdmap e758: 8 total, 8 up, 8 in 2024-03-20T08:13:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:24.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:24.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:24.025+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:24.427 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:24.025+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:24.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:24.341+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:24.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:24.341+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:24.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:24.341+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:24.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:24.499+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:24.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:24.499+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:24.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:24.606+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:24.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:24.606+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:25.030 INFO:tasks.rados.rados.0.smithi118.stdout:3488: done (2 left) 2024-03-20T08:13:25.030 INFO:tasks.rados.rados.0.smithi118.stdout:3489: delete oid 17 current snap is 341 2024-03-20T08:13:25.030 INFO:tasks.rados.rados.0.smithi118.stdout:3485: finishing rollback tid 0 to smithi11881747-23 2024-03-20T08:13:25.030 INFO:tasks.rados.rados.0.smithi118.stdout:3485: finishing rollback tid 1 to smithi11881747-23 2024-03-20T08:13:25.030 INFO:tasks.rados.rados.0.smithi118.stdout:3485: finishing rollback tid 2 to smithi11881747-23 2024-03-20T08:13:25.030 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 61745 (ObjNum 971 snap 321 seq_num 971) dirty exists 2024-03-20T08:13:25.037 INFO:tasks.rados.rados.0.smithi118.stdout:3485: done (2 left) 2024-03-20T08:13:25.037 INFO:tasks.rados.rados.0.smithi118.stdout:3486: done (1 left) 2024-03-20T08:13:25.037 INFO:tasks.rados.rados.0.smithi118.stdout:3489: done (0 left) 2024-03-20T08:13:25.037 INFO:tasks.rados.rados.0.smithi118.stdout:3490: snap_remove snap 339 2024-03-20T08:13:25.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:25.030+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:25.421 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:25.039+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:25.422 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:25.039+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:25.422 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:25.034+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:25.422 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:25.036+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:25.422 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:25.202+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:25.422 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:25.202+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:25.422 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:25.203+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: osdmap e759: 8 total, 8 up, 8 in 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:25.423 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:25.424 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:25.031+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:25.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:25.203+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:25.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:25.203+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:25.424 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:25.203+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:25.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:25.037+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:25.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:25.038+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:25.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:25.038+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:25.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:25.038+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:25.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: osdmap e759: 8 total, 8 up, 8 in 2024-03-20T08:13:25.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:25.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:25.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:25.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:25.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:25.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:25.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:25.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:25.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:25.426 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:25.031+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:25.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: osdmap e759: 8 total, 8 up, 8 in 2024-03-20T08:13:25.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:25.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:25.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:25.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:25.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:25.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:25.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:25.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:25.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:26.037 INFO:tasks.rados.rados.0.smithi118.stdout:3490: done (0 left) 2024-03-20T08:13:26.037 INFO:tasks.rados.rados.0.smithi118.stdout:3491: copy_from oid 31 from oid 28 current snap is 341 2024-03-20T08:13:26.037 INFO:tasks.rados.rados.0.smithi118.stdout:3492: copy_from oid 40 from oid 24 current snap is 341 2024-03-20T08:13:26.037 INFO:tasks.rados.rados.0.smithi118.stdout:3493: read oid 17 snap -1 2024-03-20T08:13:26.037 INFO:tasks.rados.rados.0.smithi118.stdout:3493: expect deleted 2024-03-20T08:13:26.037 INFO:tasks.rados.rados.0.smithi118.stdout:3494: rmattr oid 35 current snap is 341 2024-03-20T08:13:26.037 INFO:tasks.rados.rados.0.smithi118.stdout:3494: done (3 left) 2024-03-20T08:13:26.037 INFO:tasks.rados.rados.0.smithi118.stdout:3495: delete oid 10 current snap is 341 2024-03-20T08:13:26.044 INFO:tasks.rados.rados.0.smithi118.stdout:3491: finishing copy_from to smithi11881747-31 2024-03-20T08:13:26.044 INFO:tasks.rados.rados.0.smithi118.stdout:3491: got expected ENOENT (src dne) 2024-03-20T08:13:26.044 INFO:tasks.rados.rados.0.smithi118.stdout:3491: finishing copy_from racing read to smithi11881747-31 2024-03-20T08:13:26.044 INFO:tasks.rados.rados.0.smithi118.stdout:3491: done (3 left) 2024-03-20T08:13:26.044 INFO:tasks.rados.rados.0.smithi118.stdout:3493: done (2 left) 2024-03-20T08:13:26.044 INFO:tasks.rados.rados.0.smithi118.stdout:3495: done (1 left) 2024-03-20T08:13:26.045 INFO:tasks.rados.rados.0.smithi118.stdout:3496: delete oid 42 current snap is 341 2024-03-20T08:13:26.046 INFO:tasks.rados.rados.0.smithi118.stdout:3496: done (1 left) 2024-03-20T08:13:26.046 INFO:tasks.rados.rados.0.smithi118.stdout:3497: delete oid 36 current snap is 341 2024-03-20T08:13:26.052 INFO:tasks.rados.rados.0.smithi118.stdout:3497: done (1 left) 2024-03-20T08:13:26.052 INFO:tasks.rados.rados.0.smithi118.stdout:3498: rmattr oid 50 current snap is 341 2024-03-20T08:13:26.052 INFO:tasks.rados.rados.0.smithi118.stdout:3498: done (1 left) 2024-03-20T08:13:26.052 INFO:tasks.rados.rados.0.smithi118.stdout:3499: setattr oid 1 current snap is 341 2024-03-20T08:13:26.053 INFO:tasks.rados.rados.0.smithi118.stdout:3492: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:13:26.054 INFO:tasks.rados.rados.0.smithi118.stdout:3492: finishing copy_from to smithi11881747-40 2024-03-20T08:13:26.054 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 65296 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:13:26.055 INFO:tasks.rados.rados.0.smithi118.stdout:3492: done (1 left) 2024-03-20T08:13:26.055 INFO:tasks.rados.rados.0.smithi118.stdout:3500: rollback oid 32 current snap is 341 2024-03-20T08:13:26.055 INFO:tasks.rados.rados.0.smithi118.stdout:3500: done (1 left) 2024-03-20T08:13:26.055 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 341 2024-03-20T08:13:26.055 INFO:tasks.rados.rados.0.smithi118.stdout:3501: seq_num 1068 ranges {1736704=696320,2433024=401408} 2024-03-20T08:13:26.069 INFO:tasks.rados.rados.0.smithi118.stdout:3501: writing smithi11881747-39 from 1736704 to 2433024 tid 1 2024-03-20T08:13:26.072 INFO:tasks.rados.rados.0.smithi118.stdout:3501: writing smithi11881747-39 from 2433024 to 2834432 tid 2 2024-03-20T08:13:26.072 INFO:tasks.rados.rados.0.smithi118.stdout:3502: rollback oid 20 current snap is 341 2024-03-20T08:13:26.072 INFO:tasks.rados.rados.0.smithi118.stdout:3502: done (2 left) 2024-03-20T08:13:26.072 INFO:tasks.rados.rados.0.smithi118.stdout:3503: setattr oid 48 current snap is 341 2024-03-20T08:13:26.072 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 65287 (ObjNum 1034 snap 333 seq_num 1034) dirty exists 2024-03-20T08:13:26.072 INFO:tasks.rados.rados.0.smithi118.stdout:3501: finishing write tid 1 to smithi11881747-39 2024-03-20T08:13:26.074 INFO:tasks.rados.rados.0.smithi118.stdout:3499: done (2 left) 2024-03-20T08:13:26.074 INFO:tasks.rados.rados.0.smithi118.stdout:3504: read oid 42 snap -1 2024-03-20T08:13:26.074 INFO:tasks.rados.rados.0.smithi118.stdout:3504: expect deleted 2024-03-20T08:13:26.074 INFO:tasks.rados.rados.0.smithi118.stdout:3505: delete oid 21 current snap is 341 2024-03-20T08:13:26.075 INFO:tasks.rados.rados.0.smithi118.stdout:3501: finishing write tid 2 to smithi11881747-39 2024-03-20T08:13:26.075 INFO:tasks.rados.rados.0.smithi118.stdout:3501: finishing write tid 3 to smithi11881747-39 2024-03-20T08:13:26.076 INFO:tasks.rados.rados.0.smithi118.stdout:3501: finishing write tid 4 to smithi11881747-39 2024-03-20T08:13:26.077 INFO:tasks.rados.rados.0.smithi118.stdout:3501: oid 39 updating version 0 to 59855 2024-03-20T08:13:26.077 INFO:tasks.rados.rados.0.smithi118.stdout:3501: oid 39 updating version 59855 to 59856 2024-03-20T08:13:26.077 INFO:tasks.rados.rados.0.smithi118.stdout:3501: oid 39 updating version 59856 to 59857 2024-03-20T08:13:26.077 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 59857 (ObjNum 1068 snap 341 seq_num 1068) dirty exists 2024-03-20T08:13:26.077 INFO:tasks.rados.rados.0.smithi118.stdout:3501: left oid 39 (ObjNum 1068 snap 341 seq_num 1068) 2024-03-20T08:13:26.083 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 65297 (ObjNum 1025 snap 330 seq_num 1025) dirty exists 2024-03-20T08:13:26.083 INFO:tasks.rados.rados.0.smithi118.stdout:3501: done (3 left) 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3503: done (2 left) 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3504: done (1 left) 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3505: done (0 left) 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3506: read oid 18 snap -1 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3506: expect (ObjNum 1021 snap 329 seq_num 1021) 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3507: copy_from oid 50 from oid 38 current snap is 341 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3508: rmattr oid 29 current snap is 341 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3509: rmattr oid 4 current snap is 341 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3509: done (3 left) 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3510: read oid 31 snap -1 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3510: expect deleted 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3511: rollback oid 42 current snap is 341 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3511: done (4 left) 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3512: copy_from oid 49 from oid 32 current snap is 341 2024-03-20T08:13:26.084 INFO:tasks.rados.rados.0.smithi118.stdout:3513: delete oid 41 current snap is 341 2024-03-20T08:13:26.087 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 61746 (ObjNum 646 snap 212 seq_num 646) dirty exists 2024-03-20T08:13:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:3508: done (5 left) 2024-03-20T08:13:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:3510: done (4 left) 2024-03-20T08:13:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:3513: done (3 left) 2024-03-20T08:13:26.088 INFO:tasks.rados.rados.0.smithi118.stdout:3514: delete oid 41 current snap is 341 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:3514: done (3 left) 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:3515: read oid 29 snap -1 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:3515: expect (ObjNum 646 snap 212 seq_num 646) 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:3516: read oid 40 snap -1 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:3516: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:3517: rollback oid 12 current snap is 341 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:3517: done (5 left) 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 341 2024-03-20T08:13:26.093 INFO:tasks.rados.rados.0.smithi118.stdout:3518: seq_num 1072 ranges {270336=761856,1032192=401408} 2024-03-20T08:13:26.099 INFO:tasks.rados.rados.0.smithi118.stdout:3518: writing smithi11881747-24 from 270336 to 1032192 tid 1 2024-03-20T08:13:26.102 INFO:tasks.rados.rados.0.smithi118.stdout:3518: writing smithi11881747-24 from 1032192 to 1433600 tid 2 2024-03-20T08:13:26.102 INFO:tasks.rados.rados.0.smithi118.stdout:3519: copy_from oid 13 from oid 41 current snap is 341 2024-03-20T08:13:26.102 INFO:tasks.rados.rados.0.smithi118.stdout:3516: expect (ObjNum 589 snap 193 seq_num 589) 2024-03-20T08:13:26.118 INFO:tasks.rados.rados.0.smithi118.stdout:3515: expect (ObjNum 646 snap 212 seq_num 646) 2024-03-20T08:13:26.160 INFO:tasks.rados.rados.0.smithi118.stdout:3519: finishing copy_from to smithi11881747-13 2024-03-20T08:13:26.161 INFO:tasks.rados.rados.0.smithi118.stdout:3519: got expected ENOENT (src dne) 2024-03-20T08:13:26.161 INFO:tasks.rados.rados.0.smithi118.stdout:3519: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:13:26.161 INFO:tasks.rados.rados.0.smithi118.stdout:3506: expect (ObjNum 1021 snap 329 seq_num 1021) 2024-03-20T08:13:26.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:26.036+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:26.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:26.042+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:26.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:26.044+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:26.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:26.044+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:26.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:26.046+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:26.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:26.046+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:26.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:26.041+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:26.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:26.043+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:26.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:26.043+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: pgmap v824: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: osdmap e760: 8 total, 8 up, 8 in 2024-03-20T08:13:26.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:26.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:26.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:26.036+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:26.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:26.036+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:26.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:26.036+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:26.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:26.036+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:26.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:26.049+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:26.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:26.040+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:26.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:26.040+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:26.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:26.040+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:26.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:26.040+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:26.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:26.040+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: pgmap v824: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: osdmap e760: 8 total, 8 up, 8 in 2024-03-20T08:13:26.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: pgmap v824: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 11 op/s 2024-03-20T08:13:26.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: osdmap e760: 8 total, 8 up, 8 in 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:26.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:26.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:26.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:26.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:26.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:26.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:26.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:26.597 INFO:tasks.rados.rados.0.smithi118.stdout:3512: finishing copy_from racing read to smithi11881747-49 2024-03-20T08:13:26.597 INFO:tasks.rados.rados.0.smithi118.stdout:3518: finishing write tid 1 to smithi11881747-24 2024-03-20T08:13:26.597 INFO:tasks.rados.rados.0.smithi118.stdout:3512: finishing copy_from to smithi11881747-49 2024-03-20T08:13:26.597 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 34118 (ObjNum 1052 snap 337 seq_num 1052) dirty exists 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3507: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3518: finishing write tid 2 to smithi11881747-24 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3518: finishing write tid 3 to smithi11881747-24 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3506: done (6 left) 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3512: done (5 left) 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3515: done (4 left) 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3516: done (3 left) 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3519: done (2 left) 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 341 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3518: finishing write tid 4 to smithi11881747-24 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3518: oid 24 updating version 0 to 65305 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3518: oid 24 updating version 65305 to 65306 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3518: oid 24 version 65306 is already newer than 65304 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65306 (ObjNum 1072 snap 341 seq_num 1072) dirty exists 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3518: left oid 24 (ObjNum 1072 snap 341 seq_num 1072) 2024-03-20T08:13:26.598 INFO:tasks.rados.rados.0.smithi118.stdout:3520: seq_num 1073 ranges {1228800=778240,2007040=401408} 2024-03-20T08:13:26.607 INFO:tasks.rados.rados.0.smithi118.stdout:3520: writing smithi11881747-8 from 1228800 to 2007040 tid 1 2024-03-20T08:13:26.610 INFO:tasks.rados.rados.0.smithi118.stdout:3520: writing smithi11881747-8 from 2007040 to 2408448 tid 2 2024-03-20T08:13:26.610 INFO:tasks.rados.rados.0.smithi118.stdout:3518: done (2 left) 2024-03-20T08:13:26.610 INFO:tasks.rados.rados.0.smithi118.stdout:3521: delete oid 22 current snap is 341 2024-03-20T08:13:26.610 INFO:tasks.rados.rados.0.smithi118.stdout:3507: finishing copy_from to smithi11881747-50 2024-03-20T08:13:26.610 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 65296 (ObjNum 1060 snap 338 seq_num 1060) dirty exists 2024-03-20T08:13:26.615 INFO:tasks.rados.rados.0.smithi118.stdout:3507: done (2 left) 2024-03-20T08:13:26.615 INFO:tasks.rados.rados.0.smithi118.stdout:3521: done (1 left) 2024-03-20T08:13:26.615 INFO:tasks.rados.rados.0.smithi118.stdout:3522: read oid 6 snap -1 2024-03-20T08:13:26.615 INFO:tasks.rados.rados.0.smithi118.stdout:3522: expect (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:13:26.615 INFO:tasks.rados.rados.0.smithi118.stdout:3523: copy_from oid 37 from oid 40 current snap is 341 2024-03-20T08:13:26.615 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 341 2024-03-20T08:13:26.615 INFO:tasks.rados.rados.0.smithi118.stdout:3524: seq_num 1075 ranges {3817472=409600,4227072=671744,4898816=131072} 2024-03-20T08:13:26.638 INFO:tasks.rados.rados.0.smithi118.stdout:3524: writing smithi11881747-23 from 3817472 to 4227072 tid 1 2024-03-20T08:13:26.642 INFO:tasks.rados.rados.0.smithi118.stdout:3524: writing smithi11881747-23 from 4227072 to 4898816 tid 2 2024-03-20T08:13:26.644 INFO:tasks.rados.rados.0.smithi118.stdout:3524: writing smithi11881747-23 from 4898816 to 5029888 tid 3 2024-03-20T08:13:26.644 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 341 2024-03-20T08:13:26.644 INFO:tasks.rados.rados.0.smithi118.stdout:3520: finishing write tid 1 to smithi11881747-8 2024-03-20T08:13:26.644 INFO:tasks.rados.rados.0.smithi118.stdout:3520: finishing write tid 2 to smithi11881747-8 2024-03-20T08:13:26.644 INFO:tasks.rados.rados.0.smithi118.stdout:3520: finishing write tid 3 to smithi11881747-8 2024-03-20T08:13:26.644 INFO:tasks.rados.rados.0.smithi118.stdout:3525: seq_num 1076 ranges {344064=425984,770048=548864,1318912=253952} 2024-03-20T08:13:26.648 INFO:tasks.rados.rados.0.smithi118.stdout:3525: writing smithi11881747-20 from 344064 to 770048 tid 1 2024-03-20T08:13:26.652 INFO:tasks.rados.rados.0.smithi118.stdout:3525: writing smithi11881747-20 from 770048 to 1318912 tid 2 2024-03-20T08:13:26.654 INFO:tasks.rados.rados.0.smithi118.stdout:3525: writing smithi11881747-20 from 1318912 to 1572864 tid 3 2024-03-20T08:13:26.654 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 341 2024-03-20T08:13:26.654 INFO:tasks.rados.rados.0.smithi118.stdout:3523: finishing copy_from racing read to smithi11881747-37 2024-03-20T08:13:26.654 INFO:tasks.rados.rados.0.smithi118.stdout:3526: seq_num 1077 ranges {843776=442368,1286144=434176,1720320=368640} 2024-03-20T08:13:26.662 INFO:tasks.rados.rados.0.smithi118.stdout:3526: writing smithi11881747-43 from 843776 to 1286144 tid 1 2024-03-20T08:13:26.665 INFO:tasks.rados.rados.0.smithi118.stdout:3526: writing smithi11881747-43 from 1286144 to 1720320 tid 2 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3526: writing smithi11881747-43 from 1720320 to 2088960 tid 3 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 341 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3523: finishing copy_from to smithi11881747-37 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 65296 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3520: finishing write tid 4 to smithi11881747-8 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3520: oid 8 updating version 0 to 34121 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3520: oid 8 version 34121 is already newer than 34120 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3520: oid 8 version 34121 is already newer than 34119 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34121 (ObjNum 1073 snap 341 seq_num 1073) dirty exists 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3520: left oid 8 (ObjNum 1073 snap 341 seq_num 1073) 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3524: finishing write tid 1 to smithi11881747-23 2024-03-20T08:13:26.668 INFO:tasks.rados.rados.0.smithi118.stdout:3527: seq_num 1078 ranges {540672=458752,999424=720896,1720320=81920} 2024-03-20T08:13:26.673 INFO:tasks.rados.rados.0.smithi118.stdout:3527: writing smithi11881747-35 from 540672 to 999424 tid 1 2024-03-20T08:13:26.677 INFO:tasks.rados.rados.0.smithi118.stdout:3527: writing smithi11881747-35 from 999424 to 1720320 tid 2 2024-03-20T08:13:26.679 INFO:tasks.rados.rados.0.smithi118.stdout:3527: writing smithi11881747-35 from 1720320 to 1802240 tid 3 2024-03-20T08:13:26.679 INFO:tasks.rados.rados.0.smithi118.stdout:3520: done (6 left) 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3523: done (5 left) 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3528: read oid 11 snap -1 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3528: expect (ObjNum 969 snap 321 seq_num 969) 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3529: rollback oid 40 current snap is 341 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3529: done (6 left) 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3530: read oid 47 snap -1 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3530: expect (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3531: delete oid 8 current snap is 341 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3524: finishing write tid 2 to smithi11881747-23 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3524: finishing write tid 3 to smithi11881747-23 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3524: finishing write tid 4 to smithi11881747-23 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3524: finishing write tid 5 to smithi11881747-23 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3524: oid 23 updating version 0 to 61749 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3524: oid 23 updating version 61749 to 61751 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3524: oid 23 version 61751 is already newer than 61748 2024-03-20T08:13:26.680 INFO:tasks.rados.rados.0.smithi118.stdout:3524: oid 23 version 61751 is already newer than 61750 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 61751 (ObjNum 1075 snap 341 seq_num 1075) dirty exists 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3524: left oid 23 (ObjNum 1075 snap 341 seq_num 1075) 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: finishing write tid 1 to smithi11881747-20 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: finishing write tid 2 to smithi11881747-20 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: finishing write tid 3 to smithi11881747-20 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: finishing write tid 4 to smithi11881747-20 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: finishing write tid 5 to smithi11881747-20 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: oid 20 updating version 0 to 59859 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: oid 20 updating version 59859 to 59861 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: oid 20 version 59861 is already newer than 59860 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: oid 20 version 59861 is already newer than 59858 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 59861 (ObjNum 1076 snap 341 seq_num 1076) dirty exists 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3525: left oid 20 (ObjNum 1076 snap 341 seq_num 1076) 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3526: finishing write tid 1 to smithi11881747-43 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3526: finishing write tid 2 to smithi11881747-43 2024-03-20T08:13:26.681 INFO:tasks.rados.rados.0.smithi118.stdout:3526: finishing write tid 3 to smithi11881747-43 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:3526: finishing write tid 4 to smithi11881747-43 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:3526: finishing write tid 5 to smithi11881747-43 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:3526: oid 43 updating version 0 to 65289 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:3526: oid 43 version 65289 is already newer than 65288 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:3526: oid 43 updating version 65289 to 65290 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:3526: oid 43 updating version 65290 to 65291 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65291 (ObjNum 1077 snap 341 seq_num 1077) dirty exists 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:3526: left oid 43 (ObjNum 1077 snap 341 seq_num 1077) 2024-03-20T08:13:26.682 INFO:tasks.rados.rados.0.smithi118.stdout:3522: expect (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:13:27.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: osdmap e761: 8 total, 8 up, 8 in 2024-03-20T08:13:27.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:27.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:27.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:27.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: osdmap e761: 8 total, 8 up, 8 in 2024-03-20T08:13:27.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:27.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: osdmap e761: 8 total, 8 up, 8 in 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:27.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18a=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:27.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:27.983 INFO:tasks.rados.rados.0.smithi118.stdout:3527: finishing write tid 1 to smithi11881747-35 2024-03-20T08:13:27.983 INFO:tasks.rados.rados.0.smithi118.stdout:3522: done (7 left) 2024-03-20T08:13:27.983 INFO:tasks.rados.rados.0.smithi118.stdout:3524: done (6 left) 2024-03-20T08:13:27.983 INFO:tasks.rados.rados.0.smithi118.stdout:3525: done (5 left) 2024-03-20T08:13:27.983 INFO:tasks.rados.rados.0.smithi118.stdout:3526: done (4 left) 2024-03-20T08:13:27.983 INFO:tasks.rados.rados.0.smithi118.stdout:3531: done (3 left) 2024-03-20T08:13:27.983 INFO:tasks.rados.rados.0.smithi118.stdout:3532: copy_from oid 17 from oid 16 current snap is 341 2024-03-20T08:13:27.983 INFO:tasks.rados.rados.0.smithi118.stdout:3530: expect (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:13:28.012 INFO:tasks.rados.rados.0.smithi118.stdout:3527: finishing write tid 2 to smithi11881747-35 2024-03-20T08:13:28.012 INFO:tasks.rados.rados.0.smithi118.stdout:3527: finishing write tid 3 to smithi11881747-35 2024-03-20T08:13:28.012 INFO:tasks.rados.rados.0.smithi118.stdout:3530: done (3 left) 2024-03-20T08:13:28.012 INFO:tasks.rados.rados.0.smithi118.stdout:3533: snap_create 2024-03-20T08:13:28.012 INFO:tasks.rados.rados.0.smithi118.stdout:3527: finishing write tid 4 to smithi11881747-35 2024-03-20T08:13:28.012 INFO:tasks.rados.rados.0.smithi118.stdout:3527: finishing write tid 5 to smithi11881747-35 2024-03-20T08:13:28.012 INFO:tasks.rados.rados.0.smithi118.stdout:3527: oid 35 updating version 0 to 65298 2024-03-20T08:13:28.012 INFO:tasks.rados.rados.0.smithi118.stdout:3527: oid 35 updating version 65298 to 65300 2024-03-20T08:13:28.013 INFO:tasks.rados.rados.0.smithi118.stdout:3527: oid 35 version 65300 is already newer than 65299 2024-03-20T08:13:28.013 INFO:tasks.rados.rados.0.smithi118.stdout:3527: oid 35 version 65300 is already newer than 65297 2024-03-20T08:13:28.013 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 65300 (ObjNum 1078 snap 341 seq_num 1078) dirty exists 2024-03-20T08:13:28.013 INFO:tasks.rados.rados.0.smithi118.stdout:3527: left oid 35 (ObjNum 1078 snap 341 seq_num 1078) 2024-03-20T08:13:28.013 INFO:tasks.rados.rados.0.smithi118.stdout:3528: expect (ObjNum 969 snap 321 seq_num 969) 2024-03-20T08:13:28.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-mon[87080]: pgmap v827: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.5 MiB/s rd, 774 KiB/s wr, 14 op/s 2024-03-20T08:13:28.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:28.047+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:28.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:28.102+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:28.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:28.102+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-mon[93121]: pgmap v827: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.5 MiB/s rd, 774 KiB/s wr, 14 op/s 2024-03-20T08:13:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-mon[95692]: pgmap v827: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.5 MiB/s rd, 774 KiB/s wr, 14 op/s 2024-03-20T08:13:28.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:28.049+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:28.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:28.049+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:28.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:28.049+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:28.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:28.049+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:28.424 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:28.072+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:28.507 INFO:tasks.rados.rados.0.smithi118.stdout:3532: finishing copy_from racing read to smithi11881747-17 2024-03-20T08:13:28.507 INFO:tasks.rados.rados.0.smithi118.stdout:3532: finishing copy_from to smithi11881747-17 2024-03-20T08:13:28.507 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 61739 (ObjNum 1040 snap 336 seq_num 1040) dirty exists 2024-03-20T08:13:28.507 INFO:tasks.rados.rados.0.smithi118.stdout:3527: done (3 left) 2024-03-20T08:13:28.508 INFO:tasks.rados.rados.0.smithi118.stdout:3528: done (2 left) 2024-03-20T08:13:28.508 INFO:tasks.rados.rados.0.smithi118.stdout:3532: done (1 left) 2024-03-20T08:13:28.508 INFO:tasks.rados.rados.0.smithi118.stdout:3533: done (0 left) 2024-03-20T08:13:28.508 INFO:tasks.rados.rados.0.smithi118.stdout:3534: delete oid 13 current snap is 342 2024-03-20T08:13:28.510 INFO:tasks.rados.rados.0.smithi118.stdout:3534: done (0 left) 2024-03-20T08:13:28.510 INFO:tasks.rados.rados.0.smithi118.stdout:3535: delete oid 2 current snap is 342 2024-03-20T08:13:28.512 INFO:tasks.rados.rados.0.smithi118.stdout:3535: done (0 left) 2024-03-20T08:13:28.512 INFO:tasks.rados.rados.0.smithi118.stdout:3536: snap_create 2024-03-20T08:13:28.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:28.438+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:28.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:28.438+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:28.437+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:28.437+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:28.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:28.438+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:28.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:28.436+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:28.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:28.437+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:28.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:28.437+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:28.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:28.437+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:29.052 INFO:tasks.rados.rados.0.smithi118.stdout:3536: done (0 left) 2024-03-20T08:13:29.052 INFO:tasks.rados.rados.0.smithi118.stdout:3537: read oid 49 snap -1 2024-03-20T08:13:29.052 INFO:tasks.rados.rados.0.smithi118.stdout:3537: expect (ObjNum 1052 snap 337 seq_num 1052) 2024-03-20T08:13:29.053 INFO:tasks.rados.rados.0.smithi118.stdout:3538: snap_remove snap 342 2024-03-20T08:13:29.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:29.053+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:29.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:29.053+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:29.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:29.054+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:29.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:29.053+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:29.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:29.054+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:29.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:29.054+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:29.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: osdmap e762: 8 total, 8 up, 8 in 2024-03-20T08:13:29.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:29.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:29.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:29.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:29.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:29.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:29.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:29.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:29.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:29.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:29.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:29.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:29.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:29.052+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:29.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:29.053+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:29.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:29.060+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:29.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:29.072+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:29.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:29.073+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: osdmap e762: 8 total, 8 up, 8 in 2024-03-20T08:13:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:29.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: osdmap e762: 8 total, 8 up, 8 in 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:29.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:29.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:29.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:29.538+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:29.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:29.538+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:29.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:29.539+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:29.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:29.537+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:29.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:29.537+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:29.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:29.537+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:29.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:29.537+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:29.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:29.537+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:30.058 INFO:tasks.rados.rados.0.smithi118.stdout:3538: done (1 left) 2024-03-20T08:13:30.058 INFO:tasks.rados.rados.0.smithi118.stdout:3539: snap_remove snap 341 2024-03-20T08:13:30.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:30.058+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: pgmap v829: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 2.0 MiB/s rd, 741 KiB/s wr, 18 op/s 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: osdmap e763: 8 total, 8 up, 8 in 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:30.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:30.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:30.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:30.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:30.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:30.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:30.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:30.058+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:30.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:30.058+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: pgmap v829: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 2.0 MiB/s rd, 741 KiB/s wr, 18 op/s 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: osdmap e763: 8 total, 8 up, 8 in 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:30.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: pgmap v829: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 2.0 MiB/s rd, 741 KiB/s wr, 18 op/s 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:30.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: osdmap e763: 8 total, 8 up, 8 in 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:30.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:30.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:30.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:30.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:30.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:30.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:30.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:30.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:30.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:30.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:30.439+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:30.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:30.439+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:30.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:30.439+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:30.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:30.439+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:30.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:30.439+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:30.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:30.440+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:30.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:30.440+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:30.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:30.438+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:30.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:30.438+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:30.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:30.438+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:30.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:30.437+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:30.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:30.437+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:30.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:30.438+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:30.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:30.439+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:31.064 INFO:tasks.rados.rados.0.smithi118.stdout:3539: done (1 left) 2024-03-20T08:13:31.064 INFO:tasks.rados.rados.0.smithi118.stdout:3540: delete oid 14 current snap is 343 2024-03-20T08:13:31.064 INFO:tasks.rados.rados.0.smithi118.stdout:3537: expect (ObjNum 1052 snap 337 seq_num 1052) 2024-03-20T08:13:31.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:31.063+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:31.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:31.072+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:31.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:31.073+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:31.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:31.073+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: osdmap e764: 8 total, 8 up, 8 in 2024-03-20T08:13:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:31.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:31.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:31.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:31.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:31.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:31.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:31.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:31.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:31.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:31.073+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:31.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:31.073+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:31.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:31.064+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:31.064+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:31.064+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:31.065+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:31.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:31.065+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: osdmap e764: 8 total, 8 up, 8 in 2024-03-20T08:13:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:31.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:31.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:31.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:31.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:31.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:31.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:31.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:31.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:31.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: osdmap e764: 8 total, 8 up, 8 in 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:31.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:31.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:31.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:31.063+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:31.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:31.063+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:31.427 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:31.070+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:31.529 INFO:tasks.rados.rados.0.smithi118.stdout:3537: done (1 left) 2024-03-20T08:13:31.529 INFO:tasks.rados.rados.0.smithi118.stdout:3540: done (0 left) 2024-03-20T08:13:31.529 INFO:tasks.rados.rados.0.smithi118.stdout:3541: rollback oid 19 current snap is 343 2024-03-20T08:13:31.529 INFO:tasks.rados.rados.0.smithi118.stdout:3541: done (0 left) 2024-03-20T08:13:31.530 INFO:tasks.rados.rados.0.smithi118.stdout:3542: copy_from oid 50 from oid 14 current snap is 343 2024-03-20T08:13:31.530 INFO:tasks.rados.rados.0.smithi118.stdout:3543: read oid 7 snap -1 2024-03-20T08:13:31.530 INFO:tasks.rados.rados.0.smithi118.stdout:3543: expect (ObjNum 1035 snap 335 seq_num 1035) 2024-03-20T08:13:31.530 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 343 2024-03-20T08:13:31.530 INFO:tasks.rados.rados.0.smithi118.stdout:3544: seq_num 1083 ranges {540672=540672,1081344=589824,1671168=212992} 2024-03-20T08:13:31.536 INFO:tasks.rados.rados.0.smithi118.stdout:3544: writing smithi11881747-26 from 540672 to 1081344 tid 1 2024-03-20T08:13:31.539 INFO:tasks.rados.rados.0.smithi118.stdout:3544: writing smithi11881747-26 from 1081344 to 1671168 tid 2 2024-03-20T08:13:31.540 INFO:tasks.rados.rados.0.smithi118.stdout:3544: writing smithi11881747-26 from 1671168 to 1884160 tid 3 2024-03-20T08:13:31.540 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 343 2024-03-20T08:13:31.541 INFO:tasks.rados.rados.0.smithi118.stdout:3542: finishing copy_from to smithi11881747-50 2024-03-20T08:13:31.541 INFO:tasks.rados.rados.0.smithi118.stdout:3542: got expected ENOENT (src dne) 2024-03-20T08:13:31.541 INFO:tasks.rados.rados.0.smithi118.stdout:3545: seq_num 1084 ranges {5939200=557056,6496256=475136,6971392=327680} 2024-03-20T08:13:31.576 INFO:tasks.rados.rados.0.smithi118.stdout:3545: writing smithi11881747-18 from 5939200 to 6496256 tid 1 2024-03-20T08:13:31.579 INFO:tasks.rados.rados.0.smithi118.stdout:3545: writing smithi11881747-18 from 6496256 to 6971392 tid 2 2024-03-20T08:13:31.581 INFO:tasks.rados.rados.0.smithi118.stdout:3545: writing smithi11881747-18 from 6971392 to 7299072 tid 3 2024-03-20T08:13:31.581 INFO:tasks.rados.rados.0.smithi118.stdout:3546: delete oid 20 current snap is 343 2024-03-20T08:13:31.581 INFO:tasks.rados.rados.0.smithi118.stdout:3542: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:13:31.581 INFO:tasks.rados.rados.0.smithi118.stdout:3543: expect (ObjNum 1035 snap 335 seq_num 1035) 2024-03-20T08:13:31.604 INFO:tasks.rados.rados.0.smithi118.stdout:3544: finishing write tid 1 to smithi11881747-26 2024-03-20T08:13:31.604 INFO:tasks.rados.rados.0.smithi118.stdout:3544: finishing write tid 2 to smithi11881747-26 2024-03-20T08:13:31.604 INFO:tasks.rados.rados.0.smithi118.stdout:3544: finishing write tid 3 to smithi11881747-26 2024-03-20T08:13:31.604 INFO:tasks.rados.rados.0.smithi118.stdout:3544: finishing write tid 4 to smithi11881747-26 2024-03-20T08:13:31.604 INFO:tasks.rados.rados.0.smithi118.stdout:3544: finishing write tid 5 to smithi11881747-26 2024-03-20T08:13:31.604 INFO:tasks.rados.rados.0.smithi118.stdout:3544: oid 26 updating version 0 to 65308 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3544: oid 26 updating version 65308 to 65311 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3544: oid 26 version 65311 is already newer than 65309 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3544: oid 26 version 65311 is already newer than 65310 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65311 (ObjNum 1083 snap 343 seq_num 1083) dirty exists 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3544: left oid 26 (ObjNum 1083 snap 343 seq_num 1083) 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3545: finishing write tid 1 to smithi11881747-18 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3542: done (4 left) 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3543: done (3 left) 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3544: done (2 left) 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3546: done (1 left) 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3547: rmattr oid 8 current snap is 343 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3547: done (1 left) 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 343 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3545: finishing write tid 2 to smithi11881747-18 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3545: finishing write tid 3 to smithi11881747-18 2024-03-20T08:13:31.605 INFO:tasks.rados.rados.0.smithi118.stdout:3545: finishing write tid 4 to smithi11881747-18 2024-03-20T08:13:31.606 INFO:tasks.rados.rados.0.smithi118.stdout:3548: seq_num 1086 ranges {0=598016,598016=647168,1245184=155648} 2024-03-20T08:13:31.607 INFO:tasks.rados.rados.0.smithi118.stdout:3548: writing smithi11881747-36 from 0 to 598016 tid 1 2024-03-20T08:13:31.610 INFO:tasks.rados.rados.0.smithi118.stdout:3548: writing smithi11881747-36 from 598016 to 1245184 tid 2 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3548: writing smithi11881747-36 from 1245184 to 1400832 tid 3 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3549: rollback oid 3 current snap is 343 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3549: done (2 left) 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3550: rollback oid 24 current snap is 343 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3550: done (2 left) 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3551: read oid 21 snap -1 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3551: expect deleted 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 343 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3545: finishing write tid 5 to smithi11881747-18 2024-03-20T08:13:31.612 INFO:tasks.rados.rados.0.smithi118.stdout:3545: oid 18 updating version 0 to 59863 2024-03-20T08:13:31.613 INFO:tasks.rados.rados.0.smithi118.stdout:3545: oid 18 updating version 59863 to 59865 2024-03-20T08:13:31.613 INFO:tasks.rados.rados.0.smithi118.stdout:3545: oid 18 version 59865 is already newer than 59862 2024-03-20T08:13:31.613 INFO:tasks.rados.rados.0.smithi118.stdout:3545: oid 18 version 59865 is already newer than 59864 2024-03-20T08:13:31.613 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 59865 (ObjNum 1084 snap 343 seq_num 1084) dirty exists 2024-03-20T08:13:31.613 INFO:tasks.rados.rados.0.smithi118.stdout:3545: left oid 18 (ObjNum 1084 snap 343 seq_num 1084) 2024-03-20T08:13:31.613 INFO:tasks.rados.rados.0.smithi118.stdout:3552: seq_num 1087 ranges {11313152=614400,11927552=532480,12460032=270336} 2024-03-20T08:13:31.677 INFO:tasks.rados.rados.0.smithi118.stdout:3552: writing smithi11881747-48 from 11313152 to 11927552 tid 1 2024-03-20T08:13:31.680 INFO:tasks.rados.rados.0.smithi118.stdout:3552: writing smithi11881747-48 from 11927552 to 12460032 tid 2 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3552: writing smithi11881747-48 from 12460032 to 12730368 tid 3 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3545: done (3 left) 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3553: snap_create 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: finishing write tid 1 to smithi11881747-36 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: finishing write tid 2 to smithi11881747-36 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: finishing write tid 3 to smithi11881747-36 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: finishing write tid 4 to smithi11881747-36 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: finishing write tid 5 to smithi11881747-36 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: oid 36 updating version 0 to 45655 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: oid 36 version 45655 is already newer than 45653 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: oid 36 version 45655 is already newer than 45654 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: oid 36 version 45655 is already newer than 45652 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 45655 (ObjNum 1086 snap 343 seq_num 1086) dirty exists 2024-03-20T08:13:31.682 INFO:tasks.rados.rados.0.smithi118.stdout:3548: left oid 36 (ObjNum 1086 snap 343 seq_num 1086) 2024-03-20T08:13:31.691 INFO:tasks.rados.rados.0.smithi118.stdout:3552: finishing write tid 1 to smithi11881747-48 2024-03-20T08:13:31.695 INFO:tasks.rados.rados.0.smithi118.stdout:3552: finishing write tid 2 to smithi11881747-48 2024-03-20T08:13:31.695 INFO:tasks.rados.rados.0.smithi118.stdout:3552: finishing write tid 3 to smithi11881747-48 2024-03-20T08:13:31.695 INFO:tasks.rados.rados.0.smithi118.stdout:3552: finishing write tid 4 to smithi11881747-48 2024-03-20T08:13:31.698 INFO:tasks.rados.rados.0.smithi118.stdout:3552: finishing write tid 5 to smithi11881747-48 2024-03-20T08:13:31.699 INFO:tasks.rados.rados.0.smithi118.stdout:3552: oid 48 updating version 0 to 65302 2024-03-20T08:13:31.699 INFO:tasks.rados.rados.0.smithi118.stdout:3552: oid 48 updating version 65302 to 65303 2024-03-20T08:13:31.699 INFO:tasks.rados.rados.0.smithi118.stdout:3552: oid 48 version 65303 is already newer than 65301 2024-03-20T08:13:31.699 INFO:tasks.rados.rados.0.smithi118.stdout:3552: oid 48 version 65303 is already newer than 65300 2024-03-20T08:13:31.699 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 65303 (ObjNum 1087 snap 343 seq_num 1087) dirty exists 2024-03-20T08:13:31.699 INFO:tasks.rados.rados.0.smithi118.stdout:3552: left oid 48 (ObjNum 1087 snap 343 seq_num 1087) 2024-03-20T08:13:31.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:31.543+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:31.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:31.543+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:31.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:31.534+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:31.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:31.534+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:31.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:31.534+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:32.068 INFO:tasks.rados.rados.0.smithi118.stdout:3548: done (3 left) 2024-03-20T08:13:32.068 INFO:tasks.rados.rados.0.smithi118.stdout:3551: done (2 left) 2024-03-20T08:13:32.068 INFO:tasks.rados.rados.0.smithi118.stdout:3552: done (1 left) 2024-03-20T08:13:32.068 INFO:tasks.rados.rados.0.smithi118.stdout:3553: done (0 left) 2024-03-20T08:13:32.068 INFO:tasks.rados.rados.0.smithi118.stdout:3554: snap_create 2024-03-20T08:13:32.314 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:32.068+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:32.314 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:32.104+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:32.314 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:32.104+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:32.314 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:32.073+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:32.315 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:32.073+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:32.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:32.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:32.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:32.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:32.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:32.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: pgmap v832: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 17 MiB/s rd, 8.3 MiB/s wr, 26 op/s 2024-03-20T08:13:32.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: osdmap e765: 8 total, 8 up, 8 in 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:32.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:32.329 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:32.068+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:32.330 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:32.068+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:32.330 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:32.068+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:32.330 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:32.293+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:32.330 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:32.293+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:32.330 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:32.293+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:32.330 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:32.293+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: pgmap v832: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 17 MiB/s rd, 8.3 MiB/s wr, 26 op/s 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: osdmap e765: 8 total, 8 up, 8 in 2024-03-20T08:13:32.331 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:32.332 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: pgmap v832: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 17 MiB/s rd, 8.3 MiB/s wr, 26 op/s 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 184=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: osdmap e765: 8 total, 8 up, 8 in 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:32.333 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:32.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:32.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:32.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:32.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:32.334 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:13:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:13:32.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:32.331+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:32.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:32.331+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:32.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:32.331+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:32.674 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:32.331+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:32.674 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:32.330+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:33.069 INFO:tasks.rados.rados.0.smithi118.stdout:3554: done (0 left) 2024-03-20T08:13:33.069 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 345 2024-03-20T08:13:33.069 INFO:tasks.rados.rados.0.smithi118.stdout:3555: seq_num 1088 ranges {6488064=630784,7118848=466944,7585792=335872} 2024-03-20T08:13:33.108 INFO:tasks.rados.rados.0.smithi118.stdout:3555: writing smithi11881747-32 from 6488064 to 7118848 tid 1 2024-03-20T08:13:33.111 INFO:tasks.rados.rados.0.smithi118.stdout:3555: writing smithi11881747-32 from 7118848 to 7585792 tid 2 2024-03-20T08:13:33.114 INFO:tasks.rados.rados.0.smithi118.stdout:3555: writing smithi11881747-32 from 7585792 to 7921664 tid 3 2024-03-20T08:13:33.114 INFO:tasks.rados.rados.0.smithi118.stdout:3556: read oid 50 snap 343 2024-03-20T08:13:33.114 INFO:tasks.rados.rados.0.smithi118.stdout:3556: expect (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:13:33.114 INFO:tasks.rados.rados.0.smithi118.stdout:3557: copy_from oid 45 from oid 16 current snap is 345 2024-03-20T08:13:33.114 INFO:tasks.rados.rados.0.smithi118.stdout:3558: snap_create 2024-03-20T08:13:33.118 INFO:tasks.rados.rados.0.smithi118.stdout:3555: finishing write tid 1 to smithi11881747-32 2024-03-20T08:13:33.125 INFO:tasks.rados.rados.0.smithi118.stdout:3555: finishing write tid 2 to smithi11881747-32 2024-03-20T08:13:33.125 INFO:tasks.rados.rados.0.smithi118.stdout:3555: finishing write tid 3 to smithi11881747-32 2024-03-20T08:13:33.128 INFO:tasks.rados.rados.0.smithi118.stdout:3555: finishing write tid 4 to smithi11881747-32 2024-03-20T08:13:33.132 INFO:tasks.rados.rados.0.smithi118.stdout:3555: finishing write tid 5 to smithi11881747-32 2024-03-20T08:13:33.132 INFO:tasks.rados.rados.0.smithi118.stdout:3555: oid 32 updating version 0 to 34124 2024-03-20T08:13:33.132 INFO:tasks.rados.rados.0.smithi118.stdout:3555: oid 32 updating version 34124 to 34125 2024-03-20T08:13:33.132 INFO:tasks.rados.rados.0.smithi118.stdout:3555: oid 32 updating version 34125 to 34126 2024-03-20T08:13:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:3555: oid 32 version 34126 is already newer than 34123 2024-03-20T08:13:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34126 (ObjNum 1088 snap 345 seq_num 1088) dirty exists 2024-03-20T08:13:33.133 INFO:tasks.rados.rados.0.smithi118.stdout:3555: left oid 32 (ObjNum 1088 snap 345 seq_num 1088) 2024-03-20T08:13:33.167 INFO:tasks.rados.rados.0.smithi118.stdout:3557: finishing copy_from racing read to smithi11881747-45 2024-03-20T08:13:33.168 INFO:tasks.rados.rados.0.smithi118.stdout:3557: finishing copy_from to smithi11881747-45 2024-03-20T08:13:33.168 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 61739 (ObjNum 1040 snap 336 seq_num 1040) dirty exists 2024-03-20T08:13:33.190 INFO:tasks.rados.rados.0.smithi118.stdout:3556: expect (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:13:33.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:33.124+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:33.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:33.124+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:33.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:33.114+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:33.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:33.114+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:33.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:33.114+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:33.390 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:33.071+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:33.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:33.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:33.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: osdmap e766: 8 total, 8 up, 8 in 2024-03-20T08:13:33.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:33.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:33.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:33.113+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:33.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:33.114+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:33.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:33.069+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:33.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:33.072+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:33.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:33.072+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:33.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:33.117+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:33.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:33.118+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:33.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:33.118+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:33.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:33.118+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:33.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:33.120+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:33.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: osdmap e766: 8 total, 8 up, 8 in 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:33.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: osdmap e766: 8 total, 8 up, 8 in 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:33.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:33.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:33.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:33.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:33.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:33.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:33.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:33.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:33.394+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:33.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:33.394+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:33.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:33.394+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:13:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:13:33] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:13:34.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:34.082+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:34.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: pgmap v835: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 24 MiB/s rd, 9.1 MiB/s wr, 19 op/s 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: osdmap e767: 8 total, 8 up, 8 in 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:34.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:34.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:34.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:34.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:34.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:34.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:34.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:34.083+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:34.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:34.084+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:34.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:34.084+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: pgmap v835: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 24 MiB/s rd, 9.1 MiB/s wr, 19 op/s 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: osdmap e767: 8 total, 8 up, 8 in 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:34.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:34.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: pgmap v835: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 24 MiB/s rd, 9.1 MiB/s wr, 19 op/s 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: osdmap e767: 8 total, 8 up, 8 in 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17f=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:34.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:34.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:34.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:34.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:34.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:34.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:34.083+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:34.488 INFO:tasks.rados.rados.0.smithi118.stdout:3555: done (3 left) 2024-03-20T08:13:34.489 INFO:tasks.rados.rados.0.smithi118.stdout:3556: done (2 left) 2024-03-20T08:13:34.489 INFO:tasks.rados.rados.0.smithi118.stdout:3557: done (1 left) 2024-03-20T08:13:34.489 INFO:tasks.rados.rados.0.smithi118.stdout:3558: done (0 left) 2024-03-20T08:13:34.490 INFO:tasks.rados.rados.0.smithi118.stdout:3559: read oid 34 snap -1 2024-03-20T08:13:34.490 INFO:tasks.rados.rados.0.smithi118.stdout:3559: expect deleted 2024-03-20T08:13:34.490 INFO:tasks.rados.rados.0.smithi118.stdout:3560: copy_from oid 11 from oid 38 current snap is 346 2024-03-20T08:13:34.490 INFO:tasks.rados.rados.0.smithi118.stdout:3561: snap_create 2024-03-20T08:13:34.538 INFO:tasks.rados.rados.0.smithi118.stdout:3560: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:13:34.539 INFO:tasks.rados.rados.0.smithi118.stdout:3560: finishing copy_from to smithi11881747-11 2024-03-20T08:13:34.540 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 65296 (ObjNum 1060 snap 338 seq_num 1060) dirty exists 2024-03-20T08:13:34.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:34.521+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:34.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:34.521+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:34.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:34.521+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:34.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:34.473+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:35.088 INFO:tasks.rados.rados.0.smithi118.stdout:3559: done (2 left) 2024-03-20T08:13:35.088 INFO:tasks.rados.rados.0.smithi118.stdout:3560: done (1 left) 2024-03-20T08:13:35.088 INFO:tasks.rados.rados.0.smithi118.stdout:3561: done (0 left) 2024-03-20T08:13:35.088 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 347 2024-03-20T08:13:35.088 INFO:tasks.rados.rados.0.smithi118.stdout:3562: seq_num 1089 ranges {1884160=647168,2531328=745472,3276800=57344} 2024-03-20T08:13:35.101 INFO:tasks.rados.rados.0.smithi118.stdout:3562: writing smithi11881747-26 from 1884160 to 2531328 tid 1 2024-03-20T08:13:35.105 INFO:tasks.rados.rados.0.smithi118.stdout:3562: writing smithi11881747-26 from 2531328 to 3276800 tid 2 2024-03-20T08:13:35.106 INFO:tasks.rados.rados.0.smithi118.stdout:3562: writing smithi11881747-26 from 3276800 to 3334144 tid 3 2024-03-20T08:13:35.106 INFO:tasks.rados.rados.0.smithi118.stdout:3563: rollback oid 10 current snap is 347 2024-03-20T08:13:35.106 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 344 2024-03-20T08:13:35.106 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 347 2024-03-20T08:13:35.106 INFO:tasks.rados.rados.0.smithi118.stdout:3564: seq_num 1090 ranges {6488064=663552,7151616=630784,7782400=172032} 2024-03-20T08:13:35.147 INFO:tasks.rados.rados.0.smithi118.stdout:3564: writing smithi11881747-49 from 6488064 to 7151616 tid 1 2024-03-20T08:13:35.151 INFO:tasks.rados.rados.0.smithi118.stdout:3564: writing smithi11881747-49 from 7151616 to 7782400 tid 2 2024-03-20T08:13:35.153 INFO:tasks.rados.rados.0.smithi118.stdout:3564: writing smithi11881747-49 from 7782400 to 7954432 tid 3 2024-03-20T08:13:35.153 INFO:tasks.rados.rados.0.smithi118.stdout:3565: read oid 25 snap -1 2024-03-20T08:13:35.153 INFO:tasks.rados.rados.0.smithi118.stdout:3565: expect deleted 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3566: read oid 24 snap -1 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3566: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3567: copy_from oid 35 from oid 3 current snap is 347 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3563: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 65286 (ObjNum 33 snap 0 seq_num 321791152) dirty dne 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3562: finishing write tid 1 to smithi11881747-26 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3562: finishing write tid 2 to smithi11881747-26 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3562: finishing write tid 3 to smithi11881747-26 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3563: done (5 left) 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3568: snap_create 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3562: finishing write tid 4 to smithi11881747-26 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3562: finishing write tid 5 to smithi11881747-26 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3562: oid 26 updating version 0 to 65314 2024-03-20T08:13:35.154 INFO:tasks.rados.rados.0.smithi118.stdout:3562: oid 26 updating version 65314 to 65315 2024-03-20T08:13:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:3562: oid 26 updating version 65315 to 65316 2024-03-20T08:13:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:3562: oid 26 version 65316 is already newer than 65313 2024-03-20T08:13:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65316 (ObjNum 1089 snap 347 seq_num 1089) dirty exists 2024-03-20T08:13:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:3562: left oid 26 (ObjNum 1089 snap 347 seq_num 1089) 2024-03-20T08:13:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:3564: finishing write tid 1 to smithi11881747-49 2024-03-20T08:13:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:3564: finishing write tid 2 to smithi11881747-49 2024-03-20T08:13:35.155 INFO:tasks.rados.rados.0.smithi118.stdout:3564: finishing write tid 3 to smithi11881747-49 2024-03-20T08:13:35.156 INFO:tasks.rados.rados.0.smithi118.stdout:3564: finishing write tid 4 to smithi11881747-49 2024-03-20T08:13:35.158 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:35.089+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:35.158 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:34.905+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:35.158 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:34.905+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:35.158 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:35.106+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:35.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:35.112+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:35.159 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:35.112+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:35.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-mon[87080]: osdmap e768: 8 total, 8 up, 8 in 2024-03-20T08:13:35.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:35.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:35.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:35.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:35.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:35.160 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:35.038+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:35.160 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:35.038+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:35.160 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:35.110+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:35.160 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:35.111+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:35.161 INFO:tasks.rados.rados.0.smithi118.stdout:3564: finishing write tid 5 to smithi11881747-49 2024-03-20T08:13:35.161 INFO:tasks.rados.rados.0.smithi118.stdout:3564: oid 49 updating version 0 to 59869 2024-03-20T08:13:35.161 INFO:tasks.rados.rados.0.smithi118.stdout:3564: oid 49 version 59869 is already newer than 59868 2024-03-20T08:13:35.161 INFO:tasks.rados.rados.0.smithi118.stdout:3564: oid 49 updating version 59869 to 59870 2024-03-20T08:13:35.161 INFO:tasks.rados.rados.0.smithi118.stdout:3564: oid 49 version 59870 is already newer than 59867 2024-03-20T08:13:35.162 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 59870 (ObjNum 1090 snap 347 seq_num 1090) dirty exists 2024-03-20T08:13:35.162 INFO:tasks.rados.rados.0.smithi118.stdout:3564: left oid 49 (ObjNum 1090 snap 347 seq_num 1090) 2024-03-20T08:13:35.162 INFO:tasks.rados.rados.0.smithi118.stdout:3566: expect (ObjNum 1072 snap 341 seq_num 1072) 2024-03-20T08:13:35.211 INFO:tasks.rados.rados.0.smithi118.stdout:3567: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:13:35.211 INFO:tasks.rados.rados.0.smithi118.stdout:3567: finishing copy_from to smithi11881747-35 2024-03-20T08:13:35.211 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 34056 (ObjNum 1043 snap 336 seq_num 1043) dirty exists 2024-03-20T08:13:35.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:35.110+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:35.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:35.110+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:35.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:35.088+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:35.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:35.088+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:35.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:35.088+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[95692]: osdmap e768: 8 total, 8 up, 8 in 2024-03-20T08:13:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:35.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[93121]: osdmap e768: 8 total, 8 up, 8 in 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:35.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.037+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.037+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.038+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.038+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.160+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.160+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.161+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.162+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:35.425 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:35.162+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:35.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:35.158+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:35.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:35.158+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:35.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:35.158+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:36.095 INFO:tasks.rados.rados.0.smithi118.stdout:3562: done (5 left) 2024-03-20T08:13:36.095 INFO:tasks.rados.rados.0.smithi118.stdout:3564: done (4 left) 2024-03-20T08:13:36.095 INFO:tasks.rados.rados.0.smithi118.stdout:3565: done (3 left) 2024-03-20T08:13:36.095 INFO:tasks.rados.rados.0.smithi118.stdout:3566: done (2 left) 2024-03-20T08:13:36.095 INFO:tasks.rados.rados.0.smithi118.stdout:3567: done (1 left) 2024-03-20T08:13:36.095 INFO:tasks.rados.rados.0.smithi118.stdout:3568: done (0 left) 2024-03-20T08:13:36.095 INFO:tasks.rados.rados.0.smithi118.stdout:3569: snap_create 2024-03-20T08:13:36.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:36.139+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:36.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:36.140+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:36.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:36.140+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:36.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:36.099+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:36.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:36.099+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:36.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:36.140+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:36.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:36.140+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:36.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:36.141+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: pgmap v838: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 346 KiB/s wr, 4 op/s 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:36.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: osdmap e769: 8 total, 8 up, 8 in 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:36.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:36.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:36.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:36.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:36.393 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:36.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:36.138+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:36.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:36.138+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:36.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:36.139+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:36.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:36.094+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:36.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:36.094+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:36.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:36.095+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:36.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:36.095+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: pgmap v838: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 346 KiB/s wr, 4 op/s 2024-03-20T08:13:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: pgmap v838: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 6.0 MiB/s rd, 346 KiB/s wr, 4 op/s 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:36.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:36.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:36.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:36.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: osdmap e769: 8 total, 8 up, 8 in 2024-03-20T08:13:36.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:36.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:36.427 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:36.099+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:36.427 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:36.139+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:36.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:36.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: osdmap e769: 8 total, 8 up, 8 in 2024-03-20T08:13:36.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:36.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:37.100 INFO:tasks.rados.rados.0.smithi118.stdout:3569: done (0 left) 2024-03-20T08:13:37.100 INFO:tasks.rados.rados.0.smithi118.stdout:3570: rollback oid 9 current snap is 349 2024-03-20T08:13:37.100 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 9 to 344 2024-03-20T08:13:37.100 INFO:tasks.rados.rados.0.smithi118.stdout:3571: copy_from oid 11 from oid 31 current snap is 349 2024-03-20T08:13:37.100 INFO:tasks.rados.rados.0.smithi118.stdout:3572: read oid 13 snap -1 2024-03-20T08:13:37.100 INFO:tasks.rados.rados.0.smithi118.stdout:3572: expect deleted 2024-03-20T08:13:37.100 INFO:tasks.rados.rados.0.smithi118.stdout:3573: snap_create 2024-03-20T08:13:37.106 INFO:tasks.rados.rados.0.smithi118.stdout:3570: finishing rollback tid 0 to smithi11881747-9 2024-03-20T08:13:37.106 INFO:tasks.rados.rados.0.smithi118.stdout:3570: finishing rollback tid 1 to smithi11881747-9 2024-03-20T08:13:37.106 INFO:tasks.rados.rados.0.smithi118.stdout:3570: finishing rollback tid 2 to smithi11881747-9 2024-03-20T08:13:37.106 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 65299 (ObjNum 966 snap 320 seq_num 966) dirty exists 2024-03-20T08:13:37.107 INFO:tasks.rados.rados.0.smithi118.stdout:3571: finishing copy_from to smithi11881747-11 2024-03-20T08:13:37.107 INFO:tasks.rados.rados.0.smithi118.stdout:3571: got expected ENOENT (src dne) 2024-03-20T08:13:37.107 INFO:tasks.rados.rados.0.smithi118.stdout:3571: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:13:37.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:37.100+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:37.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:37.106+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:37.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:37.107+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:37.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:37.107+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:37.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:37.103+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:37.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:37.103+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:37.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: osdmap e770: 8 total, 8 up, 8 in 2024-03-20T08:13:37.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:37.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:37.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:37.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:37.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:37.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:37.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:37.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:37.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:37.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:37.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:37.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:37.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:37.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:37.103+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:37.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:37.109+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:37.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:37.099+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:37.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: osdmap e770: 8 total, 8 up, 8 in 2024-03-20T08:13:37.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:37.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:37.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:37.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:37.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: osdmap e770: 8 total, 8 up, 8 in 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:37.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:37.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:37.567+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:37.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:37.567+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:37.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:37.567+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:37.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:37.567+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:37.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:37.567+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:37.566+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:37.567+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:37.568+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:37.568+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:37.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:37.568+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:38.105 INFO:tasks.rados.rados.0.smithi118.stdout:3570: done (3 left) 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:3571: done (2 left) 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:3572: done (1 left) 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:3573: done (0 left) 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:3574: rollback oid 20 current snap is 350 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 20 to 346 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:3575: rmattr oid 23 current snap is 350 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:3576: rollback oid 6 current snap is 350 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 6 to 344 2024-03-20T08:13:38.106 INFO:tasks.rados.rados.0.smithi118.stdout:3577: delete oid 36 current snap is 350 2024-03-20T08:13:38.108 INFO:tasks.rados.rados.0.smithi118.stdout:3574: finishing rollback tid 1 to smithi11881747-20 2024-03-20T08:13:38.108 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 59866 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-03-20T08:13:38.113 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 61752 (ObjNum 1075 snap 341 seq_num 1075) dirty exists 2024-03-20T08:13:38.113 INFO:tasks.rados.rados.0.smithi118.stdout:3574: done (3 left) 2024-03-20T08:13:38.113 INFO:tasks.rados.rados.0.smithi118.stdout:3575: done (2 left) 2024-03-20T08:13:38.113 INFO:tasks.rados.rados.0.smithi118.stdout:3577: done (1 left) 2024-03-20T08:13:38.113 INFO:tasks.rados.rados.0.smithi118.stdout:3578: delete oid 50 current snap is 350 2024-03-20T08:13:38.116 INFO:tasks.rados.rados.0.smithi118.stdout:3576: finishing rollback tid 0 to smithi11881747-6 2024-03-20T08:13:38.116 INFO:tasks.rados.rados.0.smithi118.stdout:3576: finishing rollback tid 1 to smithi11881747-6 2024-03-20T08:13:38.116 INFO:tasks.rados.rados.0.smithi118.stdout:3576: finishing rollback tid 2 to smithi11881747-6 2024-03-20T08:13:38.116 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 65304 (ObjNum 1060 snap 338 seq_num 1060) dirty exists 2024-03-20T08:13:38.120 INFO:tasks.rados.rados.0.smithi118.stdout:3576: done (1 left) 2024-03-20T08:13:38.120 INFO:tasks.rados.rados.0.smithi118.stdout:3578: done (0 left) 2024-03-20T08:13:38.120 INFO:tasks.rados.rados.0.smithi118.stdout:3579: snap_remove snap 343 2024-03-20T08:13:38.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:38.107+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:38.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:38.110+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:38.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:38.110+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:38.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:38.111+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:38.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:38.119+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:38.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:38.119+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:38.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:38.111+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:38.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:38.119+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: pgmap v841: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 9.6 MiB/s rd, 4.1 MiB/s wr, 6 op/s 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: osdmap e771: 8 total, 8 up, 8 in 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:38.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:38.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:38.106+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:38.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:38.106+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:38.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:38.106+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:38.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:38.109+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:38.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:38.109+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:38.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:38.112+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:38.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:38.112+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:38.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: pgmap v841: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 9.6 MiB/s rd, 4.1 MiB/s wr, 6 op/s 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: osdmap e771: 8 total, 8 up, 8 in 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:38.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:38.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:38.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:38.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: pgmap v841: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.4 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 9.6 MiB/s rd, 4.1 MiB/s wr, 6 op/s 2024-03-20T08:13:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:38.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: osdmap e771: 8 total, 8 up, 8 in 2024-03-20T08:13:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:38.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:38.426 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:38.105+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:38.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:38.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:38.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:38.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:38.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:38.427 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:38.112+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:39.110 INFO:tasks.rados.rados.0.smithi118.stdout:3579: done (0 left) 2024-03-20T08:13:39.110 INFO:tasks.rados.rados.0.smithi118.stdout:3580: delete oid 17 current snap is 350 2024-03-20T08:13:39.117 INFO:tasks.rados.rados.0.smithi118.stdout:3580: done (0 left) 2024-03-20T08:13:39.117 INFO:tasks.rados.rados.0.smithi118.stdout:3581: snap_create 2024-03-20T08:13:39.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:39.111+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:39.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:39.119+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:39.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:39.119+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:39.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:39.115+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:39.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:39.117+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:39.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:39.117+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:39.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:39.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: osdmap e772: 8 total, 8 up, 8 in 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:39.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:39.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:39.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:39.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:39.110+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:39.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:39.110+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:39.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:39.112+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:39.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:39.112+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:39.118+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:39.118+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:39.118+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:39.118+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:39.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:39.118+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:39.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: osdmap e772: 8 total, 8 up, 8 in 2024-03-20T08:13:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:39.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:39.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:39.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:39.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: osdmap e772: 8 total, 8 up, 8 in 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:39.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:39.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:39.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:39.466+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:39.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:39.466+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:39.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:39.466+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:39.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:39.422+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:40.115 INFO:tasks.rados.rados.0.smithi118.stdout:3581: done (0 left) 2024-03-20T08:13:40.115 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 351 2024-03-20T08:13:40.115 INFO:tasks.rados.rados.0.smithi118.stdout:3582: seq_num 1094 ranges {0=729088,729088=573440,1302528=229376} 2024-03-20T08:13:40.118 INFO:tasks.rados.rados.0.smithi118.stdout:3582: writing smithi11881747-8 from 0 to 729088 tid 1 2024-03-20T08:13:40.122 INFO:tasks.rados.rados.0.smithi118.stdout:3582: writing smithi11881747-8 from 729088 to 1302528 tid 2 2024-03-20T08:13:40.124 INFO:tasks.rados.rados.0.smithi118.stdout:3582: writing smithi11881747-8 from 1302528 to 1531904 tid 3 2024-03-20T08:13:40.124 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 29 current snap is 351 2024-03-20T08:13:40.124 INFO:tasks.rados.rados.0.smithi118.stdout:3583: seq_num 1095 ranges {1228800=745472,1974272=499712,2473984=303104} 2024-03-20T08:13:40.135 INFO:tasks.rados.rados.0.smithi118.stdout:3583: writing smithi11881747-29 from 1228800 to 1974272 tid 1 2024-03-20T08:13:40.138 INFO:tasks.rados.rados.0.smithi118.stdout:3583: writing smithi11881747-29 from 1974272 to 2473984 tid 2 2024-03-20T08:13:40.141 INFO:tasks.rados.rados.0.smithi118.stdout:3583: writing smithi11881747-29 from 2473984 to 2777088 tid 3 2024-03-20T08:13:40.141 INFO:tasks.rados.rados.0.smithi118.stdout:3584: copy_from oid 12 from oid 17 current snap is 351 2024-03-20T08:13:40.141 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 351 2024-03-20T08:13:40.141 INFO:tasks.rados.rados.0.smithi118.stdout:3585: seq_num 1096 ranges {0=761856,761856=786432,1548288=16384} 2024-03-20T08:13:40.146 INFO:tasks.rados.rados.0.smithi118.stdout:3585: writing smithi11881747-10 from 0 to 761856 tid 1 2024-03-20T08:13:40.152 INFO:tasks.rados.rados.0.smithi118.stdout:3585: writing smithi11881747-10 from 761856 to 1548288 tid 2 2024-03-20T08:13:40.152 INFO:tasks.rados.rados.0.smithi118.stdout:3585: writing smithi11881747-10 from 1548288 to 1564672 tid 3 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3586: snap_remove snap 347 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: finishing write tid 1 to smithi11881747-8 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: finishing write tid 2 to smithi11881747-8 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: finishing write tid 3 to smithi11881747-8 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: finishing write tid 4 to smithi11881747-8 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3583: finishing write tid 1 to smithi11881747-29 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: finishing write tid 5 to smithi11881747-8 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: oid 8 updating version 0 to 34129 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: oid 8 updating version 34129 to 34130 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: oid 8 version 34130 is already newer than 34128 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: oid 8 version 34130 is already newer than 34127 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34130 (ObjNum 1094 snap 351 seq_num 1094) dirty exists 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3582: left oid 8 (ObjNum 1094 snap 351 seq_num 1094) 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3583: finishing write tid 2 to smithi11881747-29 2024-03-20T08:13:40.153 INFO:tasks.rados.rados.0.smithi118.stdout:3583: finishing write tid 3 to smithi11881747-29 2024-03-20T08:13:40.154 INFO:tasks.rados.rados.0.smithi118.stdout:3583: finishing write tid 4 to smithi11881747-29 2024-03-20T08:13:40.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:40.116+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:40.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:40.152+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:40.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:40.154+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:40.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:40.147+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:40.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:40.148+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:40.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:40.116+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:40.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:40.116+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:40.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:40.116+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: pgmap v844: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 9.2 MiB/s rd, 7.1 MiB/s wr, 7 op/s 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: osdmap e773: 8 total, 8 up, 8 in 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:40.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:40.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:40.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:40.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:40.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:40.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:40.142+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:40.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:40.126+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:40.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:40.126+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:40.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:40.127+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:40.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:40.128+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:40.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:40.115+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:40.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:40.115+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:40.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:40.115+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:40.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:40.116+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:40.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: pgmap v844: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 9.2 MiB/s rd, 7.1 MiB/s wr, 7 op/s 2024-03-20T08:13:40.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:40.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:40.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: osdmap e773: 8 total, 8 up, 8 in 2024-03-20T08:13:40.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:40.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: pgmap v844: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 9.2 MiB/s rd, 7.1 MiB/s wr, 7 op/s 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: osdmap e773: 8 total, 8 up, 8 in 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:40.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:40.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:40.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:40.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:40.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:40.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:40.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:40.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:40.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:41.122 INFO:tasks.rados.rados.0.smithi118.stdout:3582: done (4 left) 2024-03-20T08:13:41.122 INFO:tasks.rados.rados.0.smithi118.stdout:3586: done (3 left) 2024-03-20T08:13:41.122 INFO:tasks.rados.rados.0.smithi118.stdout:3587: rollback oid 40 current snap is 351 2024-03-20T08:13:41.122 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 349 2024-03-20T08:13:41.122 INFO:tasks.rados.rados.0.smithi118.stdout:3584: finishing copy_from to smithi11881747-12 2024-03-20T08:13:41.122 INFO:tasks.rados.rados.0.smithi118.stdout:3584: got expected ENOENT (src dne) 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3584: finishing copy_from racing read to smithi11881747-12 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3583: finishing write tid 5 to smithi11881747-29 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3583: oid 29 updating version 0 to 61753 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3583: oid 29 updating version 61753 to 61756 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3583: oid 29 version 61756 is already newer than 61754 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3583: oid 29 version 61756 is already newer than 61755 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 61756 (ObjNum 1095 snap 351 seq_num 1095) dirty exists 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3583: left oid 29 (ObjNum 1095 snap 351 seq_num 1095) 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3585: finishing write tid 1 to smithi11881747-10 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3585: finishing write tid 2 to smithi11881747-10 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3585: finishing write tid 3 to smithi11881747-10 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3585: finishing write tid 4 to smithi11881747-10 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3585: finishing write tid 5 to smithi11881747-10 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3585: oid 10 updating version 0 to 65292 2024-03-20T08:13:41.123 INFO:tasks.rados.rados.0.smithi118.stdout:3585: oid 10 updating version 65292 to 65294 2024-03-20T08:13:41.124 INFO:tasks.rados.rados.0.smithi118.stdout:3585: oid 10 version 65294 is already newer than 65293 2024-03-20T08:13:41.124 INFO:tasks.rados.rados.0.smithi118.stdout:3585: oid 10 updating version 65294 to 65295 2024-03-20T08:13:41.124 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 65295 (ObjNum 1096 snap 351 seq_num 1096) dirty exists 2024-03-20T08:13:41.124 INFO:tasks.rados.rados.0.smithi118.stdout:3585: left oid 10 (ObjNum 1096 snap 351 seq_num 1096) 2024-03-20T08:13:41.124 INFO:tasks.rados.rados.0.smithi118.stdout:3583: done (3 left) 2024-03-20T08:13:41.124 INFO:tasks.rados.rados.0.smithi118.stdout:3584: done (2 left) 2024-03-20T08:13:41.124 INFO:tasks.rados.rados.0.smithi118.stdout:3585: done (1 left) 2024-03-20T08:13:41.124 INFO:tasks.rados.rados.0.smithi118.stdout:3588: snap_remove snap 346 2024-03-20T08:13:41.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:41.120+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:41.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: osdmap e774: 8 total, 8 up, 8 in 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:41.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:41.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:41.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:41.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:41.129+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:41.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:41.129+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:41.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:41.120+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:41.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:41.120+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:41.125+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:41.125+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:41.125+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:41.125+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:41.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:41.139+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:41.424 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:41.122+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: osdmap e774: 8 total, 8 up, 8 in 2024-03-20T08:13:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:41.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: osdmap e774: 8 total, 8 up, 8 in 2024-03-20T08:13:41.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:41.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:41.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:41.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:41.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:41.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:41.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:41.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:41.442+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:41.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:41.442+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:41.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:41.440+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:41.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:41.440+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:41.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:41.440+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:41.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:41.441+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:41.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:41.441+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:41.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:41.441+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:42.125 INFO:tasks.rados.rados.0.smithi118.stdout:3588: done (1 left) 2024-03-20T08:13:42.125 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 351 2024-03-20T08:13:42.126 INFO:tasks.rados.rados.0.smithi118.stdout:3587: finishing rollback tid 0 to smithi11881747-40 2024-03-20T08:13:42.126 INFO:tasks.rados.rados.0.smithi118.stdout:3589: seq_num 1097 ranges {4112384=778240,4890624=671744,5562368=131072} 2024-03-20T08:13:42.152 INFO:tasks.rados.rados.0.smithi118.stdout:3589: writing smithi11881747-9 from 4112384 to 4890624 tid 1 2024-03-20T08:13:42.156 INFO:tasks.rados.rados.0.smithi118.stdout:3589: writing smithi11881747-9 from 4890624 to 5562368 tid 2 2024-03-20T08:13:42.157 INFO:tasks.rados.rados.0.smithi118.stdout:3589: writing smithi11881747-9 from 5562368 to 5693440 tid 3 2024-03-20T08:13:42.157 INFO:tasks.rados.rados.0.smithi118.stdout:3590: snap_create 2024-03-20T08:13:42.157 INFO:tasks.rados.rados.0.smithi118.stdout:3587: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:13:42.157 INFO:tasks.rados.rados.0.smithi118.stdout:3587: finishing rollback tid 2 to smithi11881747-40 2024-03-20T08:13:42.158 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 65299 (ObjNum 589 snap 193 seq_num 589) dirty exists 2024-03-20T08:13:42.160 INFO:tasks.rados.rados.0.smithi118.stdout:3589: finishing write tid 1 to smithi11881747-9 2024-03-20T08:13:42.162 INFO:tasks.rados.rados.0.smithi118.stdout:3589: finishing write tid 2 to smithi11881747-9 2024-03-20T08:13:42.162 INFO:tasks.rados.rados.0.smithi118.stdout:3589: finishing write tid 3 to smithi11881747-9 2024-03-20T08:13:42.162 INFO:tasks.rados.rados.0.smithi118.stdout:3589: finishing write tid 4 to smithi11881747-9 2024-03-20T08:13:42.164 INFO:tasks.rados.rados.0.smithi118.stdout:3589: finishing write tid 5 to smithi11881747-9 2024-03-20T08:13:42.164 INFO:tasks.rados.rados.0.smithi118.stdout:3589: oid 9 updating version 0 to 65301 2024-03-20T08:13:42.164 INFO:tasks.rados.rados.0.smithi118.stdout:3589: oid 9 version 65301 is already newer than 65300 2024-03-20T08:13:42.164 INFO:tasks.rados.rados.0.smithi118.stdout:3589: oid 9 updating version 65301 to 65302 2024-03-20T08:13:42.164 INFO:tasks.rados.rados.0.smithi118.stdout:3589: oid 9 updating version 65302 to 65303 2024-03-20T08:13:42.164 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 65303 (ObjNum 1097 snap 351 seq_num 1097) dirty exists 2024-03-20T08:13:42.164 INFO:tasks.rados.rados.0.smithi118.stdout:3589: left oid 9 (ObjNum 1097 snap 351 seq_num 1097) 2024-03-20T08:13:42.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:42.125+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:42.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:42.161+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:42.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:42.161+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:42.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:42.161+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:42.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:42.346+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:42.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:42.346+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:42.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:42.346+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:42.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:42.346+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:42.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:13:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:13:42.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:42.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:42.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: pgmap v847: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 7 op/s 2024-03-20T08:13:42.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:42.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:42.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: osdmap e775: 8 total, 8 up, 8 in 2024-03-20T08:13:42.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:42.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:42.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:42.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:42.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:42.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:42.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:42.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:42.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:42.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:42.125+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:42.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:42.156+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:42.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:42.156+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:42.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:42.164+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:42.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:42.126+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:42.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:42.343+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:42.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:42.343+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:42.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:42.343+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:42.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:42.343+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: pgmap v847: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 7 op/s 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: osdmap e775: 8 total, 8 up, 8 in 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:42.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: pgmap v847: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 7 op/s 2024-03-20T08:13:42.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: osdmap e775: 8 total, 8 up, 8 in 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:42.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:43.132 INFO:tasks.rados.rados.0.smithi118.stdout:3587: done (2 left) 2024-03-20T08:13:43.132 INFO:tasks.rados.rados.0.smithi118.stdout:3589: done (1 left) 2024-03-20T08:13:43.132 INFO:tasks.rados.rados.0.smithi118.stdout:3590: done (0 left) 2024-03-20T08:13:43.132 INFO:tasks.rados.rados.0.smithi118.stdout:3591: read oid 31 snap -1 2024-03-20T08:13:43.133 INFO:tasks.rados.rados.0.smithi118.stdout:3591: expect deleted 2024-03-20T08:13:43.133 INFO:tasks.rados.rados.0.smithi118.stdout:3592: read oid 45 snap -1 2024-03-20T08:13:43.133 INFO:tasks.rados.rados.0.smithi118.stdout:3592: expect (ObjNum 1040 snap 336 seq_num 1040) 2024-03-20T08:13:43.133 INFO:tasks.rados.rados.0.smithi118.stdout:3593: read oid 38 snap 351 2024-03-20T08:13:43.133 INFO:tasks.rados.rados.0.smithi118.stdout:3593: expect (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:13:43.133 INFO:tasks.rados.rados.0.smithi118.stdout:3594: delete oid 32 current snap is 352 2024-03-20T08:13:43.141 INFO:tasks.rados.rados.0.smithi118.stdout:3591: done (3 left) 2024-03-20T08:13:43.141 INFO:tasks.rados.rados.0.smithi118.stdout:3594: done (2 left) 2024-03-20T08:13:43.141 INFO:tasks.rados.rados.0.smithi118.stdout:3595: copy_from oid 40 from oid 26 current snap is 352 2024-03-20T08:13:43.141 INFO:tasks.rados.rados.0.smithi118.stdout:3596: snap_remove snap 345 2024-03-20T08:13:43.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:43.130+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:43.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:43.174+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:43.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:43.174+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:43.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:43.174+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:43.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:43.138+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:43.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:43.138+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:43.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:43.138+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:43.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:43.136+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:43.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:43.136+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: osdmap e776: 8 total, 8 up, 8 in 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:43.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:43.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:43.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:43.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:43.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:43.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:43.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:43.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:43.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:43.138+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:43.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:43.138+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:43.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:43.130+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:43.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:43.130+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:43.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:43.130+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:43.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:43.135+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:43.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:43.135+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:43.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:43.136+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:43.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:43.137+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:43.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:43.147+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: osdmap e776: 8 total, 8 up, 8 in 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:43.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: osdmap e776: 8 total, 8 up, 8 in 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:43.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:43.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:43.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:43.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:43.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:43.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:43.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:43.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:13:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:13:43] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:13:44.135 INFO:tasks.rados.rados.0.smithi118.stdout:3596: done (3 left) 2024-03-20T08:13:44.135 INFO:tasks.rados.rados.0.smithi118.stdout:3597: snap_remove snap 344 2024-03-20T08:13:44.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:44.137+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:44.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:44.140+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:44.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:44.140+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:44.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:44.140+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: pgmap v850: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 7 op/s 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: osdmap e777: 8 total, 8 up, 8 in 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:44.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:44.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:44.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:44.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:44.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:44.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:44.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:44.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:44.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:44.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:44.134+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:44.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:44.134+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:44.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:44.135+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:44.422 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:44.135+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: pgmap v850: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 7 op/s 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: osdmap e777: 8 total, 8 up, 8 in 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:44.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: pgmap v850: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 7 op/s 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: osdmap e777: 8 total, 8 up, 8 in 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:44.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:44.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:44.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:44.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:44.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:44.631+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:44.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:44.631+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:44.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:44.632+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:44.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:44.633+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:44.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:44.632+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:44.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:44.630+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:44.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:44.428+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:44.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:44.428+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:44.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:44.428+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:45.141 INFO:tasks.rados.rados.0.smithi118.stdout:3597: done (3 left) 2024-03-20T08:13:45.141 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 352 2024-03-20T08:13:45.141 INFO:tasks.rados.rados.0.smithi118.stdout:3592: expect (ObjNum 1040 snap 336 seq_num 1040) 2024-03-20T08:13:45.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:45.141+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:45.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:45.141+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:45.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:45.141+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: osdmap e778: 8 total, 8 up, 8 in 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:45.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:45.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:45.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:45.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:45.139+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: osdmap e778: 8 total, 8 up, 8 in 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:45.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: osdmap e778: 8 total, 8 up, 8 in 2024-03-20T08:13:45.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:45.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:45.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:45.668+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:45.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:45.668+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:45.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:45.668+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:45.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:45.505+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:45.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:45.505+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:45.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:45.505+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:45.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:45.668+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:45.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:45.669+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:45.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:45.668+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:45.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:45.668+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:45.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:45.667+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:45.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:45.669+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:45.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:45.669+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:45.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:45.670+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:45.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:45.670+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:46.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: pgmap v853: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 752 KiB/s wr, 8 op/s 2024-03-20T08:13:46.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:46.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: osdmap e779: 8 total, 8 up, 8 in 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:46.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: pgmap v853: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 752 KiB/s wr, 8 op/s 2024-03-20T08:13:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: osdmap e779: 8 total, 8 up, 8 in 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:46.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: pgmap v853: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 752 KiB/s wr, 8 op/s 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: osdmap e779: 8 total, 8 up, 8 in 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 199=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18b=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:46.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:46.520 INFO:tasks.rados.rados.0.smithi118.stdout:3595: finishing copy_from racing read to smithi11881747-40 2024-03-20T08:13:46.520 INFO:tasks.rados.rados.0.smithi118.stdout:3598: seq_num 1099 ranges {0=409600,409600=442368,851968=720896,1572864=40960} 2024-03-20T08:13:46.522 INFO:tasks.rados.rados.0.smithi118.stdout:3598: writing smithi11881747-28 from 0 to 409600 tid 1 2024-03-20T08:13:46.525 INFO:tasks.rados.rados.0.smithi118.stdout:3598: writing smithi11881747-28 from 409600 to 851968 tid 2 2024-03-20T08:13:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:3598: writing smithi11881747-28 from 851968 to 1572864 tid 3 2024-03-20T08:13:46.529 INFO:tasks.rados.rados.0.smithi118.stdout:3598: writing smithi11881747-28 from 1572864 to 1613824 tid 4 2024-03-20T08:13:46.530 INFO:tasks.rados.rados.0.smithi118.stdout:3592: done (3 left) 2024-03-20T08:13:46.530 INFO:tasks.rados.rados.0.smithi118.stdout:3599: copy_from oid 31 from oid 19 current snap is 352 2024-03-20T08:13:46.530 INFO:tasks.rados.rados.0.smithi118.stdout:3595: finishing copy_from to smithi11881747-40 2024-03-20T08:13:46.531 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 65316 (ObjNum 1089 snap 347 seq_num 1089) dirty exists 2024-03-20T08:13:46.531 INFO:tasks.rados.rados.0.smithi118.stdout:3593: expect (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:13:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:47.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:47.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:47.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:47.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:47.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:47.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:47.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:47.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:47.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:47.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:47.833 INFO:tasks.rados.rados.0.smithi118.stdout:3598: finishing write tid 1 to smithi11881747-28 2024-03-20T08:13:47.879 INFO:tasks.rados.rados.0.smithi118.stdout:3598: finishing write tid 2 to smithi11881747-28 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: finishing write tid 3 to smithi11881747-28 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: finishing write tid 4 to smithi11881747-28 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: finishing write tid 5 to smithi11881747-28 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: finishing write tid 6 to smithi11881747-28 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: oid 28 updating version 0 to 61759 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: oid 28 updating version 61759 to 61761 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: oid 28 version 61761 is already newer than 61757 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: oid 28 version 61761 is already newer than 61760 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: oid 28 version 61761 is already newer than 61758 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 61761 (ObjNum 1099 snap 352 seq_num 1099) dirty exists 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3598: left oid 28 (ObjNum 1099 snap 352 seq_num 1099) 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3599: finishing copy_from racing read to smithi11881747-31 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3599: finishing copy_from to smithi11881747-31 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 59853 (ObjNum 1022 snap 329 seq_num 1022) dirty exists 2024-03-20T08:13:47.880 INFO:tasks.rados.rados.0.smithi118.stdout:3593: done (3 left) 2024-03-20T08:13:47.881 INFO:tasks.rados.rados.0.smithi118.stdout:3595: done (2 left) 2024-03-20T08:13:47.881 INFO:tasks.rados.rados.0.smithi118.stdout:3598: done (1 left) 2024-03-20T08:13:47.881 INFO:tasks.rados.rados.0.smithi118.stdout:3599: done (0 left) 2024-03-20T08:13:47.881 INFO:tasks.rados.rados.0.smithi118.stdout:3600: read oid 32 snap 350 2024-03-20T08:13:47.881 INFO:tasks.rados.rados.0.smithi118.stdout:3600: expect (ObjNum 1088 snap 345 seq_num 1088) 2024-03-20T08:13:47.881 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 352 2024-03-20T08:13:47.881 INFO:tasks.rados.rados.0.smithi118.stdout:3601: seq_num 1100 ranges {0=425984,425984=729088,1155072=475136} 2024-03-20T08:13:47.883 INFO:tasks.rados.rados.0.smithi118.stdout:3601: writing smithi11881747-4 from 0 to 425984 tid 1 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: writing smithi11881747-4 from 425984 to 1155072 tid 2 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: writing smithi11881747-4 from 1155072 to 1630208 tid 3 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 352 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: finishing write tid 1 to smithi11881747-4 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3602: seq_num 1101 ranges {0=442368,442368=606208,1048576=598016} 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3602: writing smithi11881747-17 from 0 to 442368 tid 1 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3602: writing smithi11881747-17 from 442368 to 1048576 tid 2 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3602: writing smithi11881747-17 from 1048576 to 1646592 tid 3 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 352 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: finishing write tid 2 to smithi11881747-4 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: finishing write tid 3 to smithi11881747-4 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: finishing write tid 4 to smithi11881747-4 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: finishing write tid 5 to smithi11881747-4 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: oid 4 updating version 0 to 65263 2024-03-20T08:13:47.884 INFO:tasks.rados.rados.0.smithi118.stdout:3601: oid 4 version 65263 is already newer than 65262 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:3601: oid 4 version 65263 is already newer than 65261 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:3601: oid 4 updating version 65263 to 65264 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65264 (ObjNum 1100 snap 352 seq_num 1100) dirty exists 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:3601: left oid 4 (ObjNum 1100 snap 352 seq_num 1100) 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:3602: finishing write tid 1 to smithi11881747-17 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:3603: seq_num 1102 ranges {540672=458752,999424=491520,1490944=491520,1982464=221184} 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:3603: writing smithi11881747-7 from 540672 to 999424 tid 1 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:3603: writing smithi11881747-7 from 999424 to 1490944 tid 2 2024-03-20T08:13:47.885 INFO:tasks.rados.rados.0.smithi118.stdout:3603: writing smithi11881747-7 from 1490944 to 1982464 tid 3 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3603: writing smithi11881747-7 from 1982464 to 2203648 tid 4 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3601: done (3 left) 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3604: delete oid 5 current snap is 352 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: finishing write tid 2 to smithi11881747-17 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: finishing write tid 3 to smithi11881747-17 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: finishing write tid 4 to smithi11881747-17 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: finishing write tid 5 to smithi11881747-17 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: oid 17 updating version 0 to 61741 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: oid 17 updating version 61741 to 61744 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: oid 17 version 61744 is already newer than 61743 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: oid 17 version 61744 is already newer than 61742 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 61744 (ObjNum 1101 snap 352 seq_num 1101) dirty exists 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3602: left oid 17 (ObjNum 1101 snap 352 seq_num 1101) 2024-03-20T08:13:47.886 INFO:tasks.rados.rados.0.smithi118.stdout:3600: expect (ObjNum 1088 snap 345 seq_num 1088) 2024-03-20T08:13:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:48 smithi118 ceph-mon[93121]: pgmap v855: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 914 B/s rd, 1.3 MiB/s wr, 8 op/s 2024-03-20T08:13:48.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:48 smithi118 ceph-mon[95692]: pgmap v855: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 914 B/s rd, 1.3 MiB/s wr, 8 op/s 2024-03-20T08:13:48.481 INFO:tasks.rados.rados.0.smithi118.stdout:3603: finishing write tid 1 to smithi11881747-7 2024-03-20T08:13:48.481 INFO:tasks.rados.rados.0.smithi118.stdout:3603: finishing write tid 2 to smithi11881747-7 2024-03-20T08:13:48.481 INFO:tasks.rados.rados.0.smithi118.stdout:3603: finishing write tid 3 to smithi11881747-7 2024-03-20T08:13:48.481 INFO:tasks.rados.rados.0.smithi118.stdout:3603: finishing write tid 4 to smithi11881747-7 2024-03-20T08:13:48.481 INFO:tasks.rados.rados.0.smithi118.stdout:3603: finishing write tid 5 to smithi11881747-7 2024-03-20T08:13:48.481 INFO:tasks.rados.rados.0.smithi118.stdout:3600: done (3 left) 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3602: done (2 left) 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3604: done (1 left) 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3605: read oid 20 snap 350 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3605: expect deleted 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3606: delete oid 3 current snap is 352 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3603: finishing write tid 6 to smithi11881747-7 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3603: oid 7 updating version 0 to 65298 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3603: oid 7 version 65298 is already newer than 65297 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3603: oid 7 version 65298 is already newer than 65294 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3603: oid 7 version 65298 is already newer than 65295 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3603: oid 7 version 65298 is already newer than 65296 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 65298 (ObjNum 1102 snap 352 seq_num 1102) dirty exists 2024-03-20T08:13:48.482 INFO:tasks.rados.rados.0.smithi118.stdout:3603: left oid 7 (ObjNum 1102 snap 352 seq_num 1102) 2024-03-20T08:13:48.485 INFO:tasks.rados.rados.0.smithi118.stdout:3603: done (2 left) 2024-03-20T08:13:48.485 INFO:tasks.rados.rados.0.smithi118.stdout:3605: done (1 left) 2024-03-20T08:13:48.485 INFO:tasks.rados.rados.0.smithi118.stdout:3606: done (0 left) 2024-03-20T08:13:48.485 INFO:tasks.rados.rados.0.smithi118.stdout:3607: rollback oid 1 current snap is 352 2024-03-20T08:13:48.485 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 1 to 349 2024-03-20T08:13:48.486 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 352 2024-03-20T08:13:48.486 INFO:tasks.rados.rados.0.smithi118.stdout:3608: seq_num 1105 ranges {0=516096,516096=598016,1114112=606208} 2024-03-20T08:13:48.488 INFO:tasks.rados.rados.0.smithi118.stdout:3608: writing smithi11881747-21 from 0 to 516096 tid 1 2024-03-20T08:13:48.492 INFO:tasks.rados.rados.0.smithi118.stdout:3608: writing smithi11881747-21 from 516096 to 1114112 tid 2 2024-03-20T08:13:48.497 INFO:tasks.rados.rados.0.smithi118.stdout:3608: writing smithi11881747-21 from 1114112 to 1720320 tid 3 2024-03-20T08:13:48.497 INFO:tasks.rados.rados.0.smithi118.stdout:3609: snap_create 2024-03-20T08:13:48.497 INFO:tasks.rados.rados.0.smithi118.stdout:3607: finishing rollback tid 0 to smithi11881747-1 2024-03-20T08:13:48.497 INFO:tasks.rados.rados.0.smithi118.stdout:3607: finishing rollback tid 1 to smithi11881747-1 2024-03-20T08:13:48.497 INFO:tasks.rados.rados.0.smithi118.stdout:3607: finishing rollback tid 2 to smithi11881747-1 2024-03-20T08:13:48.497 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 65298 (ObjNum 1034 snap 333 seq_num 1034) dirty exists 2024-03-20T08:13:48.498 INFO:tasks.rados.rados.0.smithi118.stdout:3608: finishing write tid 1 to smithi11881747-21 2024-03-20T08:13:48.506 INFO:tasks.rados.rados.0.smithi118.stdout:3608: finishing write tid 2 to smithi11881747-21 2024-03-20T08:13:48.506 INFO:tasks.rados.rados.0.smithi118.stdout:3608: finishing write tid 3 to smithi11881747-21 2024-03-20T08:13:48.506 INFO:tasks.rados.rados.0.smithi118.stdout:3608: finishing write tid 4 to smithi11881747-21 2024-03-20T08:13:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:3608: finishing write tid 5 to smithi11881747-21 2024-03-20T08:13:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:3608: oid 21 updating version 0 to 65305 2024-03-20T08:13:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:3608: oid 21 updating version 65305 to 65307 2024-03-20T08:13:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:3608: oid 21 version 65307 is already newer than 65304 2024-03-20T08:13:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:3608: oid 21 version 65307 is already newer than 65306 2024-03-20T08:13:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 65307 (ObjNum 1105 snap 352 seq_num 1105) dirty exists 2024-03-20T08:13:48.510 INFO:tasks.rados.rados.0.smithi118.stdout:3608: left oid 21 (ObjNum 1105 snap 352 seq_num 1105) 2024-03-20T08:13:48.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:48 smithi196 ceph-mon[87080]: pgmap v855: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 914 B/s rd, 1.3 MiB/s wr, 8 op/s 2024-03-20T08:13:49.151 INFO:tasks.rados.rados.0.smithi118.stdout:3607: done (2 left) 2024-03-20T08:13:49.151 INFO:tasks.rados.rados.0.smithi118.stdout:3608: done (1 left) 2024-03-20T08:13:49.151 INFO:tasks.rados.rados.0.smithi118.stdout:3609: done (0 left) 2024-03-20T08:13:49.151 INFO:tasks.rados.rados.0.smithi118.stdout:3610: rollback oid 16 current snap is 353 2024-03-20T08:13:49.152 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 348 2024-03-20T08:13:49.152 INFO:tasks.rados.rados.0.smithi118.stdout:3611: setattr oid 43 current snap is 353 2024-03-20T08:13:49.152 INFO:tasks.rados.rados.0.smithi118.stdout:3612: delete oid 40 current snap is 353 2024-03-20T08:13:49.157 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65299 (ObjNum 1077 snap 341 seq_num 1077) dirty exists 2024-03-20T08:13:49.160 INFO:tasks.rados.rados.0.smithi118.stdout:3610: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:13:49.160 INFO:tasks.rados.rados.0.smithi118.stdout:3610: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:13:49.161 INFO:tasks.rados.rados.0.smithi118.stdout:3610: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:13:49.161 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 61764 (ObjNum 1040 snap 336 seq_num 1040) dirty exists 2024-03-20T08:13:49.161 INFO:tasks.rados.rados.0.smithi118.stdout:3610: done (2 left) 2024-03-20T08:13:49.161 INFO:tasks.rados.rados.0.smithi118.stdout:3611: done (1 left) 2024-03-20T08:13:49.162 INFO:tasks.rados.rados.0.smithi118.stdout:3612: done (0 left) 2024-03-20T08:13:49.162 INFO:tasks.rados.rados.0.smithi118.stdout:3613: copy_from oid 13 from oid 10 current snap is 353 2024-03-20T08:13:49.162 INFO:tasks.rados.rados.0.smithi118.stdout:3614: setattr oid 22 current snap is 353 2024-03-20T08:13:49.163 INFO:tasks.rados.rados.0.smithi118.stdout:3615: copy_from oid 50 from oid 32 current snap is 353 2024-03-20T08:13:49.163 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 353 2024-03-20T08:13:49.164 INFO:tasks.rados.rados.0.smithi118.stdout:3616: seq_num 1107 ranges {1630208=548864,2179072=770048,2949120=434176} 2024-03-20T08:13:49.177 INFO:tasks.rados.rados.0.smithi118.stdout:3616: writing smithi11881747-4 from 1630208 to 2179072 tid 1 2024-03-20T08:13:49.181 INFO:tasks.rados.rados.0.smithi118.stdout:3616: writing smithi11881747-4 from 2179072 to 2949120 tid 2 2024-03-20T08:13:49.184 INFO:tasks.rados.rados.0.smithi118.stdout:3616: writing smithi11881747-4 from 2949120 to 3383296 tid 3 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:3617: snap_create 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:3615: finishing copy_from to smithi11881747-50 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:3615: got expected ENOENT (src dne) 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:3615: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 65308 (ObjNum 33 snap 0 seq_num 323016176) dirty exists 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:3613: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:3613: finishing copy_from to smithi11881747-13 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 65295 (ObjNum 1096 snap 351 seq_num 1096) dirty exists 2024-03-20T08:13:49.185 INFO:tasks.rados.rados.0.smithi118.stdout:3616: finishing write tid 1 to smithi11881747-4 2024-03-20T08:13:49.188 INFO:tasks.rados.rados.0.smithi118.stdout:3616: finishing write tid 2 to smithi11881747-4 2024-03-20T08:13:49.188 INFO:tasks.rados.rados.0.smithi118.stdout:3616: finishing write tid 3 to smithi11881747-4 2024-03-20T08:13:49.189 INFO:tasks.rados.rados.0.smithi118.stdout:3616: finishing write tid 4 to smithi11881747-4 2024-03-20T08:13:49.189 INFO:tasks.rados.rados.0.smithi118.stdout:3616: finishing write tid 5 to smithi11881747-4 2024-03-20T08:13:49.189 INFO:tasks.rados.rados.0.smithi118.stdout:3616: oid 4 updating version 0 to 65268 2024-03-20T08:13:49.190 INFO:tasks.rados.rados.0.smithi118.stdout:3616: oid 4 version 65268 is already newer than 65267 2024-03-20T08:13:49.190 INFO:tasks.rados.rados.0.smithi118.stdout:3616: oid 4 version 65268 is already newer than 65265 2024-03-20T08:13:49.191 INFO:tasks.rados.rados.0.smithi118.stdout:3616: oid 4 version 65268 is already newer than 65266 2024-03-20T08:13:49.191 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65268 (ObjNum 1107 snap 353 seq_num 1107) dirty exists 2024-03-20T08:13:49.191 INFO:tasks.rados.rados.0.smithi118.stdout:3616: left oid 4 (ObjNum 1107 snap 353 seq_num 1107) 2024-03-20T08:13:49.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:49.158+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:49.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:49.151+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:49.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:49.151+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:49.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:49.152+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:49.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:49.157+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:49.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:49.159+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:49.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:49.160+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:49.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:49.160+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:49.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:49.162+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:49.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:49.152+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:49.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:49.157+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:49.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:49.157+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:49.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:49.158+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:49.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:49.153+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:49.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:49.156+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:49.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:49.181+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:49.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:49.181+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:50.159 INFO:tasks.rados.rados.0.smithi118.stdout:3613: done (4 left) 2024-03-20T08:13:50.159 INFO:tasks.rados.rados.0.smithi118.stdout:3614: done (3 left) 2024-03-20T08:13:50.159 INFO:tasks.rados.rados.0.smithi118.stdout:3615: done (2 left) 2024-03-20T08:13:50.159 INFO:tasks.rados.rados.0.smithi118.stdout:3616: done (1 left) 2024-03-20T08:13:50.159 INFO:tasks.rados.rados.0.smithi118.stdout:3617: done (0 left) 2024-03-20T08:13:50.159 INFO:tasks.rados.rados.0.smithi118.stdout:3618: read oid 10 snap -1 2024-03-20T08:13:50.159 INFO:tasks.rados.rados.0.smithi118.stdout:3618: expect (ObjNum 1096 snap 351 seq_num 1096) 2024-03-20T08:13:50.159 INFO:tasks.rados.rados.0.smithi118.stdout:3619: snap_create 2024-03-20T08:13:50.170 INFO:tasks.rados.rados.0.smithi118.stdout:3618: expect (ObjNum 1096 snap 351 seq_num 1096) 2024-03-20T08:13:50.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:50.159+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:50.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:50.159+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:50.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:50.159+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: pgmap v856: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 8 op/s 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: osdmap e780: 8 total, 8 up, 8 in 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: pgmap v856: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 8 op/s 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: osdmap e780: 8 total, 8 up, 8 in 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:50.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:50.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:50.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:50.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:50.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:50.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:50.159+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:50.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:50.166+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:50.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:50.166+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:50.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:50.166+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:50.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:50.166+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:50.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:50.166+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:50.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: pgmap v856: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 8 op/s 2024-03-20T08:13:50.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:50.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: osdmap e780: 8 total, 8 up, 8 in 2024-03-20T08:13:50.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:50.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:50.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:50.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:51.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:50.740+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:51.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:50.740+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:51.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:50.740+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:51.160 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:50.740+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:51.161 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:50.740+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:51.161 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:50.739+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:51.161 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:50.739+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:51.161 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:50.740+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:51.161 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:50.740+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:51.161 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:50.740+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:51.164 INFO:tasks.rados.rados.0.smithi118.stdout:3618: done (1 left) 2024-03-20T08:13:51.165 INFO:tasks.rados.rados.0.smithi118.stdout:3619: done (0 left) 2024-03-20T08:13:51.165 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 355 2024-03-20T08:13:51.165 INFO:tasks.rados.rados.0.smithi118.stdout:3620: seq_num 1108 ranges {0=565248,565248=647168,1212416=483328,1695744=73728} 2024-03-20T08:13:51.167 INFO:tasks.rados.rados.0.smithi118.stdout:3620: writing smithi11881747-42 from 0 to 565248 tid 1 2024-03-20T08:13:51.172 INFO:tasks.rados.rados.0.smithi118.stdout:3620: writing smithi11881747-42 from 565248 to 1212416 tid 2 2024-03-20T08:13:51.175 INFO:tasks.rados.rados.0.smithi118.stdout:3620: writing smithi11881747-42 from 1212416 to 1695744 tid 3 2024-03-20T08:13:51.175 INFO:tasks.rados.rados.0.smithi118.stdout:3620: writing smithi11881747-42 from 1695744 to 1769472 tid 4 2024-03-20T08:13:51.175 INFO:tasks.rados.rados.0.smithi118.stdout:3621: setattr oid 12 current snap is 355 2024-03-20T08:13:51.176 INFO:tasks.rados.rados.0.smithi118.stdout:3620: finishing write tid 1 to smithi11881747-42 2024-03-20T08:13:51.176 INFO:tasks.rados.rados.0.smithi118.stdout:3620: finishing write tid 2 to smithi11881747-42 2024-03-20T08:13:51.178 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 355 2024-03-20T08:13:51.178 INFO:tasks.rados.rados.0.smithi118.stdout:3622: seq_num 1109 ranges {0=581632,581632=532480,1114112=647168,1761280=24576} 2024-03-20T08:13:51.181 INFO:tasks.rados.rados.0.smithi118.stdout:3622: writing smithi11881747-25 from 0 to 581632 tid 1 2024-03-20T08:13:51.184 INFO:tasks.rados.rados.0.smithi118.stdout:3622: writing smithi11881747-25 from 581632 to 1114112 tid 2 2024-03-20T08:13:51.189 INFO:tasks.rados.rados.0.smithi118.stdout:3622: writing smithi11881747-25 from 1114112 to 1761280 tid 3 2024-03-20T08:13:51.189 INFO:tasks.rados.rados.0.smithi118.stdout:3622: writing smithi11881747-25 from 1761280 to 1785856 tid 4 2024-03-20T08:13:51.189 INFO:tasks.rados.rados.0.smithi118.stdout:3623: read oid 23 snap -1 2024-03-20T08:13:51.189 INFO:tasks.rados.rados.0.smithi118.stdout:3623: expect (ObjNum 1075 snap 341 seq_num 1075) 2024-03-20T08:13:51.189 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 355 2024-03-20T08:13:51.189 INFO:tasks.rados.rados.0.smithi118.stdout:3620: finishing write tid 3 to smithi11881747-42 2024-03-20T08:13:51.189 INFO:tasks.rados.rados.0.smithi118.stdout:3624: seq_num 1110 ranges {2088960=598016,2686976=466944,3153920=450560,3604480=286720} 2024-03-20T08:13:51.204 INFO:tasks.rados.rados.0.smithi118.stdout:3624: writing smithi11881747-43 from 2088960 to 2686976 tid 1 2024-03-20T08:13:51.206 INFO:tasks.rados.rados.0.smithi118.stdout:3624: writing smithi11881747-43 from 2686976 to 3153920 tid 2 2024-03-20T08:13:51.209 INFO:tasks.rados.rados.0.smithi118.stdout:3624: writing smithi11881747-43 from 3153920 to 3604480 tid 3 2024-03-20T08:13:51.211 INFO:tasks.rados.rados.0.smithi118.stdout:3624: writing smithi11881747-43 from 3604480 to 3891200 tid 4 2024-03-20T08:13:51.211 INFO:tasks.rados.rados.0.smithi118.stdout:3625: snap_create 2024-03-20T08:13:51.211 INFO:tasks.rados.rados.0.smithi118.stdout:3620: finishing write tid 4 to smithi11881747-42 2024-03-20T08:13:51.211 INFO:tasks.rados.rados.0.smithi118.stdout:3620: finishing write tid 5 to smithi11881747-42 2024-03-20T08:13:51.211 INFO:tasks.rados.rados.0.smithi118.stdout:3620: finishing write tid 6 to smithi11881747-42 2024-03-20T08:13:51.211 INFO:tasks.rados.rados.0.smithi118.stdout:3620: oid 42 updating version 0 to 65306 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3620: oid 42 updating version 65306 to 65309 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3620: oid 42 version 65309 is already newer than 65305 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3620: oid 42 version 65309 is already newer than 65308 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3620: oid 42 version 65309 is already newer than 65307 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65309 (ObjNum 1108 snap 355 seq_num 1108) dirty exists 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3620: left oid 42 (ObjNum 1108 snap 355 seq_num 1108) 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65299 (ObjNum 1056 snap 338 seq_num 1056) dirty exists 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3622: finishing write tid 1 to smithi11881747-25 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3622: finishing write tid 2 to smithi11881747-25 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3622: finishing write tid 3 to smithi11881747-25 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3622: finishing write tid 4 to smithi11881747-25 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3622: finishing write tid 5 to smithi11881747-25 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3622: finishing write tid 6 to smithi11881747-25 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3622: oid 25 updating version 0 to 65311 2024-03-20T08:13:51.212 INFO:tasks.rados.rados.0.smithi118.stdout:3622: oid 25 updating version 65311 to 65314 2024-03-20T08:13:51.213 INFO:tasks.rados.rados.0.smithi118.stdout:3622: oid 25 version 65314 is already newer than 65313 2024-03-20T08:13:51.213 INFO:tasks.rados.rados.0.smithi118.stdout:3622: oid 25 version 65314 is already newer than 65312 2024-03-20T08:13:51.213 INFO:tasks.rados.rados.0.smithi118.stdout:3622: oid 25 version 65314 is already newer than 65310 2024-03-20T08:13:51.213 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 65314 (ObjNum 1109 snap 355 seq_num 1109) dirty exists 2024-03-20T08:13:51.213 INFO:tasks.rados.rados.0.smithi118.stdout:3622: left oid 25 (ObjNum 1109 snap 355 seq_num 1109) 2024-03-20T08:13:51.213 INFO:tasks.rados.rados.0.smithi118.stdout:3623: expect (ObjNum 1075 snap 341 seq_num 1075) 2024-03-20T08:13:51.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:51.165+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:51.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:51.175+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:51.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:51.171+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:51.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:51.171+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:51.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:51.173+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:51.163+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:51.164+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:51.165+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:51.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:51.165+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: osdmap e781: 8 total, 8 up, 8 in 2024-03-20T08:13:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:51.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: osdmap e781: 8 total, 8 up, 8 in 2024-03-20T08:13:51.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:51.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:51.439 INFO:tasks.rados.rados.0.smithi118.stdout:3624: finishing write tid 1 to smithi11881747-43 2024-03-20T08:13:51.439 INFO:tasks.rados.rados.0.smithi118.stdout:3624: finishing write tid 2 to smithi11881747-43 2024-03-20T08:13:51.439 INFO:tasks.rados.rados.0.smithi118.stdout:3624: finishing write tid 3 to smithi11881747-43 2024-03-20T08:13:51.439 INFO:tasks.rados.rados.0.smithi118.stdout:3624: finishing write tid 4 to smithi11881747-43 2024-03-20T08:13:51.439 INFO:tasks.rados.rados.0.smithi118.stdout:3624: finishing write tid 5 to smithi11881747-43 2024-03-20T08:13:51.439 INFO:tasks.rados.rados.0.smithi118.stdout:3624: finishing write tid 6 to smithi11881747-43 2024-03-20T08:13:51.439 INFO:tasks.rados.rados.0.smithi118.stdout:3624: oid 43 updating version 0 to 65300 2024-03-20T08:13:51.440 INFO:tasks.rados.rados.0.smithi118.stdout:3624: oid 43 updating version 65300 to 65303 2024-03-20T08:13:51.440 INFO:tasks.rados.rados.0.smithi118.stdout:3624: oid 43 updating version 65303 to 65304 2024-03-20T08:13:51.440 INFO:tasks.rados.rados.0.smithi118.stdout:3624: oid 43 version 65304 is already newer than 65301 2024-03-20T08:13:51.440 INFO:tasks.rados.rados.0.smithi118.stdout:3624: oid 43 version 65304 is already newer than 65302 2024-03-20T08:13:51.440 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65304 (ObjNum 1110 snap 355 seq_num 1110) dirty exists 2024-03-20T08:13:51.440 INFO:tasks.rados.rados.0.smithi118.stdout:3624: left oid 43 (ObjNum 1110 snap 355 seq_num 1110) 2024-03-20T08:13:51.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:51.164+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:51.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:51.184+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:51.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:51.185+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:51.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:51.175+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:51.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:51.175+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:51.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:51.175+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:51.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:51.185+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:51.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:51.185+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: osdmap e781: 8 total, 8 up, 8 in 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:51.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:51.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:51.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:51.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:51.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:51.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:52.169 INFO:tasks.rados.rados.0.smithi118.stdout:3620: done (5 left) 2024-03-20T08:13:52.169 INFO:tasks.rados.rados.0.smithi118.stdout:3621: done (4 left) 2024-03-20T08:13:52.169 INFO:tasks.rados.rados.0.smithi118.stdout:3622: done (3 left) 2024-03-20T08:13:52.170 INFO:tasks.rados.rados.0.smithi118.stdout:3623: done (2 left) 2024-03-20T08:13:52.170 INFO:tasks.rados.rados.0.smithi118.stdout:3624: done (1 left) 2024-03-20T08:13:52.170 INFO:tasks.rados.rados.0.smithi118.stdout:3625: done (0 left) 2024-03-20T08:13:52.170 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 13 current snap is 356 2024-03-20T08:13:52.170 INFO:tasks.rados.rados.0.smithi118.stdout:3626: seq_num 1111 ranges {1564672=614400,2179072=753664,2932736=450560} 2024-03-20T08:13:52.182 INFO:tasks.rados.rados.0.smithi118.stdout:3626: writing smithi11881747-13 from 1564672 to 2179072 tid 1 2024-03-20T08:13:52.186 INFO:tasks.rados.rados.0.smithi118.stdout:3626: writing smithi11881747-13 from 2179072 to 2932736 tid 2 2024-03-20T08:13:52.190 INFO:tasks.rados.rados.0.smithi118.stdout:3626: writing smithi11881747-13 from 2932736 to 3383296 tid 3 2024-03-20T08:13:52.190 INFO:tasks.rados.rados.0.smithi118.stdout:3627: rollback oid 30 current snap is 356 2024-03-20T08:13:52.190 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 349 2024-03-20T08:13:52.190 INFO:tasks.rados.rados.0.smithi118.stdout:3628: snap_create 2024-03-20T08:13:52.190 INFO:tasks.rados.rados.0.smithi118.stdout:3626: finishing write tid 1 to smithi11881747-13 2024-03-20T08:13:52.193 INFO:tasks.rados.rados.0.smithi118.stdout:3626: finishing write tid 2 to smithi11881747-13 2024-03-20T08:13:52.194 INFO:tasks.rados.rados.0.smithi118.stdout:3626: finishing write tid 3 to smithi11881747-13 2024-03-20T08:13:52.194 INFO:tasks.rados.rados.0.smithi118.stdout:3626: finishing write tid 4 to smithi11881747-13 2024-03-20T08:13:52.195 INFO:tasks.rados.rados.0.smithi118.stdout:3627: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:13:52.195 INFO:tasks.rados.rados.0.smithi118.stdout:3627: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:13:52.195 INFO:tasks.rados.rados.0.smithi118.stdout:3627: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:13:52.195 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 65271 (ObjNum 966 snap 320 seq_num 966) dirty exists 2024-03-20T08:13:52.196 INFO:tasks.rados.rados.0.smithi118.stdout:3626: finishing write tid 5 to smithi11881747-13 2024-03-20T08:13:52.196 INFO:tasks.rados.rados.0.smithi118.stdout:3626: oid 13 updating version 0 to 65317 2024-03-20T08:13:52.196 INFO:tasks.rados.rados.0.smithi118.stdout:3626: oid 13 version 65317 is already newer than 65315 2024-03-20T08:13:52.196 INFO:tasks.rados.rados.0.smithi118.stdout:3626: oid 13 updating version 65317 to 65318 2024-03-20T08:13:52.196 INFO:tasks.rados.rados.0.smithi118.stdout:3626: oid 13 version 65318 is already newer than 65316 2024-03-20T08:13:52.196 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 65318 (ObjNum 1111 snap 356 seq_num 1111) dirty exists 2024-03-20T08:13:52.196 INFO:tasks.rados.rados.0.smithi118.stdout:3626: left oid 13 (ObjNum 1111 snap 356 seq_num 1111) 2024-03-20T08:13:52.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:52.189+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:52.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:52.185+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:52.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:52.189+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:52.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:52.170+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:52.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:52.170+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:52.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:52.168+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:52.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:52.169+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:52.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:52.169+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:52.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:52.170+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:52.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:52.170+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: pgmap v859: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 11 op/s 2024-03-20T08:13:52.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: osdmap e782: 8 total, 8 up, 8 in 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:52.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:52.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:52.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:52.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:52.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:52.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:52.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: pgmap v859: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 11 op/s 2024-03-20T08:13:52.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:52.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: osdmap e782: 8 total, 8 up, 8 in 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:52.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:52.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:52.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:52.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:52.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:52.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:52.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:52.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:52.169+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:13:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:13:52.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:52.194+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:52.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:52.195+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:52.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:52.195+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:52.190+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:52.190+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:52.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:52.190+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: pgmap v859: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 11 op/s 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: osdmap e782: 8 total, 8 up, 8 in 2024-03-20T08:13:52.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:53.176 INFO:tasks.rados.rados.0.smithi118.stdout:3626: done (2 left) 2024-03-20T08:13:53.176 INFO:tasks.rados.rados.0.smithi118.stdout:3627: done (1 left) 2024-03-20T08:13:53.176 INFO:tasks.rados.rados.0.smithi118.stdout:3628: done (0 left) 2024-03-20T08:13:53.176 INFO:tasks.rados.rados.0.smithi118.stdout:3629: read oid 41 snap -1 2024-03-20T08:13:53.176 INFO:tasks.rados.rados.0.smithi118.stdout:3629: expect deleted 2024-03-20T08:13:53.176 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 357 2024-03-20T08:13:53.176 INFO:tasks.rados.rados.0.smithi118.stdout:3630: seq_num 1112 ranges {1531904=630784,2162688=638976,2801664=466944,3268608=98304} 2024-03-20T08:13:53.188 INFO:tasks.rados.rados.0.smithi118.stdout:3630: writing smithi11881747-8 from 1531904 to 2162688 tid 1 2024-03-20T08:13:53.192 INFO:tasks.rados.rados.0.smithi118.stdout:3630: writing smithi11881747-8 from 2162688 to 2801664 tid 2 2024-03-20T08:13:53.195 INFO:tasks.rados.rados.0.smithi118.stdout:3630: writing smithi11881747-8 from 2801664 to 3268608 tid 3 2024-03-20T08:13:53.195 INFO:tasks.rados.rados.0.smithi118.stdout:3630: writing smithi11881747-8 from 3268608 to 3366912 tid 4 2024-03-20T08:13:53.196 INFO:tasks.rados.rados.0.smithi118.stdout:3631: snap_create 2024-03-20T08:13:53.203 INFO:tasks.rados.rados.0.smithi118.stdout:3630: finishing write tid 1 to smithi11881747-8 2024-03-20T08:13:53.204 INFO:tasks.rados.rados.0.smithi118.stdout:3630: finishing write tid 2 to smithi11881747-8 2024-03-20T08:13:53.204 INFO:tasks.rados.rados.0.smithi118.stdout:3630: finishing write tid 3 to smithi11881747-8 2024-03-20T08:13:53.212 INFO:tasks.rados.rados.0.smithi118.stdout:3630: finishing write tid 4 to smithi11881747-8 2024-03-20T08:13:53.213 INFO:tasks.rados.rados.0.smithi118.stdout:3630: finishing write tid 5 to smithi11881747-8 2024-03-20T08:13:53.216 INFO:tasks.rados.rados.0.smithi118.stdout:3630: finishing write tid 6 to smithi11881747-8 2024-03-20T08:13:53.217 INFO:tasks.rados.rados.0.smithi118.stdout:3630: oid 8 updating version 0 to 34134 2024-03-20T08:13:53.217 INFO:tasks.rados.rados.0.smithi118.stdout:3630: oid 8 updating version 34134 to 34135 2024-03-20T08:13:53.217 INFO:tasks.rados.rados.0.smithi118.stdout:3630: oid 8 updating version 34135 to 34136 2024-03-20T08:13:53.217 INFO:tasks.rados.rados.0.smithi118.stdout:3630: oid 8 version 34136 is already newer than 34132 2024-03-20T08:13:53.217 INFO:tasks.rados.rados.0.smithi118.stdout:3630: oid 8 version 34136 is already newer than 34133 2024-03-20T08:13:53.217 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34136 (ObjNum 1112 snap 357 seq_num 1112) dirty exists 2024-03-20T08:13:53.217 INFO:tasks.rados.rados.0.smithi118.stdout:3630: left oid 8 (ObjNum 1112 snap 357 seq_num 1112) 2024-03-20T08:13:53.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:53.196+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:53.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:53.176+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:53.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:53.174+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:53.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:53.174+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:53.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:53.174+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:53.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:53.178+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:53.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:53.179+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:53.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:53.179+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:53.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:53.180+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: osdmap e783: 8 total, 8 up, 8 in 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:53.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: osdmap e783: 8 total, 8 up, 8 in 2024-03-20T08:13:53.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:53.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:53.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:53.174+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:53.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:53.270+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:53.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:53.270+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:53.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:53.270+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:53.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:53.271+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:53.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:53.195+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:53.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:53.195+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:53.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:53.196+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:53.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: osdmap e783: 8 total, 8 up, 8 in 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:53.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:13:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:13:53] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:13:54.180 INFO:tasks.rados.rados.0.smithi118.stdout:3629: done (2 left) 2024-03-20T08:13:54.180 INFO:tasks.rados.rados.0.smithi118.stdout:3630: done (1 left) 2024-03-20T08:13:54.180 INFO:tasks.rados.rados.0.smithi118.stdout:3631: done (0 left) 2024-03-20T08:13:54.180 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 358 2024-03-20T08:13:54.180 INFO:tasks.rados.rados.0.smithi118.stdout:3632: seq_num 1113 ranges {3383296=647168,4030464=524288,4554752=622592,5177344=57344} 2024-03-20T08:13:54.201 INFO:tasks.rados.rados.0.smithi118.stdout:3632: writing smithi11881747-4 from 3383296 to 4030464 tid 1 2024-03-20T08:13:54.204 INFO:tasks.rados.rados.0.smithi118.stdout:3632: writing smithi11881747-4 from 4030464 to 4554752 tid 2 2024-03-20T08:13:54.209 INFO:tasks.rados.rados.0.smithi118.stdout:3632: writing smithi11881747-4 from 4554752 to 5177344 tid 3 2024-03-20T08:13:54.209 INFO:tasks.rados.rados.0.smithi118.stdout:3632: writing smithi11881747-4 from 5177344 to 5234688 tid 4 2024-03-20T08:13:54.209 INFO:tasks.rados.rados.0.smithi118.stdout:3633: copy_from oid 41 from oid 13 current snap is 358 2024-03-20T08:13:54.210 INFO:tasks.rados.rados.0.smithi118.stdout:3632: finishing write tid 1 to smithi11881747-4 2024-03-20T08:13:54.210 INFO:tasks.rados.rados.0.smithi118.stdout:3632: finishing write tid 2 to smithi11881747-4 2024-03-20T08:13:54.210 INFO:tasks.rados.rados.0.smithi118.stdout:3634: setattr oid 47 current snap is 358 2024-03-20T08:13:54.211 INFO:tasks.rados.rados.0.smithi118.stdout:3635: delete oid 37 current snap is 358 2024-03-20T08:13:54.213 INFO:tasks.rados.rados.0.smithi118.stdout:3632: finishing write tid 3 to smithi11881747-4 2024-03-20T08:13:54.213 INFO:tasks.rados.rados.0.smithi118.stdout:3632: finishing write tid 4 to smithi11881747-4 2024-03-20T08:13:54.215 INFO:tasks.rados.rados.0.smithi118.stdout:3632: finishing write tid 5 to smithi11881747-4 2024-03-20T08:13:54.217 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 61765 (ObjNum 1032 snap 331 seq_num 1032) dirty exists 2024-03-20T08:13:54.219 INFO:tasks.rados.rados.0.smithi118.stdout:3632: finishing write tid 6 to smithi11881747-4 2024-03-20T08:13:54.219 INFO:tasks.rados.rados.0.smithi118.stdout:3632: oid 4 updating version 0 to 65274 2024-03-20T08:13:54.219 INFO:tasks.rados.rados.0.smithi118.stdout:3632: oid 4 version 65274 is already newer than 65273 2024-03-20T08:13:54.219 INFO:tasks.rados.rados.0.smithi118.stdout:3632: oid 4 version 65274 is already newer than 65272 2024-03-20T08:13:54.219 INFO:tasks.rados.rados.0.smithi118.stdout:3632: oid 4 updating version 65274 to 65275 2024-03-20T08:13:54.219 INFO:tasks.rados.rados.0.smithi118.stdout:3632: oid 4 updating version 65275 to 65276 2024-03-20T08:13:54.219 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65276 (ObjNum 1113 snap 358 seq_num 1113) dirty exists 2024-03-20T08:13:54.219 INFO:tasks.rados.rados.0.smithi118.stdout:3632: left oid 4 (ObjNum 1113 snap 358 seq_num 1113) 2024-03-20T08:13:54.221 INFO:tasks.rados.rados.0.smithi118.stdout:3632: done (3 left) 2024-03-20T08:13:54.221 INFO:tasks.rados.rados.0.smithi118.stdout:3634: done (2 left) 2024-03-20T08:13:54.221 INFO:tasks.rados.rados.0.smithi118.stdout:3635: done (1 left) 2024-03-20T08:13:54.221 INFO:tasks.rados.rados.0.smithi118.stdout:3636: delete oid 25 current snap is 358 2024-03-20T08:13:54.225 INFO:tasks.rados.rados.0.smithi118.stdout:3636: done (1 left) 2024-03-20T08:13:54.225 INFO:tasks.rados.rados.0.smithi118.stdout:3637: snap_remove snap 349 2024-03-20T08:13:54.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:54.219+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:54.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:54.179+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:54.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:54.180+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:54.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:54.206+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:54.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:54.207+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:54.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:54.212+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:54.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:54.213+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:54.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:54.213+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:54.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:54.213+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:54.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:54.213+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: pgmap v862: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 17 op/s 2024-03-20T08:13:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:54.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: osdmap e784: 8 total, 8 up, 8 in 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:54.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: pgmap v862: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 17 op/s 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: osdmap e784: 8 total, 8 up, 8 in 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:54.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:54.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:54.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:54.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:54.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:54.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:54.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:54.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:54.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:54.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:54.608 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:54.180+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:54.609 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:54.207+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:54.609 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:54.207+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:54.609 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:54.208+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:54.609 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:54.182+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:54.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:54.182+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:54.610 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:54.182+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:54.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: pgmap v862: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 17 op/s 2024-03-20T08:13:54.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:54.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:54.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:54.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: osdmap e784: 8 total, 8 up, 8 in 2024-03-20T08:13:54.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:54.610 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:54.611 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:54.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:54.609+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:54.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:54.609+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:55.186 INFO:tasks.rados.rados.0.smithi118.stdout:3637: done (1 left) 2024-03-20T08:13:55.186 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 33 current snap is 358 2024-03-20T08:13:55.186 INFO:tasks.rados.rados.0.smithi118.stdout:3633: finishing copy_from racing read to smithi11881747-41 2024-03-20T08:13:55.186 INFO:tasks.rados.rados.0.smithi118.stdout:3638: seq_num 1116 ranges {1269760=696320,1966080=573440,2539520=450560,2990080=180224} 2024-03-20T08:13:55.196 INFO:tasks.rados.rados.0.smithi118.stdout:3638: writing smithi11881747-33 from 1269760 to 1966080 tid 1 2024-03-20T08:13:55.200 INFO:tasks.rados.rados.0.smithi118.stdout:3638: writing smithi11881747-33 from 1966080 to 2539520 tid 2 2024-03-20T08:13:55.207 INFO:tasks.rados.rados.0.smithi118.stdout:3638: writing smithi11881747-33 from 2539520 to 2990080 tid 3 2024-03-20T08:13:55.207 INFO:tasks.rados.rados.0.smithi118.stdout:3638: writing smithi11881747-33 from 2990080 to 3170304 tid 4 2024-03-20T08:13:55.207 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 358 2024-03-20T08:13:55.207 INFO:tasks.rados.rados.0.smithi118.stdout:3633: finishing copy_from to smithi11881747-41 2024-03-20T08:13:55.207 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 65318 (ObjNum 1111 snap 356 seq_num 1111) dirty exists 2024-03-20T08:13:55.207 INFO:tasks.rados.rados.0.smithi118.stdout:3639: seq_num 1117 ranges {319488=712704,1032192=458752,1490944=655360,2146304=90112} 2024-03-20T08:13:55.209 INFO:tasks.rados.rados.0.smithi118.stdout:3639: writing smithi11881747-31 from 319488 to 1032192 tid 1 2024-03-20T08:13:55.212 INFO:tasks.rados.rados.0.smithi118.stdout:3639: writing smithi11881747-31 from 1032192 to 1490944 tid 2 2024-03-20T08:13:55.216 INFO:tasks.rados.rados.0.smithi118.stdout:3639: writing smithi11881747-31 from 1490944 to 2146304 tid 3 2024-03-20T08:13:55.217 INFO:tasks.rados.rados.0.smithi118.stdout:3639: writing smithi11881747-31 from 2146304 to 2236416 tid 4 2024-03-20T08:13:55.217 INFO:tasks.rados.rados.0.smithi118.stdout:3633: done (2 left) 2024-03-20T08:13:55.217 INFO:tasks.rados.rados.0.smithi118.stdout:3640: read oid 29 snap -1 2024-03-20T08:13:55.217 INFO:tasks.rados.rados.0.smithi118.stdout:3640: expect (ObjNum 1095 snap 351 seq_num 1095) 2024-03-20T08:13:55.217 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 24 current snap is 358 2024-03-20T08:13:55.218 INFO:tasks.rados.rados.0.smithi118.stdout:3638: finishing write tid 1 to smithi11881747-33 2024-03-20T08:13:55.218 INFO:tasks.rados.rados.0.smithi118.stdout:3641: seq_num 1118 ranges {1433600=729088,2162688=794624,2957312=409600} 2024-03-20T08:13:55.230 INFO:tasks.rados.rados.0.smithi118.stdout:3641: writing smithi11881747-24 from 1433600 to 2162688 tid 1 2024-03-20T08:13:55.235 INFO:tasks.rados.rados.0.smithi118.stdout:3641: writing smithi11881747-24 from 2162688 to 2957312 tid 2 2024-03-20T08:13:55.238 INFO:tasks.rados.rados.0.smithi118.stdout:3641: writing smithi11881747-24 from 2957312 to 3366912 tid 3 2024-03-20T08:13:55.238 INFO:tasks.rados.rados.0.smithi118.stdout:3642: snap_create 2024-03-20T08:13:55.238 INFO:tasks.rados.rados.0.smithi118.stdout:3639: finishing write tid 1 to smithi11881747-31 2024-03-20T08:13:55.238 INFO:tasks.rados.rados.0.smithi118.stdout:3638: finishing write tid 2 to smithi11881747-33 2024-03-20T08:13:55.238 INFO:tasks.rados.rados.0.smithi118.stdout:3639: finishing write tid 2 to smithi11881747-31 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3638: finishing write tid 3 to smithi11881747-33 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3638: finishing write tid 4 to smithi11881747-33 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3638: finishing write tid 5 to smithi11881747-33 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: finishing write tid 3 to smithi11881747-31 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: finishing write tid 4 to smithi11881747-31 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: finishing write tid 5 to smithi11881747-31 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: finishing write tid 6 to smithi11881747-31 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: oid 31 updating version 0 to 65307 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: oid 31 updating version 65307 to 65308 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: oid 31 updating version 65308 to 65309 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: oid 31 version 65309 is already newer than 65305 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:3639: oid 31 version 65309 is already newer than 65306 2024-03-20T08:13:55.239 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 65309 (ObjNum 1117 snap 358 seq_num 1117) dirty exists 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3639: left oid 31 (ObjNum 1117 snap 358 seq_num 1117) 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3638: finishing write tid 6 to smithi11881747-33 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3638: oid 33 updating version 0 to 34060 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3638: oid 33 updating version 34060 to 34061 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3638: oid 33 updating version 34061 to 34063 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3638: oid 33 version 34063 is already newer than 34062 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3638: oid 33 updating version 34063 to 34064 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34064 (ObjNum 1116 snap 358 seq_num 1116) dirty exists 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3638: left oid 33 (ObjNum 1116 snap 358 seq_num 1116) 2024-03-20T08:13:55.240 INFO:tasks.rados.rados.0.smithi118.stdout:3640: expect (ObjNum 1095 snap 351 seq_num 1095) 2024-03-20T08:13:55.333 INFO:tasks.rados.rados.0.smithi118.stdout:3641: finishing write tid 1 to smithi11881747-24 2024-03-20T08:13:55.333 INFO:tasks.rados.rados.0.smithi118.stdout:3641: finishing write tid 2 to smithi11881747-24 2024-03-20T08:13:55.333 INFO:tasks.rados.rados.0.smithi118.stdout:3641: finishing write tid 3 to smithi11881747-24 2024-03-20T08:13:55.333 INFO:tasks.rados.rados.0.smithi118.stdout:3641: finishing write tid 4 to smithi11881747-24 2024-03-20T08:13:55.333 INFO:tasks.rados.rados.0.smithi118.stdout:3641: finishing write tid 5 to smithi11881747-24 2024-03-20T08:13:55.333 INFO:tasks.rados.rados.0.smithi118.stdout:3641: oid 24 updating version 0 to 65320 2024-03-20T08:13:55.334 INFO:tasks.rados.rados.0.smithi118.stdout:3641: oid 24 updating version 65320 to 65322 2024-03-20T08:13:55.334 INFO:tasks.rados.rados.0.smithi118.stdout:3641: oid 24 updating version 65322 to 65323 2024-03-20T08:13:55.334 INFO:tasks.rados.rados.0.smithi118.stdout:3641: oid 24 version 65323 is already newer than 65321 2024-03-20T08:13:55.334 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 24 v 65323 (ObjNum 1118 snap 358 seq_num 1118) dirty exists 2024-03-20T08:13:55.334 INFO:tasks.rados.rados.0.smithi118.stdout:3641: left oid 24 (ObjNum 1118 snap 358 seq_num 1118) 2024-03-20T08:13:55.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:55.185+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:55.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:55.202+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:55.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:55.202+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:55.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:55.186+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:55.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:55.186+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:55.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:55.186+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: osdmap e785: 8 total, 8 up, 8 in 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:55.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:55.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:55.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:55.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:55.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:55.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:55.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:55.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:55.201+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:55.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:55.235+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:55.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:55.235+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:55.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:55.236+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:55.184+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:55.185+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:55.185+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:55.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:55.185+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:55.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:55.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: osdmap e785: 8 total, 8 up, 8 in 2024-03-20T08:13:55.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:55.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: osdmap e785: 8 total, 8 up, 8 in 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:55.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:55.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:55.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:55.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:55.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:55.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:56.190 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:55.939+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:56.190 INFO:tasks.rados.rados.0.smithi118.stdout:3638: done (4 left) 2024-03-20T08:13:56.190 INFO:tasks.rados.rados.0.smithi118.stdout:3639: done (3 left) 2024-03-20T08:13:56.190 INFO:tasks.rados.rados.0.smithi118.stdout:3640: done (2 left) 2024-03-20T08:13:56.190 INFO:tasks.rados.rados.0.smithi118.stdout:3641: done (1 left) 2024-03-20T08:13:56.191 INFO:tasks.rados.rados.0.smithi118.stdout:3642: done (0 left) 2024-03-20T08:13:56.191 INFO:tasks.rados.rados.0.smithi118.stdout:3643: delete oid 10 current snap is 359 2024-03-20T08:13:56.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: pgmap v865: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-03-20T08:13:56.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:56.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:56.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:56.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:56.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:56.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: osdmap e786: 8 total, 8 up, 8 in 2024-03-20T08:13:56.191 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: pgmap v865: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:56.192 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:56.193 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: osdmap e786: 8 total, 8 up, 8 in 2024-03-20T08:13:56.193 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:56.193 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:56.193 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:56.193 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:56.193 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:56.193 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:56.193 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:56.194 INFO:tasks.rados.rados.0.smithi118.stdout:3643: done (0 left) 2024-03-20T08:13:56.194 INFO:tasks.rados.rados.0.smithi118.stdout:3644: rollback oid 48 current snap is 359 2024-03-20T08:13:56.194 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 48 to 351 2024-03-20T08:13:56.194 INFO:tasks.rados.rados.0.smithi118.stdout:3645: copy_from oid 44 from oid 42 current snap is 359 2024-03-20T08:13:56.194 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 359 2024-03-20T08:13:56.194 INFO:tasks.rados.rados.0.smithi118.stdout:3646: seq_num 1120 ranges {3383296=761856,4145152=557056,4702208=425984,5128192=221184} 2024-03-20T08:13:56.216 INFO:tasks.rados.rados.0.smithi118.stdout:3646: writing smithi11881747-41 from 3383296 to 4145152 tid 1 2024-03-20T08:13:56.220 INFO:tasks.rados.rados.0.smithi118.stdout:3646: writing smithi11881747-41 from 4145152 to 4702208 tid 2 2024-03-20T08:13:56.223 INFO:tasks.rados.rados.0.smithi118.stdout:3646: writing smithi11881747-41 from 4702208 to 5128192 tid 3 2024-03-20T08:13:56.225 INFO:tasks.rados.rados.0.smithi118.stdout:3646: writing smithi11881747-41 from 5128192 to 5349376 tid 4 2024-03-20T08:13:56.225 INFO:tasks.rados.rados.0.smithi118.stdout:3647: read oid 1 snap -1 2024-03-20T08:13:56.225 INFO:tasks.rados.rados.0.smithi118.stdout:3647: expect (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:13:56.225 INFO:tasks.rados.rados.0.smithi118.stdout:3648: read oid 47 snap -1 2024-03-20T08:13:56.225 INFO:tasks.rados.rados.0.smithi118.stdout:3648: expect (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:13:56.225 INFO:tasks.rados.rados.0.smithi118.stdout:3649: snap_remove snap 355 2024-03-20T08:13:56.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:56.191+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:56.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:56.192+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:56.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:56.193+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:56.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:56.075+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:56.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:56.075+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:56.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:56.195+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:56.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:56.195+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:56.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:56.195+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:56.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: pgmap v865: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: osdmap e786: 8 total, 8 up, 8 in 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 190=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:56.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:56.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:56.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:56.198+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:56.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:56.199+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:56.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:56.223+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:56.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:56.223+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:56.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:56.220+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:56.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:56.220+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:56.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:56.221+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:56.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:56.221+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:56.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:56.223+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:56.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:56.190+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:56.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:56.597+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:56.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:56.597+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:56.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:56.597+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:57.196 INFO:tasks.rados.rados.0.smithi118.stdout:3649: done (5 left) 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3650: snap_create 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3644: finishing rollback tid 0 to smithi11881747-48 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3644: finishing rollback tid 1 to smithi11881747-48 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3645: finishing copy_from racing read to smithi11881747-44 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3644: finishing rollback tid 2 to smithi11881747-48 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 65311 (ObjNum 1087 snap 343 seq_num 1087) dirty exists 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3645: finishing copy_from to smithi11881747-44 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 65309 (ObjNum 1108 snap 355 seq_num 1108) dirty exists 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3646: finishing write tid 1 to smithi11881747-41 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3646: finishing write tid 2 to smithi11881747-41 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3646: finishing write tid 3 to smithi11881747-41 2024-03-20T08:13:57.197 INFO:tasks.rados.rados.0.smithi118.stdout:3647: expect (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:13:57.217 INFO:tasks.rados.rados.0.smithi118.stdout:3646: finishing write tid 4 to smithi11881747-41 2024-03-20T08:13:57.217 INFO:tasks.rados.rados.0.smithi118.stdout:3646: finishing write tid 5 to smithi11881747-41 2024-03-20T08:13:57.217 INFO:tasks.rados.rados.0.smithi118.stdout:3646: finishing write tid 6 to smithi11881747-41 2024-03-20T08:13:57.217 INFO:tasks.rados.rados.0.smithi118.stdout:3646: oid 41 updating version 0 to 65322 2024-03-20T08:13:57.217 INFO:tasks.rados.rados.0.smithi118.stdout:3646: oid 41 version 65322 is already newer than 65319 2024-03-20T08:13:57.217 INFO:tasks.rados.rados.0.smithi118.stdout:3646: oid 41 updating version 65322 to 65323 2024-03-20T08:13:57.217 INFO:tasks.rados.rados.0.smithi118.stdout:3646: oid 41 version 65323 is already newer than 65320 2024-03-20T08:13:57.217 INFO:tasks.rados.rados.0.smithi118.stdout:3646: oid 41 version 65323 is already newer than 65321 2024-03-20T08:13:57.218 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 65323 (ObjNum 1120 snap 359 seq_num 1120) dirty exists 2024-03-20T08:13:57.218 INFO:tasks.rados.rados.0.smithi118.stdout:3646: left oid 41 (ObjNum 1120 snap 359 seq_num 1120) 2024-03-20T08:13:57.218 INFO:tasks.rados.rados.0.smithi118.stdout:3648: expect (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:13:57.533 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:57.196+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:57.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:57.201+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:57.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:57.201+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:57.534 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:57.201+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: osdmap e787: 8 total, 8 up, 8 in 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:57.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:57.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:57.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:57.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:57.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:57.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:57.195+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:57.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:57.195+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:57.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:57.195+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: osdmap e787: 8 total, 8 up, 8 in 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:57.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:57.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:57.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:57.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:57.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:57.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: osdmap e787: 8 total, 8 up, 8 in 2024-03-20T08:13:57.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:57.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:57.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:57.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:57.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:57.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:57.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:57.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:57.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:57.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:57.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:57.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:57.533+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:57.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:57.534+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3644: done (5 left) 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3645: done (4 left) 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3646: done (3 left) 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3647: done (2 left) 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3648: done (1 left) 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3650: done (0 left) 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3651: copy_from oid 20 from oid 16 current snap is 360 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3652: rollback oid 22 current snap is 360 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 22 to 350 2024-03-20T08:13:58.197 INFO:tasks.rados.rados.0.smithi118.stdout:3653: snap_remove snap 358 2024-03-20T08:13:58.228 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:58.201+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:58.228 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:58.201+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:58.229 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:58.196+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:58.229 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:58.196+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:58.229 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:58.197+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:58.229 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:58.197+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:58.229 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:57.965+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:58.229 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:58.199+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:58.229 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:58.199+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: pgmap v868: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 17 op/s 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: osdmap e788: 8 total, 8 up, 8 in 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: pgmap v868: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 17 op/s 2024-03-20T08:13:58.230 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:58.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:58.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: osdmap e788: 8 total, 8 up, 8 in 2024-03-20T08:13:58.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:58.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:58.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:58.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:58.231 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:58.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:58.208+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:58.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:58.204+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:58.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:58.204+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:58.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:58.297+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:58.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:58.297+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:58.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:58.202+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:58.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:58.203+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:58.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:58.203+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:58.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: pgmap v868: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 17 op/s 2024-03-20T08:13:58.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:58.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:58.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: osdmap e788: 8 total, 8 up, 8 in 2024-03-20T08:13:58.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:58.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:58.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:58.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:58.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:58.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:58.228+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:59.210 INFO:tasks.rados.rados.0.smithi118.stdout:3653: done (2 left) 2024-03-20T08:13:59.211 INFO:tasks.rados.rados.0.smithi118.stdout:3654: delete oid 3 current snap is 360 2024-03-20T08:13:59.211 INFO:tasks.rados.rados.0.smithi118.stdout:3652: finishing rollback tid 0 to smithi11881747-22 2024-03-20T08:13:59.211 INFO:tasks.rados.rados.0.smithi118.stdout:3652: finishing rollback tid 1 to smithi11881747-22 2024-03-20T08:13:59.211 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 65313 (ObjNum 65 snap 0 seq_num 1516590417) dirty dne 2024-03-20T08:13:59.211 INFO:tasks.rados.rados.0.smithi118.stdout:3651: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:13:59.211 INFO:tasks.rados.rados.0.smithi118.stdout:3651: finishing copy_from to smithi11881747-20 2024-03-20T08:13:59.211 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 61764 (ObjNum 1040 snap 336 seq_num 1040) dirty exists 2024-03-20T08:13:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:3651: done (2 left) 2024-03-20T08:13:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:3652: done (1 left) 2024-03-20T08:13:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:3654: done (0 left) 2024-03-20T08:13:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:3655: read oid 27 snap -1 2024-03-20T08:13:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:3655: expect (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:13:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:3656: copy_from oid 25 from oid 17 current snap is 360 2024-03-20T08:13:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 360 2024-03-20T08:13:59.218 INFO:tasks.rados.rados.0.smithi118.stdout:3657: seq_num 1122 ranges {319488=794624,1114112=729088,1843200=475136} 2024-03-20T08:13:59.225 INFO:tasks.rados.rados.0.smithi118.stdout:3657: writing smithi11881747-19 from 319488 to 1114112 tid 1 2024-03-20T08:13:59.230 INFO:tasks.rados.rados.0.smithi118.stdout:3657: writing smithi11881747-19 from 1114112 to 1843200 tid 2 2024-03-20T08:13:59.234 INFO:tasks.rados.rados.0.smithi118.stdout:3657: writing smithi11881747-19 from 1843200 to 2318336 tid 3 2024-03-20T08:13:59.234 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 360 2024-03-20T08:13:59.234 INFO:tasks.rados.rados.0.smithi118.stdout:3658: seq_num 1123 ranges {1769472=409600,2179072=614400,2793472=598016,3391488=393216} 2024-03-20T08:13:59.245 INFO:tasks.rados.rados.0.smithi118.stdout:3658: writing smithi11881747-42 from 1769472 to 2179072 tid 1 2024-03-20T08:13:59.249 INFO:tasks.rados.rados.0.smithi118.stdout:3658: writing smithi11881747-42 from 2179072 to 2793472 tid 2 2024-03-20T08:13:59.254 INFO:tasks.rados.rados.0.smithi118.stdout:3658: writing smithi11881747-42 from 2793472 to 3391488 tid 3 2024-03-20T08:13:59.256 INFO:tasks.rados.rados.0.smithi118.stdout:3658: writing smithi11881747-42 from 3391488 to 3784704 tid 4 2024-03-20T08:13:59.256 INFO:tasks.rados.rados.0.smithi118.stdout:3659: rollback oid 10 current snap is 360 2024-03-20T08:13:59.256 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 350 2024-03-20T08:13:59.256 INFO:tasks.rados.rados.0.smithi118.stdout:3655: expect (ObjNum 413 snap 136 seq_num 413) 2024-03-20T08:13:59.472 INFO:tasks.rados.rados.0.smithi118.stdout:3657: finishing write tid 1 to smithi11881747-19 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3656: finishing copy_from racing read to smithi11881747-25 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3656: finishing copy_from to smithi11881747-25 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 61744 (ObjNum 1101 snap 352 seq_num 1101) dirty exists 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3657: finishing write tid 2 to smithi11881747-19 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3657: finishing write tid 3 to smithi11881747-19 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3657: finishing write tid 4 to smithi11881747-19 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3655: done (4 left) 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3656: done (3 left) 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3660: read oid 47 snap -1 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3660: expect (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3661: delete oid 30 current snap is 360 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3657: finishing write tid 5 to smithi11881747-19 2024-03-20T08:13:59.473 INFO:tasks.rados.rados.0.smithi118.stdout:3657: oid 19 updating version 0 to 59857 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3657: oid 19 version 59857 is already newer than 59856 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3657: oid 19 version 59857 is already newer than 59855 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3657: oid 19 version 59857 is already newer than 59854 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 59857 (ObjNum 1122 snap 360 seq_num 1122) dirty exists 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3657: left oid 19 (ObjNum 1122 snap 360 seq_num 1122) 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3658: finishing write tid 1 to smithi11881747-42 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3658: finishing write tid 2 to smithi11881747-42 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3659: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 65310 (ObjNum 33 snap 0 seq_num 291747632) dirty dne 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3658: finishing write tid 3 to smithi11881747-42 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3658: finishing write tid 4 to smithi11881747-42 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3658: finishing write tid 5 to smithi11881747-42 2024-03-20T08:13:59.474 INFO:tasks.rados.rados.0.smithi118.stdout:3658: finishing write tid 6 to smithi11881747-42 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:3658: oid 42 updating version 0 to 65323 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:3658: oid 42 updating version 65323 to 65324 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:3658: oid 42 version 65324 is already newer than 65320 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:3658: oid 42 version 65324 is already newer than 65321 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:3658: oid 42 version 65324 is already newer than 65322 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65324 (ObjNum 1123 snap 360 seq_num 1123) dirty exists 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:3658: left oid 42 (ObjNum 1123 snap 360 seq_num 1123) 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:3657: done (4 left) 2024-03-20T08:13:59.475 INFO:tasks.rados.rados.0.smithi118.stdout:3658: done (3 left) 2024-03-20T08:13:59.476 INFO:tasks.rados.rados.0.smithi118.stdout:3659: done (2 left) 2024-03-20T08:13:59.476 INFO:tasks.rados.rados.0.smithi118.stdout:3661: done (1 left) 2024-03-20T08:13:59.476 INFO:tasks.rados.rados.0.smithi118.stdout:3662: snap_remove snap 348 2024-03-20T08:13:59.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:13:59.208+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:59.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:59.216+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:59.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:13:59.216+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:59.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:59.233+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:13:59.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:59.233+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:59.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:13:59.233+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:59.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:59.226+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:59.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:59.228+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:13:59.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:13:59.228+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: osdmap e789: 8 total, 8 up, 8 in 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:59.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:59.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:59.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:59.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:59.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:59.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:59.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:59.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:13:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:59.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:13:59.216+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:59.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:13:59.210+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:59.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:59.222+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:13:59.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:59.222+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:59.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:13:59.224+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:59.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:59.209+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:13:59.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:59.209+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:59.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:59.209+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:59.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:59.210+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:59.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:13:59.210+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:59.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:59.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: osdmap e789: 8 total, 8 up, 8 in 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:59.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:59.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: osdmap e789: 8 total, 8 up, 8 in 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:13:59.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:13:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:13:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:13:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:13:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:13:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:13:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:13:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:13:59.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:13:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:00.214 INFO:tasks.rados.rados.0.smithi118.stdout:3662: done (1 left) 2024-03-20T08:14:00.214 INFO:tasks.rados.rados.0.smithi118.stdout:3663: rollback oid 16 current snap is 360 2024-03-20T08:14:00.214 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 354 2024-03-20T08:14:00.215 INFO:tasks.rados.rados.0.smithi118.stdout:3660: expect (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:14:00.256 INFO:tasks.rados.rados.0.smithi118.stdout:3663: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:14:00.257 INFO:tasks.rados.rados.0.smithi118.stdout:3663: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:14:00.257 INFO:tasks.rados.rados.0.smithi118.stdout:3663: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:14:00.257 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 65326 (ObjNum 1040 snap 336 seq_num 1040) dirty exists 2024-03-20T08:14:00.257 INFO:tasks.rados.rados.0.smithi118.stdout:3660: done (1 left) 2024-03-20T08:14:00.257 INFO:tasks.rados.rados.0.smithi118.stdout:3663: done (0 left) 2024-03-20T08:14:00.257 INFO:tasks.rados.rados.0.smithi118.stdout:3664: delete oid 15 current snap is 360 2024-03-20T08:14:00.263 INFO:tasks.rados.rados.0.smithi118.stdout:3664: done (0 left) 2024-03-20T08:14:00.264 INFO:tasks.rados.rados.0.smithi118.stdout:3665: read oid 22 snap 356 2024-03-20T08:14:00.264 INFO:tasks.rados.rados.0.smithi118.stdout:3665: expect (ObjNum 322033952 snap 21898 seq_num 1479489888) 2024-03-20T08:14:00.264 INFO:tasks.rados.rados.0.smithi118.stdout:3666: read oid 1 snap -1 2024-03-20T08:14:00.264 INFO:tasks.rados.rados.0.smithi118.stdout:3666: expect (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:14:00.264 INFO:tasks.rados.rados.0.smithi118.stdout:3667: setattr oid 6 current snap is 360 2024-03-20T08:14:00.264 INFO:tasks.rados.rados.0.smithi118.stdout:3665: expect (ObjNum 322033952 snap 21898 seq_num 1479489888) 2024-03-20T08:14:00.267 INFO:tasks.rados.rados.0.smithi118.stdout:3665: done (2 left) 2024-03-20T08:14:00.267 INFO:tasks.rados.rados.0.smithi118.stdout:3668: delete oid 17 current snap is 360 2024-03-20T08:14:00.267 INFO:tasks.rados.rados.0.smithi118.stdout:3666: expect (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:14:00.290 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 65305 (ObjNum 1060 snap 338 seq_num 1060) dirty exists 2024-03-20T08:14:00.290 INFO:tasks.rados.rados.0.smithi118.stdout:3666: done (2 left) 2024-03-20T08:14:00.290 INFO:tasks.rados.rados.0.smithi118.stdout:3667: done (1 left) 2024-03-20T08:14:00.290 INFO:tasks.rados.rados.0.smithi118.stdout:3668: done (0 left) 2024-03-20T08:14:00.290 INFO:tasks.rados.rados.0.smithi118.stdout:3669: delete oid 30 current snap is 360 2024-03-20T08:14:00.292 INFO:tasks.rados.rados.0.smithi118.stdout:3669: done (0 left) 2024-03-20T08:14:00.292 INFO:tasks.rados.rados.0.smithi118.stdout:3670: snap_create 2024-03-20T08:14:00.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:00.215+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:00.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:00.218+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:00.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:00.218+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:00.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:00.272+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:00.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:00.273+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:00.218+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:00.219+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:00.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:00.219+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: pgmap v871: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 21 op/s 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: osdmap e790: 8 total, 8 up, 8 in 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:00.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:00.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:00.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:00.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:00.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:00.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:00.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:00.262+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:00.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:00.261+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:00.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:00.261+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:00.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:00.219+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:00.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:00.219+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:00.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:00.214+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:00.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:00.214+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:00.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:00.214+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:00.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:00.215+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: pgmap v871: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 21 op/s 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: osdmap e790: 8 total, 8 up, 8 in 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:00.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: pgmap v871: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 21 op/s 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: osdmap e790: 8 total, 8 up, 8 in 2024-03-20T08:14:00.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18c=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:00.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19b=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:00.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:01.220 INFO:tasks.rados.rados.0.smithi118.stdout:3670: done (0 left) 2024-03-20T08:14:01.221 INFO:tasks.rados.rados.0.smithi118.stdout:3671: snap_remove snap 359 2024-03-20T08:14:01.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:01.219+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:01.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:01.446+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:01.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:01.446+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:01.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:01.446+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: osdmap e791: 8 total, 8 up, 8 in 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:01.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:01.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:01.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:01.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:01.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:01.219+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:01.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:01.221+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:01.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:01.221+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:01.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:01.221+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:01.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:01.221+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:01.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:01.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:01.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: osdmap e791: 8 total, 8 up, 8 in 2024-03-20T08:14:01.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:01.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:01.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:01.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:01.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 19c=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:01.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: osdmap e791: 8 total, 8 up, 8 in 2024-03-20T08:14:01.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:01.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:01.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:01.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:01.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:01.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:01.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:01.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:01.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:01.679 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:01.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:01.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:01.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:01.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:01.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:01.681 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:01.681 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:01.220+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:02.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:01.698+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:02.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:01.698+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:02.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:01.698+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:02.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:01.698+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:02.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:01.698+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:02.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:01.698+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:02.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:01.697+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:02.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:01.697+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:02.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:01.698+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:02.226 INFO:tasks.rados.rados.0.smithi118.stdout:3671: done (0 left) 2024-03-20T08:14:02.226 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 361 2024-03-20T08:14:02.226 INFO:tasks.rados.rados.0.smithi118.stdout:3672: seq_num 1128 ranges {2203648=499712,2703360=483328,3186688=786432,3973120=335872} 2024-03-20T08:14:02.240 INFO:tasks.rados.rados.0.smithi118.stdout:3672: writing smithi11881747-7 from 2203648 to 2703360 tid 1 2024-03-20T08:14:02.243 INFO:tasks.rados.rados.0.smithi118.stdout:3672: writing smithi11881747-7 from 2703360 to 3186688 tid 2 2024-03-20T08:14:02.247 INFO:tasks.rados.rados.0.smithi118.stdout:3672: writing smithi11881747-7 from 3186688 to 3973120 tid 3 2024-03-20T08:14:02.249 INFO:tasks.rados.rados.0.smithi118.stdout:3672: writing smithi11881747-7 from 3973120 to 4308992 tid 4 2024-03-20T08:14:02.249 INFO:tasks.rados.rados.0.smithi118.stdout:3673: snap_remove snap 351 2024-03-20T08:14:02.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:02.225+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:02.541 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:02.226+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:02.542 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:02.225+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:02.542 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:02.225+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:02.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:02.243+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:02.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:02.244+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:02.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:02.244+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:02.543 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:02.253+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:02.544 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: pgmap v874: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 20 op/s 2024-03-20T08:14:02.544 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:02.544 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:02.544 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:02.544 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: osdmap e792: 8 total, 8 up, 8 in 2024-03-20T08:14:02.545 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:02.545 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:02.545 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:02.545 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:02.545 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:02.545 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:02.546 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:02.546 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: pgmap v874: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 20 op/s 2024-03-20T08:14:02.546 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:02.546 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:02.547 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:02.547 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: osdmap e792: 8 total, 8 up, 8 in 2024-03-20T08:14:02.547 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:02.547 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:02.547 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:02.547 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:02.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:02.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:02.548 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:02.549 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:14:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:14:02.550 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:02.247+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:02.550 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:02.247+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:02.551 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:02.224+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:02.551 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:02.224+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:02.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: pgmap v874: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 6.4 MiB/s rd, 6.7 MiB/s wr, 20 op/s 2024-03-20T08:14:02.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:02.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:02.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:02.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: osdmap e792: 8 total, 8 up, 8 in 2024-03-20T08:14:02.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:02.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:02.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:02.552 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:02.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:02.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:02.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:02.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:02.542+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:02.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:02.542+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:02.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:02.542+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:02.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:02.542+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:02.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:02.541+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:03.231 INFO:tasks.rados.rados.0.smithi118.stdout:3673: done (1 left) 2024-03-20T08:14:03.232 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 361 2024-03-20T08:14:03.232 INFO:tasks.rados.rados.0.smithi118.stdout:3672: finishing write tid 1 to smithi11881747-7 2024-03-20T08:14:03.232 INFO:tasks.rados.rados.0.smithi118.stdout:3674: seq_num 1129 ranges {5349376=516096,5865472=770048,6635520=598016,7233536=237568} 2024-03-20T08:14:03.265 INFO:tasks.rados.rados.0.smithi118.stdout:3674: writing smithi11881747-41 from 5349376 to 5865472 tid 1 2024-03-20T08:14:03.267 INFO:tasks.rados.rados.0.smithi118.stdout:3674: writing smithi11881747-41 from 5865472 to 6635520 tid 2 2024-03-20T08:14:03.271 INFO:tasks.rados.rados.0.smithi118.stdout:3674: writing smithi11881747-41 from 6635520 to 7233536 tid 3 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3674: writing smithi11881747-41 from 7233536 to 7471104 tid 4 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3675: read oid 43 snap 356 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3675: expect (ObjNum 1110 snap 355 seq_num 1110) 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3676: copy_from oid 29 from oid 23 current snap is 361 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3672: finishing write tid 2 to smithi11881747-7 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3672: finishing write tid 3 to smithi11881747-7 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3672: finishing write tid 4 to smithi11881747-7 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3672: finishing write tid 5 to smithi11881747-7 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3672: finishing write tid 6 to smithi11881747-7 2024-03-20T08:14:03.274 INFO:tasks.rados.rados.0.smithi118.stdout:3672: oid 7 updating version 0 to 65302 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3672: oid 7 version 65302 is already newer than 65300 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3672: oid 7 updating version 65302 to 65303 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3672: oid 7 updating version 65303 to 65304 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3672: oid 7 version 65304 is already newer than 65301 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 65304 (ObjNum 1128 snap 361 seq_num 1128) dirty exists 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3672: left oid 7 (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3674: finishing write tid 1 to smithi11881747-41 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3674: finishing write tid 2 to smithi11881747-41 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3672: done (3 left) 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3677: read oid 42 snap 360 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3677: expect (ObjNum 1123 snap 360 seq_num 1123) 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 361 2024-03-20T08:14:03.275 INFO:tasks.rados.rados.0.smithi118.stdout:3678: seq_num 1130 ranges {3334144=532480,3866624=655360,4521984=753664,5275648=196608} 2024-03-20T08:14:03.298 INFO:tasks.rados.rados.0.smithi118.stdout:3678: writing smithi11881747-26 from 3334144 to 3866624 tid 1 2024-03-20T08:14:03.301 INFO:tasks.rados.rados.0.smithi118.stdout:3678: writing smithi11881747-26 from 3866624 to 4521984 tid 2 2024-03-20T08:14:03.306 INFO:tasks.rados.rados.0.smithi118.stdout:3678: writing smithi11881747-26 from 4521984 to 5275648 tid 3 2024-03-20T08:14:03.307 INFO:tasks.rados.rados.0.smithi118.stdout:3678: writing smithi11881747-26 from 5275648 to 5472256 tid 4 2024-03-20T08:14:03.307 INFO:tasks.rados.rados.0.smithi118.stdout:3679: rollback oid 45 current snap is 361 2024-03-20T08:14:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 45 to 356 2024-03-20T08:14:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:3674: finishing write tid 3 to smithi11881747-41 2024-03-20T08:14:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:3674: finishing write tid 4 to smithi11881747-41 2024-03-20T08:14:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:3674: finishing write tid 5 to smithi11881747-41 2024-03-20T08:14:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:3674: finishing write tid 6 to smithi11881747-41 2024-03-20T08:14:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:3674: oid 41 updating version 0 to 65328 2024-03-20T08:14:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:3674: oid 41 updating version 65328 to 65331 2024-03-20T08:14:03.308 INFO:tasks.rados.rados.0.smithi118.stdout:3674: oid 41 version 65331 is already newer than 65327 2024-03-20T08:14:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:3674: oid 41 version 65331 is already newer than 65329 2024-03-20T08:14:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:3674: oid 41 version 65331 is already newer than 65330 2024-03-20T08:14:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 65331 (ObjNum 1129 snap 361 seq_num 1129) dirty exists 2024-03-20T08:14:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:3674: left oid 41 (ObjNum 1129 snap 361 seq_num 1129) 2024-03-20T08:14:03.309 INFO:tasks.rados.rados.0.smithi118.stdout:3677: expect (ObjNum 1123 snap 360 seq_num 1123) 2024-03-20T08:14:03.467 INFO:tasks.rados.rados.0.smithi118.stdout:3675: expect (ObjNum 1110 snap 355 seq_num 1110) 2024-03-20T08:14:03.585 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:03.278+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:03.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:03.230+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:03.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:03.230+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:03.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:03.230+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:03.586 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:03.232+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:03.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:03.267+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:03.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:03.267+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:03.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:03.267+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:03.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:03.267+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:03.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:03.269+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: osdmap e793: 8 total, 8 up, 8 in 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:03.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:03.588 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: osdmap e793: 8 total, 8 up, 8 in 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:03.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:03.590 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:03.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:03.231+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:03.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:03.269+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:03.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:03.269+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:03.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:03.269+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:03.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:03.276+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:03.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:03.281+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:03.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:03.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:03.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:03.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:03.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: osdmap e793: 8 total, 8 up, 8 in 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:03.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:03.647 INFO:tasks.rados.rados.0.smithi118.stdout:3676: finishing copy_from racing read to smithi11881747-29 2024-03-20T08:14:03.647 INFO:tasks.rados.rados.0.smithi118.stdout:3676: finishing copy_from to smithi11881747-29 2024-03-20T08:14:03.647 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 61752 (ObjNum 1075 snap 341 seq_num 1075) dirty exists 2024-03-20T08:14:03.647 INFO:tasks.rados.rados.0.smithi118.stdout:3678: finishing write tid 1 to smithi11881747-26 2024-03-20T08:14:03.647 INFO:tasks.rados.rados.0.smithi118.stdout:3678: finishing write tid 2 to smithi11881747-26 2024-03-20T08:14:03.647 INFO:tasks.rados.rados.0.smithi118.stdout:3674: done (5 left) 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3675: done (4 left) 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3676: done (3 left) 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3677: done (2 left) 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3680: setattr oid 9 current snap is 361 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3678: finishing write tid 3 to smithi11881747-26 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3679: finishing rollback tid 0 to smithi11881747-45 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3679: finishing rollback tid 1 to smithi11881747-45 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3678: finishing write tid 4 to smithi11881747-26 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3678: finishing write tid 5 to smithi11881747-26 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3679: finishing rollback tid 2 to smithi11881747-45 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 65321 (ObjNum 1040 snap 336 seq_num 1040) dirty exists 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3678: finishing write tid 6 to smithi11881747-26 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3678: oid 26 updating version 0 to 65331 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3678: oid 26 version 65331 is already newer than 65327 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3678: oid 26 version 65331 is already newer than 65329 2024-03-20T08:14:03.648 INFO:tasks.rados.rados.0.smithi118.stdout:3678: oid 26 version 65331 is already newer than 65328 2024-03-20T08:14:03.649 INFO:tasks.rados.rados.0.smithi118.stdout:3678: oid 26 version 65331 is already newer than 65330 2024-03-20T08:14:03.649 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65331 (ObjNum 1130 snap 361 seq_num 1130) dirty exists 2024-03-20T08:14:03.649 INFO:tasks.rados.rados.0.smithi118.stdout:3678: left oid 26 (ObjNum 1130 snap 361 seq_num 1130) 2024-03-20T08:14:03.649 INFO:tasks.rados.rados.0.smithi118.stdout:3678: done (2 left) 2024-03-20T08:14:03.649 INFO:tasks.rados.rados.0.smithi118.stdout:3679: done (1 left) 2024-03-20T08:14:03.649 INFO:tasks.rados.rados.0.smithi118.stdout:3681: snap_create 2024-03-20T08:14:03.651 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 65325 (ObjNum 1097 snap 351 seq_num 1097) dirty exists 2024-03-20T08:14:03.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:14:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:14:03] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:14:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:3680: done (1 left) 2024-03-20T08:14:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:3681: done (0 left) 2024-03-20T08:14:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:3682: read oid 20 snap -1 2024-03-20T08:14:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:3682: expect (ObjNum 1040 snap 336 seq_num 1040) 2024-03-20T08:14:04.236 INFO:tasks.rados.rados.0.smithi118.stdout:3683: snap_create 2024-03-20T08:14:04.278 INFO:tasks.rados.rados.0.smithi118.stdout:3682: expect (ObjNum 1040 snap 336 seq_num 1040) 2024-03-20T08:14:04.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:04.235+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:04.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:04.009+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:04.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:04.009+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:04.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:04.009+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: pgmap v877: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-03-20T08:14:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: osdmap e794: 8 total, 8 up, 8 in 2024-03-20T08:14:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:04.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:04.519 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:04.236+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:04.519 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:04.236+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:04.519 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:04.236+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:04.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:04.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:04.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:04.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: pgmap v877: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-03-20T08:14:04.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:04.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:04.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: osdmap e794: 8 total, 8 up, 8 in 2024-03-20T08:14:04.520 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:04.521 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: pgmap v877: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 3.8 MiB/s rd, 5.4 MiB/s wr, 11 op/s 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: osdmap e794: 8 total, 8 up, 8 in 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 194=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:04.522 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:04.523 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:04.523 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:04.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:04.672+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:04.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:04.673+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:04.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:04.519+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:04.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:04.519+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:04.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:04.519+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:04.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:04.519+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:05.548 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:05.246+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:05.548 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:05.244+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:05.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:05.244+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:05.549 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:05.244+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:05.549 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:05.242+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:05.549 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:05.242+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:05.549 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:05.244+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:05.549 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:05.244+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:05.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:05.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:05.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:05.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:05.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: osdmap e795: 8 total, 8 up, 8 in 2024-03-20T08:14:05.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:05.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:05.550 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:05.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:05.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:05.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:05.551 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:05.571 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:05.247+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:05.571 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:05.247+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:05.571 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: osdmap e795: 8 total, 8 up, 8 in 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:05.572 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: osdmap e795: 8 total, 8 up, 8 in 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:05.573 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:05.574 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:05.655 INFO:tasks.rados.rados.0.smithi118.stdout:3682: done (1 left) 2024-03-20T08:14:05.655 INFO:tasks.rados.rados.0.smithi118.stdout:3683: done (0 left) 2024-03-20T08:14:05.655 INFO:tasks.rados.rados.0.smithi118.stdout:3684: snap_create 2024-03-20T08:14:05.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:05.548+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:05.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:05.548+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:05.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:05.548+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:05.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:05.570+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:05.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:05.570+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:05.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:05.570+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:05.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:05.571+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:05.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:05 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:05.571+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:05.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:05.570+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:05.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:05.570+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:05.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:05.571+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:05.570+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:05.570+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:05.571+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:05.571+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:05.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:05 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:05.571+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:06.252 INFO:tasks.rados.rados.0.smithi118.stdout:3684: done (0 left) 2024-03-20T08:14:06.252 INFO:tasks.rados.rados.0.smithi118.stdout:3685: rmattr oid 8 current snap is 364 2024-03-20T08:14:06.252 INFO:tasks.rados.rados.0.smithi118.stdout:3686: copy_from oid 30 from oid 7 current snap is 364 2024-03-20T08:14:06.252 INFO:tasks.rados.rados.0.smithi118.stdout:3687: setattr oid 47 current snap is 364 2024-03-20T08:14:06.252 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 0 (ObjNum 1112 snap 357 seq_num 1112) dirty exists 2024-03-20T08:14:06.253 INFO:tasks.rados.rados.0.smithi118.stdout:3685: done (2 left) 2024-03-20T08:14:06.253 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 364 2024-03-20T08:14:06.253 INFO:tasks.rados.rados.0.smithi118.stdout:3688: seq_num 1131 ranges {1646592=548864,2195456=540672,2736128=565248,3301376=425984,3727360=73728} 2024-03-20T08:14:06.265 INFO:tasks.rados.rados.0.smithi118.stdout:3688: writing smithi11881747-25 from 1646592 to 2195456 tid 1 2024-03-20T08:14:06.268 INFO:tasks.rados.rados.0.smithi118.stdout:3688: writing smithi11881747-25 from 2195456 to 2736128 tid 2 2024-03-20T08:14:06.271 INFO:tasks.rados.rados.0.smithi118.stdout:3688: writing smithi11881747-25 from 2736128 to 3301376 tid 3 2024-03-20T08:14:06.274 INFO:tasks.rados.rados.0.smithi118.stdout:3688: writing smithi11881747-25 from 3301376 to 3727360 tid 4 2024-03-20T08:14:06.274 INFO:tasks.rados.rados.0.smithi118.stdout:3688: writing smithi11881747-25 from 3727360 to 3801088 tid 5 2024-03-20T08:14:06.274 INFO:tasks.rados.rados.0.smithi118.stdout:3689: snap_remove snap 362 2024-03-20T08:14:06.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:06.253+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:06.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:06.253+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:06.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:06.259+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:06.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:06.259+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:06.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:06.259+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:06.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: pgmap v880: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 608 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-03-20T08:14:06.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:06.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:06.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:06.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: osdmap e796: 8 total, 8 up, 8 in 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:06.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:06.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:06.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:06.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:06.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:06.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:06.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:06.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:06.268+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:06.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:06.271+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:06.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:06.256+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:06.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:06.256+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:06.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:06.256+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:06.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:06.257+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:06.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:06.257+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:06.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:06.259+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:06.675 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:06.251+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:06.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: pgmap v880: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 608 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-03-20T08:14:06.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:06.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:06.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:06.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:06.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:06.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:06.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:06.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:06.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: osdmap e796: 8 total, 8 up, 8 in 2024-03-20T08:14:06.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:06.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:06.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: pgmap v880: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 608 KiB/s rd, 1.4 MiB/s wr, 6 op/s 2024-03-20T08:14:06.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:06.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:06.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:06.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:06.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:06.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:06.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:06.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:06.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: osdmap e796: 8 total, 8 up, 8 in 2024-03-20T08:14:06.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:06.680 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:06.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:06.680 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:06.681 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:06.681 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:06.681 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:06.681 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:06.681 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:06.682 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:06.682 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:06.682 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:06.682 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:06.682 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:06.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:06.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:06.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:06.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:06.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:06.683 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:07.257 INFO:tasks.rados.rados.0.smithi118.stdout:3689: done (3 left) 2024-03-20T08:14:07.258 INFO:tasks.rados.rados.0.smithi118.stdout:3690: read oid 4 snap 357 2024-03-20T08:14:07.258 INFO:tasks.rados.rados.0.smithi118.stdout:3690: expect (ObjNum 1107 snap 353 seq_num 1107) 2024-03-20T08:14:07.258 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 17 current snap is 364 2024-03-20T08:14:07.258 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 65332 (ObjNum 1032 snap 331 seq_num 1032) dirty exists 2024-03-20T08:14:07.258 INFO:tasks.rados.rados.0.smithi118.stdout:3691: seq_num 1132 ranges {0=565248,565248=425984,991232=770048,1761280=409600} 2024-03-20T08:14:07.260 INFO:tasks.rados.rados.0.smithi118.stdout:3691: writing smithi11881747-17 from 0 to 565248 tid 1 2024-03-20T08:14:07.263 INFO:tasks.rados.rados.0.smithi118.stdout:3691: writing smithi11881747-17 from 565248 to 991232 tid 2 2024-03-20T08:14:07.267 INFO:tasks.rados.rados.0.smithi118.stdout:3691: writing smithi11881747-17 from 991232 to 1761280 tid 3 2024-03-20T08:14:07.270 INFO:tasks.rados.rados.0.smithi118.stdout:3691: writing smithi11881747-17 from 1761280 to 2170880 tid 4 2024-03-20T08:14:07.270 INFO:tasks.rados.rados.0.smithi118.stdout:3687: done (4 left) 2024-03-20T08:14:07.270 INFO:tasks.rados.rados.0.smithi118.stdout:3692: snap_remove snap 356 2024-03-20T08:14:07.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:07.258+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:07.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:07.268+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:07.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:07.279+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:07.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:07.279+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:07.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:07.272+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:07.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:07.272+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:07.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:07.265+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:07.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:07.277+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:07.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:07.277+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: osdmap e797: 8 total, 8 up, 8 in 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:07.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:07.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:07.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:07.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:07.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:07.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:07.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:07.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:07.261+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:07.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:07.257+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:07.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:07.268+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:07.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:07.268+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:07.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:07.269+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:07.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:07.269+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:07.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:07.269+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:07.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:07.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: osdmap e797: 8 total, 8 up, 8 in 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:07.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: osdmap e797: 8 total, 8 up, 8 in 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:07.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:07.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:08.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:07.699+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:08.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:07.699+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:08.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:07.699+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:08.262 INFO:tasks.rados.rados.0.smithi118.stdout:3692: done (4 left) 2024-03-20T08:14:08.262 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 364 2024-03-20T08:14:08.262 INFO:tasks.rados.rados.0.smithi118.stdout:3688: finishing write tid 1 to smithi11881747-25 2024-03-20T08:14:08.262 INFO:tasks.rados.rados.0.smithi118.stdout:3693: seq_num 1133 ranges {7954432=581632,8536064=753664,9289728=573440,9863168=278528} 2024-03-20T08:14:08.308 INFO:tasks.rados.rados.0.smithi118.stdout:3693: writing smithi11881747-49 from 7954432 to 8536064 tid 1 2024-03-20T08:14:08.312 INFO:tasks.rados.rados.0.smithi118.stdout:3693: writing smithi11881747-49 from 8536064 to 9289728 tid 2 2024-03-20T08:14:08.316 INFO:tasks.rados.rados.0.smithi118.stdout:3693: writing smithi11881747-49 from 9289728 to 9863168 tid 3 2024-03-20T08:14:08.318 INFO:tasks.rados.rados.0.smithi118.stdout:3693: writing smithi11881747-49 from 9863168 to 10141696 tid 4 2024-03-20T08:14:08.318 INFO:tasks.rados.rados.0.smithi118.stdout:3694: snap_create 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: finishing write tid 2 to smithi11881747-25 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: finishing write tid 3 to smithi11881747-25 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3686: finishing copy_from racing read to smithi11881747-30 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: finishing write tid 4 to smithi11881747-25 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3686: finishing copy_from to smithi11881747-30 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 65304 (ObjNum 1128 snap 361 seq_num 1128) dirty exists 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: finishing write tid 5 to smithi11881747-25 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: finishing write tid 6 to smithi11881747-25 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: finishing write tid 7 to smithi11881747-25 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: oid 25 updating version 0 to 65328 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: oid 25 updating version 65328 to 65329 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: oid 25 updating version 65329 to 65331 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: oid 25 version 65331 is already newer than 65327 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: oid 25 version 65331 is already newer than 65330 2024-03-20T08:14:08.319 INFO:tasks.rados.rados.0.smithi118.stdout:3688: oid 25 version 65331 is already newer than 65326 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 65331 (ObjNum 1131 snap 364 seq_num 1131) dirty exists 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3688: left oid 25 (ObjNum 1131 snap 364 seq_num 1131) 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: finishing write tid 1 to smithi11881747-17 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: finishing write tid 2 to smithi11881747-17 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: finishing write tid 3 to smithi11881747-17 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: finishing write tid 4 to smithi11881747-17 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: finishing write tid 5 to smithi11881747-17 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: finishing write tid 6 to smithi11881747-17 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: oid 17 updating version 0 to 61748 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: oid 17 version 61748 is already newer than 61747 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: oid 17 updating version 61748 to 61750 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: oid 17 version 61750 is already newer than 61746 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: oid 17 version 61750 is already newer than 61749 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 17 v 61750 (ObjNum 1132 snap 364 seq_num 1132) dirty exists 2024-03-20T08:14:08.320 INFO:tasks.rados.rados.0.smithi118.stdout:3691: left oid 17 (ObjNum 1132 snap 364 seq_num 1132) 2024-03-20T08:14:08.321 INFO:tasks.rados.rados.0.smithi118.stdout:3690: expect (ObjNum 1107 snap 353 seq_num 1107) 2024-03-20T08:14:08.446 INFO:tasks.rados.rados.0.smithi118.stdout:3693: finishing write tid 1 to smithi11881747-49 2024-03-20T08:14:08.446 INFO:tasks.rados.rados.0.smithi118.stdout:3693: finishing write tid 2 to smithi11881747-49 2024-03-20T08:14:08.446 INFO:tasks.rados.rados.0.smithi118.stdout:3693: finishing write tid 3 to smithi11881747-49 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: finishing write tid 4 to smithi11881747-49 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: finishing write tid 5 to smithi11881747-49 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: finishing write tid 6 to smithi11881747-49 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: oid 49 updating version 0 to 61769 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: oid 49 version 61769 is already newer than 61767 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: oid 49 version 61769 is already newer than 61766 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: oid 49 version 61769 is already newer than 61768 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: oid 49 version 61769 is already newer than 61765 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 61769 (ObjNum 1133 snap 364 seq_num 1133) dirty exists 2024-03-20T08:14:08.447 INFO:tasks.rados.rados.0.smithi118.stdout:3693: left oid 49 (ObjNum 1133 snap 364 seq_num 1133) 2024-03-20T08:14:08.535 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:08.341+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:08.535 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:08.342+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:08.535 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:08.262+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:08.536 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:08.315+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:08.536 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:08.315+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:08.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: pgmap v883: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-03-20T08:14:08.536 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: osdmap e798: 8 total, 8 up, 8 in 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:08.537 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: pgmap v883: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: osdmap e798: 8 total, 8 up, 8 in 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:08.538 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:08.539 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:08.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:08.344+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:08.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:08.344+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:08.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:08.263+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:08.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:08.344+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:08.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:08.344+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:08.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:08.263+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:08.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:08.263+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:08.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:08.263+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: pgmap v883: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 1.0 MiB/s wr, 19 op/s 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: osdmap e798: 8 total, 8 up, 8 in 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:08.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:08.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:08.535+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:08.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:08.535+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:08.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:08.535+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:08.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:08.535+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:09.268 INFO:tasks.rados.rados.0.smithi118.stdout:3686: done (5 left) 2024-03-20T08:14:09.268 INFO:tasks.rados.rados.0.smithi118.stdout:3688: done (4 left) 2024-03-20T08:14:09.268 INFO:tasks.rados.rados.0.smithi118.stdout:3690: done (3 left) 2024-03-20T08:14:09.268 INFO:tasks.rados.rados.0.smithi118.stdout:3691: done (2 left) 2024-03-20T08:14:09.268 INFO:tasks.rados.rados.0.smithi118.stdout:3693: done (1 left) 2024-03-20T08:14:09.268 INFO:tasks.rados.rados.0.smithi118.stdout:3694: done (0 left) 2024-03-20T08:14:09.268 INFO:tasks.rados.rados.0.smithi118.stdout:3695: snap_create 2024-03-20T08:14:09.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:09.267+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:09.575 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:09.267+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:09.576 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:09.268+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:09.576 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:09.268+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:09.576 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:09.268+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:09.576 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:09.576 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:09.576 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:09.576 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:09.576 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:09.576 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: osdmap e799: 8 total, 8 up, 8 in 2024-03-20T08:14:09.576 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:09.577 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: osdmap e799: 8 total, 8 up, 8 in 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:09.578 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:09.579 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:09.579 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:09.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:09.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:09.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:09.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:09.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:09.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:09.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:09.269+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:09.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:09.269+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:09.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:09.269+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:09.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:09.269+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:09.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:09.310+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:09.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:09.310+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:09.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:09.310+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:09.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:09.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: osdmap e799: 8 total, 8 up, 8 in 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:09.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:09.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:09.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:09.922 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:09.575+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:09.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:09.649+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:09.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:09.648+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:09.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:09.648+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:09.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:09.649+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:10.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:09.649+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:10.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:09.649+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:10.269 INFO:tasks.rados.rados.0.smithi118.stdout:3695: done (0 left) 2024-03-20T08:14:10.269 INFO:tasks.rados.rados.0.smithi118.stdout:3696: rollback oid 29 current snap is 366 2024-03-20T08:14:10.269 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 365 2024-03-20T08:14:10.269 INFO:tasks.rados.rados.0.smithi118.stdout:3697: rmattr oid 15 current snap is 366 2024-03-20T08:14:10.269 INFO:tasks.rados.rados.0.smithi118.stdout:3697: done (1 left) 2024-03-20T08:14:10.269 INFO:tasks.rados.rados.0.smithi118.stdout:3698: rmattr oid 40 current snap is 366 2024-03-20T08:14:10.269 INFO:tasks.rados.rados.0.smithi118.stdout:3698: done (1 left) 2024-03-20T08:14:10.269 INFO:tasks.rados.rados.0.smithi118.stdout:3699: snap_remove snap 353 2024-03-20T08:14:10.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:10.269+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:10.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:10.270+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:10.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:10.270+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:10.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:10.271+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: pgmap v886: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: osdmap e800: 8 total, 8 up, 8 in 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:10.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:10.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:10.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:10.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:10.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:10.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:10.269+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:10.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:10.269+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:10.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:10.275+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:10.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:10.275+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:10.272+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:10.272+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:10.272+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:10.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:10.276+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: pgmap v886: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: osdmap e800: 8 total, 8 up, 8 in 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: pgmap v886: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: osdmap e800: 8 total, 8 up, 8 in 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:10.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:10.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:10.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:10.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:11.262 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:10.998+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:11.263 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:10.935+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:11.263 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:10.935+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:11.263 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:10.999+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:11.263 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:10.999+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:11.275 INFO:tasks.rados.rados.0.smithi118.stdout:3699: done (1 left) 2024-03-20T08:14:11.275 INFO:tasks.rados.rados.0.smithi118.stdout:3700: delete oid 12 current snap is 366 2024-03-20T08:14:11.275 INFO:tasks.rados.rados.0.smithi118.stdout:3696: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:14:11.275 INFO:tasks.rados.rados.0.smithi118.stdout:3696: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:14:11.275 INFO:tasks.rados.rados.0.smithi118.stdout:3696: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:14:11.276 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 65335 (ObjNum 1075 snap 341 seq_num 1075) dirty exists 2024-03-20T08:14:11.284 INFO:tasks.rados.rados.0.smithi118.stdout:3696: done (1 left) 2024-03-20T08:14:11.284 INFO:tasks.rados.rados.0.smithi118.stdout:3700: done (0 left) 2024-03-20T08:14:11.285 INFO:tasks.rados.rados.0.smithi118.stdout:3701: snap_create 2024-03-20T08:14:11.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:11.273+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:11.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:11.371+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:11.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:11.371+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:11.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:11.371+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:11.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:11.281+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:11.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:11.283+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:11.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:11.283+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:11.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:11.284+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:11.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:11.285+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: osdmap e801: 8 total, 8 up, 8 in 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:11.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:11.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:11.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:11.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:11.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:11 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: osdmap e801: 8 total, 8 up, 8 in 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:11.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:11.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:11.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:11.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:11.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:11.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:11.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: osdmap e801: 8 total, 8 up, 8 in 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:11.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:14:11.676 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:11.273+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:11.676 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:11.273+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:11.676 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:11.274+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:11.676 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:11.371+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:11.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:11.278+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:11.676 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:11.278+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:11.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:11.283+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:11.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:11.283+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:11.677 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:11.283+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:11.677 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:11.275+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:11.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:11.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:11.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:11.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:11.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:11 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:14:12.275 INFO:tasks.rados.rados.0.smithi118.stdout:3701: done (0 left) 2024-03-20T08:14:12.275 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 367 2024-03-20T08:14:12.276 INFO:tasks.rados.rados.0.smithi118.stdout:3702: seq_num 1135 ranges {2318336=614400,2932736=524288,3457024=540672,3997696=483328,4481024=57344} 2024-03-20T08:14:12.291 INFO:tasks.rados.rados.0.smithi118.stdout:3702: writing smithi11881747-19 from 2318336 to 2932736 tid 1 2024-03-20T08:14:12.294 INFO:tasks.rados.rados.0.smithi118.stdout:3702: writing smithi11881747-19 from 2932736 to 3457024 tid 2 2024-03-20T08:14:12.297 INFO:tasks.rados.rados.0.smithi118.stdout:3702: writing smithi11881747-19 from 3457024 to 3997696 tid 3 2024-03-20T08:14:12.300 INFO:tasks.rados.rados.0.smithi118.stdout:3702: writing smithi11881747-19 from 3997696 to 4481024 tid 4 2024-03-20T08:14:12.300 INFO:tasks.rados.rados.0.smithi118.stdout:3702: writing smithi11881747-19 from 4481024 to 4538368 tid 5 2024-03-20T08:14:12.300 INFO:tasks.rados.rados.0.smithi118.stdout:3703: copy_from oid 20 from oid 49 current snap is 367 2024-03-20T08:14:12.300 INFO:tasks.rados.rados.0.smithi118.stdout:3704: snap_remove snap 363 2024-03-20T08:14:12.558 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:12.301+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:12.559 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:12.303+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:12.559 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:12.276+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:12.559 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:12.274+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:12.559 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:12.274+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: pgmap v889: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 11 op/s 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:12.560 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: osdmap e802: 8 total, 8 up, 8 in 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: pgmap v889: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 11 op/s 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:12.561 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: osdmap e802: 8 total, 8 up, 8 in 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:12.562 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:12.563 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:12.564 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:12.564 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:12.564 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:12.564 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:12.564 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:12.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:12.274+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:14:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:14:12.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:12.299+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:12.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:12.313+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:12.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:12.329+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:12.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:12.329+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:12.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:12.329+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: pgmap v889: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 11 op/s 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: osdmap e802: 8 total, 8 up, 8 in 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:12.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:12.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:12.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:12.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:12.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:12.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:12.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:12.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:12 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:12.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:12.558+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:12.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:12.558+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:12.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:12.558+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:12.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:12.559+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:13.276 INFO:tasks.rados.rados.0.smithi118.stdout:3704: done (2 left) 2024-03-20T08:14:13.276 INFO:tasks.rados.rados.0.smithi118.stdout:3705: rollback oid 39 current snap is 367 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 352 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: finishing write tid 1 to smithi11881747-19 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: finishing write tid 2 to smithi11881747-19 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: finishing write tid 3 to smithi11881747-19 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: finishing write tid 4 to smithi11881747-19 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: finishing write tid 5 to smithi11881747-19 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: finishing write tid 6 to smithi11881747-19 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: finishing write tid 7 to smithi11881747-19 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: oid 19 updating version 0 to 59861 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: oid 19 version 59861 is already newer than 59859 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: oid 19 updating version 59861 to 59862 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: oid 19 updating version 59862 to 59864 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: oid 19 version 59864 is already newer than 59860 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: oid 19 version 59864 is already newer than 59863 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 59864 (ObjNum 1135 snap 367 seq_num 1135) dirty exists 2024-03-20T08:14:13.277 INFO:tasks.rados.rados.0.smithi118.stdout:3702: left oid 19 (ObjNum 1135 snap 367 seq_num 1135) 2024-03-20T08:14:13.278 INFO:tasks.rados.rados.0.smithi118.stdout:3703: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:14:13.278 INFO:tasks.rados.rados.0.smithi118.stdout:3703: finishing copy_from to smithi11881747-20 2024-03-20T08:14:13.278 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 61769 (ObjNum 1133 snap 364 seq_num 1133) dirty exists 2024-03-20T08:14:13.278 INFO:tasks.rados.rados.0.smithi118.stdout:3702: done (2 left) 2024-03-20T08:14:13.278 INFO:tasks.rados.rados.0.smithi118.stdout:3703: done (1 left) 2024-03-20T08:14:13.278 INFO:tasks.rados.rados.0.smithi118.stdout:3706: copy_from oid 27 from oid 46 current snap is 367 2024-03-20T08:14:13.278 INFO:tasks.rados.rados.0.smithi118.stdout:3707: delete oid 4 current snap is 367 2024-03-20T08:14:13.281 INFO:tasks.rados.rados.0.smithi118.stdout:3705: finishing rollback tid 0 to smithi11881747-39 2024-03-20T08:14:13.281 INFO:tasks.rados.rados.0.smithi118.stdout:3707: done (2 left) 2024-03-20T08:14:13.281 INFO:tasks.rados.rados.0.smithi118.stdout:3708: rollback oid 38 current snap is 367 2024-03-20T08:14:13.281 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 38 to 365 2024-03-20T08:14:13.281 INFO:tasks.rados.rados.0.smithi118.stdout:3705: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:14:13.281 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 23 current snap is 367 2024-03-20T08:14:13.281 INFO:tasks.rados.rados.0.smithi118.stdout:3709: seq_num 1137 ranges {5029888=647168,5677056=696320,6373376=557056,6930432=352256} 2024-03-20T08:14:13.313 INFO:tasks.rados.rados.0.smithi118.stdout:3709: writing smithi11881747-23 from 5029888 to 5677056 tid 1 2024-03-20T08:14:13.318 INFO:tasks.rados.rados.0.smithi118.stdout:3709: writing smithi11881747-23 from 5677056 to 6373376 tid 2 2024-03-20T08:14:13.321 INFO:tasks.rados.rados.0.smithi118.stdout:3709: writing smithi11881747-23 from 6373376 to 6930432 tid 3 2024-03-20T08:14:13.324 INFO:tasks.rados.rados.0.smithi118.stdout:3709: writing smithi11881747-23 from 6930432 to 7282688 tid 4 2024-03-20T08:14:13.324 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 367 2024-03-20T08:14:13.324 INFO:tasks.rados.rados.0.smithi118.stdout:3705: finishing rollback tid 2 to smithi11881747-39 2024-03-20T08:14:13.324 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 61772 (ObjNum 1068 snap 341 seq_num 1068) dirty exists 2024-03-20T08:14:13.324 INFO:tasks.rados.rados.0.smithi118.stdout:3710: seq_num 1138 ranges {3891200=663552,4554752=581632,5136384=712704,5849088=311296} 2024-03-20T08:14:13.349 INFO:tasks.rados.rados.0.smithi118.stdout:3710: writing smithi11881747-43 from 3891200 to 4554752 tid 1 2024-03-20T08:14:13.352 INFO:tasks.rados.rados.0.smithi118.stdout:3710: writing smithi11881747-43 from 4554752 to 5136384 tid 2 2024-03-20T08:14:13.356 INFO:tasks.rados.rados.0.smithi118.stdout:3710: writing smithi11881747-43 from 5136384 to 5849088 tid 3 2024-03-20T08:14:13.358 INFO:tasks.rados.rados.0.smithi118.stdout:3710: writing smithi11881747-43 from 5849088 to 6160384 tid 4 2024-03-20T08:14:13.358 INFO:tasks.rados.rados.0.smithi118.stdout:3705: done (4 left) 2024-03-20T08:14:13.358 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 34 current snap is 367 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3708: finishing rollback tid 0 to smithi11881747-38 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3708: finishing rollback tid 1 to smithi11881747-38 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3708: finishing rollback tid 2 to smithi11881747-38 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 38 v 65316 (ObjNum 1060 snap 338 seq_num 1060) dirty exists 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: finishing write tid 1 to smithi11881747-23 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: finishing write tid 2 to smithi11881747-23 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: finishing write tid 3 to smithi11881747-23 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: finishing write tid 4 to smithi11881747-23 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: finishing write tid 5 to smithi11881747-23 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: finishing write tid 6 to smithi11881747-23 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: oid 23 updating version 0 to 65339 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: oid 23 updating version 65339 to 65340 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: oid 23 version 65340 is already newer than 65336 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: oid 23 version 65340 is already newer than 65338 2024-03-20T08:14:13.359 INFO:tasks.rados.rados.0.smithi118.stdout:3709: oid 23 version 65340 is already newer than 65337 2024-03-20T08:14:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 23 v 65340 (ObjNum 1137 snap 367 seq_num 1137) dirty exists 2024-03-20T08:14:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:3709: left oid 23 (ObjNum 1137 snap 367 seq_num 1137) 2024-03-20T08:14:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:3706: finishing copy_from racing read to smithi11881747-27 2024-03-20T08:14:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:3706: finishing copy_from to smithi11881747-27 2024-03-20T08:14:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 65293 (ObjNum 1057 snap 338 seq_num 1057) dirty exists 2024-03-20T08:14:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:3710: finishing write tid 1 to smithi11881747-43 2024-03-20T08:14:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:3710: finishing write tid 2 to smithi11881747-43 2024-03-20T08:14:13.360 INFO:tasks.rados.rados.0.smithi118.stdout:3711: seq_num 1139 ranges {0=679936,679936=458752,1138688=524288,1662976=540672,2203648=81920} 2024-03-20T08:14:13.361 INFO:tasks.rados.rados.0.smithi118.stdout:3711: writing smithi11881747-34 from 0 to 679936 tid 1 2024-03-20T08:14:13.364 INFO:tasks.rados.rados.0.smithi118.stdout:3711: writing smithi11881747-34 from 679936 to 1138688 tid 2 2024-03-20T08:14:13.367 INFO:tasks.rados.rados.0.smithi118.stdout:3711: writing smithi11881747-34 from 1138688 to 1662976 tid 3 2024-03-20T08:14:13.371 INFO:tasks.rados.rados.0.smithi118.stdout:3711: writing smithi11881747-34 from 1662976 to 2203648 tid 4 2024-03-20T08:14:13.372 INFO:tasks.rados.rados.0.smithi118.stdout:3711: writing smithi11881747-34 from 2203648 to 2285568 tid 5 2024-03-20T08:14:13.372 INFO:tasks.rados.rados.0.smithi118.stdout:3706: done (4 left) 2024-03-20T08:14:13.372 INFO:tasks.rados.rados.0.smithi118.stdout:3708: done (3 left) 2024-03-20T08:14:13.372 INFO:tasks.rados.rados.0.smithi118.stdout:3709: done (2 left) 2024-03-20T08:14:13.372 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 367 2024-03-20T08:14:13.372 INFO:tasks.rados.rados.0.smithi118.stdout:3710: finishing write tid 3 to smithi11881747-43 2024-03-20T08:14:13.372 INFO:tasks.rados.rados.0.smithi118.stdout:3710: finishing write tid 4 to smithi11881747-43 2024-03-20T08:14:13.372 INFO:tasks.rados.rados.0.smithi118.stdout:3712: seq_num 1140 ranges {12730368=696320,13426688=745472,14172160=729088,14901248=131072} 2024-03-20T08:14:13.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:13.275+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:13.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:13.287+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:13.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:13.287+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:13.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:13.279+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:13.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:13.281+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:13.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:13.281+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:13.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:13.072+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:13.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:13.072+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:13.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:13.283+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:13.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:13.283+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:13.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:13.283+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: osdmap e803: 8 total, 8 up, 8 in 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:13.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:13 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:13.445 INFO:tasks.rados.rados.0.smithi118.stdout:3712: writing smithi11881747-48 from 12730368 to 13426688 tid 1 2024-03-20T08:14:13.449 INFO:tasks.rados.rados.0.smithi118.stdout:3712: writing smithi11881747-48 from 13426688 to 14172160 tid 2 2024-03-20T08:14:13.452 INFO:tasks.rados.rados.0.smithi118.stdout:3712: writing smithi11881747-48 from 14172160 to 14901248 tid 3 2024-03-20T08:14:13.454 INFO:tasks.rados.rados.0.smithi118.stdout:3712: writing smithi11881747-48 from 14901248 to 15032320 tid 4 2024-03-20T08:14:13.454 INFO:tasks.rados.rados.0.smithi118.stdout:3713: delete oid 9 current snap is 367 2024-03-20T08:14:13.454 INFO:tasks.rados.rados.0.smithi118.stdout:3710: finishing write tid 5 to smithi11881747-43 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3710: finishing write tid 6 to smithi11881747-43 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3710: oid 43 updating version 0 to 65312 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3710: oid 43 updating version 65312 to 65313 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3710: oid 43 updating version 65313 to 65315 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3710: oid 43 version 65315 is already newer than 65311 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3710: oid 43 version 65315 is already newer than 65314 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65315 (ObjNum 1138 snap 367 seq_num 1138) dirty exists 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3710: left oid 43 (ObjNum 1138 snap 367 seq_num 1138) 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3711: finishing write tid 1 to smithi11881747-34 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3711: finishing write tid 2 to smithi11881747-34 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3711: finishing write tid 3 to smithi11881747-34 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3711: finishing write tid 4 to smithi11881747-34 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3711: finishing write tid 5 to smithi11881747-34 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3711: finishing write tid 6 to smithi11881747-34 2024-03-20T08:14:13.455 INFO:tasks.rados.rados.0.smithi118.stdout:3711: finishing write tid 7 to smithi11881747-34 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3711: oid 34 updating version 0 to 45661 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3711: oid 34 version 45661 is already newer than 45659 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3711: oid 34 version 45661 is already newer than 45658 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3711: oid 34 updating version 45661 to 45662 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3711: oid 34 version 45662 is already newer than 45657 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3711: oid 34 version 45662 is already newer than 45660 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45662 (ObjNum 1139 snap 367 seq_num 1139) dirty exists 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3711: left oid 34 (ObjNum 1139 snap 367 seq_num 1139) 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3710: done (3 left) 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3711: done (2 left) 2024-03-20T08:14:13.456 INFO:tasks.rados.rados.0.smithi118.stdout:3713: done (1 left) 2024-03-20T08:14:13.457 INFO:tasks.rados.rados.0.smithi118.stdout:3714: delete oid 1 current snap is 367 2024-03-20T08:14:13.460 INFO:tasks.rados.rados.0.smithi118.stdout:3714: done (1 left) 2024-03-20T08:14:13.460 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 367 2024-03-20T08:14:13.460 INFO:tasks.rados.rados.0.smithi118.stdout:3715: seq_num 1143 ranges {0=753664,753664=450560,1204224=507904,1712128=589824,2301952=57344} 2024-03-20T08:14:13.464 INFO:tasks.rados.rados.0.smithi118.stdout:3715: writing smithi11881747-2 from 0 to 753664 tid 1 2024-03-20T08:14:13.466 INFO:tasks.rados.rados.0.smithi118.stdout:3715: writing smithi11881747-2 from 753664 to 1204224 tid 2 2024-03-20T08:14:13.470 INFO:tasks.rados.rados.0.smithi118.stdout:3715: writing smithi11881747-2 from 1204224 to 1712128 tid 3 2024-03-20T08:14:13.473 INFO:tasks.rados.rados.0.smithi118.stdout:3715: writing smithi11881747-2 from 1712128 to 2301952 tid 4 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3715: writing smithi11881747-2 from 2301952 to 2359296 tid 5 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3716: snap_create 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3712: finishing write tid 1 to smithi11881747-48 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3712: finishing write tid 2 to smithi11881747-48 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3712: finishing write tid 3 to smithi11881747-48 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3712: finishing write tid 4 to smithi11881747-48 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3715: finishing write tid 1 to smithi11881747-2 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3715: finishing write tid 2 to smithi11881747-2 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3712: finishing write tid 5 to smithi11881747-48 2024-03-20T08:14:13.474 INFO:tasks.rados.rados.0.smithi118.stdout:3715: finishing write tid 3 to smithi11881747-2 2024-03-20T08:14:13.477 INFO:tasks.rados.rados.0.smithi118.stdout:3715: finishing write tid 4 to smithi11881747-2 2024-03-20T08:14:13.477 INFO:tasks.rados.rados.0.smithi118.stdout:3715: finishing write tid 5 to smithi11881747-2 2024-03-20T08:14:13.477 INFO:tasks.rados.rados.0.smithi118.stdout:3715: finishing write tid 6 to smithi11881747-2 2024-03-20T08:14:13.479 INFO:tasks.rados.rados.0.smithi118.stdout:3712: finishing write tid 6 to smithi11881747-48 2024-03-20T08:14:13.479 INFO:tasks.rados.rados.0.smithi118.stdout:3712: oid 48 updating version 0 to 65321 2024-03-20T08:14:13.479 INFO:tasks.rados.rados.0.smithi118.stdout:3712: oid 48 version 65321 is already newer than 65319 2024-03-20T08:14:13.479 INFO:tasks.rados.rados.0.smithi118.stdout:3712: oid 48 version 65321 is already newer than 65317 2024-03-20T08:14:13.479 INFO:tasks.rados.rados.0.smithi118.stdout:3712: oid 48 version 65321 is already newer than 65318 2024-03-20T08:14:13.479 INFO:tasks.rados.rados.0.smithi118.stdout:3712: oid 48 version 65321 is already newer than 65320 2024-03-20T08:14:13.479 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 65321 (ObjNum 1140 snap 367 seq_num 1140) dirty exists 2024-03-20T08:14:13.479 INFO:tasks.rados.rados.0.smithi118.stdout:3712: left oid 48 (ObjNum 1140 snap 367 seq_num 1140) 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:3715: finishing write tid 7 to smithi11881747-2 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:3715: oid 2 updating version 0 to 65309 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:3715: oid 2 updating version 65309 to 65311 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:3715: oid 2 version 65311 is already newer than 65308 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:3715: oid 2 version 65311 is already newer than 65306 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:3715: oid 2 version 65311 is already newer than 65307 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:3715: oid 2 version 65311 is already newer than 65310 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65311 (ObjNum 1143 snap 367 seq_num 1143) dirty exists 2024-03-20T08:14:13.481 INFO:tasks.rados.rados.0.smithi118.stdout:3715: left oid 2 (ObjNum 1143 snap 367 seq_num 1143) 2024-03-20T08:14:13.585 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:13.275+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:13.585 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:13.276+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:13.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:13.285+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:13.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:13.285+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:13.585 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:13.286+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:13.319+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:13.322+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:13.322+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:13.323+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:13.324+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:13.586 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: osdmap e803: 8 total, 8 up, 8 in 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:13.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: osdmap e803: 8 total, 8 up, 8 in 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:13.588 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:14:13.589 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:14:13.590 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:13.590 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:13.590 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:13.590 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:14:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:14:13] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3712: done (2 left) 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3715: done (1 left) 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3716: done (0 left) 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3717: read oid 25 snap -1 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3717: expect (ObjNum 1131 snap 364 seq_num 1131) 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3718: rollback oid 21 current snap is 368 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 367 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3719: read oid 35 snap 357 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3719: expect (ObjNum 1043 snap 336 seq_num 1043) 2024-03-20T08:14:14.407 INFO:tasks.rados.rados.0.smithi118.stdout:3720: snap_create 2024-03-20T08:14:14.421 INFO:tasks.rados.rados.0.smithi118.stdout:3718: finishing rollback tid 0 to smithi11881747-21 2024-03-20T08:14:14.421 INFO:tasks.rados.rados.0.smithi118.stdout:3718: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:14:14.423 INFO:tasks.rados.rados.0.smithi118.stdout:3717: expect (ObjNum 1131 snap 364 seq_num 1131) 2024-03-20T08:14:14.550 INFO:tasks.rados.rados.0.smithi118.stdout:3718: finishing rollback tid 2 to smithi11881747-21 2024-03-20T08:14:14.551 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 65324 (ObjNum 1105 snap 352 seq_num 1105) dirty exists 2024-03-20T08:14:14.551 INFO:tasks.rados.rados.0.smithi118.stdout:3719: expect (ObjNum 1043 snap 336 seq_num 1043) 2024-03-20T08:14:14.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:14.406+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:14.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:14.412+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:14.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:14.414+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:14.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:14.413+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:14.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:14.413+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:14.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: pgmap v892: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: osdmap e804: 8 total, 8 up, 8 in 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:14.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:14.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:14.409+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:14.413+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:14.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:14.405+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:14.405+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:14.406+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:14.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:14.407+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: pgmap v892: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-03-20T08:14:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: osdmap e804: 8 total, 8 up, 8 in 2024-03-20T08:14:14.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: pgmap v892: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 6 op/s 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: osdmap e804: 8 total, 8 up, 8 in 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18f=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:14.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:14.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:14.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:14.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:14.677 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:14.413+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:14.677 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:14.406+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:14.677 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:14.406+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:15.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:14.743+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:15.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:14.743+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:15.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:14.743+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:15.412 INFO:tasks.rados.rados.0.smithi118.stdout:3717: done (3 left) 2024-03-20T08:14:15.412 INFO:tasks.rados.rados.0.smithi118.stdout:3718: done (2 left) 2024-03-20T08:14:15.412 INFO:tasks.rados.rados.0.smithi118.stdout:3719: done (1 left) 2024-03-20T08:14:15.412 INFO:tasks.rados.rados.0.smithi118.stdout:3720: done (0 left) 2024-03-20T08:14:15.412 INFO:tasks.rados.rados.0.smithi118.stdout:3721: setattr oid 4 current snap is 369 2024-03-20T08:14:15.412 INFO:tasks.rados.rados.0.smithi118.stdout:3722: setattr oid 39 current snap is 369 2024-03-20T08:14:15.413 INFO:tasks.rados.rados.0.smithi118.stdout:3723: snap_create 2024-03-20T08:14:15.417 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 65294 (ObjNum 1068 snap 341 seq_num 1068) dirty exists 2024-03-20T08:14:15.418 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65306 (ObjNum 33 snap 0 seq_num 1268395351) dirty exists 2024-03-20T08:14:15.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:15.412+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:15.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:15.412+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:15.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:15.412+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:15.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:15.411+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:15.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:15.418+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:15.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: osdmap e805: 8 total, 8 up, 8 in 2024-03-20T08:14:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:15.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:15.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:15.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: osdmap e805: 8 total, 8 up, 8 in 2024-03-20T08:14:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:15.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:15.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:15.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:15.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:15.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: pgmap v895: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 255 B/s rd, 526 KiB/s wr, 2 op/s 2024-03-20T08:14:15.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:15.678 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:15.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:15.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:15.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:15.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: pgmap v895: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 255 B/s rd, 526 KiB/s wr, 2 op/s 2024-03-20T08:14:15.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:15.679 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:15.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:15.413+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:15.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:15.419+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:15.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:15.420+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:15.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:15.420+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:15.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:15.416+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:15.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:15.418+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:15.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:15.418+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:15.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:15.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:15.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:15.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: osdmap e805: 8 total, 8 up, 8 in 2024-03-20T08:14:15.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: pgmap v895: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 255 B/s rd, 526 KiB/s wr, 2 op/s 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:15.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:16.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:15.698+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:16.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:15.698+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:16.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:15.698+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:16.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:15.698+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:16.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:15.698+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:16.412 INFO:tasks.rados.rados.0.smithi118.stdout:3721: done (2 left) 2024-03-20T08:14:16.412 INFO:tasks.rados.rados.0.smithi118.stdout:3722: done (1 left) 2024-03-20T08:14:16.412 INFO:tasks.rados.rados.0.smithi118.stdout:3723: done (0 left) 2024-03-20T08:14:16.412 INFO:tasks.rados.rados.0.smithi118.stdout:3724: rollback oid 29 current snap is 370 2024-03-20T08:14:16.412 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 29 to 365 2024-03-20T08:14:16.412 INFO:tasks.rados.rados.0.smithi118.stdout:3725: rollback oid 14 current snap is 370 2024-03-20T08:14:16.412 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 14 to 360 2024-03-20T08:14:16.412 INFO:tasks.rados.rados.0.smithi118.stdout:3726: snap_remove snap 354 2024-03-20T08:14:16.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:16.420+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:16.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:16.420+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:16.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:16.412+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:16.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:16.414+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:16.673 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:16.414+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:16.415+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:16.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:16.416+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:16.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:16.416+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:16.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:16.423+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: osdmap e806: 8 total, 8 up, 8 in 2024-03-20T08:14:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:16.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:16.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: osdmap e806: 8 total, 8 up, 8 in 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:16.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:16.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:16.414+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:16.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:16.420+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:16.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:16.420+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:16.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:16.421+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:16.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:16.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: osdmap e806: 8 total, 8 up, 8 in 2024-03-20T08:14:16.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:16.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:16.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:16.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:16.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:16.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:16.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:16.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:16.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:16.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:16.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:16.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:3726: done (2 left) 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:3727: rmattr oid 43 current snap is 370 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:3725: finishing rollback tid 1 to smithi11881747-14 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 34113 (ObjNum 97 snap 0 seq_num 321656656) dirty dne 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:3724: finishing rollback tid 0 to smithi11881747-29 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:3724: finishing rollback tid 1 to smithi11881747-29 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:3724: finishing rollback tid 2 to smithi11881747-29 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 65343 (ObjNum 1075 snap 341 seq_num 1075) dirty exists 2024-03-20T08:14:17.426 INFO:tasks.rados.rados.0.smithi118.stdout:3724: done (2 left) 2024-03-20T08:14:17.427 INFO:tasks.rados.rados.0.smithi118.stdout:3725: done (1 left) 2024-03-20T08:14:17.427 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 370 2024-03-20T08:14:17.427 INFO:tasks.rados.rados.0.smithi118.stdout:3728: seq_num 1144 ranges {0=770048,770048=737280,1507328=712704,2220032=155648} 2024-03-20T08:14:17.429 INFO:tasks.rados.rados.0.smithi118.stdout:3728: writing smithi11881747-4 from 0 to 770048 tid 1 2024-03-20T08:14:17.433 INFO:tasks.rados.rados.0.smithi118.stdout:3728: writing smithi11881747-4 from 770048 to 1507328 tid 2 2024-03-20T08:14:17.437 INFO:tasks.rados.rados.0.smithi118.stdout:3728: writing smithi11881747-4 from 1507328 to 2220032 tid 3 2024-03-20T08:14:17.438 INFO:tasks.rados.rados.0.smithi118.stdout:3728: writing smithi11881747-4 from 2220032 to 2375680 tid 4 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:3729: read oid 23 snap -1 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:3729: expect (ObjNum 1137 snap 367 seq_num 1137) 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:3730: rollback oid 47 current snap is 370 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 47 to 369 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65317 (ObjNum 1138 snap 367 seq_num 1138) dirty exists 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:3727: done (3 left) 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:3731: setattr oid 34 current snap is 370 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:3728: finishing write tid 1 to smithi11881747-4 2024-03-20T08:14:17.439 INFO:tasks.rados.rados.0.smithi118.stdout:3728: finishing write tid 2 to smithi11881747-4 2024-03-20T08:14:17.440 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 370 2024-03-20T08:14:17.440 INFO:tasks.rados.rados.0.smithi118.stdout:3732: seq_num 1145 ranges {5472256=786432,6258688=622592,6881280=516096,7397376=417792,7815168=49152} 2024-03-20T08:14:17.475 INFO:tasks.rados.rados.0.smithi118.stdout:3732: writing smithi11881747-26 from 5472256 to 6258688 tid 1 2024-03-20T08:14:17.479 INFO:tasks.rados.rados.0.smithi118.stdout:3732: writing smithi11881747-26 from 6258688 to 6881280 tid 2 2024-03-20T08:14:17.481 INFO:tasks.rados.rados.0.smithi118.stdout:3732: writing smithi11881747-26 from 6881280 to 7397376 tid 3 2024-03-20T08:14:17.486 INFO:tasks.rados.rados.0.smithi118.stdout:3732: writing smithi11881747-26 from 7397376 to 7815168 tid 4 2024-03-20T08:14:17.486 INFO:tasks.rados.rados.0.smithi118.stdout:3732: writing smithi11881747-26 from 7815168 to 7864320 tid 5 2024-03-20T08:14:17.486 INFO:tasks.rados.rados.0.smithi118.stdout:3733: read oid 1 snap 352 2024-03-20T08:14:17.486 INFO:tasks.rados.rados.0.smithi118.stdout:3733: expect (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:14:17.486 INFO:tasks.rados.rados.0.smithi118.stdout:3734: rollback oid 22 current snap is 370 2024-03-20T08:14:17.486 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 22 to 360 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: finishing write tid 3 to smithi11881747-4 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: finishing write tid 4 to smithi11881747-4 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: finishing write tid 5 to smithi11881747-4 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: finishing write tid 6 to smithi11881747-4 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: oid 4 updating version 0 to 65307 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: oid 4 updating version 65307 to 65308 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: oid 4 updating version 65308 to 65310 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: oid 4 updating version 65310 to 65311 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: oid 4 version 65311 is already newer than 65309 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65311 (ObjNum 1144 snap 370 seq_num 1144) dirty exists 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: left oid 4 (ObjNum 1144 snap 370 seq_num 1144) 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 34 v 45663 (ObjNum 1139 snap 367 seq_num 1139) dirty exists 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3730: finishing rollback tid 0 to smithi11881747-47 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3730: finishing rollback tid 1 to smithi11881747-47 2024-03-20T08:14:17.487 INFO:tasks.rados.rados.0.smithi118.stdout:3728: done (6 left) 2024-03-20T08:14:17.488 INFO:tasks.rados.rados.0.smithi118.stdout:3731: done (5 left) 2024-03-20T08:14:17.488 INFO:tasks.rados.rados.0.smithi118.stdout:3735: rollback oid 13 current snap is 370 2024-03-20T08:14:17.488 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 13 to 364 2024-03-20T08:14:17.488 INFO:tasks.rados.rados.0.smithi118.stdout:3729: expect (ObjNum 1137 snap 367 seq_num 1137) 2024-03-20T08:14:17.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:17.432+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:17.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:17.436+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:17.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:17.437+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:17.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:17.437+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:17.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:17.250+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:17.641 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:17.251+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:17.641 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:17.424+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:17.641 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:17.424+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:17.641 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:17.424+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:17.642 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:17.251+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:17.642 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:17.251+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:17.642 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:17.429+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:17.642 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:17.431+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:17.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:17.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: osdmap e807: 8 total, 8 up, 8 in 2024-03-20T08:14:17.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:17.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:17.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:17.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:17.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:17.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:17.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:17.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:17.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:17.644 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:17.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:17 smithi196 ceph-mon[87080]: pgmap v898: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 12 op/s 2024-03-20T08:14:17.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:17.490+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:17.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:17.445+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:17.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:17.449+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:17.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:17.449+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:17.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:17.424+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:17.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:17.424+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:17.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:17.425+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:17.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:17.425+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:17.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:17.425+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:17.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: osdmap e807: 8 total, 8 up, 8 in 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:17.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:17.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[95692]: pgmap v898: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 12 op/s 2024-03-20T08:14:17.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:17.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: osdmap e807: 8 total, 8 up, 8 in 2024-03-20T08:14:17.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:17.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:17.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:17.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:17.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:17.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:17.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:17.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:17.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:17.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:17.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-mon[93121]: pgmap v898: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 4.8 MiB/s rd, 7.6 MiB/s wr, 12 op/s 2024-03-20T08:14:17.678 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:17.425+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:17.861 INFO:tasks.rados.rados.0.smithi118.stdout:3730: finishing rollback tid 2 to smithi11881747-47 2024-03-20T08:14:17.861 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 65346 (ObjNum 1032 snap 331 seq_num 1032) dirty exists 2024-03-20T08:14:17.861 INFO:tasks.rados.rados.0.smithi118.stdout:3735: finishing rollback tid 0 to smithi11881747-13 2024-03-20T08:14:17.861 INFO:tasks.rados.rados.0.smithi118.stdout:3735: finishing rollback tid 1 to smithi11881747-13 2024-03-20T08:14:17.861 INFO:tasks.rados.rados.0.smithi118.stdout:3729: done (5 left) 2024-03-20T08:14:17.861 INFO:tasks.rados.rados.0.smithi118.stdout:3730: done (4 left) 2024-03-20T08:14:17.861 INFO:tasks.rados.rados.0.smithi118.stdout:3736: rollback oid 25 current snap is 370 2024-03-20T08:14:17.861 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 25 to 368 2024-03-20T08:14:17.862 INFO:tasks.rados.rados.0.smithi118.stdout:3735: finishing rollback tid 2 to smithi11881747-13 2024-03-20T08:14:17.862 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 65335 (ObjNum 1111 snap 356 seq_num 1111) dirty exists 2024-03-20T08:14:17.862 INFO:tasks.rados.rados.0.smithi118.stdout:3733: expect (ObjNum 1034 snap 333 seq_num 1034) 2024-03-20T08:14:17.879 INFO:tasks.rados.rados.0.smithi118.stdout:3734: finishing rollback tid 1 to smithi11881747-22 2024-03-20T08:14:17.879 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 65313 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-03-20T08:14:17.879 INFO:tasks.rados.rados.0.smithi118.stdout:3732: finishing write tid 1 to smithi11881747-26 2024-03-20T08:14:17.879 INFO:tasks.rados.rados.0.smithi118.stdout:3732: finishing write tid 2 to smithi11881747-26 2024-03-20T08:14:17.879 INFO:tasks.rados.rados.0.smithi118.stdout:3732: finishing write tid 3 to smithi11881747-26 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: finishing write tid 4 to smithi11881747-26 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: finishing write tid 5 to smithi11881747-26 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: finishing write tid 6 to smithi11881747-26 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: finishing write tid 7 to smithi11881747-26 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: oid 26 updating version 0 to 65335 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: oid 26 version 65335 is already newer than 65334 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: oid 26 version 65335 is already newer than 65333 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: oid 26 updating version 65335 to 65336 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: oid 26 updating version 65336 to 65338 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: oid 26 version 65338 is already newer than 65337 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65338 (ObjNum 1145 snap 370 seq_num 1145) dirty exists 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: left oid 26 (ObjNum 1145 snap 370 seq_num 1145) 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3736: finishing rollback tid 0 to smithi11881747-25 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3736: finishing rollback tid 1 to smithi11881747-25 2024-03-20T08:14:17.880 INFO:tasks.rados.rados.0.smithi118.stdout:3732: done (4 left) 2024-03-20T08:14:17.881 INFO:tasks.rados.rados.0.smithi118.stdout:3733: done (3 left) 2024-03-20T08:14:17.881 INFO:tasks.rados.rados.0.smithi118.stdout:3734: done (2 left) 2024-03-20T08:14:17.881 INFO:tasks.rados.rados.0.smithi118.stdout:3735: done (1 left) 2024-03-20T08:14:17.881 INFO:tasks.rados.rados.0.smithi118.stdout:3737: delete oid 40 current snap is 370 2024-03-20T08:14:17.881 INFO:tasks.rados.rados.0.smithi118.stdout:3736: finishing rollback tid 2 to smithi11881747-25 2024-03-20T08:14:17.881 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 65338 (ObjNum 1131 snap 364 seq_num 1131) dirty exists 2024-03-20T08:14:17.886 INFO:tasks.rados.rados.0.smithi118.stdout:3736: done (1 left) 2024-03-20T08:14:17.886 INFO:tasks.rados.rados.0.smithi118.stdout:3737: done (0 left) 2024-03-20T08:14:17.886 INFO:tasks.rados.rados.0.smithi118.stdout:3738: delete oid 28 current snap is 370 2024-03-20T08:14:17.890 INFO:tasks.rados.rados.0.smithi118.stdout:3738: done (0 left) 2024-03-20T08:14:17.890 INFO:tasks.rados.rados.0.smithi118.stdout:3739: rmattr oid 44 current snap is 370 2024-03-20T08:14:17.890 INFO:tasks.rados.rados.0.smithi118.stdout:3739: done (0 left) 2024-03-20T08:14:17.890 INFO:tasks.rados.rados.0.smithi118.stdout:3740: read oid 7 snap -1 2024-03-20T08:14:17.890 INFO:tasks.rados.rados.0.smithi118.stdout:3740: expect (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:14:17.890 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 46 current snap is 370 2024-03-20T08:14:17.890 INFO:tasks.rados.rados.0.smithi118.stdout:3741: seq_num 1148 ranges {12697600=32768} 2024-03-20T08:14:17.959 INFO:tasks.rados.rados.0.smithi118.stdout:3741: writing smithi11881747-46 from 12697600 to 12730368 tid 1 2024-03-20T08:14:17.959 INFO:tasks.rados.rados.0.smithi118.stdout:3742: copy_from oid 36 from oid 4 current snap is 370 2024-03-20T08:14:17.959 INFO:tasks.rados.rados.0.smithi118.stdout:3740: expect (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3741: finishing write tid 1 to smithi11881747-46 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3741: finishing write tid 2 to smithi11881747-46 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3740: done (2 left) 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3743: read oid 12 snap -1 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3743: expect deleted 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3744: read oid 8 snap -1 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3744: expect (ObjNum 1112 snap 357 seq_num 1112) 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3745: copy_from oid 9 from oid 14 current snap is 370 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3741: finishing write tid 3 to smithi11881747-46 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3741: oid 46 updating version 0 to 65313 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:3741: oid 46 version 65313 is already newer than 65312 2024-03-20T08:14:18.120 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 65313 (ObjNum 1148 snap 370 seq_num 1148) dirty exists 2024-03-20T08:14:18.121 INFO:tasks.rados.rados.0.smithi118.stdout:3741: left oid 46 (ObjNum 1148 snap 370 seq_num 1148) 2024-03-20T08:14:18.121 INFO:tasks.rados.rados.0.smithi118.stdout:3742: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:14:18.121 INFO:tasks.rados.rados.0.smithi118.stdout:3742: finishing copy_from to smithi11881747-36 2024-03-20T08:14:18.121 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 65311 (ObjNum 1144 snap 370 seq_num 1144) dirty exists 2024-03-20T08:14:18.121 INFO:tasks.rados.rados.0.smithi118.stdout:3741: done (4 left) 2024-03-20T08:14:18.121 INFO:tasks.rados.rados.0.smithi118.stdout:3742: done (3 left) 2024-03-20T08:14:18.121 INFO:tasks.rados.rados.0.smithi118.stdout:3746: delete oid 45 current snap is 370 2024-03-20T08:14:18.123 INFO:tasks.rados.rados.0.smithi118.stdout:3745: finishing copy_from to smithi11881747-9 2024-03-20T08:14:18.123 INFO:tasks.rados.rados.0.smithi118.stdout:3745: got expected ENOENT (src dne) 2024-03-20T08:14:18.123 INFO:tasks.rados.rados.0.smithi118.stdout:3745: finishing copy_from racing read to smithi11881747-9 2024-03-20T08:14:18.125 INFO:tasks.rados.rados.0.smithi118.stdout:3743: done (3 left) 2024-03-20T08:14:18.125 INFO:tasks.rados.rados.0.smithi118.stdout:3745: done (2 left) 2024-03-20T08:14:18.125 INFO:tasks.rados.rados.0.smithi118.stdout:3746: done (1 left) 2024-03-20T08:14:18.125 INFO:tasks.rados.rados.0.smithi118.stdout:3747: snap_create 2024-03-20T08:14:18.131 INFO:tasks.rados.rados.0.smithi118.stdout:3744: expect (ObjNum 1112 snap 357 seq_num 1112) 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:3744: done (1 left) 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:3747: done (0 left) 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:3748: read oid 47 snap 365 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:3748: expect (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:3749: read oid 48 snap 370 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:3749: expect (ObjNum 1140 snap 367 seq_num 1140) 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:3750: copy_from oid 18 from oid 7 current snap is 371 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 371 2024-03-20T08:14:18.430 INFO:tasks.rados.rados.0.smithi118.stdout:3751: seq_num 1150 ranges {1720320=65536} 2024-03-20T08:14:18.440 INFO:tasks.rados.rados.0.smithi118.stdout:3751: writing smithi11881747-21 from 1720320 to 1785856 tid 1 2024-03-20T08:14:18.440 INFO:tasks.rados.rados.0.smithi118.stdout:3752: copy_from oid 3 from oid 10 current snap is 371 2024-03-20T08:14:18.440 INFO:tasks.rados.rados.0.smithi118.stdout:3748: expect (ObjNum 1032 snap 331 seq_num 1032) 2024-03-20T08:14:18.474 INFO:tasks.rados.rados.0.smithi118.stdout:3752: finishing copy_from to smithi11881747-3 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3752: got expected ENOENT (src dne) 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3752: finishing copy_from racing read to smithi11881747-3 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3750: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3748: done (4 left) 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3752: done (3 left) 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3753: rollback oid 4 current snap is 371 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 4 to 367 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3750: finishing copy_from to smithi11881747-18 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 65304 (ObjNum 1128 snap 361 seq_num 1128) dirty exists 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3750: done (3 left) 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:3754: rollback oid 2 current snap is 371 2024-03-20T08:14:18.475 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 2 to 369 2024-03-20T08:14:18.476 INFO:tasks.rados.rados.0.smithi118.stdout:3755: copy_from oid 47 from oid 31 current snap is 371 2024-03-20T08:14:18.476 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 35 current snap is 371 2024-03-20T08:14:18.476 INFO:tasks.rados.rados.0.smithi118.stdout:3756: seq_num 1151 ranges {2883584=81920} 2024-03-20T08:14:18.491 INFO:tasks.rados.rados.0.smithi118.stdout:3756: writing smithi11881747-35 from 2883584 to 2965504 tid 1 2024-03-20T08:14:18.492 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 8 current snap is 371 2024-03-20T08:14:18.492 INFO:tasks.rados.rados.0.smithi118.stdout:3754: finishing rollback tid 0 to smithi11881747-2 2024-03-20T08:14:18.492 INFO:tasks.rados.rados.0.smithi118.stdout:3753: finishing rollback tid 0 to smithi11881747-4 2024-03-20T08:14:18.492 INFO:tasks.rados.rados.0.smithi118.stdout:3753: finishing rollback tid 1 to smithi11881747-4 2024-03-20T08:14:18.492 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65313 (ObjNum 177 snap 0 seq_num 322544416) dirty dne 2024-03-20T08:14:18.492 INFO:tasks.rados.rados.0.smithi118.stdout:3754: finishing rollback tid 1 to smithi11881747-2 2024-03-20T08:14:18.492 INFO:tasks.rados.rados.0.smithi118.stdout:3757: seq_num 1152 ranges {3366912=98304} 2024-03-20T08:14:18.511 INFO:tasks.rados.rados.0.smithi118.stdout:3757: writing smithi11881747-8 from 3366912 to 3465216 tid 1 2024-03-20T08:14:18.512 INFO:tasks.rados.rados.0.smithi118.stdout:3753: done (6 left) 2024-03-20T08:14:18.512 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 371 2024-03-20T08:14:18.512 INFO:tasks.rados.rados.0.smithi118.stdout:3754: finishing rollback tid 2 to smithi11881747-2 2024-03-20T08:14:18.512 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65316 (ObjNum 1143 snap 367 seq_num 1143) dirty exists 2024-03-20T08:14:18.512 INFO:tasks.rados.rados.0.smithi118.stdout:3755: finishing copy_from racing read to smithi11881747-47 2024-03-20T08:14:18.512 INFO:tasks.rados.rados.0.smithi118.stdout:3755: finishing copy_from to smithi11881747-47 2024-03-20T08:14:18.512 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 47 v 65309 (ObjNum 1117 snap 358 seq_num 1117) dirty exists 2024-03-20T08:14:18.512 INFO:tasks.rados.rados.0.smithi118.stdout:3758: seq_num 1153 ranges {7864320=114688} 2024-03-20T08:14:18.554 INFO:tasks.rados.rados.0.smithi118.stdout:3758: writing smithi11881747-26 from 7864320 to 7979008 tid 1 2024-03-20T08:14:18.554 INFO:tasks.rados.rados.0.smithi118.stdout:3754: done (6 left) 2024-03-20T08:14:18.555 INFO:tasks.rados.rados.0.smithi118.stdout:3755: done (5 left) 2024-03-20T08:14:18.555 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 371 2024-03-20T08:14:18.555 INFO:tasks.rados.rados.0.smithi118.stdout:3759: seq_num 1154 ranges {2359296=131072} 2024-03-20T08:14:18.568 INFO:tasks.rados.rados.0.smithi118.stdout:3759: writing smithi11881747-2 from 2359296 to 2490368 tid 1 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3760: read oid 16 snap -1 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3760: expect (ObjNum 1040 snap 336 seq_num 1040) 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3761: copy_from oid 36 from oid 37 current snap is 371 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3756: finishing write tid 1 to smithi11881747-35 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3756: finishing write tid 2 to smithi11881747-35 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3756: finishing write tid 3 to smithi11881747-35 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3756: oid 35 updating version 0 to 65318 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3756: oid 35 version 65318 is already newer than 65317 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 65318 (ObjNum 1151 snap 371 seq_num 1151) dirty exists 2024-03-20T08:14:18.569 INFO:tasks.rados.rados.0.smithi118.stdout:3756: left oid 35 (ObjNum 1151 snap 371 seq_num 1151) 2024-03-20T08:14:18.570 INFO:tasks.rados.rados.0.smithi118.stdout:3756: done (7 left) 2024-03-20T08:14:18.570 INFO:tasks.rados.rados.0.smithi118.stdout:3762: snap_remove snap 364 2024-03-20T08:14:18.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:18.433+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:18.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:18.433+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:18.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:18.429+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:18.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:18.429+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:18.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:18.450+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:18.435+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:18.673 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:18.435+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:18.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:18.435+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:18.674 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:18.436+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:18.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: osdmap e808: 8 total, 8 up, 8 in 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:18.675 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: osdmap e808: 8 total, 8 up, 8 in 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:18.676 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:18.677 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:18.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:18.678 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:18.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:18.430+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:18.890 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:18.430+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:18.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:18.431+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:18.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:18.431+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:18.891 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:18.438+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:18.892 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:18.439+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:18.892 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:18.445+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:18.893 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:18.445+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:18.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:18.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:18.894 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:18.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:18.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:18.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 18d=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:18.895 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a4=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:18.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:18.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: osdmap e808: 8 total, 8 up, 8 in 2024-03-20T08:14:18.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:18.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:18.896 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:18.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:18.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:18.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:18.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:18.897 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:19.436 INFO:tasks.rados.rados.0.smithi118.stdout:3762: done (7 left) 2024-03-20T08:14:19.436 INFO:tasks.rados.rados.0.smithi118.stdout:3763: copy_from oid 5 from oid 4 current snap is 371 2024-03-20T08:14:19.436 INFO:tasks.rados.rados.0.smithi118.stdout:3749: expect (ObjNum 1140 snap 367 seq_num 1140) 2024-03-20T08:14:19.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:19.436+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:19.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:19.444+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:19.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:19.444+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:19.891 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:19.444+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:19.891 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:19.444+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:19.891 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:19.444+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:19.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:19.442+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:19.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:19.443+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:19.891 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:19.443+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: osdmap e809: 8 total, 8 up, 8 in 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:19.892 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:19.893 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:19 smithi196 ceph-mon[87080]: pgmap v901: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 17 op/s 2024-03-20T08:14:19.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:19.437+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:19.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:19.439+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:19.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:19.439+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:19.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:19.439+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:19.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:19.433+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:19.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:19.434+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:19.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:19.435+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:19.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:19.435+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:19.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:19.435+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:19.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:19.443+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: osdmap e809: 8 total, 8 up, 8 in 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:19.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: osdmap e809: 8 total, 8 up, 8 in 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:19.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 192=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:19.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[95692]: pgmap v901: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 17 op/s 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:19.927 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:19 smithi118 ceph-mon[93121]: pgmap v901: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 17 op/s 2024-03-20T08:14:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:20.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: osdmap e810: 8 total, 8 up, 8 in 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:20.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:20.891 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:20.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: osdmap e810: 8 total, 8 up, 8 in 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:20.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:20.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:20.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:20.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:20.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:20.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:20.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: osdmap e810: 8 total, 8 up, 8 in 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:20.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:3751: finishing write tid 1 to smithi11881747-21 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:3749: done (7 left) 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:3764: rollback oid 27 current snap is 371 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 27 to 369 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:3751: finishing write tid 2 to smithi11881747-21 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:3751: finishing write tid 3 to smithi11881747-21 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:3751: oid 21 updating version 0 to 65325 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:3751: oid 21 updating version 65325 to 65326 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 65326 (ObjNum 1150 snap 371 seq_num 1150) dirty exists 2024-03-20T08:14:21.358 INFO:tasks.rados.rados.0.smithi118.stdout:3751: left oid 21 (ObjNum 1150 snap 371 seq_num 1150) 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3757: finishing write tid 1 to smithi11881747-8 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3757: finishing write tid 2 to smithi11881747-8 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3757: finishing write tid 3 to smithi11881747-8 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3757: oid 8 updating version 0 to 34137 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3757: oid 8 updating version 34137 to 34138 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 8 v 34138 (ObjNum 1152 snap 371 seq_num 1152) dirty exists 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3757: left oid 8 (ObjNum 1152 snap 371 seq_num 1152) 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3758: finishing write tid 1 to smithi11881747-26 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3758: finishing write tid 2 to smithi11881747-26 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3758: finishing write tid 3 to smithi11881747-26 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3758: oid 26 updating version 0 to 65341 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3758: oid 26 version 65341 is already newer than 65340 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65341 (ObjNum 1153 snap 371 seq_num 1153) dirty exists 2024-03-20T08:14:21.359 INFO:tasks.rados.rados.0.smithi118.stdout:3758: left oid 26 (ObjNum 1153 snap 371 seq_num 1153) 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3761: finishing copy_from to smithi11881747-36 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3761: got expected ENOENT (src dne) 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3761: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3759: finishing write tid 1 to smithi11881747-2 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3759: finishing write tid 2 to smithi11881747-2 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3759: finishing write tid 3 to smithi11881747-2 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3759: oid 2 updating version 0 to 65319 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3759: oid 2 updating version 65319 to 65320 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65320 (ObjNum 1154 snap 371 seq_num 1154) dirty exists 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3759: left oid 2 (ObjNum 1154 snap 371 seq_num 1154) 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3751: done (7 left) 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3757: done (6 left) 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3758: done (5 left) 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3759: done (4 left) 2024-03-20T08:14:21.360 INFO:tasks.rados.rados.0.smithi118.stdout:3761: done (3 left) 2024-03-20T08:14:21.361 INFO:tasks.rados.rados.0.smithi118.stdout:3765: copy_from oid 50 from oid 7 current snap is 371 2024-03-20T08:14:21.361 INFO:tasks.rados.rados.0.smithi118.stdout:3760: expect (ObjNum 1040 snap 336 seq_num 1040) 2024-03-20T08:14:21.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:21 smithi196 ceph-mon[87080]: pgmap v903: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 5.4 MiB/s rd, 3.2 MiB/s wr, 26 op/s 2024-03-20T08:14:21.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:21 smithi118 ceph-mon[93121]: pgmap v903: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 5.4 MiB/s rd, 3.2 MiB/s wr, 26 op/s 2024-03-20T08:14:21.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:21 smithi118 ceph-mon[95692]: pgmap v903: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 5.4 MiB/s rd, 3.2 MiB/s wr, 26 op/s 2024-03-20T08:14:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:14:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:14:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:14:22.745 INFO:tasks.rados.rados.0.smithi118.stdout:3763: finishing copy_from to smithi11881747-5 2024-03-20T08:14:22.745 INFO:tasks.rados.rados.0.smithi118.stdout:3763: got expected ENOENT (src dne) 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3763: finishing copy_from racing read to smithi11881747-5 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3764: finishing rollback tid 0 to smithi11881747-27 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3764: finishing rollback tid 1 to smithi11881747-27 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3760: done (3 left) 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3763: done (2 left) 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3766: rmattr oid 28 current snap is 371 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3766: done (2 left) 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 4 current snap is 371 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3764: finishing rollback tid 2 to smithi11881747-27 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 65307 (ObjNum 1057 snap 338 seq_num 1057) dirty exists 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3765: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:14:22.746 INFO:tasks.rados.rados.0.smithi118.stdout:3767: seq_num 1155 ranges {0=147456} 2024-03-20T08:14:22.747 INFO:tasks.rados.rados.0.smithi118.stdout:3767: writing smithi11881747-4 from 0 to 147456 tid 1 2024-03-20T08:14:22.747 INFO:tasks.rados.rados.0.smithi118.stdout:3764: done (2 left) 2024-03-20T08:14:22.747 INFO:tasks.rados.rados.0.smithi118.stdout:3768: snap_remove snap 360 2024-03-20T08:14:23.668 INFO:tasks.rados.rados.0.smithi118.stdout:3768: done (2 left) 2024-03-20T08:14:23.668 INFO:tasks.rados.rados.0.smithi118.stdout:3769: snap_create 2024-03-20T08:14:23.668 INFO:tasks.rados.rados.0.smithi118.stdout:3765: finishing copy_from to smithi11881747-50 2024-03-20T08:14:23.668 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 65304 (ObjNum 1128 snap 361 seq_num 1128) dirty exists 2024-03-20T08:14:23.668 INFO:tasks.rados.rados.0.smithi118.stdout:3767: finishing write tid 1 to smithi11881747-4 2024-03-20T08:14:23.669 INFO:tasks.rados.rados.0.smithi118.stdout:3767: finishing write tid 2 to smithi11881747-4 2024-03-20T08:14:23.669 INFO:tasks.rados.rados.0.smithi118.stdout:3767: finishing write tid 3 to smithi11881747-4 2024-03-20T08:14:23.669 INFO:tasks.rados.rados.0.smithi118.stdout:3767: oid 4 updating version 0 to 65315 2024-03-20T08:14:23.669 INFO:tasks.rados.rados.0.smithi118.stdout:3767: oid 4 version 65315 is already newer than 65314 2024-03-20T08:14:23.669 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65315 (ObjNum 1155 snap 371 seq_num 1155) dirty exists 2024-03-20T08:14:23.669 INFO:tasks.rados.rados.0.smithi118.stdout:3767: left oid 4 (ObjNum 1155 snap 371 seq_num 1155) 2024-03-20T08:14:23.912 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:23 smithi196 ceph-mon[87080]: pgmap v904: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 13 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-03-20T08:14:23.912 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:23.667+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:23.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:23 smithi118 ceph-mon[93121]: pgmap v904: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 13 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-03-20T08:14:23.922 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:23 smithi118 ceph-mon[95692]: pgmap v904: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 13 MiB/s rd, 2.6 MiB/s wr, 25 op/s 2024-03-20T08:14:23.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:14:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:14:23] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:14:23.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:23.668+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:23.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:23.668+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:23.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:23.668+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:23.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:23.668+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:24.337 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:23.912+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:24.338 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:23.912+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:24.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:24.077+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:24.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:24.337+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:24.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:24.337+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:24.673 INFO:tasks.rados.rados.0.smithi118.stdout:3765: done (2 left) 2024-03-20T08:14:24.673 INFO:tasks.rados.rados.0.smithi118.stdout:3767: done (1 left) 2024-03-20T08:14:24.673 INFO:tasks.rados.rados.0.smithi118.stdout:3769: done (0 left) 2024-03-20T08:14:24.673 INFO:tasks.rados.rados.0.smithi118.stdout:3770: rollback oid 42 current snap is 372 2024-03-20T08:14:24.673 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 42 to 352 2024-03-20T08:14:24.674 INFO:tasks.rados.rados.0.smithi118.stdout:3771: read oid 27 snap -1 2024-03-20T08:14:24.674 INFO:tasks.rados.rados.0.smithi118.stdout:3771: expect (ObjNum 1057 snap 338 seq_num 1057) 2024-03-20T08:14:24.674 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 372 2024-03-20T08:14:24.674 INFO:tasks.rados.rados.0.smithi118.stdout:3772: seq_num 1156 ranges {0=163840} 2024-03-20T08:14:24.674 INFO:tasks.rados.rados.0.smithi118.stdout:3772: writing smithi11881747-32 from 0 to 163840 tid 1 2024-03-20T08:14:24.674 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 1 current snap is 372 2024-03-20T08:14:24.674 INFO:tasks.rados.rados.0.smithi118.stdout:3773: seq_num 1157 ranges {0=180224} 2024-03-20T08:14:24.675 INFO:tasks.rados.rados.0.smithi118.stdout:3773: writing smithi11881747-1 from 0 to 180224 tid 1 2024-03-20T08:14:24.675 INFO:tasks.rados.rados.0.smithi118.stdout:3774: delete oid 7 current snap is 372 2024-03-20T08:14:24.681 INFO:tasks.rados.rados.0.smithi118.stdout:3773: finishing write tid 1 to smithi11881747-1 2024-03-20T08:14:24.681 INFO:tasks.rados.rados.0.smithi118.stdout:3773: finishing write tid 2 to smithi11881747-1 2024-03-20T08:14:24.681 INFO:tasks.rados.rados.0.smithi118.stdout:3770: finishing rollback tid 0 to smithi11881747-42 2024-03-20T08:14:24.681 INFO:tasks.rados.rados.0.smithi118.stdout:3770: finishing rollback tid 1 to smithi11881747-42 2024-03-20T08:14:24.682 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65340 (ObjNum 65 snap 0 seq_num 1634164852) dirty dne 2024-03-20T08:14:24.683 INFO:tasks.rados.rados.0.smithi118.stdout:3773: finishing write tid 3 to smithi11881747-1 2024-03-20T08:14:24.683 INFO:tasks.rados.rados.0.smithi118.stdout:3773: oid 1 updating version 0 to 65319 2024-03-20T08:14:24.683 INFO:tasks.rados.rados.0.smithi118.stdout:3773: oid 1 version 65319 is already newer than 65318 2024-03-20T08:14:24.683 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 1 v 65319 (ObjNum 1157 snap 372 seq_num 1157) dirty exists 2024-03-20T08:14:24.683 INFO:tasks.rados.rados.0.smithi118.stdout:3773: left oid 1 (ObjNum 1157 snap 372 seq_num 1157) 2024-03-20T08:14:24.684 INFO:tasks.rados.rados.0.smithi118.stdout:3770: done (4 left) 2024-03-20T08:14:24.684 INFO:tasks.rados.rados.0.smithi118.stdout:3773: done (3 left) 2024-03-20T08:14:24.684 INFO:tasks.rados.rados.0.smithi118.stdout:3774: done (2 left) 2024-03-20T08:14:24.684 INFO:tasks.rados.rados.0.smithi118.stdout:3775: snap_remove snap 352 2024-03-20T08:14:24.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:24.674+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:24.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:24.675+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:24.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:24.675+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:24.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:24.681+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:24.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[93121]: osdmap e811: 8 total, 8 up, 8 in 2024-03-20T08:14:24.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:24.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:24.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:24.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:24.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:24.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[95692]: osdmap e811: 8 total, 8 up, 8 in 2024-03-20T08:14:24.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:24.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:24.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:24.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:24.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:24.925 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:24.677+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:24.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:24.678+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:24.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:24.678+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:24.926 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:24.685+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:24.926 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:24.672+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:25.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:24.672+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:25.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:24.676+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:25.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:24.676+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:25.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:24.676+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:25.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:24.677+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:25.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:24.677+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:25.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:24.676+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:25.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:24.678+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:25.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:24.678+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:25.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-mon[87080]: osdmap e811: 8 total, 8 up, 8 in 2024-03-20T08:14:25.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:25.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:25.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:25.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:25.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:24 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3775: done (2 left) 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3776: read oid 5 snap -1 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3776: expect deleted 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3777: snap_create 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3772: finishing write tid 1 to smithi11881747-32 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3772: finishing write tid 2 to smithi11881747-32 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3772: finishing write tid 3 to smithi11881747-32 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3772: oid 32 updating version 0 to 34139 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:3772: oid 32 updating version 34139 to 34140 2024-03-20T08:14:25.679 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34140 (ObjNum 1156 snap 372 seq_num 1156) dirty exists 2024-03-20T08:14:25.680 INFO:tasks.rados.rados.0.smithi118.stdout:3772: left oid 32 (ObjNum 1156 snap 372 seq_num 1156) 2024-03-20T08:14:25.680 INFO:tasks.rados.rados.0.smithi118.stdout:3771: expect (ObjNum 1057 snap 338 seq_num 1057) 2024-03-20T08:14:25.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:25.681+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:25.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:25.682+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:25.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:25.679+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:25.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:25.679+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:25.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:25.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:25.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:25.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:25.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: pgmap v906: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-03-20T08:14:25.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:25.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:25.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: osdmap e812: 8 total, 8 up, 8 in 2024-03-20T08:14:25.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:25.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:25.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:25.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:25.677+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:25.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:25.677+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:25.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:25.677+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:25.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:25.678+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:25.924 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:25.678+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: pgmap v906: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: osdmap e812: 8 total, 8 up, 8 in 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:25.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:25.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:25.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:26.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:25.678+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:26.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: pgmap v906: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 23 op/s 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: osdmap e812: 8 total, 8 up, 8 in 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:26.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:26.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:26.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:26.202+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:26.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:26.202+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:26.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:26.201+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:26.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:26.201+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:26.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:26.202+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:26.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:26.202+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:26.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:26.202+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:26.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:26.199+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:26.790 INFO:tasks.rados.rados.0.smithi118.stdout:3771: done (3 left) 2024-03-20T08:14:26.790 INFO:tasks.rados.rados.0.smithi118.stdout:3772: done (2 left) 2024-03-20T08:14:26.790 INFO:tasks.rados.rados.0.smithi118.stdout:3776: done (1 left) 2024-03-20T08:14:26.790 INFO:tasks.rados.rados.0.smithi118.stdout:3777: done (0 left) 2024-03-20T08:14:26.790 INFO:tasks.rados.rados.0.smithi118.stdout:3778: snap_create 2024-03-20T08:14:26.932 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:26.679+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: osdmap e813: 8 total, 8 up, 8 in 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:26.933 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:27.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:26.680+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:27.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:26.678+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:27.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:26.679+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:27.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:26.680+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:27.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:26.680+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: osdmap e813: 8 total, 8 up, 8 in 2024-03-20T08:14:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:27.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: osdmap e813: 8 total, 8 up, 8 in 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:27.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:27.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:27.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:27.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:27.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:27.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:27.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:27.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:26 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:27.301 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:27.051+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:27.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:27.052+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:27.402 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:27.052+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:27.402 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:26.932+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:27.403 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:26.932+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:27.403 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:26.932+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:27.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:27.301+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:27.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:27.301+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:27.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:27.301+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:27.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:27.300+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:27.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:27.300+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:27.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:27.301+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:27.692 INFO:tasks.rados.rados.0.smithi118.stdout:3778: done (0 left) 2024-03-20T08:14:27.692 INFO:tasks.rados.rados.0.smithi118.stdout:3779: rmattr oid 7 current snap is 374 2024-03-20T08:14:27.692 INFO:tasks.rados.rados.0.smithi118.stdout:3779: done (0 left) 2024-03-20T08:14:27.692 INFO:tasks.rados.rados.0.smithi118.stdout:3780: read oid 17 snap 371 2024-03-20T08:14:27.692 INFO:tasks.rados.rados.0.smithi118.stdout:3780: expect (ObjNum 1132 snap 364 seq_num 1132) 2024-03-20T08:14:27.692 INFO:tasks.rados.rados.0.smithi118.stdout:3781: read oid 46 snap -1 2024-03-20T08:14:27.693 INFO:tasks.rados.rados.0.smithi118.stdout:3781: expect (ObjNum 1148 snap 370 seq_num 1148) 2024-03-20T08:14:27.693 INFO:tasks.rados.rados.0.smithi118.stdout:3782: read oid 49 snap -1 2024-03-20T08:14:27.693 INFO:tasks.rados.rados.0.smithi118.stdout:3782: expect (ObjNum 1133 snap 364 seq_num 1133) 2024-03-20T08:14:27.693 INFO:tasks.rados.rados.0.smithi118.stdout:3783: snap_create 2024-03-20T08:14:27.721 INFO:tasks.rados.rados.0.smithi118.stdout:3780: expect (ObjNum 1132 snap 364 seq_num 1132) 2024-03-20T08:14:27.786 INFO:tasks.rados.rados.0.smithi118.stdout:3782: expect (ObjNum 1133 snap 364 seq_num 1133) 2024-03-20T08:14:28.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:27.693+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:28.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:27.706+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:28.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:27.734+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: pgmap v909: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 23 MiB/s rd, 1.5 MiB/s wr, 12 op/s 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: osdmap e814: 8 total, 8 up, 8 in 2024-03-20T08:14:28.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:28.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:27 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:28.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:28.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:28.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:28.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:28.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: pgmap v909: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 23 MiB/s rd, 1.5 MiB/s wr, 12 op/s 2024-03-20T08:14:28.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:28.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:28.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: osdmap e814: 8 total, 8 up, 8 in 2024-03-20T08:14:28.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:28.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:28.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:28.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:28.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:28.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:28.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:28.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:28.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: pgmap v909: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 23 MiB/s rd, 1.5 MiB/s wr, 12 op/s 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: osdmap e814: 8 total, 8 up, 8 in 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:28.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:28.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:28.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:28.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:28.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:28.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:27.691+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:28.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:27.691+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:28.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:27.691+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:28.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:27.692+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:28.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:27.692+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:28.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:27.704+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:28.178 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:27.705+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:28.178 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:27 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:27.700+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:28.584 INFO:tasks.rados.rados.0.smithi118.stdout:3781: expect (ObjNum 1148 snap 370 seq_num 1148) 2024-03-20T08:14:28.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:28.175+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:28.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:28.175+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:28.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:28.175+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:28.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:28.175+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:28.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:28.175+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:28.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:28.173+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:28.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:28.174+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:29.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:28.697+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:29.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:28.938+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:29.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:28.938+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: osdmap e815: 8 total, 8 up, 8 in 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:29.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:29.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:29.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:28.698+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:29.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:28.695+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:29.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:28.695+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:29.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:28.696+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:29.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:28.777+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:29.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:29.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:29.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:29.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:29.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: osdmap e815: 8 total, 8 up, 8 in 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:29.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: osdmap e815: 8 total, 8 up, 8 in 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:29.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:29.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:29.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:29.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:29.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:29.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:29.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:29.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:29.891 INFO:tasks.rados.rados.0.smithi118.stdout:3780: done (3 left) 2024-03-20T08:14:29.891 INFO:tasks.rados.rados.0.smithi118.stdout:3781: done (2 left) 2024-03-20T08:14:29.891 INFO:tasks.rados.rados.0.smithi118.stdout:3782: done (1 left) 2024-03-20T08:14:29.891 INFO:tasks.rados.rados.0.smithi118.stdout:3783: done (0 left) 2024-03-20T08:14:29.891 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 375 2024-03-20T08:14:29.891 INFO:tasks.rados.rados.0.smithi118.stdout:3784: seq_num 1159 ranges {0=212992} 2024-03-20T08:14:29.892 INFO:tasks.rados.rados.0.smithi118.stdout:3784: writing smithi11881747-9 from 0 to 212992 tid 1 2024-03-20T08:14:29.892 INFO:tasks.rados.rados.0.smithi118.stdout:3785: setattr oid 3 current snap is 375 2024-03-20T08:14:29.895 INFO:tasks.rados.rados.0.smithi118.stdout:3786: rollback oid 19 current snap is 375 2024-03-20T08:14:29.895 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 19 to 366 2024-03-20T08:14:29.895 INFO:tasks.rados.rados.0.smithi118.stdout:3787: snap_remove snap 373 2024-03-20T08:14:29.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:29.560+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:29.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:29.560+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:29.922 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:29.560+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:29.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:29.560+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:29.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:29.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:29.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: pgmap v912: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 4 op/s 2024-03-20T08:14:29.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:29.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:29.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:29.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: osdmap e816: 8 total, 8 up, 8 in 2024-03-20T08:14:29.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:29.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:29.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:29.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:29.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:29.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:29.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: pgmap v912: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 4 op/s 2024-03-20T08:14:29.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:29.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:29.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:29.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: osdmap e816: 8 total, 8 up, 8 in 2024-03-20T08:14:29.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:29.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:29.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:29.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:29 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:30.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:29.897+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:30.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:29.897+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:30.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:29.897+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: pgmap v912: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 4 op/s 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: osdmap e816: 8 total, 8 up, 8 in 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:30.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:29 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:30.703 INFO:tasks.rados.rados.0.smithi118.stdout:3787: done (3 left) 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3788: rmattr oid 26 current snap is 375 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3784: finishing write tid 1 to smithi11881747-9 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3784: finishing write tid 2 to smithi11881747-9 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3784: finishing write tid 3 to smithi11881747-9 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3784: oid 9 updating version 0 to 65342 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3784: oid 9 version 65342 is already newer than 65341 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 65342 (ObjNum 1159 snap 375 seq_num 1159) dirty exists 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3784: left oid 9 (ObjNum 1159 snap 375 seq_num 1159) 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 3 v 34065 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3786: finishing rollback tid 0 to smithi11881747-19 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3786: finishing rollback tid 1 to smithi11881747-19 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3786: finishing rollback tid 2 to smithi11881747-19 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 59867 (ObjNum 1122 snap 360 seq_num 1122) dirty exists 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3784: done (3 left) 2024-03-20T08:14:30.704 INFO:tasks.rados.rados.0.smithi118.stdout:3785: done (2 left) 2024-03-20T08:14:30.705 INFO:tasks.rados.rados.0.smithi118.stdout:3786: done (1 left) 2024-03-20T08:14:30.705 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 10 current snap is 375 2024-03-20T08:14:30.705 INFO:tasks.rados.rados.0.smithi118.stdout:3789: seq_num 1160 ranges {0=229376} 2024-03-20T08:14:30.705 INFO:tasks.rados.rados.0.smithi118.stdout:3789: writing smithi11881747-10 from 0 to 229376 tid 1 2024-03-20T08:14:30.705 INFO:tasks.rados.rados.0.smithi118.stdout:3790: rmattr oid 9 current snap is 375 2024-03-20T08:14:30.705 INFO:tasks.rados.rados.0.smithi118.stdout:3790: done (2 left) 2024-03-20T08:14:30.705 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:30.444+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:30.705 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:30.444+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:30.706 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:30.444+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:30.706 INFO:tasks.rados.rados.0.smithi118.stdout:3791: read oid 25 snap 369 2024-03-20T08:14:30.708 INFO:tasks.rados.rados.0.smithi118.stdout:3791: expect (ObjNum 1131 snap 364 seq_num 1131) 2024-03-20T08:14:30.708 INFO:tasks.rados.rados.0.smithi118.stdout:3792: read oid 13 snap -1 2024-03-20T08:14:30.709 INFO:tasks.rados.rados.0.smithi118.stdout:3792: expect (ObjNum 1111 snap 356 seq_num 1111) 2024-03-20T08:14:30.709 INFO:tasks.rados.rados.0.smithi118.stdout:3793: rollback oid 12 current snap is 375 2024-03-20T08:14:30.709 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 12 to 361 2024-03-20T08:14:30.709 INFO:tasks.rados.rados.0.smithi118.stdout:3794: copy_from oid 20 from oid 24 current snap is 375 2024-03-20T08:14:30.709 INFO:tasks.rados.rados.0.smithi118.stdout:3795: copy_from oid 42 from oid 2 current snap is 375 2024-03-20T08:14:30.709 INFO:tasks.rados.rados.0.smithi118.stdout:3796: snap_create 2024-03-20T08:14:30.718 INFO:tasks.rados.rados.0.smithi118.stdout:3789: finishing write tid 1 to smithi11881747-10 2024-03-20T08:14:30.719 INFO:tasks.rados.rados.0.smithi118.stdout:3789: finishing write tid 2 to smithi11881747-10 2024-03-20T08:14:30.721 INFO:tasks.rados.rados.0.smithi118.stdout:3793: finishing rollback tid 1 to smithi11881747-12 2024-03-20T08:14:30.721 INFO:tasks.rados.rados.0.smithi118.stdout:3789: finishing write tid 3 to smithi11881747-10 2024-03-20T08:14:30.721 INFO:tasks.rados.rados.0.smithi118.stdout:3789: oid 10 updating version 0 to 65321 2024-03-20T08:14:30.722 INFO:tasks.rados.rados.0.smithi118.stdout:3789: oid 10 version 65321 is already newer than 65320 2024-03-20T08:14:30.722 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 65321 (ObjNum 1160 snap 375 seq_num 1160) dirty exists 2024-03-20T08:14:30.722 INFO:tasks.rados.rados.0.smithi118.stdout:3789: left oid 10 (ObjNum 1160 snap 375 seq_num 1160) 2024-03-20T08:14:30.722 INFO:tasks.rados.rados.0.smithi118.stdout:3793: finishing rollback tid 2 to smithi11881747-12 2024-03-20T08:14:30.722 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65308 (ObjNum 1056 snap 338 seq_num 1056) dirty exists 2024-03-20T08:14:30.722 INFO:tasks.rados.rados.0.smithi118.stdout:3791: expect (ObjNum 1131 snap 364 seq_num 1131) 2024-03-20T08:14:30.915 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65344 (ObjNum 1153 snap 371 seq_num 1153) dirty exists 2024-03-20T08:14:30.916 INFO:tasks.rados.rados.0.smithi118.stdout:3792: expect (ObjNum 1111 snap 356 seq_num 1111) 2024-03-20T08:14:31.085 INFO:tasks.rados.rados.0.smithi118.stdout:3794: finishing copy_from racing read to smithi11881747-20 2024-03-20T08:14:31.085 INFO:tasks.rados.rados.0.smithi118.stdout:3795: finishing copy_from racing read to smithi11881747-42 2024-03-20T08:14:31.085 INFO:tasks.rados.rados.0.smithi118.stdout:3794: finishing copy_from to smithi11881747-20 2024-03-20T08:14:31.085 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 65323 (ObjNum 1118 snap 358 seq_num 1118) dirty exists 2024-03-20T08:14:31.085 INFO:tasks.rados.rados.0.smithi118.stdout:3795: finishing copy_from to smithi11881747-42 2024-03-20T08:14:31.086 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65320 (ObjNum 1154 snap 371 seq_num 1154) dirty exists 2024-03-20T08:14:31.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:30.703+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:31.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:30.713+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b2=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:31.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:30.717+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:31.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:30.714+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:31.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:30.716+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:31.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:30.712+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:31.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:30.712+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:31.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:30.721+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:31.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:31.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:30 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:31.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:30.706+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:31.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:30.706+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:31.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:30.706+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:31.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:30.727+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:31.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:30.701+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:31.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:31.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:31.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:31.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:31.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:31.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:31.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:31.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:31.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:31.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:31.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:31.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:31.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:30.707+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:31.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:30.707+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:31.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:30.707+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:31.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:30.707+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:31.177 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:30.717+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:31.709 INFO:tasks.rados.rados.0.smithi118.stdout:3788: done (7 left) 2024-03-20T08:14:31.709 INFO:tasks.rados.rados.0.smithi118.stdout:3789: done (6 left) 2024-03-20T08:14:31.709 INFO:tasks.rados.rados.0.smithi118.stdout:3791: done (5 left) 2024-03-20T08:14:31.709 INFO:tasks.rados.rados.0.smithi118.stdout:3792: done (4 left) 2024-03-20T08:14:31.709 INFO:tasks.rados.rados.0.smithi118.stdout:3793: done (3 left) 2024-03-20T08:14:31.709 INFO:tasks.rados.rados.0.smithi118.stdout:3794: done (2 left) 2024-03-20T08:14:31.709 INFO:tasks.rados.rados.0.smithi118.stdout:3795: done (1 left) 2024-03-20T08:14:31.709 INFO:tasks.rados.rados.0.smithi118.stdout:3796: done (0 left) 2024-03-20T08:14:31.710 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 376 2024-03-20T08:14:31.710 INFO:tasks.rados.rados.0.smithi118.stdout:3797: seq_num 1161 ranges {6430720=245760} 2024-03-20T08:14:31.745 INFO:tasks.rados.rados.0.smithi118.stdout:3797: writing smithi11881747-12 from 6430720 to 6676480 tid 1 2024-03-20T08:14:31.745 INFO:tasks.rados.rados.0.smithi118.stdout:3798: copy_from oid 6 from oid 39 current snap is 376 2024-03-20T08:14:31.746 INFO:tasks.rados.rados.0.smithi118.stdout:3799: copy_from oid 15 from oid 41 current snap is 376 2024-03-20T08:14:31.746 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 376 2024-03-20T08:14:31.746 INFO:tasks.rados.rados.0.smithi118.stdout:3800: seq_num 1162 ranges {1785856=270336} 2024-03-20T08:14:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:3800: writing smithi11881747-21 from 1785856 to 2056192 tid 1 2024-03-20T08:14:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 45 current snap is 376 2024-03-20T08:14:31.757 INFO:tasks.rados.rados.0.smithi118.stdout:3801: seq_num 1163 ranges {0=286720} 2024-03-20T08:14:31.759 INFO:tasks.rados.rados.0.smithi118.stdout:3801: writing smithi11881747-45 from 0 to 286720 tid 1 2024-03-20T08:14:31.759 INFO:tasks.rados.rados.0.smithi118.stdout:3802: rollback oid 30 current snap is 376 2024-03-20T08:14:31.759 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 30 to 375 2024-03-20T08:14:31.759 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 376 2024-03-20T08:14:31.759 INFO:tasks.rados.rados.0.smithi118.stdout:3803: seq_num 1164 ranges {6160384=303104} 2024-03-20T08:14:31.795 INFO:tasks.rados.rados.0.smithi118.stdout:3803: writing smithi11881747-43 from 6160384 to 6463488 tid 1 2024-03-20T08:14:31.795 INFO:tasks.rados.rados.0.smithi118.stdout:3804: read oid 34 snap 375 2024-03-20T08:14:31.795 INFO:tasks.rados.rados.0.smithi118.stdout:3804: expect (ObjNum 1139 snap 367 seq_num 1139) 2024-03-20T08:14:31.795 INFO:tasks.rados.rados.0.smithi118.stdout:3805: read oid 14 snap -1 2024-03-20T08:14:31.796 INFO:tasks.rados.rados.0.smithi118.stdout:3805: expect deleted 2024-03-20T08:14:31.796 INFO:tasks.rados.rados.0.smithi118.stdout:3806: snap_remove snap 367 2024-03-20T08:14:32.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:31.708+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:32.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:31.750+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:32.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:31.750+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b2=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:32.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:31.750+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:32.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:31.750+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:32.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:31.750+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:32.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:31.754+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:32.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:31.755+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:32.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:31.755+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:32.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:32.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:32.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: pgmap v914: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 32 KiB/s wr, 3 op/s 2024-03-20T08:14:32.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: osdmap e817: 8 total, 8 up, 8 in 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b2=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:32.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:32.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:32.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:31 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:32.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:31.707+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:32.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:31.708+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:32.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:31.747+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:32.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:31.748+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:32.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:31.748+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:32.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:31.759+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:32.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:31.763+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:32.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:31.763+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:32.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:31.764+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: pgmap v914: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 32 KiB/s wr, 3 op/s 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: osdmap e817: 8 total, 8 up, 8 in 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:32.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: pgmap v914: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 32 KiB/s wr, 3 op/s 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: osdmap e817: 8 total, 8 up, 8 in 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:32.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b2=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:32.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b2=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:32.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:31 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:14:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:14:32.717 INFO:tasks.rados.rados.0.smithi118.stdout:3806: done (9 left) 2024-03-20T08:14:32.717 INFO:tasks.rados.rados.0.smithi118.stdout:3807: copy_from oid 33 from oid 22 current snap is 376 2024-03-20T08:14:32.717 INFO:tasks.rados.rados.0.smithi118.stdout:3797: finishing write tid 1 to smithi11881747-12 2024-03-20T08:14:32.717 INFO:tasks.rados.rados.0.smithi118.stdout:3797: finishing write tid 2 to smithi11881747-12 2024-03-20T08:14:32.717 INFO:tasks.rados.rados.0.smithi118.stdout:3798: finishing copy_from racing read to smithi11881747-6 2024-03-20T08:14:32.717 INFO:tasks.rados.rados.0.smithi118.stdout:3802: finishing rollback tid 0 to smithi11881747-30 2024-03-20T08:14:32.717 INFO:tasks.rados.rados.0.smithi118.stdout:3802: finishing rollback tid 1 to smithi11881747-30 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3802: finishing rollback tid 2 to smithi11881747-30 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 65318 (ObjNum 1128 snap 361 seq_num 1128) dirty exists 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3797: finishing write tid 3 to smithi11881747-12 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3797: oid 12 updating version 0 to 65309 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3797: oid 12 updating version 65309 to 65310 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65310 (ObjNum 1161 snap 376 seq_num 1161) dirty exists 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3797: left oid 12 (ObjNum 1161 snap 376 seq_num 1161) 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3800: finishing write tid 1 to smithi11881747-21 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3797: done (9 left) 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3802: done (8 left) 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3808: read oid 30 snap 365 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3808: expect (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3809: rollback oid 4 current snap is 376 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 4 to 370 2024-03-20T08:14:32.718 INFO:tasks.rados.rados.0.smithi118.stdout:3800: finishing write tid 2 to smithi11881747-21 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3801: finishing write tid 1 to smithi11881747-45 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3798: finishing copy_from to smithi11881747-6 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 6 v 65294 (ObjNum 1068 snap 341 seq_num 1068) dirty exists 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3800: finishing write tid 3 to smithi11881747-21 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3800: oid 21 updating version 0 to 65328 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3800: oid 21 version 65328 is already newer than 65327 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 65328 (ObjNum 1162 snap 376 seq_num 1162) dirty exists 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3800: left oid 21 (ObjNum 1162 snap 376 seq_num 1162) 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3801: finishing write tid 2 to smithi11881747-45 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3798: done (9 left) 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3800: done (8 left) 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 376 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3801: finishing write tid 3 to smithi11881747-45 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3801: oid 45 updating version 0 to 65324 2024-03-20T08:14:32.719 INFO:tasks.rados.rados.0.smithi118.stdout:3801: oid 45 version 65324 is already newer than 65323 2024-03-20T08:14:32.720 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 65324 (ObjNum 1163 snap 376 seq_num 1163) dirty exists 2024-03-20T08:14:32.720 INFO:tasks.rados.rados.0.smithi118.stdout:3801: left oid 45 (ObjNum 1163 snap 376 seq_num 1163) 2024-03-20T08:14:32.720 INFO:tasks.rados.rados.0.smithi118.stdout:3810: seq_num 1165 ranges {2318336=319488} 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3810: writing smithi11881747-19 from 2318336 to 2637824 tid 1 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3801: done (8 left) 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3811: snap_create 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3799: finishing copy_from racing read to smithi11881747-15 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3803: finishing write tid 1 to smithi11881747-43 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3803: finishing write tid 2 to smithi11881747-43 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3799: finishing copy_from to smithi11881747-15 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 65331 (ObjNum 1129 snap 361 seq_num 1129) dirty exists 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3803: finishing write tid 3 to smithi11881747-43 2024-03-20T08:14:32.731 INFO:tasks.rados.rados.0.smithi118.stdout:3803: oid 43 updating version 0 to 65323 2024-03-20T08:14:32.732 INFO:tasks.rados.rados.0.smithi118.stdout:3803: oid 43 version 65323 is already newer than 65322 2024-03-20T08:14:32.732 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65323 (ObjNum 1164 snap 376 seq_num 1164) dirty exists 2024-03-20T08:14:32.732 INFO:tasks.rados.rados.0.smithi118.stdout:3803: left oid 43 (ObjNum 1164 snap 376 seq_num 1164) 2024-03-20T08:14:32.732 INFO:tasks.rados.rados.0.smithi118.stdout:3804: expect (ObjNum 1139 snap 367 seq_num 1139) 2024-03-20T08:14:32.800 INFO:tasks.rados.rados.0.smithi118.stdout:3807: finishing copy_from to smithi11881747-33 2024-03-20T08:14:32.801 INFO:tasks.rados.rados.0.smithi118.stdout:3807: got expected ENOENT (src dne) 2024-03-20T08:14:32.801 INFO:tasks.rados.rados.0.smithi118.stdout:3807: finishing copy_from racing read to smithi11881747-33 2024-03-20T08:14:32.801 INFO:tasks.rados.rados.0.smithi118.stdout:3810: finishing write tid 1 to smithi11881747-19 2024-03-20T08:14:32.801 INFO:tasks.rados.rados.0.smithi118.stdout:3810: finishing write tid 2 to smithi11881747-19 2024-03-20T08:14:32.801 INFO:tasks.rados.rados.0.smithi118.stdout:3808: expect (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:3809: finishing rollback tid 0 to smithi11881747-4 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:3809: finishing rollback tid 1 to smithi11881747-4 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:3809: finishing rollback tid 2 to smithi11881747-4 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65321 (ObjNum 1144 snap 370 seq_num 1144) dirty exists 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:3810: finishing write tid 3 to smithi11881747-19 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:3810: oid 19 updating version 0 to 65333 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:3810: oid 19 version 65333 is already newer than 65332 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 65333 (ObjNum 1165 snap 376 seq_num 1165) dirty exists 2024-03-20T08:14:32.970 INFO:tasks.rados.rados.0.smithi118.stdout:3810: left oid 19 (ObjNum 1165 snap 376 seq_num 1165) 2024-03-20T08:14:33.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:32.714+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:33.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:32.726+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:33.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:32.727+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:33.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:32.719+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:33.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:32.737+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:33.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:32.738+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:33.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:32.743+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:33.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:32.743+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: osdmap e818: 8 total, 8 up, 8 in 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b2=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:33.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:33.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:33.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:33.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:33.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:32 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:33.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:32.714+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:33.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:32.714+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:33.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:32.738+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:33.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:32.714+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:33.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:32.714+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:33.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:33.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: osdmap e818: 8 total, 8 up, 8 in 2024-03-20T08:14:33.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:33.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:33.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:33.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:33.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:33.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:33.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:33.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:32.717+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:33.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:32.718+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:33.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:32.718+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:33.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:32.718+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:33.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:32.718+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:33.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:33.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b2=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:33.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:33.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:33.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:33.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:33.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:33.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: osdmap e818: 8 total, 8 up, 8 in 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b2=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:33.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:33.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:32 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:33.720 INFO:tasks.rados.rados.0.smithi118.stdout:3799: done (8 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3803: done (7 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3804: done (6 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3805: done (5 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3807: done (4 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3808: done (3 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3809: done (2 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3810: done (1 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3811: done (0 left) 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3812: copy_from oid 13 from oid 29 current snap is 377 2024-03-20T08:14:33.721 INFO:tasks.rados.rados.0.smithi118.stdout:3813: setattr oid 33 current snap is 377 2024-03-20T08:14:33.722 INFO:tasks.rados.rados.0.smithi118.stdout:3814: setattr oid 30 current snap is 377 2024-03-20T08:14:33.724 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 7 current snap is 377 2024-03-20T08:14:33.724 INFO:tasks.rados.rados.0.smithi118.stdout:3815: seq_num 1166 ranges {0=335872} 2024-03-20T08:14:33.726 INFO:tasks.rados.rados.0.smithi118.stdout:3815: writing smithi11881747-7 from 0 to 335872 tid 1 2024-03-20T08:14:33.726 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 27 current snap is 377 2024-03-20T08:14:33.726 INFO:tasks.rados.rados.0.smithi118.stdout:3816: seq_num 1167 ranges {12697600=352256} 2024-03-20T08:14:33.799 INFO:tasks.rados.rados.0.smithi118.stdout:3816: writing smithi11881747-27 from 12697600 to 13049856 tid 1 2024-03-20T08:14:33.799 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 377 2024-03-20T08:14:33.799 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 30 v 65322 (ObjNum 1128 snap 361 seq_num 1128) dirty exists 2024-03-20T08:14:33.799 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 33 v 34066 (ObjNum 1116 snap 358 seq_num 1116) dirty exists 2024-03-20T08:14:33.799 INFO:tasks.rados.rados.0.smithi118.stdout:3815: finishing write tid 1 to smithi11881747-7 2024-03-20T08:14:33.799 INFO:tasks.rados.rados.0.smithi118.stdout:3815: finishing write tid 2 to smithi11881747-7 2024-03-20T08:14:33.799 INFO:tasks.rados.rados.0.smithi118.stdout:3815: finishing write tid 3 to smithi11881747-7 2024-03-20T08:14:33.800 INFO:tasks.rados.rados.0.smithi118.stdout:3815: oid 7 updating version 0 to 65312 2024-03-20T08:14:33.800 INFO:tasks.rados.rados.0.smithi118.stdout:3815: oid 7 version 65312 is already newer than 65311 2024-03-20T08:14:33.800 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 65312 (ObjNum 1166 snap 377 seq_num 1166) dirty exists 2024-03-20T08:14:33.800 INFO:tasks.rados.rados.0.smithi118.stdout:3815: left oid 7 (ObjNum 1166 snap 377 seq_num 1166) 2024-03-20T08:14:33.800 INFO:tasks.rados.rados.0.smithi118.stdout:3812: finishing copy_from racing read to smithi11881747-13 2024-03-20T08:14:33.800 INFO:tasks.rados.rados.0.smithi118.stdout:3812: finishing copy_from to smithi11881747-13 2024-03-20T08:14:33.800 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 13 v 65343 (ObjNum 1075 snap 341 seq_num 1075) dirty exists 2024-03-20T08:14:33.800 INFO:tasks.rados.rados.0.smithi118.stdout:3817: seq_num 1168 ranges {2490368=368640} 2024-03-20T08:14:33.816 INFO:tasks.rados.rados.0.smithi118.stdout:3817: writing smithi11881747-2 from 2490368 to 2859008 tid 1 2024-03-20T08:14:33.816 INFO:tasks.rados.rados.0.smithi118.stdout:3812: done (5 left) 2024-03-20T08:14:33.816 INFO:tasks.rados.rados.0.smithi118.stdout:3813: done (4 left) 2024-03-20T08:14:33.816 INFO:tasks.rados.rados.0.smithi118.stdout:3814: done (3 left) 2024-03-20T08:14:33.816 INFO:tasks.rados.rados.0.smithi118.stdout:3815: done (2 left) 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3818: read oid 34 snap -1 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3818: expect (ObjNum 1139 snap 367 seq_num 1139) 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3819: rmattr oid 31 current snap is 377 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3819: done (3 left) 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3820: read oid 45 snap -1 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3820: expect (ObjNum 1163 snap 376 seq_num 1163) 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3821: read oid 30 snap -1 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3821: expect (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3816: finishing write tid 1 to smithi11881747-27 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3816: finishing write tid 2 to smithi11881747-27 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3816: finishing write tid 3 to smithi11881747-27 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3816: oid 27 updating version 0 to 65324 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:3816: oid 27 updating version 65324 to 65325 2024-03-20T08:14:33.817 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 27 v 65325 (ObjNum 1167 snap 377 seq_num 1167) dirty exists 2024-03-20T08:14:33.818 INFO:tasks.rados.rados.0.smithi118.stdout:3816: left oid 27 (ObjNum 1167 snap 377 seq_num 1167) 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3822: read oid 11 snap 368 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3822: expect (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3816: done (5 left) 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3823: copy_from oid 18 from oid 14 current snap is 377 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3824: delete oid 13 current snap is 377 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3823: finishing copy_from to smithi11881747-18 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3823: got expected ENOENT (src dne) 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3823: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:14:33.819 INFO:tasks.rados.rados.0.smithi118.stdout:3820: expect (ObjNum 1163 snap 376 seq_num 1163) 2024-03-20T08:14:33.831 INFO:tasks.rados.rados.0.smithi118.stdout:3817: finishing write tid 1 to smithi11881747-2 2024-03-20T08:14:33.831 INFO:tasks.rados.rados.0.smithi118.stdout:3817: finishing write tid 2 to smithi11881747-2 2024-03-20T08:14:33.831 INFO:tasks.rados.rados.0.smithi118.stdout:3817: finishing write tid 3 to smithi11881747-2 2024-03-20T08:14:33.831 INFO:tasks.rados.rados.0.smithi118.stdout:3817: oid 2 updating version 0 to 65322 2024-03-20T08:14:33.831 INFO:tasks.rados.rados.0.smithi118.stdout:3817: oid 2 version 65322 is already newer than 65321 2024-03-20T08:14:33.832 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65322 (ObjNum 1168 snap 377 seq_num 1168) dirty exists 2024-03-20T08:14:33.832 INFO:tasks.rados.rados.0.smithi118.stdout:3817: left oid 2 (ObjNum 1168 snap 377 seq_num 1168) 2024-03-20T08:14:33.832 INFO:tasks.rados.rados.0.smithi118.stdout:3818: expect (ObjNum 1139 snap 367 seq_num 1139) 2024-03-20T08:14:33.901 INFO:tasks.rados.rados.0.smithi118.stdout:3821: expect (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:14:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:14:33] "GET /metrics HTTP/1.1" 200 36407 "" "Prometheus/2.43.0" 2024-03-20T08:14:33.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:33.724+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:33.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:33.719+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:33.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:33.719+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:33.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:33.720+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:33.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:33.723+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:33.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:33.725+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:33.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:33.725+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:33.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:33.730+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:33.924 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:33.729+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:33.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: pgmap v917: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 590 KiB/s wr, 5 op/s 2024-03-20T08:14:33.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:33.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:33.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:33.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:33.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: osdmap e819: 8 total, 8 up, 8 in 2024-03-20T08:14:33.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: pgmap v917: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 590 KiB/s wr, 5 op/s 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: osdmap e819: 8 total, 8 up, 8 in 2024-03-20T08:14:33.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:33.926 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:33.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:33.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:33.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:33.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:33.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:33.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:33 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:34.059 INFO:tasks.rados.rados.0.smithi118.stdout:3822: expect (ObjNum 1060 snap 338 seq_num 1060) 2024-03-20T08:14:34.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:33.722+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:34.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:33.732+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:34.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:33.732+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:34.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:33.732+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:34.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:33.728+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:34.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:33.728+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:34.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:33.731+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:34.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:33.729+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:34.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:33.729+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:34.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: pgmap v917: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 19 MiB/s rd, 590 KiB/s wr, 5 op/s 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: osdmap e819: 8 total, 8 up, 8 in 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:34.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:34.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:34.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:34.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:33 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:35.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:35.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: osdmap e820: 8 total, 8 up, 8 in 2024-03-20T08:14:35.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:35.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:35.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:35.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:34 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:35.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:35.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: osdmap e820: 8 total, 8 up, 8 in 2024-03-20T08:14:35.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:35.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:35.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: osdmap e820: 8 total, 8 up, 8 in 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:35.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:35.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:35.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:35.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:35.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:35.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:35.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:35.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:34 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:35.337 INFO:tasks.rados.rados.0.smithi118.stdout:3817: done (6 left) 2024-03-20T08:14:35.337 INFO:tasks.rados.rados.0.smithi118.stdout:3818: done (5 left) 2024-03-20T08:14:35.337 INFO:tasks.rados.rados.0.smithi118.stdout:3820: done (4 left) 2024-03-20T08:14:35.337 INFO:tasks.rados.rados.0.smithi118.stdout:3821: done (3 left) 2024-03-20T08:14:35.337 INFO:tasks.rados.rados.0.smithi118.stdout:3822: done (2 left) 2024-03-20T08:14:35.338 INFO:tasks.rados.rados.0.smithi118.stdout:3823: done (1 left) 2024-03-20T08:14:35.338 INFO:tasks.rados.rados.0.smithi118.stdout:3824: done (0 left) 2024-03-20T08:14:35.338 INFO:tasks.rados.rados.0.smithi118.stdout:3825: snap_remove snap 366 2024-03-20T08:14:35.731 INFO:tasks.rados.rados.0.smithi118.stdout:3825: done (0 left) 2024-03-20T08:14:35.731 INFO:tasks.rados.rados.0.smithi118.stdout:3826: setattr oid 36 current snap is 377 2024-03-20T08:14:35.733 INFO:tasks.rados.rados.0.smithi118.stdout:3827: delete oid 33 current snap is 377 2024-03-20T08:14:35.742 INFO:tasks.rados.rados.0.smithi118.stdout:3827: done (1 left) 2024-03-20T08:14:35.742 INFO:tasks.rados.rados.0.smithi118.stdout:3828: read oid 49 snap -1 2024-03-20T08:14:35.742 INFO:tasks.rados.rados.0.smithi118.stdout:3828: expect (ObjNum 1133 snap 364 seq_num 1133) 2024-03-20T08:14:35.742 INFO:tasks.rados.rados.0.smithi118.stdout:3829: rollback oid 21 current snap is 377 2024-03-20T08:14:35.743 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 21 to 376 2024-03-20T08:14:35.743 INFO:tasks.rados.rados.0.smithi118.stdout:3830: read oid 4 snap -1 2024-03-20T08:14:35.743 INFO:tasks.rados.rados.0.smithi118.stdout:3830: expect (ObjNum 1144 snap 370 seq_num 1144) 2024-03-20T08:14:35.743 INFO:tasks.rados.rados.0.smithi118.stdout:3831: read oid 40 snap -1 2024-03-20T08:14:35.743 INFO:tasks.rados.rados.0.smithi118.stdout:3831: expect deleted 2024-03-20T08:14:35.743 INFO:tasks.rados.rados.0.smithi118.stdout:3832: snap_create 2024-03-20T08:14:35.743 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 65312 (ObjNum 1144 snap 370 seq_num 1144) dirty exists 2024-03-20T08:14:35.756 INFO:tasks.rados.rados.0.smithi118.stdout:3829: finishing rollback tid 0 to smithi11881747-21 2024-03-20T08:14:35.756 INFO:tasks.rados.rados.0.smithi118.stdout:3829: finishing rollback tid 1 to smithi11881747-21 2024-03-20T08:14:35.757 INFO:tasks.rados.rados.0.smithi118.stdout:3830: expect (ObjNum 1144 snap 370 seq_num 1144) 2024-03-20T08:14:35.825 INFO:tasks.rados.rados.0.smithi118.stdout:3829: finishing rollback tid 2 to smithi11881747-21 2024-03-20T08:14:35.825 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 65331 (ObjNum 1162 snap 376 seq_num 1162) dirty exists 2024-03-20T08:14:35.825 INFO:tasks.rados.rados.0.smithi118.stdout:3828: expect (ObjNum 1133 snap 364 seq_num 1133) 2024-03-20T08:14:36.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-mon[87080]: pgmap v920: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 5.1 MiB/s rd, 856 KiB/s wr, 10 op/s 2024-03-20T08:14:36.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:35.734+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:36.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:35.775+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:36.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:35.775+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:36.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:35.746+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:36.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:35.749+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:36.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:35.749+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:36.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:35.741+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:36.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:35.741+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:36.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-mon[93121]: pgmap v920: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 5.1 MiB/s rd, 856 KiB/s wr, 10 op/s 2024-03-20T08:14:36.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:35.737+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:36.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:35.742+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:36.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-mon[95692]: pgmap v920: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 5.1 MiB/s rd, 856 KiB/s wr, 10 op/s 2024-03-20T08:14:36.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:35.734+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:36.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:35.734+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:36.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:35.736+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:36.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:35.737+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:36.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:35.737+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:36.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:35.742+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:36.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:35.742+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:36.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:35 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:35.732+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:36.739 INFO:tasks.rados.rados.0.smithi118.stdout:3826: done (5 left) 2024-03-20T08:14:36.739 INFO:tasks.rados.rados.0.smithi118.stdout:3828: done (4 left) 2024-03-20T08:14:36.739 INFO:tasks.rados.rados.0.smithi118.stdout:3829: done (3 left) 2024-03-20T08:14:36.739 INFO:tasks.rados.rados.0.smithi118.stdout:3830: done (2 left) 2024-03-20T08:14:36.739 INFO:tasks.rados.rados.0.smithi118.stdout:3831: done (1 left) 2024-03-20T08:14:36.739 INFO:tasks.rados.rados.0.smithi118.stdout:3832: done (0 left) 2024-03-20T08:14:36.739 INFO:tasks.rados.rados.0.smithi118.stdout:3833: snap_remove snap 377 2024-03-20T08:14:37.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:36.739+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:37.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:36.813+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:37.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:36.813+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:37.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:36.813+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: osdmap e821: 8 total, 8 up, 8 in 2024-03-20T08:14:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:36 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:37.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:36.739+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:37.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:36.737+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:37.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:36.738+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:37.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:36.738+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: osdmap e821: 8 total, 8 up, 8 in 2024-03-20T08:14:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:37.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: osdmap e821: 8 total, 8 up, 8 in 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:37.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:37.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:36 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:37.743 INFO:tasks.rados.rados.0.smithi118.stdout:3833: done (0 left) 2024-03-20T08:14:37.743 INFO:tasks.rados.rados.0.smithi118.stdout:3834: read oid 42 snap 372 2024-03-20T08:14:37.743 INFO:tasks.rados.rados.0.smithi118.stdout:3834: expect deleted 2024-03-20T08:14:37.743 INFO:tasks.rados.rados.0.smithi118.stdout:3835: snap_remove snap 374 2024-03-20T08:14:37.801 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:37.743+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:37.801 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:37.744+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:37.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.496+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:37.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.496+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:37.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.496+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:37.801 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.496+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:37.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[93121]: pgmap v922: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 26 op/s 2024-03-20T08:14:37.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:37.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:37.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:37.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[93121]: osdmap e822: 8 total, 8 up, 8 in 2024-03-20T08:14:37.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:37.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:37.802 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:37.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[95692]: pgmap v922: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 26 op/s 2024-03-20T08:14:37.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:37.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:37.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:37.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[95692]: osdmap e822: 8 total, 8 up, 8 in 2024-03-20T08:14:37.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:37.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:37.803 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:37.803 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:37.579+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:37.804 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:37.742+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:37.804 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:37.742+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:38.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:37.743+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:38.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:37.802+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:38.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:37.803+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:38.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:37.803+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-mon[87080]: pgmap v922: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 26 op/s 2024-03-20T08:14:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-mon[87080]: osdmap e822: 8 total, 8 up, 8 in 2024-03-20T08:14:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:38.141 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:37.803+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:38.141 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:37.803+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:38.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.801+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:38.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.801+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:38.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.801+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:38.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.801+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:38.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:37 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:37.801+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:38.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:38.303+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:38.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:38.303+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:38.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:38.302+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:38.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:38.302+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:38.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:38.302+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:38.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:38.302+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:38.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:38.303+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:38.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:38.301+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:38.749 INFO:tasks.rados.rados.0.smithi118.stdout:3835: done (1 left) 2024-03-20T08:14:38.749 INFO:tasks.rados.rados.0.smithi118.stdout:3836: rollback oid 46 current snap is 378 2024-03-20T08:14:38.750 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 375 2024-03-20T08:14:38.750 INFO:tasks.rados.rados.0.smithi118.stdout:3834: done (1 left) 2024-03-20T08:14:38.750 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 31 current snap is 378 2024-03-20T08:14:38.750 INFO:tasks.rados.rados.0.smithi118.stdout:3837: seq_num 1171 ranges {2236416=417792} 2024-03-20T08:14:38.764 INFO:tasks.rados.rados.0.smithi118.stdout:3837: writing smithi11881747-31 from 2236416 to 2654208 tid 1 2024-03-20T08:14:38.764 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 37 current snap is 378 2024-03-20T08:14:38.764 INFO:tasks.rados.rados.0.smithi118.stdout:3836: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:14:38.764 INFO:tasks.rados.rados.0.smithi118.stdout:3838: seq_num 1172 ranges {0=434176} 2024-03-20T08:14:38.767 INFO:tasks.rados.rados.0.smithi118.stdout:3838: writing smithi11881747-37 from 0 to 434176 tid 1 2024-03-20T08:14:38.767 INFO:tasks.rados.rados.0.smithi118.stdout:3839: read oid 5 snap -1 2024-03-20T08:14:38.767 INFO:tasks.rados.rados.0.smithi118.stdout:3839: expect deleted 2024-03-20T08:14:38.767 INFO:tasks.rados.rados.0.smithi118.stdout:3840: snap_create 2024-03-20T08:14:38.767 INFO:tasks.rados.rados.0.smithi118.stdout:3836: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:14:38.767 INFO:tasks.rados.rados.0.smithi118.stdout:3836: finishing rollback tid 2 to smithi11881747-46 2024-03-20T08:14:38.767 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 65325 (ObjNum 1148 snap 370 seq_num 1148) dirty exists 2024-03-20T08:14:38.771 INFO:tasks.rados.rados.0.smithi118.stdout:3837: finishing write tid 1 to smithi11881747-31 2024-03-20T08:14:38.771 INFO:tasks.rados.rados.0.smithi118.stdout:3837: finishing write tid 2 to smithi11881747-31 2024-03-20T08:14:38.773 INFO:tasks.rados.rados.0.smithi118.stdout:3837: finishing write tid 3 to smithi11881747-31 2024-03-20T08:14:38.773 INFO:tasks.rados.rados.0.smithi118.stdout:3837: oid 31 updating version 0 to 65324 2024-03-20T08:14:38.773 INFO:tasks.rados.rados.0.smithi118.stdout:3837: oid 31 updating version 65324 to 65325 2024-03-20T08:14:38.773 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 65325 (ObjNum 1171 snap 378 seq_num 1171) dirty exists 2024-03-20T08:14:38.773 INFO:tasks.rados.rados.0.smithi118.stdout:3837: left oid 31 (ObjNum 1171 snap 378 seq_num 1171) 2024-03-20T08:14:38.777 INFO:tasks.rados.rados.0.smithi118.stdout:3838: finishing write tid 1 to smithi11881747-37 2024-03-20T08:14:38.778 INFO:tasks.rados.rados.0.smithi118.stdout:3838: finishing write tid 2 to smithi11881747-37 2024-03-20T08:14:38.782 INFO:tasks.rados.rados.0.smithi118.stdout:3838: finishing write tid 3 to smithi11881747-37 2024-03-20T08:14:38.782 INFO:tasks.rados.rados.0.smithi118.stdout:3838: oid 37 updating version 0 to 65326 2024-03-20T08:14:38.782 INFO:tasks.rados.rados.0.smithi118.stdout:3838: oid 37 version 65326 is already newer than 65325 2024-03-20T08:14:38.782 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 65326 (ObjNum 1172 snap 378 seq_num 1172) dirty exists 2024-03-20T08:14:38.782 INFO:tasks.rados.rados.0.smithi118.stdout:3838: left oid 37 (ObjNum 1172 snap 378 seq_num 1172) 2024-03-20T08:14:39.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:38.748+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:39.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:38.753+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:39.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:38.753+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:39.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:38.754+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:39.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:38.772+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:39.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:38.772+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:39.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:38.754+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:39.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:38.754+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:39.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:38.758+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: osdmap e823: 8 total, 8 up, 8 in 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:39.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:38 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:39.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:38.748+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:39.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:38.748+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:39.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:38.749+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:39.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:38.770+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:39.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:38.770+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:39.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:38.771+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:39.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:38.778+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:39.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:39.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:39.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: osdmap e823: 8 total, 8 up, 8 in 2024-03-20T08:14:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:39.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: osdmap e823: 8 total, 8 up, 8 in 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:39.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:39.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:39.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:39.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:39.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:39.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:39.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:39.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:39.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:39.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:39.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:39.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:39.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:39.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:39.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:39.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:39.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:39.180 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:38.749+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:39.180 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:38.757+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:39.754 INFO:tasks.rados.rados.0.smithi118.stdout:3836: done (4 left) 2024-03-20T08:14:39.754 INFO:tasks.rados.rados.0.smithi118.stdout:3837: done (3 left) 2024-03-20T08:14:39.754 INFO:tasks.rados.rados.0.smithi118.stdout:3838: done (2 left) 2024-03-20T08:14:39.754 INFO:tasks.rados.rados.0.smithi118.stdout:3839: done (1 left) 2024-03-20T08:14:39.754 INFO:tasks.rados.rados.0.smithi118.stdout:3840: done (0 left) 2024-03-20T08:14:39.755 INFO:tasks.rados.rados.0.smithi118.stdout:3841: delete oid 39 current snap is 379 2024-03-20T08:14:39.758 INFO:tasks.rados.rados.0.smithi118.stdout:3841: done (0 left) 2024-03-20T08:14:39.758 INFO:tasks.rados.rados.0.smithi118.stdout:3842: read oid 44 snap 378 2024-03-20T08:14:39.758 INFO:tasks.rados.rados.0.smithi118.stdout:3842: expect (ObjNum 1108 snap 355 seq_num 1108) 2024-03-20T08:14:39.758 INFO:tasks.rados.rados.0.smithi118.stdout:3843: read oid 10 snap -1 2024-03-20T08:14:39.758 INFO:tasks.rados.rados.0.smithi118.stdout:3843: expect (ObjNum 1160 snap 375 seq_num 1160) 2024-03-20T08:14:39.758 INFO:tasks.rados.rados.0.smithi118.stdout:3844: setattr oid 43 current snap is 379 2024-03-20T08:14:39.759 INFO:tasks.rados.rados.0.smithi118.stdout:3845: copy_from oid 18 from oid 5 current snap is 379 2024-03-20T08:14:39.759 INFO:tasks.rados.rados.0.smithi118.stdout:3846: rmattr oid 12 current snap is 379 2024-03-20T08:14:39.759 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 379 2024-03-20T08:14:39.759 INFO:tasks.rados.rados.0.smithi118.stdout:3847: seq_num 1174 ranges {0=466944} 2024-03-20T08:14:39.762 INFO:tasks.rados.rados.0.smithi118.stdout:3847: writing smithi11881747-28 from 0 to 466944 tid 1 2024-03-20T08:14:39.762 INFO:tasks.rados.rados.0.smithi118.stdout:3848: copy_from oid 50 from oid 3 current snap is 379 2024-03-20T08:14:39.762 INFO:tasks.rados.rados.0.smithi118.stdout:3849: setattr oid 42 current snap is 379 2024-03-20T08:14:39.764 INFO:tasks.rados.rados.0.smithi118.stdout:3850: rollback oid 16 current snap is 379 2024-03-20T08:14:39.764 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 16 to 368 2024-03-20T08:14:39.764 INFO:tasks.rados.rados.0.smithi118.stdout:3845: finishing copy_from to smithi11881747-18 2024-03-20T08:14:39.764 INFO:tasks.rados.rados.0.smithi118.stdout:3845: got expected ENOENT (src dne) 2024-03-20T08:14:39.764 INFO:tasks.rados.rados.0.smithi118.stdout:3851: read oid 2 snap 357 2024-03-20T08:14:39.765 INFO:tasks.rados.rados.0.smithi118.stdout:3851: expect deleted 2024-03-20T08:14:39.765 INFO:tasks.rados.rados.0.smithi118.stdout:3852: delete oid 19 current snap is 379 2024-03-20T08:14:39.765 INFO:tasks.rados.rados.0.smithi118.stdout:3845: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:14:39.768 INFO:tasks.rados.rados.0.smithi118.stdout:3843: expect (ObjNum 1160 snap 375 seq_num 1160) 2024-03-20T08:14:39.777 INFO:tasks.rados.rados.0.smithi118.stdout:3842: expect (ObjNum 1108 snap 355 seq_num 1108) 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65313 (ObjNum 1161 snap 376 seq_num 1161) dirty exists 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:3847: finishing write tid 1 to smithi11881747-28 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:3847: finishing write tid 2 to smithi11881747-28 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65345 (ObjNum 1154 snap 371 seq_num 1154) dirty exists 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:3850: finishing rollback tid 0 to smithi11881747-16 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:3850: finishing rollback tid 1 to smithi11881747-16 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65326 (ObjNum 1164 snap 376 seq_num 1164) dirty exists 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:3850: finishing rollback tid 2 to smithi11881747-16 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 65352 (ObjNum 1040 snap 336 seq_num 1040) dirty exists 2024-03-20T08:14:39.848 INFO:tasks.rados.rados.0.smithi118.stdout:3847: finishing write tid 3 to smithi11881747-28 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3847: oid 28 updating version 0 to 65349 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3847: oid 28 version 65349 is already newer than 65348 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 65349 (ObjNum 1174 snap 379 seq_num 1174) dirty exists 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3847: left oid 28 (ObjNum 1174 snap 379 seq_num 1174) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3842: done (10 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3843: done (9 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3844: done (8 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3845: done (7 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3846: done (6 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3847: done (5 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3849: done (4 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3850: done (3 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3851: done (2 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3852: done (1 left) 2024-03-20T08:14:39.849 INFO:tasks.rados.rados.0.smithi118.stdout:3853: snap_remove snap 368 2024-03-20T08:14:40.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:39.754+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:40.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:39.758+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:40.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:39.758+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:40.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:39.759+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:40.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:39.774+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:40.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:39.775+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:40.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:39.769+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:40.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:39.771+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: pgmap v925: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 4.0 MiB/s wr, 26 op/s 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: osdmap e824: 8 total, 8 up, 8 in 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:40.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:40.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:40.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:40.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:40.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:40.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:39.764+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:40.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:39.764+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:40.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:39.754+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:40.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:39.755+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:40.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:39.755+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:40.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:39.761+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:40.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:39.762+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:40.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:39.762+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:40.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:39.763+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:40.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:39.770+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: pgmap v925: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 4.0 MiB/s wr, 26 op/s 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:40.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: osdmap e824: 8 total, 8 up, 8 in 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ad=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: pgmap v925: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 15 MiB/s rd, 4.0 MiB/s wr, 26 op/s 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: osdmap e824: 8 total, 8 up, 8 in 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:40.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:40.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:40.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:40.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:3853: done (1 left) 2024-03-20T08:14:40.760 INFO:tasks.rados.rados.0.smithi118.stdout:3854: snap_remove snap 378 2024-03-20T08:14:41.102 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:40.759+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:41.102 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:40.794+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:41.102 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:40.794+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:41.102 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:40.794+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:41.103 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:40.759+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:41.103 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:40.760+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:41.103 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:40.760+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:41.103 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:40.760+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:41.103 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: osdmap e825: 8 total, 8 up, 8 in 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:41.104 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: osdmap e825: 8 total, 8 up, 8 in 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:41.105 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:41.106 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:41.106 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:41.106 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:41.106 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:41.106 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:41.106 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:41.106 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:41.106 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:40 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:41.107 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:40.759+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: osdmap e825: 8 total, 8 up, 8 in 2024-03-20T08:14:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:41.107 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a3=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:41.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:41.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:41.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:41.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:41.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:41.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:41.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:41.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:40 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:41.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:41.105+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:41.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:41.105+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:41.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:41.103+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:41.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:41.103+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:41.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:41.103+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:41.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:41.104+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:41.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:41.104+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:41.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:41.104+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:41.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:41.102+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:41.761 INFO:tasks.rados.rados.0.smithi118.stdout:3854: done (1 left) 2024-03-20T08:14:41.761 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 16 current snap is 379 2024-03-20T08:14:41.761 INFO:tasks.rados.rados.0.smithi118.stdout:3848: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:14:41.761 INFO:tasks.rados.rados.0.smithi118.stdout:3855: seq_num 1176 ranges {11460608=499712} 2024-03-20T08:14:41.826 INFO:tasks.rados.rados.0.smithi118.stdout:3855: writing smithi11881747-16 from 11460608 to 11960320 tid 1 2024-03-20T08:14:41.826 INFO:tasks.rados.rados.0.smithi118.stdout:3856: setattr oid 45 current snap is 379 2024-03-20T08:14:41.826 INFO:tasks.rados.rados.0.smithi118.stdout:3848: finishing copy_from to smithi11881747-50 2024-03-20T08:14:41.826 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 34065 (ObjNum 65 snap 0 seq_num 1801941333) dirty exists 2024-03-20T08:14:41.827 INFO:tasks.rados.rados.0.smithi118.stdout:3848: done (2 left) 2024-03-20T08:14:41.827 INFO:tasks.rados.rados.0.smithi118.stdout:3857: read oid 21 snap -1 2024-03-20T08:14:41.828 INFO:tasks.rados.rados.0.smithi118.stdout:3857: expect (ObjNum 1162 snap 376 seq_num 1162) 2024-03-20T08:14:41.828 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 14 current snap is 379 2024-03-20T08:14:41.828 INFO:tasks.rados.rados.0.smithi118.stdout:3858: seq_num 1177 ranges {0=516096} 2024-03-20T08:14:41.831 INFO:tasks.rados.rados.0.smithi118.stdout:3858: writing smithi11881747-14 from 0 to 516096 tid 1 2024-03-20T08:14:41.831 INFO:tasks.rados.rados.0.smithi118.stdout:3859: snap_create 2024-03-20T08:14:41.834 INFO:tasks.rados.rados.0.smithi118.stdout:3855: finishing write tid 1 to smithi11881747-16 2024-03-20T08:14:41.834 INFO:tasks.rados.rados.0.smithi118.stdout:3855: finishing write tid 2 to smithi11881747-16 2024-03-20T08:14:41.836 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 45 v 65327 (ObjNum 1163 snap 376 seq_num 1163) dirty exists 2024-03-20T08:14:41.839 INFO:tasks.rados.rados.0.smithi118.stdout:3855: finishing write tid 3 to smithi11881747-16 2024-03-20T08:14:41.839 INFO:tasks.rados.rados.0.smithi118.stdout:3855: oid 16 updating version 0 to 65354 2024-03-20T08:14:41.839 INFO:tasks.rados.rados.0.smithi118.stdout:3855: oid 16 version 65354 is already newer than 65353 2024-03-20T08:14:41.839 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 65354 (ObjNum 1176 snap 379 seq_num 1176) dirty exists 2024-03-20T08:14:41.839 INFO:tasks.rados.rados.0.smithi118.stdout:3855: left oid 16 (ObjNum 1176 snap 379 seq_num 1176) 2024-03-20T08:14:41.842 INFO:tasks.rados.rados.0.smithi118.stdout:3858: finishing write tid 1 to smithi11881747-14 2024-03-20T08:14:41.842 INFO:tasks.rados.rados.0.smithi118.stdout:3858: finishing write tid 2 to smithi11881747-14 2024-03-20T08:14:41.843 INFO:tasks.rados.rados.0.smithi118.stdout:3857: expect (ObjNum 1162 snap 376 seq_num 1162) 2024-03-20T08:14:41.954 INFO:tasks.rados.rados.0.smithi118.stdout:3858: finishing write tid 3 to smithi11881747-14 2024-03-20T08:14:41.954 INFO:tasks.rados.rados.0.smithi118.stdout:3858: oid 14 updating version 0 to 34142 2024-03-20T08:14:41.954 INFO:tasks.rados.rados.0.smithi118.stdout:3858: oid 14 version 34142 is already newer than 34141 2024-03-20T08:14:41.955 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 14 v 34142 (ObjNum 1177 snap 379 seq_num 1177) dirty exists 2024-03-20T08:14:41.955 INFO:tasks.rados.rados.0.smithi118.stdout:3858: left oid 14 (ObjNum 1177 snap 379 seq_num 1177) 2024-03-20T08:14:42.114 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:41.760+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:42.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:41.833+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:42.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:41.833+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:42.115 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:41.833+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:42.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: pgmap v928: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 184 KiB/s wr, 17 op/s 2024-03-20T08:14:42.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:42.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:42.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:42.115 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: osdmap e826: 8 total, 8 up, 8 in 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:42.116 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:41 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:42.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:41.760+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:42.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:41.761+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:42.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:41.761+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:41.833+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:41.833+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:41.833+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:41.833+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:42.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:41.835+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:42.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:41.835+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:42.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:41.836+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: pgmap v928: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 184 KiB/s wr, 17 op/s 2024-03-20T08:14:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:42.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: osdmap e826: 8 total, 8 up, 8 in 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:42.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: pgmap v928: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 9.3 MiB/s rd, 184 KiB/s wr, 17 op/s 2024-03-20T08:14:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b3=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:42.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: osdmap e826: 8 total, 8 up, 8 in 2024-03-20T08:14:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:42.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:41 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:42.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:14:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:14:42.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:42.114+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:42.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:42.114+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:42.770 INFO:tasks.rados.rados.0.smithi118.stdout:3855: done (4 left) 2024-03-20T08:14:42.770 INFO:tasks.rados.rados.0.smithi118.stdout:3856: done (3 left) 2024-03-20T08:14:42.770 INFO:tasks.rados.rados.0.smithi118.stdout:3857: done (2 left) 2024-03-20T08:14:42.771 INFO:tasks.rados.rados.0.smithi118.stdout:3858: done (1 left) 2024-03-20T08:14:42.771 INFO:tasks.rados.rados.0.smithi118.stdout:3859: done (0 left) 2024-03-20T08:14:42.771 INFO:tasks.rados.rados.0.smithi118.stdout:3860: setattr oid 16 current snap is 380 2024-03-20T08:14:42.771 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:42.517+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:42.771 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:42.517+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:42.772 INFO:tasks.rados.rados.0.smithi118.stdout:3861: read oid 45 snap -1 2024-03-20T08:14:42.772 INFO:tasks.rados.rados.0.smithi118.stdout:3861: expect (ObjNum 1163 snap 376 seq_num 1163) 2024-03-20T08:14:42.772 INFO:tasks.rados.rados.0.smithi118.stdout:3862: rmattr oid 43 current snap is 380 2024-03-20T08:14:42.773 INFO:tasks.rados.rados.0.smithi118.stdout:3863: rmattr oid 1 current snap is 380 2024-03-20T08:14:42.773 INFO:tasks.rados.rados.0.smithi118.stdout:3863: done (3 left) 2024-03-20T08:14:42.773 INFO:tasks.rados.rados.0.smithi118.stdout:3864: snap_create 2024-03-20T08:14:42.777 INFO:tasks.rados.rados.0.smithi118.stdout:3861: expect (ObjNum 1163 snap 376 seq_num 1163) 2024-03-20T08:14:42.788 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 65355 (ObjNum 1176 snap 379 seq_num 1176) dirty exists 2024-03-20T08:14:42.788 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65327 (ObjNum 1164 snap 376 seq_num 1164) dirty exists 2024-03-20T08:14:43.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:42.771+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:43.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:42.780+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:43.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:42.783+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:43.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:42.786+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:43.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:42.787+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:43.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:42.787+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:43.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:42.777+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:43.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:42.777+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:43.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:42.777+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: osdmap e827: 8 total, 8 up, 8 in 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:43.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:43.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:43.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:42 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:43.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:42.778+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:43.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:42.778+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:43.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:42.778+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:43.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:42.773+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:43.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:42.879+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:43.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:43.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: osdmap e827: 8 total, 8 up, 8 in 2024-03-20T08:14:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:43.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:43.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:42.769+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:43.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:42.769+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:42.770+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:43.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:42.770+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:43.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:43.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:43.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 17e=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:43.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:43.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: osdmap e827: 8 total, 8 up, 8 in 2024-03-20T08:14:43.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:43.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:43.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:43.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:43.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:42 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:3860: done (3 left) 2024-03-20T08:14:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:3861: done (2 left) 2024-03-20T08:14:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:3862: done (1 left) 2024-03-20T08:14:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:3864: done (0 left) 2024-03-20T08:14:43.776 INFO:tasks.rados.rados.0.smithi118.stdout:3865: snap_remove snap 350 2024-03-20T08:14:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:14:43] "GET /metrics HTTP/1.1" 200 36413 "" "Prometheus/2.43.0" 2024-03-20T08:14:43.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:43.777+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:43.923 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:43.777+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:43.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:43.776+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:43.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:43.776+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:43.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:43.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:43.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: pgmap v931: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 228 KiB/s wr, 16 op/s 2024-03-20T08:14:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:43.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: osdmap e828: 8 total, 8 up, 8 in 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: pgmap v931: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 228 KiB/s wr, 16 op/s 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:43.925 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: osdmap e828: 8 total, 8 up, 8 in 2024-03-20T08:14:43.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:43.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:43.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:43.929 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:43.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:43.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:43.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:43.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:43.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:43.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:43.930 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:43.931 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:43.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:43.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:43.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:43.932 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:44.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:43.775+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:44.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:43.813+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:44.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:43.813+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: pgmap v931: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 228 KiB/s wr, 16 op/s 2024-03-20T08:14:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:44.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: osdmap e828: 8 total, 8 up, 8 in 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:44.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:44.777 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:44.630+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:44.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:44.448+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:44.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:44.448+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:44.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:44.449+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:44.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:44.449+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:44.778 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:44.449+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:44.780 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:44.518+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:44.780 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:44.518+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:44.780 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:44.519+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:44.781 INFO:tasks.rados.rados.0.smithi118.stdout:3865: done (0 left) 2024-03-20T08:14:44.781 INFO:tasks.rados.rados.0.smithi118.stdout:3866: snap_remove snap 365 2024-03-20T08:14:45.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:44.780+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:45.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:44.783+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:45.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:44.783+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:45.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-mon[87080]: osdmap e829: 8 total, 8 up, 8 in 2024-03-20T08:14:45.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:45.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:45.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:45.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:45.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:45.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:44 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:45.143 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[93121]: osdmap e829: 8 total, 8 up, 8 in 2024-03-20T08:14:45.143 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:45.144 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:45.144 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:45.144 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:45.144 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:45.144 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:45.144 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:44.780+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:45.144 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:44.780+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:45.144 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:44.780+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:45.144 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[95692]: osdmap e829: 8 total, 8 up, 8 in 2024-03-20T08:14:45.145 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:45.145 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:45.145 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:45.145 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:45.145 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:45.145 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:44 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:45.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:45.144+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:45.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:45.153+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:45.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:45.153+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:45.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:45.153+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.441+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.442+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.442+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:45.745 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.442+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:45.781 INFO:tasks.rados.rados.0.smithi118.stdout:3866: done (0 left) 2024-03-20T08:14:45.782 INFO:tasks.rados.rados.0.smithi118.stdout:3867: snap_create 2024-03-20T08:14:46.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:45.781+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:46.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:45.782+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:46.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:45.782+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:46.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:45.939+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:46.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:45.939+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:46.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:45.941+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:46.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:45.744+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:46.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:45.745+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:46.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:45.745+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:46.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:45.940+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:46.142 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:45.940+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:46.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:46.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:46.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: pgmap v934: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.7 MiB/s rd, 468 KiB/s wr, 24 op/s 2024-03-20T08:14:46.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:46.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:46.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:46.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:46.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: osdmap e830: 8 total, 8 up, 8 in 2024-03-20T08:14:46.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:46.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:46.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:46.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:46.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:45 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:46.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:45.865+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:46.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:45.865+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.780+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.781+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.781+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.781+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:46.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:45.781+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:46.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:46.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: pgmap v934: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.7 MiB/s rd, 468 KiB/s wr, 24 op/s 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: osdmap e830: 8 total, 8 up, 8 in 2024-03-20T08:14:46.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:46.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:45.745+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:46.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:45.940+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:46.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:45.940+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:46.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:45.940+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:46.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:46.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:46.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:46.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:46.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: pgmap v934: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.7 MiB/s rd, 468 KiB/s wr, 24 op/s 2024-03-20T08:14:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:46.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:46.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:46.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:46.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: osdmap e830: 8 total, 8 up, 8 in 2024-03-20T08:14:46.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:46.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:46.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:46.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:46.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:45 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:46.793 INFO:tasks.rados.rados.0.smithi118.stdout:3867: done (0 left) 2024-03-20T08:14:46.793 INFO:tasks.rados.rados.0.smithi118.stdout:3868: read oid 36 snap 380 2024-03-20T08:14:46.793 INFO:tasks.rados.rados.0.smithi118.stdout:3868: expect (ObjNum 1144 snap 370 seq_num 1144) 2024-03-20T08:14:46.794 INFO:tasks.rados.rados.0.smithi118.stdout:3869: rollback oid 26 current snap is 382 2024-03-20T08:14:46.794 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 361 2024-03-20T08:14:46.794 INFO:tasks.rados.rados.0.smithi118.stdout:3870: read oid 28 snap 369 2024-03-20T08:14:46.794 INFO:tasks.rados.rados.0.smithi118.stdout:3870: expect (ObjNum 1099 snap 352 seq_num 1099) 2024-03-20T08:14:46.794 INFO:tasks.rados.rados.0.smithi118.stdout:3871: snap_remove snap 371 2024-03-20T08:14:47.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:46.791+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:47.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:46.844+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:47.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:46.845+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:47.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:46.845+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:47.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:46.797+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:47.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:46.797+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:47.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:46.792+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:47.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:46.793+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:47.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:46.793+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:47.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: osdmap e831: 8 total, 8 up, 8 in 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:47.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:46 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:47.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:46.797+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:47.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:46.801+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:47.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:46.801+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:46.796+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:46.797+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:46.798+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:47.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:46.799+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:47.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:47.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:47.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:46.791+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:47.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:46.801+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:47.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:47.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: osdmap e831: 8 total, 8 up, 8 in 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:47.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: osdmap e831: 8 total, 8 up, 8 in 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:47.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:46 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:47.797 INFO:tasks.rados.rados.0.smithi118.stdout:3871: done (3 left) 2024-03-20T08:14:47.797 INFO:tasks.rados.rados.0.smithi118.stdout:3872: snap_create 2024-03-20T08:14:47.797 INFO:tasks.rados.rados.0.smithi118.stdout:3870: expect (ObjNum 1099 snap 352 seq_num 1099) 2024-03-20T08:14:47.848 INFO:tasks.rados.rados.0.smithi118.stdout:3869: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:14:47.848 INFO:tasks.rados.rados.0.smithi118.stdout:3868: expect (ObjNum 1144 snap 370 seq_num 1144) 2024-03-20T08:14:47.922 INFO:tasks.rados.rados.0.smithi118.stdout:3869: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:14:47.922 INFO:tasks.rados.rados.0.smithi118.stdout:3869: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:14:47.922 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65350 (ObjNum 1130 snap 361 seq_num 1130) dirty exists 2024-03-20T08:14:48.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:47.796+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:48.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: pgmap v937: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.5 MiB/s rd, 342 KiB/s wr, 26 op/s 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: osdmap e832: 8 total, 8 up, 8 in 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:48.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:48.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:48.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:48.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:48.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:48.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:47 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:48.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:47.798+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:48.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:47.798+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:48.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:47.797+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:48.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:48.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:48.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: pgmap v937: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.5 MiB/s rd, 342 KiB/s wr, 26 op/s 2024-03-20T08:14:48.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:48.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:48.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: osdmap e832: 8 total, 8 up, 8 in 2024-03-20T08:14:48.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:48.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: pgmap v937: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.5 MiB/s rd, 342 KiB/s wr, 26 op/s 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: osdmap e832: 8 total, 8 up, 8 in 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:48.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:48.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:47 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:3868: done (3 left) 2024-03-20T08:14:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:3869: done (2 left) 2024-03-20T08:14:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:3870: done (1 left) 2024-03-20T08:14:48.799 INFO:tasks.rados.rados.0.smithi118.stdout:3872: done (0 left) 2024-03-20T08:14:48.800 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 383 2024-03-20T08:14:48.800 INFO:tasks.rados.rados.0.smithi118.stdout:3873: seq_num 1178 ranges {466944=532480} 2024-03-20T08:14:48.805 INFO:tasks.rados.rados.0.smithi118.stdout:3873: writing smithi11881747-28 from 466944 to 999424 tid 1 2024-03-20T08:14:48.805 INFO:tasks.rados.rados.0.smithi118.stdout:3874: snap_remove snap 357 2024-03-20T08:14:48.812 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-mon[87080]: osdmap e833: 8 total, 8 up, 8 in 2024-03-20T08:14:48.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:48.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:48.813 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:48.813 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:48.781+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:48.813 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:48.781+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:48.813 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:48.554+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:48.554+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:48.555+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:48.812+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:48.812+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:48.781+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:48.781+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:48.798+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:48.798+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:48.814 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:48.798+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:49.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:48.812+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-mon[93121]: osdmap e833: 8 total, 8 up, 8 in 2024-03-20T08:14:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:49.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:49.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-mon[95692]: osdmap e833: 8 total, 8 up, 8 in 2024-03-20T08:14:49.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:49.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:49.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:49.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:48.780+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:49.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:48.780+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:49.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:48.798+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:49.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:48.799+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:49.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:48.799+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:49.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:48.800+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:49.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.779+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:49.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.780+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:49.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.781+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:49.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.781+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:49.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.781+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:49.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.808+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:49.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.809+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:49.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.809+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:49.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:48 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:48.811+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:49.810 INFO:tasks.rados.rados.0.smithi118.stdout:3874: done (1 left) 2024-03-20T08:14:49.810 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 9 current snap is 383 2024-03-20T08:14:49.810 INFO:tasks.rados.rados.0.smithi118.stdout:3875: seq_num 1179 ranges {212992=548864} 2024-03-20T08:14:49.814 INFO:tasks.rados.rados.0.smithi118.stdout:3875: writing smithi11881747-9 from 212992 to 761856 tid 1 2024-03-20T08:14:49.814 INFO:tasks.rados.rados.0.smithi118.stdout:3876: rollback oid 32 current snap is 383 2024-03-20T08:14:49.814 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 32 to 376 2024-03-20T08:14:49.814 INFO:tasks.rados.rados.0.smithi118.stdout:3873: finishing write tid 1 to smithi11881747-28 2024-03-20T08:14:49.814 INFO:tasks.rados.rados.0.smithi118.stdout:3873: finishing write tid 2 to smithi11881747-28 2024-03-20T08:14:49.815 INFO:tasks.rados.rados.0.smithi118.stdout:3873: finishing write tid 3 to smithi11881747-28 2024-03-20T08:14:49.815 INFO:tasks.rados.rados.0.smithi118.stdout:3873: oid 28 updating version 0 to 65356 2024-03-20T08:14:49.815 INFO:tasks.rados.rados.0.smithi118.stdout:3873: oid 28 updating version 65356 to 65357 2024-03-20T08:14:49.815 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 65357 (ObjNum 1178 snap 383 seq_num 1178) dirty exists 2024-03-20T08:14:49.815 INFO:tasks.rados.rados.0.smithi118.stdout:3873: left oid 28 (ObjNum 1178 snap 383 seq_num 1178) 2024-03-20T08:14:49.815 INFO:tasks.rados.rados.0.smithi118.stdout:3873: done (2 left) 2024-03-20T08:14:49.815 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 42 current snap is 383 2024-03-20T08:14:49.815 INFO:tasks.rados.rados.0.smithi118.stdout:3877: seq_num 1180 ranges {2490368=565248} 2024-03-20T08:14:49.832 INFO:tasks.rados.rados.0.smithi118.stdout:3877: writing smithi11881747-42 from 2490368 to 3055616 tid 1 2024-03-20T08:14:49.832 INFO:tasks.rados.rados.0.smithi118.stdout:3878: copy_from oid 50 from oid 16 current snap is 383 2024-03-20T08:14:49.832 INFO:tasks.rados.rados.0.smithi118.stdout:3875: finishing write tid 1 to smithi11881747-9 2024-03-20T08:14:49.832 INFO:tasks.rados.rados.0.smithi118.stdout:3875: finishing write tid 2 to smithi11881747-9 2024-03-20T08:14:49.832 INFO:tasks.rados.rados.0.smithi118.stdout:3875: finishing write tid 3 to smithi11881747-9 2024-03-20T08:14:49.832 INFO:tasks.rados.rados.0.smithi118.stdout:3875: oid 9 updating version 0 to 65346 2024-03-20T08:14:49.832 INFO:tasks.rados.rados.0.smithi118.stdout:3875: oid 9 updating version 65346 to 65347 2024-03-20T08:14:49.832 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 9 v 65347 (ObjNum 1179 snap 383 seq_num 1179) dirty exists 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3875: left oid 9 (ObjNum 1179 snap 383 seq_num 1179) 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3876: finishing rollback tid 0 to smithi11881747-32 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3876: finishing rollback tid 1 to smithi11881747-32 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3876: finishing rollback tid 2 to smithi11881747-32 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 34145 (ObjNum 1156 snap 372 seq_num 1156) dirty exists 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3875: done (3 left) 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3876: done (2 left) 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3879: read oid 1 snap -1 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3879: expect (ObjNum 1157 snap 372 seq_num 1157) 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3880: read oid 8 snap -1 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3880: expect (ObjNum 1152 snap 371 seq_num 1152) 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 383 2024-03-20T08:14:49.833 INFO:tasks.rados.rados.0.smithi118.stdout:3881: seq_num 1181 ranges {6463488=581632} 2024-03-20T08:14:49.871 INFO:tasks.rados.rados.0.smithi118.stdout:3881: writing smithi11881747-43 from 6463488 to 7045120 tid 1 2024-03-20T08:14:49.872 INFO:tasks.rados.rados.0.smithi118.stdout:3882: snap_remove snap 375 2024-03-20T08:14:49.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:49.811+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:49.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:49.811+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:49.836+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:49.839+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:49.839+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:49.839+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:49.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:49.839+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:49.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:49.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:49.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: pgmap v940: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:49.924 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: pgmap v940: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:49.925 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:49.926 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:49.810+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:49.926 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:49.811+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:49.926 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:49.655+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:49.926 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:49.827+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:49.926 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: osdmap e834: 8 total, 8 up, 8 in 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:49.927 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: osdmap e834: 8 total, 8 up, 8 in 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:49.928 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:49.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:49.929 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:49 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:50.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:49.820+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:50.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:49.820+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:50.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:49.820+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:50.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:49.657+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:50.140 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:49.810+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:50.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:49.657+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:50.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:49.657+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:50.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:49.835+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:50.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:49.839+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: pgmap v940: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 2.4 MiB/s rd, 0 B/s wr, 9 op/s 2024-03-20T08:14:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:50.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: osdmap e834: 8 total, 8 up, 8 in 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:50.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:50.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:50.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:50.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:50.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:50.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:50.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:49 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:50.813 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:50.503+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:50.813 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:50.503+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:50.815 INFO:tasks.rados.rados.0.smithi118.stdout:3882: done (5 left) 2024-03-20T08:14:50.815 INFO:tasks.rados.rados.0.smithi118.stdout:3883: snap_remove snap 381 2024-03-20T08:14:51.076 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:51.044+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:51.076 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:50.814+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: osdmap e835: 8 total, 8 up, 8 in 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:51.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: osdmap e835: 8 total, 8 up, 8 in 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:51.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:51.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:51.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:51.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:51.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:51.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:51.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:51.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:51.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:51.080 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:51.080 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:51.080 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:51.080 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:51.080 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:51.080 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:51.080 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:50 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:51.080 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:50.815+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:51.081 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:50.816+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:51.081 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:50.816+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:51.081 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: osdmap e835: 8 total, 8 up, 8 in 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:51.082 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:51.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:51.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:51.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:51.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:51.085 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:51.086 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:50 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:51.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:51.077+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:51.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:51.077+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:51.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:51.077+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:51.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:51.078+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:51.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:51.078+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:51.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:51.078+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:51.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:51.077+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:51.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:51.077+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:51.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:51.078+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.076+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.077+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.078+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:51.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.078+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:51.821 INFO:tasks.rados.rados.0.smithi118.stdout:3883: done (5 left) 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:3884: copy_from oid 18 from oid 37 current snap is 383 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:3877: finishing write tid 1 to smithi11881747-42 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:3877: finishing write tid 2 to smithi11881747-42 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:3877: finishing write tid 3 to smithi11881747-42 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:3877: oid 42 updating version 0 to 65348 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:3877: oid 42 updating version 65348 to 65349 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 42 v 65349 (ObjNum 1180 snap 383 seq_num 1180) dirty exists 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:3877: left oid 42 (ObjNum 1180 snap 383 seq_num 1180) 2024-03-20T08:14:51.822 INFO:tasks.rados.rados.0.smithi118.stdout:3879: expect (ObjNum 1157 snap 372 seq_num 1157) 2024-03-20T08:14:51.826 INFO:tasks.rados.rados.0.smithi118.stdout:3880: expect (ObjNum 1152 snap 371 seq_num 1152) 2024-03-20T08:14:51.997 INFO:tasks.rados.rados.0.smithi118.stdout:3881: finishing write tid 1 to smithi11881747-43 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3881: finishing write tid 2 to smithi11881747-43 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3881: finishing write tid 3 to smithi11881747-43 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3881: oid 43 updating version 0 to 65328 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3881: oid 43 updating version 65328 to 65329 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65329 (ObjNum 1181 snap 383 seq_num 1181) dirty exists 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3881: left oid 43 (ObjNum 1181 snap 383 seq_num 1181) 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3878: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3877: done (5 left) 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3879: done (4 left) 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3880: done (3 left) 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3881: done (2 left) 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3885: copy_from oid 29 from oid 48 current snap is 383 2024-03-20T08:14:51.998 INFO:tasks.rados.rados.0.smithi118.stdout:3878: finishing copy_from to smithi11881747-50 2024-03-20T08:14:51.999 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 65355 (ObjNum 1176 snap 379 seq_num 1176) dirty exists 2024-03-20T08:14:51.999 INFO:tasks.rados.rados.0.smithi118.stdout:3884: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:14:51.999 INFO:tasks.rados.rados.0.smithi118.stdout:3884: finishing copy_from to smithi11881747-18 2024-03-20T08:14:51.999 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 65326 (ObjNum 1172 snap 378 seq_num 1172) dirty exists 2024-03-20T08:14:51.999 INFO:tasks.rados.rados.0.smithi118.stdout:3878: done (2 left) 2024-03-20T08:14:51.999 INFO:tasks.rados.rados.0.smithi118.stdout:3884: done (1 left) 2024-03-20T08:14:51.999 INFO:tasks.rados.rados.0.smithi118.stdout:3886: setattr oid 5 current snap is 383 2024-03-20T08:14:51.999 INFO:tasks.rados.rados.0.smithi118.stdout:3887: delete oid 3 current snap is 383 2024-03-20T08:14:52.003 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 65350 (ObjNum 273 snap 0 seq_num 0) dirty exists 2024-03-20T08:14:52.004 INFO:tasks.rados.rados.0.smithi118.stdout:3886: done (2 left) 2024-03-20T08:14:52.004 INFO:tasks.rados.rados.0.smithi118.stdout:3887: done (1 left) 2024-03-20T08:14:52.004 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 383 2024-03-20T08:14:52.005 INFO:tasks.rados.rados.0.smithi118.stdout:3888: seq_num 1183 ranges {3801088=622592} 2024-03-20T08:14:52.031 INFO:tasks.rados.rados.0.smithi118.stdout:3888: writing smithi11881747-25 from 3801088 to 4423680 tid 1 2024-03-20T08:14:52.031 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 383 2024-03-20T08:14:52.031 INFO:tasks.rados.rados.0.smithi118.stdout:3889: seq_num 1184 ranges {999424=638976} 2024-03-20T08:14:52.040 INFO:tasks.rados.rados.0.smithi118.stdout:3889: writing smithi11881747-28 from 999424 to 1638400 tid 1 2024-03-20T08:14:52.040 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 383 2024-03-20T08:14:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3888: finishing write tid 1 to smithi11881747-25 2024-03-20T08:14:52.041 INFO:tasks.rados.rados.0.smithi118.stdout:3890: seq_num 1185 ranges {3366912=655360} 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:3890: writing smithi11881747-20 from 3366912 to 4022272 tid 1 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 12 current snap is 383 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:3888: finishing write tid 2 to smithi11881747-25 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:3888: finishing write tid 3 to smithi11881747-25 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:3888: oid 25 updating version 0 to 65351 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:3888: oid 25 updating version 65351 to 65352 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 65352 (ObjNum 1183 snap 383 seq_num 1183) dirty exists 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:3888: left oid 25 (ObjNum 1183 snap 383 seq_num 1183) 2024-03-20T08:14:52.063 INFO:tasks.rados.rados.0.smithi118.stdout:3891: seq_num 1186 ranges {6676480=671744} 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3891: writing smithi11881747-12 from 6676480 to 7348224 tid 1 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3888: done (4 left) 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3892: snap_create 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3885: finishing copy_from racing read to smithi11881747-29 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3889: finishing write tid 1 to smithi11881747-28 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3889: finishing write tid 2 to smithi11881747-28 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3889: finishing write tid 3 to smithi11881747-28 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3889: oid 28 updating version 0 to 65359 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:3889: oid 28 version 65359 is already newer than 65358 2024-03-20T08:14:52.105 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 65359 (ObjNum 1184 snap 383 seq_num 1184) dirty exists 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:3889: left oid 28 (ObjNum 1184 snap 383 seq_num 1184) 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:3885: finishing copy_from to smithi11881747-29 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 29 v 65321 (ObjNum 1140 snap 367 seq_num 1140) dirty exists 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:3890: finishing write tid 1 to smithi11881747-20 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:3890: finishing write tid 2 to smithi11881747-20 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:3890: finishing write tid 3 to smithi11881747-20 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:3890: oid 20 updating version 0 to 65328 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:3890: oid 20 version 65328 is already newer than 65327 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 65328 (ObjNum 1185 snap 383 seq_num 1185) dirty exists 2024-03-20T08:14:52.106 INFO:tasks.rados.rados.0.smithi118.stdout:3890: left oid 20 (ObjNum 1185 snap 383 seq_num 1185) 2024-03-20T08:14:52.107 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:51.821+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:52.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:51.831+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:52.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:51.831+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:52.107 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:51.831+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:52.108 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:51.821+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:52.108 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:51.821+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:52.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:52.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:52.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: pgmap v943: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:14:52.108 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:52.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:52.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: osdmap e836: 8 total, 8 up, 8 in 2024-03-20T08:14:52.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:52.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:52.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:52.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:52.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:52.109 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:52.110 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:51 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:52.114 INFO:tasks.rados.rados.0.smithi118.stdout:3891: finishing write tid 1 to smithi11881747-12 2024-03-20T08:14:52.114 INFO:tasks.rados.rados.0.smithi118.stdout:3891: finishing write tid 2 to smithi11881747-12 2024-03-20T08:14:52.117 INFO:tasks.rados.rados.0.smithi118.stdout:3891: finishing write tid 3 to smithi11881747-12 2024-03-20T08:14:52.117 INFO:tasks.rados.rados.0.smithi118.stdout:3891: oid 12 updating version 0 to 65314 2024-03-20T08:14:52.118 INFO:tasks.rados.rados.0.smithi118.stdout:3891: oid 12 updating version 65314 to 65315 2024-03-20T08:14:52.118 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65315 (ObjNum 1186 snap 383 seq_num 1186) dirty exists 2024-03-20T08:14:52.118 INFO:tasks.rados.rados.0.smithi118.stdout:3891: left oid 12 (ObjNum 1186 snap 383 seq_num 1186) 2024-03-20T08:14:52.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:51.819+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:52.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:51.827+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:52.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:51.828+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:52.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:51.823+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:52.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:51.823+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:52.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.825+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:52.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.825+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:52.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.826+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:52.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.826+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:52.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:51.826+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:52.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:52.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:52.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: pgmap v943: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:14:52.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:52.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:52.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: osdmap e836: 8 total, 8 up, 8 in 2024-03-20T08:14:52.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:52.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:52.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:52.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:52.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:52.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:52.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:52.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:52.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:52.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: pgmap v943: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 1 op/s 2024-03-20T08:14:52.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:52.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:52.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: osdmap e836: 8 total, 8 up, 8 in 2024-03-20T08:14:52.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:52.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:52.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:52.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:52.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:52.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:52.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:51 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:52.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:14:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:14:52.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:52.107+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:52.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:52.107+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:52.826 INFO:tasks.rados.rados.0.smithi118.stdout:3885: done (4 left) 2024-03-20T08:14:52.826 INFO:tasks.rados.rados.0.smithi118.stdout:3889: done (3 left) 2024-03-20T08:14:52.827 INFO:tasks.rados.rados.0.smithi118.stdout:3890: done (2 left) 2024-03-20T08:14:52.827 INFO:tasks.rados.rados.0.smithi118.stdout:3891: done (1 left) 2024-03-20T08:14:52.827 INFO:tasks.rados.rados.0.smithi118.stdout:3892: done (0 left) 2024-03-20T08:14:52.827 INFO:tasks.rados.rados.0.smithi118.stdout:3893: read oid 22 snap -1 2024-03-20T08:14:52.827 INFO:tasks.rados.rados.0.smithi118.stdout:3893: expect deleted 2024-03-20T08:14:52.827 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 384 2024-03-20T08:14:52.827 INFO:tasks.rados.rados.0.smithi118.stdout:3894: seq_num 1187 ranges {2859008=688128} 2024-03-20T08:14:52.845 INFO:tasks.rados.rados.0.smithi118.stdout:3894: writing smithi11881747-2 from 2859008 to 3547136 tid 1 2024-03-20T08:14:52.846 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 44 current snap is 384 2024-03-20T08:14:52.846 INFO:tasks.rados.rados.0.smithi118.stdout:3895: seq_num 1188 ranges {1769472=704512} 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:3895: writing smithi11881747-44 from 1769472 to 2473984 tid 1 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:3896: copy_from oid 50 from oid 29 current snap is 384 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:3894: finishing write tid 1 to smithi11881747-2 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:3894: finishing write tid 2 to smithi11881747-2 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:3894: finishing write tid 3 to smithi11881747-2 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:3894: oid 2 updating version 0 to 65326 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:3894: oid 2 updating version 65326 to 65327 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65327 (ObjNum 1187 snap 384 seq_num 1187) dirty exists 2024-03-20T08:14:52.859 INFO:tasks.rados.rados.0.smithi118.stdout:3894: left oid 2 (ObjNum 1187 snap 384 seq_num 1187) 2024-03-20T08:14:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3893: done (3 left) 2024-03-20T08:14:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3894: done (2 left) 2024-03-20T08:14:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3897: read oid 41 snap 376 2024-03-20T08:14:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3897: expect (ObjNum 1129 snap 361 seq_num 1129) 2024-03-20T08:14:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3898: copy_from oid 11 from oid 49 current snap is 384 2024-03-20T08:14:52.860 INFO:tasks.rados.rados.0.smithi118.stdout:3899: snap_create 2024-03-20T08:14:52.870 INFO:tasks.rados.rados.0.smithi118.stdout:3895: finishing write tid 1 to smithi11881747-44 2024-03-20T08:14:52.870 INFO:tasks.rados.rados.0.smithi118.stdout:3895: finishing write tid 2 to smithi11881747-44 2024-03-20T08:14:52.875 INFO:tasks.rados.rados.0.smithi118.stdout:3895: finishing write tid 3 to smithi11881747-44 2024-03-20T08:14:52.875 INFO:tasks.rados.rados.0.smithi118.stdout:3895: oid 44 updating version 0 to 65357 2024-03-20T08:14:52.875 INFO:tasks.rados.rados.0.smithi118.stdout:3895: oid 44 version 65357 is already newer than 65356 2024-03-20T08:14:52.875 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 65357 (ObjNum 1188 snap 384 seq_num 1188) dirty exists 2024-03-20T08:14:52.875 INFO:tasks.rados.rados.0.smithi118.stdout:3895: left oid 44 (ObjNum 1188 snap 384 seq_num 1188) 2024-03-20T08:14:52.897 INFO:tasks.rados.rados.0.smithi118.stdout:3897: expect (ObjNum 1129 snap 361 seq_num 1129) 2024-03-20T08:14:53.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:52.825+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:53.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:52.849+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:53.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:52.849+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:53.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:52.852+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:53.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:52.829+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:53.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:52.829+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:53.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:52.829+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:53.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:52.864+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:53.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:52.864+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: osdmap e837: 8 total, 8 up, 8 in 2024-03-20T08:14:53.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:53.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:52 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:53.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:52.825+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:53.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:52.867+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:53.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:52.862+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:53.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:52.863+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:53.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:52.863+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:53.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:52.862+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:52.863+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:52.863+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:52.868+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: osdmap e837: 8 total, 8 up, 8 in 2024-03-20T08:14:53.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: osdmap e837: 8 total, 8 up, 8 in 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:53.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:53.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ae=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:53.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:52 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:53.263 INFO:tasks.rados.rados.0.smithi118.stdout:3898: finishing copy_from racing read to smithi11881747-11 2024-03-20T08:14:53.263 INFO:tasks.rados.rados.0.smithi118.stdout:3898: finishing copy_from to smithi11881747-11 2024-03-20T08:14:53.263 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 11 v 61769 (ObjNum 1133 snap 364 seq_num 1133) dirty exists 2024-03-20T08:14:53.264 INFO:tasks.rados.rados.0.smithi118.stdout:3896: finishing copy_from racing read to smithi11881747-50 2024-03-20T08:14:53.264 INFO:tasks.rados.rados.0.smithi118.stdout:3896: finishing copy_from to smithi11881747-50 2024-03-20T08:14:53.264 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 65321 (ObjNum 1140 snap 367 seq_num 1140) dirty exists 2024-03-20T08:14:53.827 INFO:tasks.rados.rados.0.smithi118.stdout:3895: done (4 left) 2024-03-20T08:14:53.827 INFO:tasks.rados.rados.0.smithi118.stdout:3896: done (3 left) 2024-03-20T08:14:53.828 INFO:tasks.rados.rados.0.smithi118.stdout:3897: done (2 left) 2024-03-20T08:14:53.828 INFO:tasks.rados.rados.0.smithi118.stdout:3898: done (1 left) 2024-03-20T08:14:53.828 INFO:tasks.rados.rados.0.smithi118.stdout:3899: done (0 left) 2024-03-20T08:14:53.828 INFO:tasks.rados.rados.0.smithi118.stdout:3900: rollback oid 40 current snap is 385 2024-03-20T08:14:53.828 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 40 to 369 2024-03-20T08:14:53.828 INFO:tasks.rados.rados.0.smithi118.stdout:3901: read oid 10 snap -1 2024-03-20T08:14:53.828 INFO:tasks.rados.rados.0.smithi118.stdout:3901: expect (ObjNum 1160 snap 375 seq_num 1160) 2024-03-20T08:14:53.829 INFO:tasks.rados.rados.0.smithi118.stdout:3902: rollback oid 39 current snap is 385 2024-03-20T08:14:53.829 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 39 to 380 2024-03-20T08:14:53.829 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 50 current snap is 385 2024-03-20T08:14:53.829 INFO:tasks.rados.rados.0.smithi118.stdout:3903: seq_num 1189 ranges {15032320=720896} 2024-03-20T08:14:53.837 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:14:53] "GET /metrics HTTP/1.1" 200 36413 "" "Prometheus/2.43.0" 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: pgmap v946: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 4.2 MiB/s rd, 272 KiB/s wr, 4 op/s 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: osdmap e838: 8 total, 8 up, 8 in 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.838 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:53.839 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: pgmap v946: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 4.2 MiB/s rd, 272 KiB/s wr, 4 op/s 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: osdmap e838: 8 total, 8 up, 8 in 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:53.840 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:53.841 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:53.841 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:53.841 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:53.841 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:53.826+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:53.841 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:53.827+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:53.841 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:53.827+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:53.917 INFO:tasks.rados.rados.0.smithi118.stdout:3903: writing smithi11881747-50 from 15032320 to 15753216 tid 1 2024-03-20T08:14:53.917 INFO:tasks.rados.rados.0.smithi118.stdout:3904: read oid 22 snap -1 2024-03-20T08:14:53.917 INFO:tasks.rados.rados.0.smithi118.stdout:3904: expect deleted 2024-03-20T08:14:53.917 INFO:tasks.rados.rados.0.smithi118.stdout:3905: snap_create 2024-03-20T08:14:53.917 INFO:tasks.rados.rados.0.smithi118.stdout:3902: finishing rollback tid 1 to smithi11881747-39 2024-03-20T08:14:53.917 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 65326 (ObjNum 33 snap 0 seq_num 322447664) dirty dne 2024-03-20T08:14:53.918 INFO:tasks.rados.rados.0.smithi118.stdout:3901: expect (ObjNum 1160 snap 375 seq_num 1160) 2024-03-20T08:14:53.923 INFO:tasks.rados.rados.0.smithi118.stdout:3900: finishing rollback tid 1 to smithi11881747-40 2024-03-20T08:14:53.923 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 40 v 65317 (ObjNum 161 snap 0 seq_num 0) dirty dne 2024-03-20T08:14:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:3903: finishing write tid 1 to smithi11881747-50 2024-03-20T08:14:53.927 INFO:tasks.rados.rados.0.smithi118.stdout:3903: finishing write tid 2 to smithi11881747-50 2024-03-20T08:14:53.930 INFO:tasks.rados.rados.0.smithi118.stdout:3903: finishing write tid 3 to smithi11881747-50 2024-03-20T08:14:53.931 INFO:tasks.rados.rados.0.smithi118.stdout:3903: oid 50 updating version 0 to 65360 2024-03-20T08:14:53.931 INFO:tasks.rados.rados.0.smithi118.stdout:3903: oid 50 version 65360 is already newer than 65359 2024-03-20T08:14:53.931 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 65360 (ObjNum 1189 snap 385 seq_num 1189) dirty exists 2024-03-20T08:14:53.931 INFO:tasks.rados.rados.0.smithi118.stdout:3903: left oid 50 (ObjNum 1189 snap 385 seq_num 1189) 2024-03-20T08:14:54.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:53.827+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:54.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:54.041+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:54.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:54.041+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:53.832+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:53.832+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:54.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:53.832+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:54.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:53.839+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:54.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:53.840+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:54.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: pgmap v946: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 4.2 MiB/s rd, 272 KiB/s wr, 4 op/s 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: osdmap e838: 8 total, 8 up, 8 in 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:54.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:54.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:54.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:54.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:54.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:54.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:53 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:53.841+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:54.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:53.842+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:53.838+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:53.838+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:53.838+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:53.838+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:54.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:53.839+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:54.837 INFO:tasks.rados.rados.0.smithi118.stdout:3900: done (5 left) 2024-03-20T08:14:54.837 INFO:tasks.rados.rados.0.smithi118.stdout:3901: done (4 left) 2024-03-20T08:14:54.837 INFO:tasks.rados.rados.0.smithi118.stdout:3902: done (3 left) 2024-03-20T08:14:54.837 INFO:tasks.rados.rados.0.smithi118.stdout:3903: done (2 left) 2024-03-20T08:14:54.837 INFO:tasks.rados.rados.0.smithi118.stdout:3904: done (1 left) 2024-03-20T08:14:54.838 INFO:tasks.rados.rados.0.smithi118.stdout:3905: done (0 left) 2024-03-20T08:14:54.838 INFO:tasks.rados.rados.0.smithi118.stdout:3906: snap_remove snap 382 2024-03-20T08:14:55.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:54.838+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:55.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:55.078+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:55.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:55.079+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:55.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:55.079+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:55.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:55.080+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:55.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:55.081+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:55.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:55.081+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:55.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:55.080+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:55.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:55.080+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: osdmap e839: 8 total, 8 up, 8 in 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:55.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:55.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:55.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:54 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:55.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:55.078+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:55.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:55.078+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:55.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:55.078+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:55.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:55.078+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:55.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:54.836+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:55.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:54.837+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:55.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:54.837+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:55.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:54.837+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:55.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:54.838+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: osdmap e839: 8 total, 8 up, 8 in 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:55.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:55.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:55.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: osdmap e839: 8 total, 8 up, 8 in 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:55.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:54 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:55.842 INFO:tasks.rados.rados.0.smithi118.stdout:3906: done (0 left) 2024-03-20T08:14:55.842 INFO:tasks.rados.rados.0.smithi118.stdout:3907: setattr oid 44 current snap is 386 2024-03-20T08:14:55.843 INFO:tasks.rados.rados.0.smithi118.stdout:3908: copy_from oid 35 from oid 34 current snap is 386 2024-03-20T08:14:55.843 INFO:tasks.rados.rados.0.smithi118.stdout:3909: read oid 22 snap -1 2024-03-20T08:14:55.843 INFO:tasks.rados.rados.0.smithi118.stdout:3909: expect deleted 2024-03-20T08:14:55.843 INFO:tasks.rados.rados.0.smithi118.stdout:3910: snap_remove snap 380 2024-03-20T08:14:56.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:55.848+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:56.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:55.854+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:56.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:55.854+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:56.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:55.843+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:56.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:55.843+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:56.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:55.849+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:56.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:55.849+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:56.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:55.856+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:56.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:56.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:56.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: pgmap v949: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-03-20T08:14:56.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:56.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:56.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:56.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: osdmap e840: 8 total, 8 up, 8 in 2024-03-20T08:14:56.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:56.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:56.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:56.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:56.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:56.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:56.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:56.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:56.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:56.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:55 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:56.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:55.847+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:56.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:55.847+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:56.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:55.846+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:56.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:55.846+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:56.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:55.842+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:56.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.055+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:56.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.056+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:56.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.056+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:56.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.056+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:56.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.056+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: pgmap v949: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-03-20T08:14:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:56.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: osdmap e840: 8 total, 8 up, 8 in 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: pgmap v949: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.6 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 4.2 MiB/s rd, 4.3 MiB/s wr, 6 op/s 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:56.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: osdmap e840: 8 total, 8 up, 8 in 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:56.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:56.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:55 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:56.848 INFO:tasks.rados.rados.0.smithi118.stdout:3910: done (3 left) 2024-03-20T08:14:56.848 INFO:tasks.rados.rados.0.smithi118.stdout:3911: rollback oid 12 current snap is 386 2024-03-20T08:14:56.848 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 12 to 361 2024-03-20T08:14:56.848 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 44 v 65362 (ObjNum 1188 snap 384 seq_num 1188) dirty exists 2024-03-20T08:14:56.848 INFO:tasks.rados.rados.0.smithi118.stdout:3908: finishing copy_from racing read to smithi11881747-35 2024-03-20T08:14:56.848 INFO:tasks.rados.rados.0.smithi118.stdout:3908: finishing copy_from to smithi11881747-35 2024-03-20T08:14:56.848 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 45663 (ObjNum 1139 snap 367 seq_num 1139) dirty exists 2024-03-20T08:14:56.849 INFO:tasks.rados.rados.0.smithi118.stdout:3907: done (3 left) 2024-03-20T08:14:56.849 INFO:tasks.rados.rados.0.smithi118.stdout:3908: done (2 left) 2024-03-20T08:14:56.849 INFO:tasks.rados.rados.0.smithi118.stdout:3909: done (1 left) 2024-03-20T08:14:56.849 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 2 current snap is 386 2024-03-20T08:14:56.849 INFO:tasks.rados.rados.0.smithi118.stdout:3912: seq_num 1190 ranges {3547136=737280} 2024-03-20T08:14:56.871 INFO:tasks.rados.rados.0.smithi118.stdout:3912: writing smithi11881747-2 from 3547136 to 4284416 tid 1 2024-03-20T08:14:56.871 INFO:tasks.rados.rados.0.smithi118.stdout:3913: snap_remove snap 379 2024-03-20T08:14:57.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:56.849+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:57.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:56.849+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:57.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:56.850+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:57.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:56.850+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:57.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:56.855+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:57.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:56.855+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:57.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:56.854+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:14:57.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:56.854+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:57.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:56.855+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:57.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:57.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:57.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: osdmap e841: 8 total, 8 up, 8 in 2024-03-20T08:14:57.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:57.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:57.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:57.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:57.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:57.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:57.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:57.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:57.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:57.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:57.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:56 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:57.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:56.848+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:57.172 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.850+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:57.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.850+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:57.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.851+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:57.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:56.863+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: osdmap e841: 8 total, 8 up, 8 in 2024-03-20T08:14:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:57.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:57.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: osdmap e841: 8 total, 8 up, 8 in 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:57.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:57.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:56 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:57.672 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:57.377+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:57.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:57.346+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:57.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:57.346+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:57.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:57.346+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:57.854 INFO:tasks.rados.rados.0.smithi118.stdout:3913: done (2 left) 2024-03-20T08:14:57.854 INFO:tasks.rados.rados.0.smithi118.stdout:3914: read oid 18 snap -1 2024-03-20T08:14:57.854 INFO:tasks.rados.rados.0.smithi118.stdout:3914: expect (ObjNum 1172 snap 378 seq_num 1172) 2024-03-20T08:14:57.854 INFO:tasks.rados.rados.0.smithi118.stdout:3915: setattr oid 20 current snap is 386 2024-03-20T08:14:57.854 INFO:tasks.rados.rados.0.smithi118.stdout:3911: finishing rollback tid 0 to smithi11881747-12 2024-03-20T08:14:57.854 INFO:tasks.rados.rados.0.smithi118.stdout:3911: finishing rollback tid 1 to smithi11881747-12 2024-03-20T08:14:57.854 INFO:tasks.rados.rados.0.smithi118.stdout:3911: finishing rollback tid 2 to smithi11881747-12 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 12 v 65318 (ObjNum 1056 snap 338 seq_num 1056) dirty exists 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:3912: finishing write tid 1 to smithi11881747-2 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:3912: finishing write tid 2 to smithi11881747-2 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:3912: finishing write tid 3 to smithi11881747-2 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:3912: oid 2 updating version 0 to 65328 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:3912: oid 2 updating version 65328 to 65329 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65329 (ObjNum 1190 snap 386 seq_num 1190) dirty exists 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:3912: left oid 2 (ObjNum 1190 snap 386 seq_num 1190) 2024-03-20T08:14:57.855 INFO:tasks.rados.rados.0.smithi118.stdout:3911: done (3 left) 2024-03-20T08:14:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:3912: done (2 left) 2024-03-20T08:14:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 49 current snap is 386 2024-03-20T08:14:57.856 INFO:tasks.rados.rados.0.smithi118.stdout:3916: seq_num 1191 ranges {10141696=753664} 2024-03-20T08:14:57.915 INFO:tasks.rados.rados.0.smithi118.stdout:3916: writing smithi11881747-49 from 10141696 to 10895360 tid 1 2024-03-20T08:14:57.915 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 28 current snap is 386 2024-03-20T08:14:57.915 INFO:tasks.rados.rados.0.smithi118.stdout:3917: seq_num 1192 ranges {1638400=770048} 2024-03-20T08:14:57.929 INFO:tasks.rados.rados.0.smithi118.stdout:3917: writing smithi11881747-28 from 1638400 to 2408448 tid 1 2024-03-20T08:14:57.929 INFO:tasks.rados.rados.0.smithi118.stdout:3918: copy_from oid 36 from oid 5 current snap is 386 2024-03-20T08:14:57.929 INFO:tasks.rados.rados.0.smithi118.stdout:3914: expect (ObjNum 1172 snap 378 seq_num 1172) 2024-03-20T08:14:57.945 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 65329 (ObjNum 1185 snap 383 seq_num 1185) dirty exists 2024-03-20T08:14:57.945 INFO:tasks.rados.rados.0.smithi118.stdout:3916: finishing write tid 1 to smithi11881747-49 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3916: finishing write tid 2 to smithi11881747-49 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3914: done (4 left) 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3915: done (3 left) 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 25 current snap is 386 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3916: finishing write tid 3 to smithi11881747-49 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3916: oid 49 updating version 0 to 65330 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3916: oid 49 updating version 65330 to 65331 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 49 v 65331 (ObjNum 1191 snap 386 seq_num 1191) dirty exists 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3916: left oid 49 (ObjNum 1191 snap 386 seq_num 1191) 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3917: finishing write tid 1 to smithi11881747-28 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3918: finishing copy_from racing read to smithi11881747-36 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3917: finishing write tid 2 to smithi11881747-28 2024-03-20T08:14:57.946 INFO:tasks.rados.rados.0.smithi118.stdout:3918: finishing copy_from to smithi11881747-36 2024-03-20T08:14:57.947 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 65350 (ObjNum 465 snap 0 seq_num 322110528) dirty exists 2024-03-20T08:14:57.947 INFO:tasks.rados.rados.0.smithi118.stdout:3917: finishing write tid 3 to smithi11881747-28 2024-03-20T08:14:57.947 INFO:tasks.rados.rados.0.smithi118.stdout:3917: oid 28 updating version 0 to 65360 2024-03-20T08:14:57.947 INFO:tasks.rados.rados.0.smithi118.stdout:3917: oid 28 updating version 65360 to 65361 2024-03-20T08:14:57.947 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 28 v 65361 (ObjNum 1192 snap 386 seq_num 1192) dirty exists 2024-03-20T08:14:57.947 INFO:tasks.rados.rados.0.smithi118.stdout:3917: left oid 28 (ObjNum 1192 snap 386 seq_num 1192) 2024-03-20T08:14:57.947 INFO:tasks.rados.rados.0.smithi118.stdout:3919: seq_num 1193 ranges {4423680=786432} 2024-03-20T08:14:57.974 INFO:tasks.rados.rados.0.smithi118.stdout:3919: writing smithi11881747-25 from 4423680 to 5210112 tid 1 2024-03-20T08:14:57.974 INFO:tasks.rados.rados.0.smithi118.stdout:3916: done (3 left) 2024-03-20T08:14:57.974 INFO:tasks.rados.rados.0.smithi118.stdout:3917: done (2 left) 2024-03-20T08:14:57.974 INFO:tasks.rados.rados.0.smithi118.stdout:3918: done (1 left) 2024-03-20T08:14:57.974 INFO:tasks.rados.rados.0.smithi118.stdout:3920: copy_from oid 32 from oid 45 current snap is 386 2024-03-20T08:14:57.974 INFO:tasks.rados.rados.0.smithi118.stdout:3921: snap_remove snap 361 2024-03-20T08:14:58.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:57.855+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:58.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:57.863+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:58.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:57.863+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:58.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:57.863+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: pgmap v952: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 303 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-03-20T08:14:58.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: osdmap e842: 8 total, 8 up, 8 in 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:58.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:58.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:57 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:58.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:57.933+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:58.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:57.934+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:57.933+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:57.937+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:57.938+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:14:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:57.938+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:58.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:57.940+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:58.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:57.853+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: pgmap v952: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 303 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: osdmap e842: 8 total, 8 up, 8 in 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:58.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:58.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:57.861+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:58.175 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:57.861+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: pgmap v952: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 303 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-03-20T08:14:58.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: osdmap e842: 8 total, 8 up, 8 in 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:58.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:58.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:58.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:58.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:58.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:58.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:58.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:58.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:57 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:58.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:58.181+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:14:58.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:58.181+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:14:58.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:58.182+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:58.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:58.182+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:58.860 INFO:tasks.rados.rados.0.smithi118.stdout:3921: done (2 left) 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3922: rollback oid 50 current snap is 386 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 50 to 383 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3919: finishing write tid 1 to smithi11881747-25 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3919: finishing write tid 2 to smithi11881747-25 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3919: finishing write tid 3 to smithi11881747-25 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3919: oid 25 updating version 0 to 65354 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3919: oid 25 version 65354 is already newer than 65353 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 25 v 65354 (ObjNum 1193 snap 386 seq_num 1193) dirty exists 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3919: left oid 25 (ObjNum 1193 snap 386 seq_num 1193) 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3920: finishing copy_from racing read to smithi11881747-32 2024-03-20T08:14:58.861 INFO:tasks.rados.rados.0.smithi118.stdout:3920: finishing copy_from to smithi11881747-32 2024-03-20T08:14:58.862 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 65327 (ObjNum 1163 snap 376 seq_num 1163) dirty exists 2024-03-20T08:14:58.862 INFO:tasks.rados.rados.0.smithi118.stdout:3919: done (2 left) 2024-03-20T08:14:58.862 INFO:tasks.rados.rados.0.smithi118.stdout:3920: done (1 left) 2024-03-20T08:14:58.862 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 32 current snap is 386 2024-03-20T08:14:58.862 INFO:tasks.rados.rados.0.smithi118.stdout:3923: seq_num 1194 ranges {286720=802816} 2024-03-20T08:14:58.866 INFO:tasks.rados.rados.0.smithi118.stdout:3923: writing smithi11881747-32 from 286720 to 1089536 tid 1 2024-03-20T08:14:58.866 INFO:tasks.rados.rados.0.smithi118.stdout:3924: read oid 7 snap 376 2024-03-20T08:14:58.866 INFO:tasks.rados.rados.0.smithi118.stdout:3924: expect deleted 2024-03-20T08:14:58.867 INFO:tasks.rados.rados.0.smithi118.stdout:3925: snap_remove snap 369 2024-03-20T08:14:59.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:14:58.859+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:59.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:58.866+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:14:59.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:14:58.867+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:14:59.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:58.863+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:59.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:58.876+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:14:59.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:58.876+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: osdmap e843: 8 total, 8 up, 8 in 2024-03-20T08:14:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:59.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:59.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:59.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:58.859+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:59.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:58.860+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:59.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:58.865+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:14:59.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:58.865+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:59.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:14:58.866+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:58.870+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:58.870+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:58.870+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:59.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:58.871+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: osdmap e843: 8 total, 8 up, 8 in 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:59.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: osdmap e843: 8 total, 8 up, 8 in 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:14:59.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:14:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:14:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:14:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:14:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:14:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:14:59.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:14:59.865 INFO:tasks.rados.rados.0.smithi118.stdout:3925: done (3 left) 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3926: copy_from oid 16 from oid 30 current snap is 386 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3922: finishing rollback tid 0 to smithi11881747-50 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3922: finishing rollback tid 1 to smithi11881747-50 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3923: finishing write tid 1 to smithi11881747-32 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3922: finishing rollback tid 2 to smithi11881747-50 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 50 v 65366 (ObjNum 1176 snap 379 seq_num 1176) dirty exists 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3923: finishing write tid 2 to smithi11881747-32 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3923: finishing write tid 3 to smithi11881747-32 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3923: oid 32 updating version 0 to 65328 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3923: oid 32 updating version 65328 to 65329 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 32 v 65329 (ObjNum 1194 snap 386 seq_num 1194) dirty exists 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3923: left oid 32 (ObjNum 1194 snap 386 seq_num 1194) 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3922: done (3 left) 2024-03-20T08:14:59.866 INFO:tasks.rados.rados.0.smithi118.stdout:3923: done (2 left) 2024-03-20T08:14:59.867 INFO:tasks.rados.rados.0.smithi118.stdout:3924: done (1 left) 2024-03-20T08:14:59.867 INFO:tasks.rados.rados.0.smithi118.stdout:3927: snap_remove snap 372 2024-03-20T08:15:00.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:59.871+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:00.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:59.871+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:00.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:14:59.871+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:00.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:59.647+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:00.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:59.647+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:00.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:59.647+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:00.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:59.865+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:00.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:14:59.865+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:00.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:00.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:00.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:00.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:00.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: pgmap v955: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 5.6 MiB/s rd, 9.9 MiB/s wr, 14 op/s 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: osdmap e844: 8 total, 8 up, 8 in 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:00.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:14:59 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:00.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:14:59.864+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:00.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:59.865+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:00.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:14:59.866+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:59.868+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:59.868+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:59.868+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:59.869+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:00.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:14:59.869+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:00.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: pgmap v955: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 5.6 MiB/s rd, 9.9 MiB/s wr, 14 op/s 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: osdmap e844: 8 total, 8 up, 8 in 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:00.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: pgmap v955: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 5.6 MiB/s rd, 9.9 MiB/s wr, 14 op/s 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: osdmap e844: 8 total, 8 up, 8 in 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:00.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:00.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:00.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:14:59 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:00.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:00.217+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:00.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:00.217+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:00.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:00.217+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:00.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:00.216+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:00.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:00.216+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:00.870 INFO:tasks.rados.rados.0.smithi118.stdout:3927: done (1 left) 2024-03-20T08:15:00.871 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 43 current snap is 386 2024-03-20T08:15:00.871 INFO:tasks.rados.rados.0.smithi118.stdout:3926: finishing copy_from racing read to smithi11881747-16 2024-03-20T08:15:00.871 INFO:tasks.rados.rados.0.smithi118.stdout:3928: seq_num 1195 ranges {7045120=417792,7462912=401408} 2024-03-20T08:15:00.911 INFO:tasks.rados.rados.0.smithi118.stdout:3928: writing smithi11881747-43 from 7045120 to 7462912 tid 1 2024-03-20T08:15:00.913 INFO:tasks.rados.rados.0.smithi118.stdout:3928: writing smithi11881747-43 from 7462912 to 7864320 tid 2 2024-03-20T08:15:00.913 INFO:tasks.rados.rados.0.smithi118.stdout:3929: snap_create 2024-03-20T08:15:00.913 INFO:tasks.rados.rados.0.smithi118.stdout:3926: finishing copy_from to smithi11881747-16 2024-03-20T08:15:00.913 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 16 v 65322 (ObjNum 1128 snap 361 seq_num 1128) dirty exists 2024-03-20T08:15:00.921 INFO:tasks.rados.rados.0.smithi118.stdout:3928: finishing write tid 1 to smithi11881747-43 2024-03-20T08:15:00.921 INFO:tasks.rados.rados.0.smithi118.stdout:3928: finishing write tid 2 to smithi11881747-43 2024-03-20T08:15:00.922 INFO:tasks.rados.rados.0.smithi118.stdout:3928: finishing write tid 3 to smithi11881747-43 2024-03-20T08:15:00.924 INFO:tasks.rados.rados.0.smithi118.stdout:3928: finishing write tid 4 to smithi11881747-43 2024-03-20T08:15:00.924 INFO:tasks.rados.rados.0.smithi118.stdout:3928: oid 43 updating version 0 to 65330 2024-03-20T08:15:00.924 INFO:tasks.rados.rados.0.smithi118.stdout:3928: oid 43 updating version 65330 to 65332 2024-03-20T08:15:00.924 INFO:tasks.rados.rados.0.smithi118.stdout:3928: oid 43 version 65332 is already newer than 65331 2024-03-20T08:15:00.924 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65332 (ObjNum 1195 snap 386 seq_num 1195) dirty exists 2024-03-20T08:15:00.924 INFO:tasks.rados.rados.0.smithi118.stdout:3928: left oid 43 (ObjNum 1195 snap 386 seq_num 1195) 2024-03-20T08:15:01.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:00.870+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:01.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:00.919+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:01.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:00.919+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:01.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:00.919+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:01.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:00.916+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:01.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:00.916+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:01.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:01.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: osdmap e845: 8 total, 8 up, 8 in 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:01.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:01.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:00 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:01.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:00.873+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:01.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:00.869+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:01.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:00.869+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:01.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:00.869+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: osdmap e845: 8 total, 8 up, 8 in 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:01.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: osdmap e845: 8 total, 8 up, 8 in 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:01.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b7=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:01.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:00 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:01.871 INFO:tasks.rados.rados.0.smithi118.stdout:3926: done (2 left) 2024-03-20T08:15:01.871 INFO:tasks.rados.rados.0.smithi118.stdout:3928: done (1 left) 2024-03-20T08:15:01.871 INFO:tasks.rados.rados.0.smithi118.stdout:3929: done (0 left) 2024-03-20T08:15:01.871 INFO:tasks.rados.rados.0.smithi118.stdout:3930: snap_remove snap 386 2024-03-20T08:15:02.044 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:01.675+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:02.044 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:01.870+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:02.044 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:02.044 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:02.044 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:02.044 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: pgmap v958: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 5.7 MiB/s rd, 922 KiB/s wr, 14 op/s 2024-03-20T08:15:02.044 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: osdmap e846: 8 total, 8 up, 8 in 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:02.045 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: pgmap v958: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 5.7 MiB/s rd, 922 KiB/s wr, 14 op/s 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: osdmap e846: 8 total, 8 up, 8 in 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:02.046 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:01 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:02.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:01.872+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:02.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:01.871+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:02.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:01.871+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:02.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:01.872+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:02.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:02.044+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:02.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:02.044+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:02.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:02.044+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:02.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:02.045+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:02.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:02.045+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: pgmap v958: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 5.7 MiB/s rd, 922 KiB/s wr, 14 op/s 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: osdmap e846: 8 total, 8 up, 8 in 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:02.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:01 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:02.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:02.281+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:02.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:02.281+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:02.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:02.281+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:02.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:02.281+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:02.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.043+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:02.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.043+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:02.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.043+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:02.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.043+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:02.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.203+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:02.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.203+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:02.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.203+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:02.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.203+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:02.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.203+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:15:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:15:02.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:02.281+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:02.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:02.281+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:02.882 INFO:tasks.rados.rados.0.smithi118.stdout:3930: done (0 left) 2024-03-20T08:15:02.882 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 20 current snap is 387 2024-03-20T08:15:02.882 INFO:tasks.rados.rados.0.smithi118.stdout:3931: seq_num 1196 ranges {4022272=434176,4456448=401408} 2024-03-20T08:15:02.905 INFO:tasks.rados.rados.0.smithi118.stdout:3931: writing smithi11881747-20 from 4022272 to 4456448 tid 1 2024-03-20T08:15:02.908 INFO:tasks.rados.rados.0.smithi118.stdout:3931: writing smithi11881747-20 from 4456448 to 4857856 tid 2 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3932: read oid 45 snap -1 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3932: expect (ObjNum 1163 snap 376 seq_num 1163) 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3933: read oid 26 snap -1 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3933: expect (ObjNum 1130 snap 361 seq_num 1130) 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3934: read oid 37 snap 383 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3934: expect (ObjNum 1172 snap 378 seq_num 1172) 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3935: read oid 49 snap -1 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3935: expect (ObjNum 1191 snap 386 seq_num 1191) 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3936: copy_from oid 2 from oid 18 current snap is 387 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 21 current snap is 387 2024-03-20T08:15:02.909 INFO:tasks.rados.rados.0.smithi118.stdout:3937: seq_num 1197 ranges {2056192=450560,2506752=401408} 2024-03-20T08:15:02.923 INFO:tasks.rados.rados.0.smithi118.stdout:3937: writing smithi11881747-21 from 2056192 to 2506752 tid 1 2024-03-20T08:15:02.926 INFO:tasks.rados.rados.0.smithi118.stdout:3937: writing smithi11881747-21 from 2506752 to 2908160 tid 2 2024-03-20T08:15:02.927 INFO:tasks.rados.rados.0.smithi118.stdout:3938: read oid 48 snap 384 2024-03-20T08:15:02.927 INFO:tasks.rados.rados.0.smithi118.stdout:3938: expect (ObjNum 1140 snap 367 seq_num 1140) 2024-03-20T08:15:02.927 INFO:tasks.rados.rados.0.smithi118.stdout:3939: snap_create 2024-03-20T08:15:02.927 INFO:tasks.rados.rados.0.smithi118.stdout:3931: finishing write tid 1 to smithi11881747-20 2024-03-20T08:15:02.927 INFO:tasks.rados.rados.0.smithi118.stdout:3932: expect (ObjNum 1163 snap 376 seq_num 1163) 2024-03-20T08:15:02.941 INFO:tasks.rados.rados.0.smithi118.stdout:3931: finishing write tid 2 to smithi11881747-20 2024-03-20T08:15:02.941 INFO:tasks.rados.rados.0.smithi118.stdout:3934: expect (ObjNum 1172 snap 378 seq_num 1172) 2024-03-20T08:15:02.955 INFO:tasks.rados.rados.0.smithi118.stdout:3933: expect (ObjNum 1130 snap 361 seq_num 1130) 2024-03-20T08:15:03.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:02.881+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:03.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:02.916+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:03.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:02.916+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:03.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:02.910+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:03.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:02.917+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:03.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:02.917+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:03.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:02.914+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:03.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:02.914+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:03.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:02.914+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: osdmap e847: 8 total, 8 up, 8 in 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:03.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:02 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:03.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:02.881+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:03.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:02.881+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:03.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:02.881+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:03.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:02.881+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:03.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:02.910+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:03.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.913+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:03.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.914+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:03.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.914+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:03.178 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:02.922+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:03.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:03.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:03.179 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: osdmap e847: 8 total, 8 up, 8 in 2024-03-20T08:15:03.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:03.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:03.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:03.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:03.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:03.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:03.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:03.180 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: osdmap e847: 8 total, 8 up, 8 in 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:03.181 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:03.182 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:02 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:03.200 INFO:tasks.rados.rados.0.smithi118.stdout:3937: finishing write tid 1 to smithi11881747-21 2024-03-20T08:15:03.201 INFO:tasks.rados.rados.0.smithi118.stdout:3937: finishing write tid 2 to smithi11881747-21 2024-03-20T08:15:03.201 INFO:tasks.rados.rados.0.smithi118.stdout:3931: finishing write tid 3 to smithi11881747-20 2024-03-20T08:15:03.201 INFO:tasks.rados.rados.0.smithi118.stdout:3935: expect (ObjNum 1191 snap 386 seq_num 1191) 2024-03-20T08:15:03.878 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:15:03] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:15:04.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:03.886+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: pgmap v961: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: osdmap e848: 8 total, 8 up, 8 in 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:04.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:04.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:04.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:03 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:03.887+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:04.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:03.888+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:03.888+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:03.888+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:03.888+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:03.888+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:04.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:03.888+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: pgmap v961: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-03-20T08:15:04.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: osdmap e848: 8 total, 8 up, 8 in 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: pgmap v961: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: osdmap e848: 8 total, 8 up, 8 in 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:04.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:04.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:04.179 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:03 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3931: finishing write tid 4 to smithi11881747-20 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3931: oid 20 updating version 0 to 65333 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3931: oid 20 version 65333 is already newer than 65332 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3931: oid 20 updating version 65333 to 65334 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 20 v 65334 (ObjNum 1196 snap 387 seq_num 1196) dirty exists 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3931: left oid 20 (ObjNum 1196 snap 387 seq_num 1196) 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3936: finishing copy_from racing read to smithi11881747-2 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3936: finishing copy_from to smithi11881747-2 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65326 (ObjNum 1172 snap 378 seq_num 1172) dirty exists 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3937: finishing write tid 3 to smithi11881747-21 2024-03-20T08:15:04.297 INFO:tasks.rados.rados.0.smithi118.stdout:3938: expect (ObjNum 1140 snap 367 seq_num 1140) 2024-03-20T08:15:04.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:04.247+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:04.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:04.248+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:05.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:04.844+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:05.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:04.844+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:05.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:05.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: osdmap e849: 8 total, 8 up, 8 in 2024-03-20T08:15:05.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:05.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:05.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:05.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:05.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:05.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:05.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:04 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:05.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:05.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: osdmap e849: 8 total, 8 up, 8 in 2024-03-20T08:15:05.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:05.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: osdmap e849: 8 total, 8 up, 8 in 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:05.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:05.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:04 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:06.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:05 smithi118 ceph-mon[93121]: pgmap v964: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 3 op/s 2024-03-20T08:15:06.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:05 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:06.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:05 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:06.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:05 smithi118 ceph-mon[95692]: pgmap v964: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 3 op/s 2024-03-20T08:15:06.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:05 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:06.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:05 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:06.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:05 smithi196 ceph-mon[87080]: pgmap v964: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 767 B/s rd, 1.7 MiB/s wr, 3 op/s 2024-03-20T08:15:06.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:05 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:06.179 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:05 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:06.179 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:06.117+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:06.179 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:06.117+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:3937: finishing write tid 4 to smithi11881747-21 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:3937: oid 21 updating version 0 to 65333 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:3937: oid 21 version 65333 is already newer than 65332 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:3937: oid 21 updating version 65333 to 65334 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 21 v 65334 (ObjNum 1197 snap 387 seq_num 1197) dirty exists 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:3937: left oid 21 (ObjNum 1197 snap 387 seq_num 1197) 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:3931: done (8 left) 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:3932: done (7 left) 2024-03-20T08:15:06.236 INFO:tasks.rados.rados.0.smithi118.stdout:3933: done (6 left) 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3934: done (5 left) 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3935: done (4 left) 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3936: done (3 left) 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3937: done (2 left) 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3938: done (1 left) 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3939: done (0 left) 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3940: read oid 23 snap -1 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3940: expect (ObjNum 1137 snap 367 seq_num 1137) 2024-03-20T08:15:06.237 INFO:tasks.rados.rados.0.smithi118.stdout:3941: snap_remove snap 383 2024-03-20T08:15:06.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:06.178+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:06.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:06.179+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:06.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:06.179+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:06.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:06.179+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:06.897 INFO:tasks.rados.rados.0.smithi118.stdout:3941: done (1 left) 2024-03-20T08:15:06.898 INFO:tasks.rados.rados.0.smithi118.stdout:3942: rmattr oid 43 current snap is 388 2024-03-20T08:15:06.898 INFO:tasks.rados.rados.0.smithi118.stdout:3942: done (1 left) 2024-03-20T08:15:06.898 INFO:tasks.rados.rados.0.smithi118.stdout:3943: rollback oid 22 current snap is 388 2024-03-20T08:15:06.898 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 22 to 384 2024-03-20T08:15:06.898 INFO:tasks.rados.rados.0.smithi118.stdout:3940: expect (ObjNum 1137 snap 367 seq_num 1137) 2024-03-20T08:15:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:07.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:07.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:07.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:06.905+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:07.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:06.905+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:07.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:06.906+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:07.174 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:06.905+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:07.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:06.896+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:07.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:06.896+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:07.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:06.897+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:07.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:06.897+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:07.174 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:06 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:06.898+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3943: finishing rollback tid 1 to smithi11881747-22 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 65313 (ObjNum 113 snap 0 seq_num 322654400) dirty dne 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3940: done (1 left) 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3943: done (0 left) 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3944: read oid 15 snap -1 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3944: expect (ObjNum 1129 snap 361 seq_num 1129) 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3945: rollback oid 4 current snap is 388 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 4 to 385 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3946: read oid 6 snap -1 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3946: expect (ObjNum 1068 snap 341 seq_num 1068) 2024-03-20T08:15:07.268 INFO:tasks.rados.rados.0.smithi118.stdout:3947: setattr oid 36 current snap is 388 2024-03-20T08:15:07.269 INFO:tasks.rados.rados.0.smithi118.stdout:3948: delete oid 42 current snap is 388 2024-03-20T08:15:07.274 INFO:tasks.rados.rados.0.smithi118.stdout:3948: done (4 left) 2024-03-20T08:15:07.274 INFO:tasks.rados.rados.0.smithi118.stdout:3949: delete oid 23 current snap is 388 2024-03-20T08:15:07.277 INFO:tasks.rados.rados.0.smithi118.stdout:3945: finishing rollback tid 0 to smithi11881747-4 2024-03-20T08:15:07.279 INFO:tasks.rados.rados.0.smithi118.stdout:3949: done (4 left) 2024-03-20T08:15:07.279 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 41 current snap is 388 2024-03-20T08:15:07.279 INFO:tasks.rados.rados.0.smithi118.stdout:3950: seq_num 1200 ranges {7471104=499712,7970816=401408} 2024-03-20T08:15:07.323 INFO:tasks.rados.rados.0.smithi118.stdout:3950: writing smithi11881747-41 from 7471104 to 7970816 tid 1 2024-03-20T08:15:07.325 INFO:tasks.rados.rados.0.smithi118.stdout:3950: writing smithi11881747-41 from 7970816 to 8372224 tid 2 2024-03-20T08:15:07.325 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 19 current snap is 388 2024-03-20T08:15:07.325 INFO:tasks.rados.rados.0.smithi118.stdout:3951: seq_num 1201 ranges {0=524288,524288=401408} 2024-03-20T08:15:07.328 INFO:tasks.rados.rados.0.smithi118.stdout:3951: writing smithi11881747-19 from 0 to 524288 tid 1 2024-03-20T08:15:07.331 INFO:tasks.rados.rados.0.smithi118.stdout:3951: writing smithi11881747-19 from 524288 to 925696 tid 2 2024-03-20T08:15:07.331 INFO:tasks.rados.rados.0.smithi118.stdout:3952: snap_create 2024-03-20T08:15:07.331 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 65351 (ObjNum 177 snap 0 seq_num 322852512) dirty exists 2024-03-20T08:15:07.331 INFO:tasks.rados.rados.0.smithi118.stdout:3945: finishing rollback tid 1 to smithi11881747-4 2024-03-20T08:15:07.331 INFO:tasks.rados.rados.0.smithi118.stdout:3945: finishing rollback tid 2 to smithi11881747-4 2024-03-20T08:15:07.331 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 4 v 65325 (ObjNum 1144 snap 370 seq_num 1144) dirty exists 2024-03-20T08:15:07.331 INFO:tasks.rados.rados.0.smithi118.stdout:3946: expect (ObjNum 1068 snap 341 seq_num 1068) 2024-03-20T08:15:07.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:06.907+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:07.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:06.907+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:07.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:07.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:07.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:07.390 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:06.897+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:07.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:06.905+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:07.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:06.907+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:07.391 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:06.907+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:07.391 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:06.906+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:07.391 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:06.906+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:07.391 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:06 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:06.906+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:07.446 INFO:tasks.rados.rados.0.smithi118.stdout:3944: expect (ObjNum 1129 snap 361 seq_num 1129) 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3950: finishing write tid 1 to smithi11881747-41 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3950: finishing write tid 2 to smithi11881747-41 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3950: finishing write tid 3 to smithi11881747-41 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3950: finishing write tid 4 to smithi11881747-41 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3950: oid 41 updating version 0 to 65365 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3950: oid 41 version 65365 is already newer than 65364 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3950: oid 41 version 65365 is already newer than 65363 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 41 v 65365 (ObjNum 1200 snap 388 seq_num 1200) dirty exists 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3950: left oid 41 (ObjNum 1200 snap 388 seq_num 1200) 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3951: finishing write tid 1 to smithi11881747-19 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3951: finishing write tid 2 to smithi11881747-19 2024-03-20T08:15:07.902 INFO:tasks.rados.rados.0.smithi118.stdout:3951: finishing write tid 3 to smithi11881747-19 2024-03-20T08:15:07.903 INFO:tasks.rados.rados.0.smithi118.stdout:3951: finishing write tid 4 to smithi11881747-19 2024-03-20T08:15:07.903 INFO:tasks.rados.rados.0.smithi118.stdout:3951: oid 19 updating version 0 to 65336 2024-03-20T08:15:07.903 INFO:tasks.rados.rados.0.smithi118.stdout:3951: oid 19 version 65336 is already newer than 65335 2024-03-20T08:15:07.903 INFO:tasks.rados.rados.0.smithi118.stdout:3951: oid 19 updating version 65336 to 65337 2024-03-20T08:15:07.903 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 19 v 65337 (ObjNum 1201 snap 388 seq_num 1201) dirty exists 2024-03-20T08:15:07.903 INFO:tasks.rados.rados.0.smithi118.stdout:3951: left oid 19 (ObjNum 1201 snap 388 seq_num 1201) 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3944: done (6 left) 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3945: done (5 left) 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3946: done (4 left) 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3947: done (3 left) 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3950: done (2 left) 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3951: done (1 left) 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3952: done (0 left) 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3953: rollback oid 22 current snap is 389 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 22 to 376 2024-03-20T08:15:07.905 INFO:tasks.rados.rados.0.smithi118.stdout:3954: snap_create 2024-03-20T08:15:07.912 INFO:tasks.rados.rados.0.smithi118.stdout:3953: finishing rollback tid 1 to smithi11881747-22 2024-03-20T08:15:07.912 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 22 v 65313 (ObjNum 97 snap 0 seq_num 0) dirty dne 2024-03-20T08:15:08.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:07.905+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:08.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:07.911+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:08.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:07.911+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:08.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:07.904+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:08.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:07.904+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:08.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: pgmap v965: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 8.2 MiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: osdmap e850: 8 total, 8 up, 8 in 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:08.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: pgmap v965: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 8.2 MiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: osdmap e850: 8 total, 8 up, 8 in 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:08.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:08.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:08.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:08.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:08.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:08.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:07 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:08.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:07.904+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:08.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:07.912+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:08.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:07.912+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:08.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:08.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:08.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: pgmap v965: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 8.2 MiB/s rd, 1.5 MiB/s wr, 5 op/s 2024-03-20T08:15:08.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:08.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:08.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: osdmap e850: 8 total, 8 up, 8 in 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:08.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:08.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:07 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:08.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:08.450+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:08.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:08.450+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:08.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:08.450+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:08.889 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:08.646+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:08.890 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:08.646+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:08.905 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.645+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:08.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.646+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:08.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.646+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:08.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.646+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:08.906 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.646+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:08.910 INFO:tasks.rados.rados.0.smithi118.stdout:3953: done (1 left) 2024-03-20T08:15:08.910 INFO:tasks.rados.rados.0.smithi118.stdout:3954: done (0 left) 2024-03-20T08:15:08.910 INFO:tasks.rados.rados.0.smithi118.stdout:3955: copy_from oid 7 from oid 45 current snap is 390 2024-03-20T08:15:08.910 INFO:tasks.rados.rados.0.smithi118.stdout:3956: snap_remove snap 388 2024-03-20T08:15:09.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:08.947+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:09.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:08.911+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.908+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.909+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.909+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:09.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:08.909+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[95692]: osdmap e851: 8 total, 8 up, 8 in 2024-03-20T08:15:09.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[93121]: osdmap e851: 8 total, 8 up, 8 in 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:09.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:09.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:09.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:09.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:09.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:09.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:08.914+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:09.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:08.914+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:09.175 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:08 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:08.915+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:09.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:08.910+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:09.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:08.948+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:09.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:08.948+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:09.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:08.948+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:09.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:08.918+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:09.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:08.918+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:09.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:08.917+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:09.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:08.917+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:09.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:08.918+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:09.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-mon[87080]: osdmap e851: 8 total, 8 up, 8 in 2024-03-20T08:15:09.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:09.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:09.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:09.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:09.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:09.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:09.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:08 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:09.911 INFO:tasks.rados.rados.0.smithi118.stdout:3956: done (1 left) 2024-03-20T08:15:09.911 INFO:tasks.rados.rados.0.smithi118.stdout:3957: read oid 16 snap 389 2024-03-20T08:15:09.911 INFO:tasks.rados.rados.0.smithi118.stdout:3957: expect (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:15:09.911 INFO:tasks.rados.rados.0.smithi118.stdout:3958: snap_remove snap 376 2024-03-20T08:15:10.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:09.910+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:10.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:09.910+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:10.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:09.912+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:10.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:09.912+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:10.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:09.914+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:10.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:09.917+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:10.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:09.917+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:10.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:09.917+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:10.174 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:09.917+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: pgmap v968: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 20 MiB/s rd, 654 KiB/s wr, 7 op/s 2024-03-20T08:15:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:10.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: pgmap v968: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 20 MiB/s rd, 654 KiB/s wr, 7 op/s 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:10.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: osdmap e852: 8 total, 8 up, 8 in 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:10.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: osdmap e852: 8 total, 8 up, 8 in 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:10.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:10.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:10.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:10.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:10.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:10.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:10.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:10.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:09 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:10.332 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:09.912+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:10.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:09.917+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:10.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:09.917+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:10.332 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:09.917+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: pgmap v968: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 20 MiB/s rd, 654 KiB/s wr, 7 op/s 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:10.333 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: osdmap e852: 8 total, 8 up, 8 in 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:10.334 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:09 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:10.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:10.332+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:10.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:10.332+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:10.923 INFO:tasks.rados.rados.0.smithi118.stdout:3958: done (2 left) 2024-03-20T08:15:10.923 INFO:tasks.rados.rados.0.smithi118.stdout:3959: read oid 10 snap -1 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:3959: expect (ObjNum 1160 snap 375 seq_num 1160) 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:3960: rollback oid 36 current snap is 390 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 36 to 384 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:3955: finishing copy_from racing read to smithi11881747-7 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:3955: finishing copy_from to smithi11881747-7 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 65327 (ObjNum 1163 snap 376 seq_num 1163) dirty exists 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:3955: done (3 left) 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:3961: rollback oid 26 current snap is 390 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 26 to 387 2024-03-20T08:15:10.924 INFO:tasks.rados.rados.0.smithi118.stdout:3957: expect (ObjNum 1128 snap 361 seq_num 1128) 2024-03-20T08:15:11.087 INFO:tasks.rados.rados.0.smithi118.stdout:3959: expect (ObjNum 1160 snap 375 seq_num 1160) 2024-03-20T08:15:11.093 INFO:tasks.rados.rados.0.smithi118.stdout:3960: finishing rollback tid 0 to smithi11881747-36 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3960: finishing rollback tid 1 to smithi11881747-36 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3960: finishing rollback tid 2 to smithi11881747-36 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 65354 (ObjNum 1144 snap 370 seq_num 1144) dirty exists 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3961: finishing rollback tid 0 to smithi11881747-26 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3957: done (3 left) 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3959: done (2 left) 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3960: done (1 left) 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3962: delete oid 12 current snap is 390 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3961: finishing rollback tid 1 to smithi11881747-26 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:3961: finishing rollback tid 2 to smithi11881747-26 2024-03-20T08:15:11.094 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65370 (ObjNum 1130 snap 361 seq_num 1130) dirty exists 2024-03-20T08:15:11.099 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:10.921+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:11.099 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:10.745+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:11.099 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:10.745+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:11.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:10.927+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:11.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:10.941+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:11.100 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:10.941+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:11.100 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:10.930+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:11.100 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:10.930+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:11.100 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:11.100 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:11.100 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: osdmap e853: 8 total, 8 up, 8 in 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:11.101 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:10 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:3961: done (1 left) 2024-03-20T08:15:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:3962: done (0 left) 2024-03-20T08:15:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:3963: rmattr oid 43 current snap is 390 2024-03-20T08:15:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:3963: done (0 left) 2024-03-20T08:15:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:3964: rollback oid 7 current snap is 390 2024-03-20T08:15:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 7 to 384 2024-03-20T08:15:11.102 INFO:tasks.rados.rados.0.smithi118.stdout:3965: snap_remove snap 387 2024-03-20T08:15:11.172 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:10.922+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:11.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:10.926+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:11.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:10.936+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:11.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:10.936+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:11.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:10.745+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:11.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:10.921+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:11.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:10.922+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:11.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:10.923+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:11.173 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:10.923+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: osdmap e853: 8 total, 8 up, 8 in 2024-03-20T08:15:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:11.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: osdmap e853: 8 total, 8 up, 8 in 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:11.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:11.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:10 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:11.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:11.098+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:11.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:11.098+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:11.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:11.098+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:11.928 INFO:tasks.rados.rados.0.smithi118.stdout:3965: done (1 left) 2024-03-20T08:15:11.928 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 39 current snap is 390 2024-03-20T08:15:11.928 INFO:tasks.rados.rados.0.smithi118.stdout:3966: seq_num 1203 ranges {0=557056,557056=401408} 2024-03-20T08:15:11.931 INFO:tasks.rados.rados.0.smithi118.stdout:3966: writing smithi11881747-39 from 0 to 557056 tid 1 2024-03-20T08:15:11.934 INFO:tasks.rados.rados.0.smithi118.stdout:3966: writing smithi11881747-39 from 557056 to 958464 tid 2 2024-03-20T08:15:11.934 INFO:tasks.rados.rados.0.smithi118.stdout:3964: finishing rollback tid 0 to smithi11881747-7 2024-03-20T08:15:11.934 INFO:tasks.rados.rados.0.smithi118.stdout:3964: finishing rollback tid 1 to smithi11881747-7 2024-03-20T08:15:11.934 INFO:tasks.rados.rados.0.smithi118.stdout:3964: finishing rollback tid 2 to smithi11881747-7 2024-03-20T08:15:11.934 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 7 v 65331 (ObjNum 1166 snap 377 seq_num 1166) dirty exists 2024-03-20T08:15:11.934 INFO:tasks.rados.rados.0.smithi118.stdout:3964: done (1 left) 2024-03-20T08:15:11.934 INFO:tasks.rados.rados.0.smithi118.stdout:3967: snap_create 2024-03-20T08:15:11.935 INFO:tasks.rados.rados.0.smithi118.stdout:3966: finishing write tid 1 to smithi11881747-39 2024-03-20T08:15:11.937 INFO:tasks.rados.rados.0.smithi118.stdout:3966: finishing write tid 2 to smithi11881747-39 2024-03-20T08:15:11.937 INFO:tasks.rados.rados.0.smithi118.stdout:3966: finishing write tid 3 to smithi11881747-39 2024-03-20T08:15:11.939 INFO:tasks.rados.rados.0.smithi118.stdout:3966: finishing write tid 4 to smithi11881747-39 2024-03-20T08:15:11.939 INFO:tasks.rados.rados.0.smithi118.stdout:3966: oid 39 updating version 0 to 65335 2024-03-20T08:15:11.939 INFO:tasks.rados.rados.0.smithi118.stdout:3966: oid 39 updating version 65335 to 65337 2024-03-20T08:15:11.939 INFO:tasks.rados.rados.0.smithi118.stdout:3966: oid 39 version 65337 is already newer than 65336 2024-03-20T08:15:11.939 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 39 v 65337 (ObjNum 1203 snap 390 seq_num 1203) dirty exists 2024-03-20T08:15:11.939 INFO:tasks.rados.rados.0.smithi118.stdout:3966: left oid 39 (ObjNum 1203 snap 390 seq_num 1203) 2024-03-20T08:15:12.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:11.957+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:12.172 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:11.957+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:12.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:11.936+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:12.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:11.929+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:12.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:11.929+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: pgmap v971: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 24 MiB/s rd, 654 KiB/s wr, 9 op/s 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:12.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: pgmap v971: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 24 MiB/s rd, 654 KiB/s wr, 9 op/s 2024-03-20T08:15:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:12.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:12.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:11.926+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:12.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:11.927+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:12.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:11.927+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:12.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:11.927+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:12.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:11.927+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:12.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:12.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:12.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: osdmap e854: 8 total, 8 up, 8 in 2024-03-20T08:15:12.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:12.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:12.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:12.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:12.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: osdmap e854: 8 total, 8 up, 8 in 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:12.177 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:12.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:12.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:12.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:12.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:12.178 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:11 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:12.313 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:11.929+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:12.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:11.935+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:12.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:11.935+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:12.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:11.935+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:12.315 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:11.958+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:12.315 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:11.958+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:12.315 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:11.958+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:12.316 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:11.958+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:12.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: pgmap v971: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 24 MiB/s rd, 654 KiB/s wr, 9 op/s 2024-03-20T08:15:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:12.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:12.318 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:12.318 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:12.318 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:12.318 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:12.318 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: osdmap e854: 8 total, 8 up, 8 in 2024-03-20T08:15:12.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:12.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:12.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:12.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:12.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:12.319 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:12.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:12.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:12.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:12.320 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:11 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:15:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:15:12.934 INFO:tasks.rados.rados.0.smithi118.stdout:3966: done (1 left) 2024-03-20T08:15:12.934 INFO:tasks.rados.rados.0.smithi118.stdout:3967: done (0 left) 2024-03-20T08:15:12.934 INFO:tasks.rados.rados.0.smithi118.stdout:3968: snap_create 2024-03-20T08:15:13.305 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:12.934+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:13.305 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:12.932+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:13.305 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:13.305 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: osdmap e855: 8 total, 8 up, 8 in 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:13.306 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: osdmap e855: 8 total, 8 up, 8 in 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:13.307 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:13.308 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:12 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:15:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:13.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: osdmap e855: 8 total, 8 up, 8 in 2024-03-20T08:15:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:13.390 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:12.933+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:13.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:13.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:12 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:15:13.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.305+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:13.585 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.305+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.305+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:13.586 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.305+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:15:13] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:15:13.932 INFO:tasks.rados.rados.0.smithi118.stdout:3968: done (0 left) 2024-03-20T08:15:13.932 INFO:tasks.rados.rados.0.smithi118.stdout:3969: snap_remove snap 389 2024-03-20T08:15:13.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:13.657+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:13.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:13.657+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:13.933 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:13.657+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:13.933 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:13.717+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:13.933 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:13.717+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:13.933 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:13.717+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:13.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.932+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:13.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.932+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:13.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.932+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:13.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.932+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:13.935 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:13.932+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:13.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: pgmap v974: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 7.4 MiB/s rd, 226 KiB/s wr, 9 op/s 2024-03-20T08:15:13.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:13.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: osdmap e856: 8 total, 8 up, 8 in 2024-03-20T08:15:13.935 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:13.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:13.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:13.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:13.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:13.936 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:15:14.218 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:15:14.218 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:14.218 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:14.218 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:14.218 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: pgmap v974: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 7.4 MiB/s rd, 226 KiB/s wr, 9 op/s 2024-03-20T08:15:14.218 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:14.218 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: osdmap e856: 8 total, 8 up, 8 in 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:14.219 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:13 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:14.220 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:13.933+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:14.220 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:13.933+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:14.220 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:13.933+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:14.220 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:13.933+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:14.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: pgmap v974: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 7.4 MiB/s rd, 226 KiB/s wr, 9 op/s 2024-03-20T08:15:14.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:14.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: osdmap e856: 8 total, 8 up, 8 in 2024-03-20T08:15:14.220 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:14.221 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:13 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:14.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:14.219+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:14.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:14.219+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:14.640 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:14.218+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:14.641 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:14.218+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:14.216+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:14.216+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:14.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:14.217+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:14.944 INFO:tasks.rados.rados.0.smithi118.stdout:3969: done (0 left) 2024-03-20T08:15:14.945 INFO:tasks.rados.rados.0.smithi118.stdout:3970: snap_remove snap 385 2024-03-20T08:15:15.172 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:14.942+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:15.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:14.945+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:15.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:14.945+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:15.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:14.847+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:15.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:14.847+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:15.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:14.947+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:15.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:14.947+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:15.173 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:14.947+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:15.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:15.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:15.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:15.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:15.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:15.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:15.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:15.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:15.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:15.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:15.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: osdmap e857: 8 total, 8 up, 8 in 2024-03-20T08:15:15.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:15.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:14.947+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:15.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:14.947+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:15.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:14.948+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:15.176 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:14.948+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:15.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:15.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:15.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:15.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:15.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:15.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: osdmap e857: 8 total, 8 up, 8 in 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:15.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:15.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:15.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:15.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:15.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:15.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:14 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:15.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:14.943+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:15.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:14.948+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:15.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:14.949+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:15.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:14.949+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:15.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:14.951+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:15.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:14.951+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:15.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:14.950+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:15.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:14.950+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:15.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:14.950+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: osdmap e857: 8 total, 8 up, 8 in 2024-03-20T08:15:15.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:15.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:14 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:15.943 INFO:tasks.rados.rados.0.smithi118.stdout:3970: done (0 left) 2024-03-20T08:15:15.943 INFO:tasks.rados.rados.0.smithi118.stdout:3971: rmattr oid 1 current snap is 392 2024-03-20T08:15:15.943 INFO:tasks.rados.rados.0.smithi118.stdout:3971: done (0 left) 2024-03-20T08:15:15.943 INFO:tasks.rados.rados.0.smithi118.stdout:3972: snap_remove snap 384 2024-03-20T08:15:16.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:15.945+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:16.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:16.082+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:16.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:16.082+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:16.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:16.082+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:16.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:16.083+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:16.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:16.083+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:16.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:16.082+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:16.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:16.082+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: pgmap v977: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 3.1 MiB/s rd, 70 KiB/s wr, 8 op/s 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: osdmap e858: 8 total, 8 up, 8 in 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:16.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:16.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:16.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:16.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:16.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:15 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:16.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:16.079+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:16.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:16.079+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:16.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:15.942+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.079+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.080+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.080+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.080+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:16.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.080+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:16.424 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:15.944+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: pgmap v977: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 3.1 MiB/s rd, 70 KiB/s wr, 8 op/s 2024-03-20T08:15:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:16.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: osdmap e858: 8 total, 8 up, 8 in 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: pgmap v977: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 3.1 MiB/s rd, 70 KiB/s wr, 8 op/s 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:16.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: osdmap e858: 8 total, 8 up, 8 in 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:16.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:16.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:16.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:16.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:16.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:15 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:16.956 INFO:tasks.rados.rados.0.smithi118.stdout:3972: done (0 left) 2024-03-20T08:15:16.956 INFO:tasks.rados.rados.0.smithi118.stdout:3973: delete oid 18 current snap is 392 2024-03-20T08:15:16.959 INFO:tasks.rados.rados.0.smithi118.stdout:3973: done (0 left) 2024-03-20T08:15:16.959 INFO:tasks.rados.rados.0.smithi118.stdout:3974: setattr oid 31 current snap is 392 2024-03-20T08:15:16.961 INFO:tasks.rados.rados.0.smithi118.stdout:3975: copy_from oid 2 from oid 7 current snap is 392 2024-03-20T08:15:16.961 INFO:tasks.rados.rados.0.smithi118.stdout:3976: read oid 50 snap -1 2024-03-20T08:15:16.962 INFO:tasks.rados.rados.0.smithi118.stdout:3976: expect (ObjNum 1176 snap 379 seq_num 1176) 2024-03-20T08:15:16.962 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 18 current snap is 392 2024-03-20T08:15:16.962 INFO:tasks.rados.rados.0.smithi118.stdout:3977: seq_num 1205 ranges {0=589824,589824=401408} 2024-03-20T08:15:16.964 INFO:tasks.rados.rados.0.smithi118.stdout:3977: writing smithi11881747-18 from 0 to 589824 tid 1 2024-03-20T08:15:16.967 INFO:tasks.rados.rados.0.smithi118.stdout:3977: writing smithi11881747-18 from 589824 to 991232 tid 2 2024-03-20T08:15:16.967 INFO:tasks.rados.rados.0.smithi118.stdout:3978: read oid 21 snap -1 2024-03-20T08:15:16.968 INFO:tasks.rados.rados.0.smithi118.stdout:3978: expect (ObjNum 1197 snap 387 seq_num 1197) 2024-03-20T08:15:16.968 INFO:tasks.rados.rados.0.smithi118.stdout:3979: delete oid 23 current snap is 392 2024-03-20T08:15:16.973 INFO:tasks.rados.rados.0.smithi118.stdout:3977: finishing write tid 1 to smithi11881747-18 2024-03-20T08:15:16.973 INFO:tasks.rados.rados.0.smithi118.stdout:3979: done (5 left) 2024-03-20T08:15:16.973 INFO:tasks.rados.rados.0.smithi118.stdout:3980: setattr oid 43 current snap is 392 2024-03-20T08:15:16.973 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 31 v 65333 (ObjNum 1171 snap 378 seq_num 1171) dirty exists 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3977: finishing write tid 2 to smithi11881747-18 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3977: finishing write tid 3 to smithi11881747-18 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3977: finishing write tid 4 to smithi11881747-18 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3977: oid 18 updating version 0 to 65340 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3977: oid 18 updating version 65340 to 65341 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3977: oid 18 version 65341 is already newer than 65339 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 65341 (ObjNum 1205 snap 392 seq_num 1205) dirty exists 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3977: left oid 18 (ObjNum 1205 snap 392 seq_num 1205) 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3974: done (5 left) 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3977: done (4 left) 2024-03-20T08:15:16.974 INFO:tasks.rados.rados.0.smithi118.stdout:3981: snap_create 2024-03-20T08:15:16.976 INFO:tasks.rados.rados.0.smithi118.stdout:3975: finishing copy_from racing read to smithi11881747-2 2024-03-20T08:15:16.977 INFO:tasks.rados.rados.0.smithi118.stdout:3975: finishing copy_from to smithi11881747-2 2024-03-20T08:15:16.977 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 2 v 65331 (ObjNum 1166 snap 377 seq_num 1166) dirty exists 2024-03-20T08:15:16.982 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 43 v 65334 (ObjNum 1195 snap 386 seq_num 1195) dirty exists 2024-03-20T08:15:16.988 INFO:tasks.rados.rados.0.smithi118.stdout:3978: expect (ObjNum 1197 snap 387 seq_num 1197) 2024-03-20T08:15:17.161 INFO:tasks.rados.rados.0.smithi118.stdout:3976: expect (ObjNum 1176 snap 379 seq_num 1176) 2024-03-20T08:15:17.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:16.956+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:17.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:16.971+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:17.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:16.971+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:17.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:16.960+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:17.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:16.961+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:17.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:16.961+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:17.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:16.966+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:17.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:16.967+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:17.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:16.967+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: osdmap e859: 8 total, 8 up, 8 in 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:17.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:17.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:16 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:17.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:16.959+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:17.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:16.956+0000 7f0cf2464700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:17.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:16.956+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:17.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:16.956+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:17.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:16.954+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:17.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:16.954+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:17.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.966+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:17.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.967+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:17.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.968+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:17.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:16.968+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:17.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:17.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:17.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:17.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:17.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: osdmap e859: 8 total, 8 up, 8 in 2024-03-20T08:15:17.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:17.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:17.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:17.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:17.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:17.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:17.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:17.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:17.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: osdmap e859: 8 total, 8 up, 8 in 2024-03-20T08:15:17.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:17.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:17.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:17.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:17.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:17.428 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:16 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:18.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.004+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:18.348 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.004+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:18.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.004+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:18.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.004+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:18.349 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.004+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:18.349 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: pgmap v980: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 3.3 MiB/s rd, 304 KiB/s wr, 4 op/s 2024-03-20T08:15:18.349 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:18.349 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: osdmap e860: 8 total, 8 up, 8 in 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:18.350 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:18.351 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:18.351 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:18.351 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:18.351 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:18.351 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:17.959+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c3=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:18.351 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:17.960+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: pgmap v980: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 3.3 MiB/s rd, 304 KiB/s wr, 4 op/s 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: osdmap e860: 8 total, 8 up, 8 in 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:18.352 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:18.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:18.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:18.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:18.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:18.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:18.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:18.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:18.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:17 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:18.353 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:17.960+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: pgmap v980: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 3.3 MiB/s rd, 304 KiB/s wr, 4 op/s 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1ba=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: osdmap e860: 8 total, 8 up, 8 in 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:18.354 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:18.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:18.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:18.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:18.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:18.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:18.355 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:17 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:18.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:18.349+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:18.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:18.351+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:18.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:18.350+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:18.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:18.350+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:18.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:18.348+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:18.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:18.348+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:18.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:18.350+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:18.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:18.347+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:18.672 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:18.347+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:18.748 INFO:tasks.rados.rados.0.smithi118.stdout:3975: done (4 left) 2024-03-20T08:15:18.748 INFO:tasks.rados.rados.0.smithi118.stdout:3976: done (3 left) 2024-03-20T08:15:18.748 INFO:tasks.rados.rados.0.smithi118.stdout:3978: done (2 left) 2024-03-20T08:15:18.748 INFO:tasks.rados.rados.0.smithi118.stdout:3980: done (1 left) 2024-03-20T08:15:18.748 INFO:tasks.rados.rados.0.smithi118.stdout:3981: done (0 left) 2024-03-20T08:15:18.748 INFO:tasks.rados.rados.0.smithi118.stdout:3982: copy_from oid 15 from oid 17 current snap is 393 2024-03-20T08:15:18.748 INFO:tasks.rados.rados.0.smithi118.stdout:3983: snap_remove snap 392 2024-03-20T08:15:18.965 INFO:tasks.rados.rados.0.smithi118.stdout:3983: done (1 left) 2024-03-20T08:15:18.966 INFO:tasks.rados.rados.0.smithi118.stdout:3984: copy_from oid 18 from oid 46 current snap is 393 2024-03-20T08:15:18.966 INFO:tasks.rados.rados.0.smithi118.stdout:3982: finishing copy_from racing read to smithi11881747-15 2024-03-20T08:15:18.966 INFO:tasks.rados.rados.0.smithi118.stdout:3982: finishing copy_from to smithi11881747-15 2024-03-20T08:15:18.966 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 15 v 61750 (ObjNum 1132 snap 364 seq_num 1132) dirty exists 2024-03-20T08:15:18.966 INFO:tasks.rados.rados.0.smithi118.stdout:3982: done (1 left) 2024-03-20T08:15:18.966 INFO:tasks.rados.rados.0.smithi118.stdout:3985: snap_create 2024-03-20T08:15:19.019 INFO:tasks.rados.rados.0.smithi118.stdout:3984: finishing copy_from racing read to smithi11881747-18 2024-03-20T08:15:19.019 INFO:tasks.rados.rados.0.smithi118.stdout:3984: finishing copy_from to smithi11881747-18 2024-03-20T08:15:19.019 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 18 v 65325 (ObjNum 1148 snap 370 seq_num 1148) dirty exists 2024-03-20T08:15:19.216 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:18.973+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:19.217 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:18.999+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:19.217 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:18.999+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:19.217 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:18.999+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:19.217 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:18.967+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:19.217 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:18.967+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:19.217 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:18.967+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c3=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: osdmap e861: 8 total, 8 up, 8 in 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:19.218 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:18 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:19.422 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:18.998+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:19.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:18.969+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:19.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:18.969+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c3=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:19.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.964+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:19.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.964+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:19.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.965+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:19.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:18.965+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c3=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: osdmap e861: 8 total, 8 up, 8 in 2024-03-20T08:15:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:19.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c3=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:19.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: osdmap e861: 8 total, 8 up, 8 in 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:19.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:18 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:19.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:19.216+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:19.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:19.216+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:19.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:19.544+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:19.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:19.544+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:19.922 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:19.544+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:19.971 INFO:tasks.rados.rados.0.smithi118.stdout:3984: done (1 left) 2024-03-20T08:15:19.971 INFO:tasks.rados.rados.0.smithi118.stdout:3985: done (0 left) 2024-03-20T08:15:19.971 INFO:tasks.rados.rados.0.smithi118.stdout:3986: read oid 18 snap -1 2024-03-20T08:15:19.972 INFO:tasks.rados.rados.0.smithi118.stdout:3986: expect (ObjNum 1148 snap 370 seq_num 1148) 2024-03-20T08:15:19.972 INFO:tasks.rados.rados.0.smithi118.stdout:3987: copy_from oid 37 from oid 43 current snap is 394 2024-03-20T08:15:19.972 INFO:tasks.rados.rados.0.smithi118.stdout:3988: rollback oid 10 current snap is 394 2024-03-20T08:15:19.972 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 10 to 390 2024-03-20T08:15:19.972 INFO:tasks.rados.rados.0.smithi118.stdout:3989: snap_remove snap 391 2024-03-20T08:15:20.383 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:19.972+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:20.384 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:19.981+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:20.384 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:19.986+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:20.384 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:19.977+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:20.384 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:19.977+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:20.384 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:19.977+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: pgmap v983: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 234 KiB/s wr, 3 op/s 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: osdmap e862: 8 total, 8 up, 8 in 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:20.385 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c3=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:20.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:20.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:20.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:20.386 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:19 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:20.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:20.019+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:20.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:20.008+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:20.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:20.009+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:20.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:19.977+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:19.969+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:19.969+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:19.970+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:20.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:19.970+0000 7f831a6a6700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:20.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:19.970+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: pgmap v983: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 234 KiB/s wr, 3 op/s 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: osdmap e862: 8 total, 8 up, 8 in 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:20.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: pgmap v983: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 2.2 MiB/s rd, 234 KiB/s wr, 3 op/s 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: osdmap e862: 8 total, 8 up, 8 in 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:20.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:20.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c3=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:20.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:20.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:20.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c3=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:20.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:20.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:20.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:20.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:19 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:20.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:20.383+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:20.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:20.383+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:20.978 INFO:tasks.rados.rados.0.smithi118.stdout:3989: done (3 left) 2024-03-20T08:15:20.978 INFO:tasks.rados.rados.0.smithi118.stdout:3990: snap_create 2024-03-20T08:15:20.978 INFO:tasks.rados.rados.0.smithi118.stdout:3988: finishing rollback tid 0 to smithi11881747-10 2024-03-20T08:15:20.978 INFO:tasks.rados.rados.0.smithi118.stdout:3988: finishing rollback tid 1 to smithi11881747-10 2024-03-20T08:15:20.978 INFO:tasks.rados.rados.0.smithi118.stdout:3988: finishing rollback tid 2 to smithi11881747-10 2024-03-20T08:15:20.978 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 10 v 65337 (ObjNum 1160 snap 375 seq_num 1160) dirty exists 2024-03-20T08:15:20.978 INFO:tasks.rados.rados.0.smithi118.stdout:3987: finishing copy_from racing read to smithi11881747-37 2024-03-20T08:15:20.978 INFO:tasks.rados.rados.0.smithi118.stdout:3987: finishing copy_from to smithi11881747-37 2024-03-20T08:15:20.979 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 37 v 65334 (ObjNum 1195 snap 386 seq_num 1195) dirty exists 2024-03-20T08:15:20.979 INFO:tasks.rados.rados.0.smithi118.stdout:3986: expect (ObjNum 1148 snap 370 seq_num 1148) 2024-03-20T08:15:21.246 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:20.976+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:21.247 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:21.119+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:21.247 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:21.119+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:21.247 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:21.119+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:21.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:21.247 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: osdmap e863: 8 total, 8 up, 8 in 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:21.248 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:21.249 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:21.249 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:21.249 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:21.249 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:20 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:21.380 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:20.977+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:21.381 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:20.977+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:21.381 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:20.978+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:21.381 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:21.381 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:21.381 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:21.381 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:21.381 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: osdmap e863: 8 total, 8 up, 8 in 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:21.382 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: osdmap e863: 8 total, 8 up, 8 in 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:21.383 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:21.384 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:21.384 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:21.384 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:21.384 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:20.977+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:21.384 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:20.977+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:21.384 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:21.384 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:21.384 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:21.384 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:21.385 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:21.385 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:21.385 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:21.385 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:20 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:21.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:21.246+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:21.639 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:21.246+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:21.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:21.258+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:21.639 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:21.258+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:21.640 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:21.258+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:21.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:21.380+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:21.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.381+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:21.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.381+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:21.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.381+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:21.672 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.382+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:22.309 INFO:tasks.rados.rados.0.smithi118.stdout:3986: done (3 left) 2024-03-20T08:15:22.309 INFO:tasks.rados.rados.0.smithi118.stdout:3987: done (2 left) 2024-03-20T08:15:22.309 INFO:tasks.rados.rados.0.smithi118.stdout:3988: done (1 left) 2024-03-20T08:15:22.310 INFO:tasks.rados.rados.0.smithi118.stdout:3990: done (0 left) 2024-03-20T08:15:22.310 INFO:tasks.rados.rados.0.smithi118.stdout:3991: setattr oid 5 current snap is 395 2024-03-20T08:15:22.310 INFO:tasks.rados.rados.0.smithi118.stdout:3992: snap_remove snap 390 2024-03-20T08:15:22.314 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:21.979+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:22.314 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:22.108+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:22.314 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:22.108+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:22.314 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:22.107+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:22.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:22.107+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:22.315 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:22.107+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:22.315 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:22.108+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:22.315 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:22.108+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:22.315 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: pgmap v986: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 14 op/s 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: osdmap e864: 8 total, 8 up, 8 in 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:22.316 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:22.317 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:21 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:22.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:22.106+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:22.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:22.106+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:22.423 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:22.106+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:22.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:21.980+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:22.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.977+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:22.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.978+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:22.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.978+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:22.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.978+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:22.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:21.979+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: pgmap v986: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 14 op/s 2024-03-20T08:15:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: osdmap e864: 8 total, 8 up, 8 in 2024-03-20T08:15:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:22.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:22.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: pgmap v986: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 14 op/s 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: osdmap e864: 8 total, 8 up, 8 in 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:22.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:22.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:21 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:15:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:15:22.990 INFO:tasks.rados.rados.0.smithi118.stdout:3992: done (1 left) 2024-03-20T08:15:22.991 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 36 current snap is 395 2024-03-20T08:15:22.991 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 5 v 65356 (ObjNum 289 snap 0 seq_num 0) dirty exists 2024-03-20T08:15:22.991 INFO:tasks.rados.rados.0.smithi118.stdout:3993: seq_num 1207 ranges {2375680=622592,2998272=401408} 2024-03-20T08:15:23.007 INFO:tasks.rados.rados.0.smithi118.stdout:3993: writing smithi11881747-36 from 2375680 to 2998272 tid 1 2024-03-20T08:15:23.009 INFO:tasks.rados.rados.0.smithi118.stdout:3993: writing smithi11881747-36 from 2998272 to 3399680 tid 2 2024-03-20T08:15:23.009 INFO:tasks.rados.rados.0.smithi118.stdout:3991: done (1 left) 2024-03-20T08:15:23.009 INFO:tasks.rados.rados.0.smithi118.stdout:3994: setattr oid 35 current snap is 395 2024-03-20T08:15:23.011 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 26 current snap is 395 2024-03-20T08:15:23.012 INFO:tasks.rados.rados.0.smithi118.stdout:3995: seq_num 1208 ranges {5472256=638976,6111232=401408} 2024-03-20T08:15:23.046 INFO:tasks.rados.rados.0.smithi118.stdout:3995: writing smithi11881747-26 from 5472256 to 6111232 tid 1 2024-03-20T08:15:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:3995: writing smithi11881747-26 from 6111232 to 6512640 tid 2 2024-03-20T08:15:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:3996: snap_create 2024-03-20T08:15:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 35 v 65332 (ObjNum 1139 snap 367 seq_num 1139) dirty exists 2024-03-20T08:15:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:3993: finishing write tid 1 to smithi11881747-36 2024-03-20T08:15:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:3993: finishing write tid 2 to smithi11881747-36 2024-03-20T08:15:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:3993: finishing write tid 3 to smithi11881747-36 2024-03-20T08:15:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:3993: finishing write tid 4 to smithi11881747-36 2024-03-20T08:15:23.049 INFO:tasks.rados.rados.0.smithi118.stdout:3993: oid 36 updating version 0 to 65355 2024-03-20T08:15:23.050 INFO:tasks.rados.rados.0.smithi118.stdout:3993: oid 36 updating version 65355 to 65357 2024-03-20T08:15:23.050 INFO:tasks.rados.rados.0.smithi118.stdout:3993: oid 36 version 65357 is already newer than 65356 2024-03-20T08:15:23.050 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 36 v 65357 (ObjNum 1207 snap 395 seq_num 1207) dirty exists 2024-03-20T08:15:23.050 INFO:tasks.rados.rados.0.smithi118.stdout:3993: left oid 36 (ObjNum 1207 snap 395 seq_num 1207) 2024-03-20T08:15:23.062 INFO:tasks.rados.rados.0.smithi118.stdout:3995: finishing write tid 1 to smithi11881747-26 2024-03-20T08:15:23.063 INFO:tasks.rados.rados.0.smithi118.stdout:3995: finishing write tid 2 to smithi11881747-26 2024-03-20T08:15:23.063 INFO:tasks.rados.rados.0.smithi118.stdout:3995: finishing write tid 3 to smithi11881747-26 2024-03-20T08:15:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:3995: finishing write tid 4 to smithi11881747-26 2024-03-20T08:15:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:3995: oid 26 updating version 0 to 65374 2024-03-20T08:15:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:3995: oid 26 version 65374 is already newer than 65372 2024-03-20T08:15:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:3995: oid 26 version 65374 is already newer than 65373 2024-03-20T08:15:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 26 v 65374 (ObjNum 1208 snap 395 seq_num 1208) dirty exists 2024-03-20T08:15:23.067 INFO:tasks.rados.rados.0.smithi118.stdout:3995: left oid 26 (ObjNum 1208 snap 395 seq_num 1208) 2024-03-20T08:15:23.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:22.990+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:23.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:23.054+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:23.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:23.054+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:23.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:23.051+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:23.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:23.063+0000 7fa5d3038700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:23.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:23.063+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:23.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:23.016+0000 7ffb40715700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:15:23.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:23.016+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:23.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:23.018+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: osdmap e865: 8 total, 8 up, 8 in 2024-03-20T08:15:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:23.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:23.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:22 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:23.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:23.011+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:23.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:23.026+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:15:23.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:23.026+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:23.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:22.993+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:23.424 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:22.993+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:22.989+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:22.989+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:22.989+0000 7f831aea7700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:23.424 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:22.990+0000 7f831a6a6700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:23.425 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:23.020+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:23.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: osdmap e865: 8 total, 8 up, 8 in 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:23.426 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:23.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: osdmap e865: 8 total, 8 up, 8 in 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:23.428 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:22 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:15:23] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:15:23.991 INFO:tasks.rados.rados.0.smithi118.stdout:3993: done (3 left) 2024-03-20T08:15:23.991 INFO:tasks.rados.rados.0.smithi118.stdout:3994: done (2 left) 2024-03-20T08:15:23.991 INFO:tasks.rados.rados.0.smithi118.stdout:3995: done (1 left) 2024-03-20T08:15:23.991 INFO:tasks.rados.rados.0.smithi118.stdout:3996: done (0 left) 2024-03-20T08:15:23.991 INFO:tasks.rados.rados.0.smithi118.stdout:3997: rollback oid 46 current snap is 396 2024-03-20T08:15:23.992 INFO:tasks.rados.rados.0.smithi118.stdout:rollback oid 46 to 394 2024-03-20T08:15:23.992 INFO:tasks.rados.rados.0.smithi118.stdout:append oid 48 current snap is 396 2024-03-20T08:15:23.992 INFO:tasks.rados.rados.0.smithi118.stdout:3998: seq_num 1209 ranges {15032320=655360,15687680=401408} 2024-03-20T08:15:24.076 INFO:tasks.rados.rados.0.smithi118.stdout:3998: writing smithi11881747-48 from 15032320 to 15687680 tid 1 2024-03-20T08:15:24.079 INFO:tasks.rados.rados.0.smithi118.stdout:3998: writing smithi11881747-48 from 15687680 to 16089088 tid 2 2024-03-20T08:15:24.079 INFO:tasks.rados.rados.0.smithi118.stdout:3999: read oid 33 snap -1 2024-03-20T08:15:24.079 INFO:tasks.rados.rados.0.smithi118.stdout:3999: expect deleted 2024-03-20T08:15:24.079 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 3 2024-03-20T08:15:24.079 INFO:tasks.rados.rados.0.smithi118.stdout:3997: finishing rollback tid 0 to smithi11881747-46 2024-03-20T08:15:24.080 INFO:tasks.rados.rados.0.smithi118.stdout:3997: finishing rollback tid 1 to smithi11881747-46 2024-03-20T08:15:24.080 INFO:tasks.rados.rados.0.smithi118.stdout:3997: finishing rollback tid 2 to smithi11881747-46 2024-03-20T08:15:24.080 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 46 v 65335 (ObjNum 1148 snap 370 seq_num 1148) dirty exists 2024-03-20T08:15:24.080 INFO:tasks.rados.rados.0.smithi118.stdout:3997: done (2 left) 2024-03-20T08:15:24.080 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 2 2024-03-20T08:15:24.080 INFO:tasks.rados.rados.0.smithi118.stdout:3999: done (1 left) 2024-03-20T08:15:24.080 INFO:tasks.rados.rados.0.smithi118.stdout: waiting on 1 2024-03-20T08:15:24.093 INFO:tasks.rados.rados.0.smithi118.stdout:3998: finishing write tid 1 to smithi11881747-48 2024-03-20T08:15:24.093 INFO:tasks.rados.rados.0.smithi118.stdout:3998: finishing write tid 2 to smithi11881747-48 2024-03-20T08:15:24.093 INFO:tasks.rados.rados.0.smithi118.stdout:3998: finishing write tid 3 to smithi11881747-48 2024-03-20T08:15:24.097 INFO:tasks.rados.rados.0.smithi118.stdout:3998: finishing write tid 4 to smithi11881747-48 2024-03-20T08:15:24.097 INFO:tasks.rados.rados.0.smithi118.stdout:3998: oid 48 updating version 0 to 65335 2024-03-20T08:15:24.097 INFO:tasks.rados.rados.0.smithi118.stdout:3998: oid 48 updating version 65335 to 65336 2024-03-20T08:15:24.097 INFO:tasks.rados.rados.0.smithi118.stdout:3998: oid 48 updating version 65336 to 65337 2024-03-20T08:15:24.098 INFO:tasks.rados.rados.0.smithi118.stdout:update_object_version oid 48 v 65337 (ObjNum 1209 snap 396 seq_num 1209) dirty exists 2024-03-20T08:15:24.098 INFO:tasks.rados.rados.0.smithi118.stdout:3998: left oid 48 (ObjNum 1209 snap 396 seq_num 1209) 2024-03-20T08:15:24.098 INFO:tasks.rados.rados.0.smithi118.stdout:3998: done (0 left) 2024-03-20T08:15:24.100 INFO:tasks.rados.rados.0.smithi118.stderr:0 errors. 2024-03-20T08:15:24.101 INFO:tasks.rados.rados.0.smithi118.stderr: 2024-03-20T08:15:24.103 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-20T08:15:24.356 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:15:24.389 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:15:23.991+0000 7fafa2d84700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:24.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:24.082+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:24.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:15:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:15:24.082+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:24.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:23.998+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:24.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:15:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:15:24.003+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:24.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:24.149+0000 7fa5d583d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:15:24.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:24.149+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:24.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:15:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:15:24.149+0000 7fa5d3038700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:24.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: pgmap v989: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: osdmap e866: 8 total, 8 up, 8 in 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:24.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:24.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:24.392 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:23 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:24.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:24.082+0000 7f0cf2464700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:24.422 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:15:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:15:24.082+0000 7f0ced45a700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:24.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:15:23.991+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:24.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:23.991+0000 7f83186a2700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:24.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:23.991+0000 7f83196a4700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:24.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:23.991+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:24.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:23.992+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:15:24.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:15:23.992+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:24.424 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:15:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:15:24.001+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: pgmap v989: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-03-20T08:15:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:24.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: osdmap e866: 8 total, 8 up, 8 in 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:24.425 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: pgmap v989: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 13 op/s 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: osdmap e866: 8 total, 8 up, 8 in 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b5=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:24.426 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:24.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:24.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:24.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:24.427 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:23 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:25.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:25.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:25.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: osdmap e867: 8 total, 8 up, 8 in 2024-03-20T08:15:25.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:25.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:25.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:25.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:25.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: osdmap e867: 8 total, 8 up, 8 in 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:25.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:25.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:25.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:25.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:25.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:25 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:15:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:15:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: osdmap e867: 8 total, 8 up, 8 in 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:15:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:25 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:15:26.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:26 smithi118 ceph-mon[93121]: pgmap v992: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 0 op/s 2024-03-20T08:15:26.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:26 smithi118 ceph-mon[95692]: pgmap v992: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 0 op/s 2024-03-20T08:15:26.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:26 smithi196 ceph-mon[87080]: pgmap v992: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 0 B/s rd, 1.9 MiB/s wr, 0 op/s 2024-03-20T08:15:26.438 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:15:26.438 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":867,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T08:15:23.987668+0000","last_up_change":"2024-03-20T08:09:45.750191+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-03-20T07:59:00.573805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"unique_pool_0","create_time":"2024-03-20T08:01:24.424653+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"867","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":455,"snap_epoch":867,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"teuthologyprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":289,"up_from":300,"up_thru":551,"down_at":290,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3093978881},{"type":"v1","addr":"172.21.15.118:6803","nonce":3093978881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3093978881},{"type":"v1","addr":"172.21.15.118:6805","nonce":3093978881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":3093978881},{"type":"v1","addr":"172.21.15.118:6809","nonce":3093978881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3093978881},{"type":"v1","addr":"172.21.15.118:6807","nonce":3093978881}]},"public_addr":"172.21.15.118:6803/3093978881","cluster_addr":"172.21.15.118:6805/3093978881","heartbeat_back_addr":"172.21.15.118:6809/3093978881","heartbeat_front_addr":"172.21.15.118:6807/3093978881","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":325,"up_from":336,"up_thru":551,"down_at":326,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":406990489},{"type":"v1","addr":"172.21.15.118:6811","nonce":406990489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":406990489},{"type":"v1","addr":"172.21.15.118:6813","nonce":406990489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":406990489},{"type":"v1","addr":"172.21.15.118:6817","nonce":406990489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":406990489},{"type":"v1","addr":"172.21.15.118:6815","nonce":406990489}]},"public_addr":"172.21.15.118:6811/406990489","cluster_addr":"172.21.15.118:6813/406990489","heartbeat_back_addr":"172.21.15.118:6817/406990489","heartbeat_front_addr":"172.21.15.118:6815/406990489","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":363,"up_from":376,"up_thru":551,"down_at":364,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":3195797555},{"type":"v1","addr":"172.21.15.118:6819","nonce":3195797555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3195797555},{"type":"v1","addr":"172.21.15.118:6821","nonce":3195797555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3195797555},{"type":"v1","addr":"172.21.15.118:6825","nonce":3195797555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":3195797555},{"type":"v1","addr":"172.21.15.118:6823","nonce":3195797555}]},"public_addr":"172.21.15.118:6819/3195797555","cluster_addr":"172.21.15.118:6821/3195797555","heartbeat_back_addr":"172.21.15.118:6825/3195797555","heartbeat_front_addr":"172.21.15.118:6823/3195797555","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":396,"up_from":408,"up_thru":551,"down_at":397,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2186452731},{"type":"v1","addr":"172.21.15.118:6827","nonce":2186452731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2186452731},{"type":"v1","addr":"172.21.15.118:6829","nonce":2186452731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2186452731},{"type":"v1","addr":"172.21.15.118:6833","nonce":2186452731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2186452731},{"type":"v1","addr":"172.21.15.118:6831","nonce":2186452731}]},"public_addr":"172.21.15.118:6827/2186452731","cluster_addr":"172.21.15.118:6829/2186452731","heartbeat_back_addr":"172.21.15.118:6833/2186452731","heartbeat_front_addr":"172.21.15.118:6831/2186452731","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":31,"last_clean_end":434,"up_from":445,"up_thru":551,"down_at":435,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":502714599},{"type":"v1","addr":"172.21.15.196:6801","nonce":502714599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":502714599},{"type":"v1","addr":"172.21.15.196:6803","nonce":502714599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":502714599},{"type":"v1","addr":"172.21.15.196:6807","nonce":502714599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":502714599},{"type":"v1","addr":"172.21.15.196:6805","nonce":502714599}]},"public_addr":"172.21.15.196:6801/502714599","cluster_addr":"172.21.15.196:6803/502714599","heartbeat_back_addr":"172.21.15.196:6807/502714599","heartbeat_front_addr":"172.21.15.196:6805/502714599","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":456,"up_from":470,"up_thru":551,"down_at":457,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":4232618693},{"type":"v1","addr":"172.21.15.196:6809","nonce":4232618693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4232618693},{"type":"v1","addr":"172.21.15.196:6811","nonce":4232618693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4232618693},{"type":"v1","addr":"172.21.15.196:6815","nonce":4232618693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4232618693},{"type":"v1","addr":"172.21.15.196:6813","nonce":4232618693}]},"public_addr":"172.21.15.196:6809/4232618693","cluster_addr":"172.21.15.196:6811/4232618693","heartbeat_back_addr":"172.21.15.196:6815/4232618693","heartbeat_front_addr":"172.21.15.196:6813/4232618693","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":47,"last_clean_end":492,"up_from":505,"up_thru":551,"down_at":493,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":812001496},{"type":"v1","addr":"172.21.15.196:6817","nonce":812001496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":812001496},{"type":"v1","addr":"172.21.15.196:6819","nonce":812001496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":812001496},{"type":"v1","addr":"172.21.15.196:6823","nonce":812001496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":812001496},{"type":"v1","addr":"172.21.15.196:6821","nonce":812001496}]},"public_addr":"172.21.15.196:6817/812001496","cluster_addr":"172.21.15.196:6819/812001496","heartbeat_back_addr":"172.21.15.196:6823/812001496","heartbeat_front_addr":"172.21.15.196:6821/812001496","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":55,"last_clean_end":534,"up_from":551,"up_thru":551,"down_at":535,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":4230059391},{"type":"v1","addr":"172.21.15.196:6825","nonce":4230059391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":4230059391},{"type":"v1","addr":"172.21.15.196:6827","nonce":4230059391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":4230059391},{"type":"v1","addr":"172.21.15.196:6831","nonce":4230059391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":4230059391},{"type":"v1","addr":"172.21.15.196:6829","nonce":4230059391}]},"public_addr":"172.21.15.196:6825/4230059391","cluster_addr":"172.21.15.196:6827/4230059391","heartbeat_back_addr":"172.21.15.196:6831/4230059391","heartbeat_front_addr":"172.21.15.196:6829/4230059391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":289},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":325},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":363},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":396},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":434},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":456},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":492},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":534}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":7,"to":2}]},{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1},{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":0,"to":1},{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6832/5972345":"2024-03-21T08:10:17.017317+0000","172.21.15.118:6801/1685672110":"2024-03-21T08:03:58.165635+0000","172.21.15.118:6800/4166913641":"2024-03-21T08:02:37.593461+0000","172.21.15.118:0/1828201096":"2024-03-21T08:02:37.593461+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/4166913641":"2024-03-21T08:02:37.593461+0000","172.21.15.196:6835/1340665682":"2024-03-21T08:03:13.331084+0000","172.21.15.118:0/3938132886":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6800/1685672110":"2024-03-21T08:03:58.165635+0000","172.21.15.196:0/2412915018":"2024-03-21T08:03:13.331084+0000","172.21.15.196:6833/5972345":"2024-03-21T08:10:17.017317+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2517893481":"2024-03-21T08:02:37.593461+0000","172.21.15.196:6834/1340665682":"2024-03-21T08:03:13.331084+0000","172.21.15.118:0/1838641107":"2024-03-21T08:03:58.165635+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2649753508":"2024-03-21T07:58:59.897028+0000","172.21.15.196:0/3478301470":"2024-03-21T08:03:13.331084+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/237356808":"2024-03-21T08:02:37.593461+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1237762586":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2856673731":"2024-03-21T08:03:58.165635+0000","172.21.15.118:0/2551714175":"2024-03-21T08:02:37.593461+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.196:0/3067939912":"2024-03-21T08:03:13.331084+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.196:0/2774265323":"2024-03-21T08:03:13.331084+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2309848212":"2024-03-21T08:02:37.593461+0000","172.21.15.118:0/184826175":"2024-03-21T07:58:59.897028+0000","172.21.15.196:0/4207256264":"2024-03-21T08:03:13.331084+0000","172.21.15.118:6800/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/1635512832":"2024-03-21T08:03:58.165635+0000","172.21.15.118:0/462969649":"2024-03-21T08:03:58.165635+0000","172.21.15.196:0/3872526590":"2024-03-21T08:03:13.331084+0000","172.21.15.118:0/2179849791":"2024-03-21T08:03:58.165635+0000","172.21.15.118:0/3072694613":"2024-03-21T08:03:58.165635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"teuthologyprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van","w":"8"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":50,"length":1},{"begin":80,"length":1},{"begin":101,"length":1},{"begin":107,"length":1},{"begin":131,"length":1},{"begin":137,"length":3},{"begin":145,"length":1},{"begin":149,"length":1},{"begin":157,"length":1},{"begin":159,"length":1},{"begin":162,"length":5},{"begin":168,"length":2},{"begin":171,"length":2},{"begin":175,"length":255},{"begin":431,"length":22}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T08:15:27.025 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:15:27.236 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:15:27.365 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:27 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/206616276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:15:27.365 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:27 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/206616276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:15:27.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:27 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/206616276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:15:28.352 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:28 smithi118 ceph-mon[93121]: pgmap v993: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 6.5 MiB/s rd, 1.7 MiB/s wr, 10 op/s 2024-03-20T08:15:28.353 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:28 smithi118 ceph-mon[95692]: pgmap v993: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 6.5 MiB/s rd, 1.7 MiB/s wr, 10 op/s 2024-03-20T08:15:28.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:28 smithi196 ceph-mon[87080]: pgmap v993: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 6.5 MiB/s rd, 1.7 MiB/s wr, 10 op/s 2024-03-20T08:15:29.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:29.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:29.199 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:15:29.201 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:15:29.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:29 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:29.867 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":994,"stamp":"2024-03-20T08:15:28.623036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10985472,"num_objects":8,"num_object_clones":8,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":37340,"num_write":54,"num_write_kb":10728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.646722"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999130+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999130+0000","last_peered":"2024-03-20T08:15:23.999130+0000","last_clean":"2024-03-20T08:15:23.999130+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999130+0000","last_undegraded":"2024-03-20T08:15:23.999130+0000","last_fullsized":"2024-03-20T08:15:23.999130+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1339,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002139+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002139+0000","last_peered":"2024-03-20T08:15:24.002139+0000","last_clean":"2024-03-20T08:15:24.002139+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002139+0000","last_undegraded":"2024-03-20T08:15:24.002139+0000","last_fullsized":"2024-03-20T08:15:24.002139+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082706+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082706+0000","last_peered":"2024-03-20T08:15:24.082706+0000","last_clean":"2024-03-20T08:15:24.082706+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082706+0000","last_undegraded":"2024-03-20T08:15:24.082706+0000","last_fullsized":"2024-03-20T08:15:24.082706+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086423+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086423+0000","last_peered":"2024-03-20T08:15:24.086423+0000","last_clean":"2024-03-20T08:15:24.086423+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086423+0000","last_undegraded":"2024-03-20T08:15:24.086423+0000","last_fullsized":"2024-03-20T08:15:24.086423+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998738+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998738+0000","last_peered":"2024-03-20T08:15:23.998738+0000","last_clean":"2024-03-20T08:15:23.998738+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998738+0000","last_undegraded":"2024-03-20T08:15:23.998738+0000","last_fullsized":"2024-03-20T08:15:23.998738+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991415+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991415+0000","last_peered":"2024-03-20T08:15:23.991415+0000","last_clean":"2024-03-20T08:15:23.991415+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991415+0000","last_undegraded":"2024-03-20T08:15:23.991415+0000","last_fullsized":"2024-03-20T08:15:23.991415+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991194+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991194+0000","last_peered":"2024-03-20T08:15:23.991194+0000","last_clean":"2024-03-20T08:15:23.991194+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991194+0000","last_undegraded":"2024-03-20T08:15:23.991194+0000","last_fullsized":"2024-03-20T08:15:23.991194+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992438+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992438+0000","last_peered":"2024-03-20T08:15:23.992438+0000","last_clean":"2024-03-20T08:15:23.992438+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992438+0000","last_undegraded":"2024-03-20T08:15:23.992438+0000","last_fullsized":"2024-03-20T08:15:23.992438+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082177+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082177+0000","last_peered":"2024-03-20T08:15:24.082177+0000","last_clean":"2024-03-20T08:15:24.082177+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082177+0000","last_undegraded":"2024-03-20T08:15:24.082177+0000","last_fullsized":"2024-03-20T08:15:24.082177+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1299,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991364+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991364+0000","last_peered":"2024-03-20T08:15:23.991364+0000","last_clean":"2024-03-20T08:15:23.991364+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991364+0000","last_undegraded":"2024-03-20T08:15:23.991364+0000","last_fullsized":"2024-03-20T08:15:23.991364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999389+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999389+0000","last_peered":"2024-03-20T08:15:23.999389+0000","last_clean":"2024-03-20T08:15:23.999389+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999389+0000","last_undegraded":"2024-03-20T08:15:23.999389+0000","last_fullsized":"2024-03-20T08:15:23.999389+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1391,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083281+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083281+0000","last_peered":"2024-03-20T08:15:24.083281+0000","last_clean":"2024-03-20T08:15:24.083281+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083281+0000","last_undegraded":"2024-03-20T08:15:24.083281+0000","last_fullsized":"2024-03-20T08:15:24.083281+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082846+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082846+0000","last_peered":"2024-03-20T08:15:24.082846+0000","last_clean":"2024-03-20T08:15:24.082846+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082846+0000","last_undegraded":"2024-03-20T08:15:24.082846+0000","last_fullsized":"2024-03-20T08:15:24.082846+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1225,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082834+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082834+0000","last_peered":"2024-03-20T08:15:24.082834+0000","last_clean":"2024-03-20T08:15:24.082834+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082834+0000","last_undegraded":"2024-03-20T08:15:24.082834+0000","last_fullsized":"2024-03-20T08:15:24.082834+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991223+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991223+0000","last_peered":"2024-03-20T08:15:23.991223+0000","last_clean":"2024-03-20T08:15:23.991223+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991223+0000","last_undegraded":"2024-03-20T08:15:23.991223+0000","last_fullsized":"2024-03-20T08:15:23.991223+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991397+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991397+0000","last_peered":"2024-03-20T08:15:23.991397+0000","last_clean":"2024-03-20T08:15:23.991397+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991397+0000","last_undegraded":"2024-03-20T08:15:23.991397+0000","last_fullsized":"2024-03-20T08:15:23.991397+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991466+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991466+0000","last_peered":"2024-03-20T08:15:23.991466+0000","last_clean":"2024-03-20T08:15:23.991466+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991466+0000","last_undegraded":"2024-03-20T08:15:23.991466+0000","last_fullsized":"2024-03-20T08:15:23.991466+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082450+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082450+0000","last_peered":"2024-03-20T08:15:24.082450+0000","last_clean":"2024-03-20T08:15:24.082450+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082450+0000","last_undegraded":"2024-03-20T08:15:24.082450+0000","last_fullsized":"2024-03-20T08:15:24.082450+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1339,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002365+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002365+0000","last_peered":"2024-03-20T08:15:24.002365+0000","last_clean":"2024-03-20T08:15:24.002365+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002365+0000","last_undegraded":"2024-03-20T08:15:24.002365+0000","last_fullsized":"2024-03-20T08:15:24.002365+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991749+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991749+0000","last_peered":"2024-03-20T08:15:23.991749+0000","last_clean":"2024-03-20T08:15:23.991749+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991749+0000","last_undegraded":"2024-03-20T08:15:23.991749+0000","last_fullsized":"2024-03-20T08:15:23.991749+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083070+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083070+0000","last_peered":"2024-03-20T08:15:24.083070+0000","last_clean":"2024-03-20T08:15:24.083070+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083070+0000","last_undegraded":"2024-03-20T08:15:24.083070+0000","last_fullsized":"2024-03-20T08:15:24.083070+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082887+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082887+0000","last_peered":"2024-03-20T08:15:24.082887+0000","last_clean":"2024-03-20T08:15:24.082887+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082887+0000","last_undegraded":"2024-03-20T08:15:24.082887+0000","last_fullsized":"2024-03-20T08:15:24.082887+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002126+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002126+0000","last_peered":"2024-03-20T08:15:24.002126+0000","last_clean":"2024-03-20T08:15:24.002126+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002126+0000","last_undegraded":"2024-03-20T08:15:24.002126+0000","last_fullsized":"2024-03-20T08:15:24.002126+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1219,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991508+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991508+0000","last_peered":"2024-03-20T08:15:23.991508+0000","last_clean":"2024-03-20T08:15:23.991508+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991508+0000","last_undegraded":"2024-03-20T08:15:23.991508+0000","last_fullsized":"2024-03-20T08:15:23.991508+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998902+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998902+0000","last_peered":"2024-03-20T08:15:23.998902+0000","last_clean":"2024-03-20T08:15:23.998902+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998902+0000","last_undegraded":"2024-03-20T08:15:23.998902+0000","last_fullsized":"2024-03-20T08:15:23.998902+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082824+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082824+0000","last_peered":"2024-03-20T08:15:24.082824+0000","last_clean":"2024-03-20T08:15:24.082824+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082824+0000","last_undegraded":"2024-03-20T08:15:24.082824+0000","last_fullsized":"2024-03-20T08:15:24.082824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085437+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085437+0000","last_peered":"2024-03-20T08:15:24.085437+0000","last_clean":"2024-03-20T08:15:24.085437+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085437+0000","last_undegraded":"2024-03-20T08:15:24.085437+0000","last_fullsized":"2024-03-20T08:15:24.085437+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998772+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998772+0000","last_peered":"2024-03-20T08:15:23.998772+0000","last_clean":"2024-03-20T08:15:23.998772+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998772+0000","last_undegraded":"2024-03-20T08:15:23.998772+0000","last_fullsized":"2024-03-20T08:15:23.998772+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990901+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990901+0000","last_peered":"2024-03-20T08:15:23.990901+0000","last_clean":"2024-03-20T08:15:23.990901+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990901+0000","last_undegraded":"2024-03-20T08:15:23.990901+0000","last_fullsized":"2024-03-20T08:15:23.990901+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990851+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990851+0000","last_peered":"2024-03-20T08:15:23.990851+0000","last_clean":"2024-03-20T08:15:23.990851+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990851+0000","last_undegraded":"2024-03-20T08:15:23.990851+0000","last_fullsized":"2024-03-20T08:15:23.990851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1315,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990647+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990647+0000","last_peered":"2024-03-20T08:15:23.990647+0000","last_clean":"2024-03-20T08:15:23.990647+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990647+0000","last_undegraded":"2024-03-20T08:15:23.990647+0000","last_fullsized":"2024-03-20T08:15:23.990647+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998183+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998183+0000","last_peered":"2024-03-20T08:15:23.998183+0000","last_clean":"2024-03-20T08:15:23.998183+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998183+0000","last_undegraded":"2024-03-20T08:15:23.998183+0000","last_fullsized":"2024-03-20T08:15:23.998183+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150274+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150274+0000","last_peered":"2024-03-20T08:15:24.150274+0000","last_clean":"2024-03-20T08:15:24.150274+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150274+0000","last_undegraded":"2024-03-20T08:15:24.150274+0000","last_fullsized":"2024-03-20T08:15:24.150274+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998895+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998895+0000","last_peered":"2024-03-20T08:15:23.998895+0000","last_clean":"2024-03-20T08:15:23.998895+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998895+0000","last_undegraded":"2024-03-20T08:15:23.998895+0000","last_fullsized":"2024-03-20T08:15:23.998895+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11687,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.082829+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.082829+0000","last_peered":"2024-03-20T08:15:24.082829+0000","last_clean":"2024-03-20T08:15:24.082829+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.082829+0000","last_undegraded":"2024-03-20T08:15:24.082829+0000","last_fullsized":"2024-03-20T08:15:24.082829+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001728+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001728+0000","last_peered":"2024-03-20T08:15:24.001728+0000","last_clean":"2024-03-20T08:15:24.001728+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001728+0000","last_undegraded":"2024-03-20T08:15:24.001728+0000","last_fullsized":"2024-03-20T08:15:24.001728+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001977+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.001977+0000","last_peered":"2024-03-20T08:15:24.001977+0000","last_clean":"2024-03-20T08:15:24.001977+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.001977+0000","last_undegraded":"2024-03-20T08:15:24.001977+0000","last_fullsized":"2024-03-20T08:15:24.001977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082803+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082803+0000","last_peered":"2024-03-20T08:15:24.082803+0000","last_clean":"2024-03-20T08:15:24.082803+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082803+0000","last_undegraded":"2024-03-20T08:15:24.082803+0000","last_fullsized":"2024-03-20T08:15:24.082803+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5949,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083173+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083173+0000","last_peered":"2024-03-20T08:15:24.083173+0000","last_clean":"2024-03-20T08:15:24.083173+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083173+0000","last_undegraded":"2024-03-20T08:15:24.083173+0000","last_fullsized":"2024-03-20T08:15:24.083173+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002077+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002077+0000","last_peered":"2024-03-20T08:15:24.002077+0000","last_clean":"2024-03-20T08:15:24.002077+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002077+0000","last_undegraded":"2024-03-20T08:15:24.002077+0000","last_fullsized":"2024-03-20T08:15:24.002077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082497+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082497+0000","last_peered":"2024-03-20T08:15:24.082497+0000","last_clean":"2024-03-20T08:15:24.082497+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082497+0000","last_undegraded":"2024-03-20T08:15:24.082497+0000","last_fullsized":"2024-03-20T08:15:24.082497+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149780+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149780+0000","last_peered":"2024-03-20T08:15:24.149780+0000","last_clean":"2024-03-20T08:15:24.149780+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149780+0000","last_undegraded":"2024-03-20T08:15:24.149780+0000","last_fullsized":"2024-03-20T08:15:24.149780+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1332,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001769+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001769+0000","last_peered":"2024-03-20T08:15:24.001769+0000","last_clean":"2024-03-20T08:15:24.001769+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001769+0000","last_undegraded":"2024-03-20T08:15:24.001769+0000","last_fullsized":"2024-03-20T08:15:24.001769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001346+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001346+0000","last_peered":"2024-03-20T08:15:24.001346+0000","last_clean":"2024-03-20T08:15:24.001346+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001346+0000","last_undegraded":"2024-03-20T08:15:24.001346+0000","last_fullsized":"2024-03-20T08:15:24.001346+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990310+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990310+0000","last_peered":"2024-03-20T08:15:23.990310+0000","last_clean":"2024-03-20T08:15:23.990310+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990310+0000","last_undegraded":"2024-03-20T08:15:23.990310+0000","last_fullsized":"2024-03-20T08:15:23.990310+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3977,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999530+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999530+0000","last_peered":"2024-03-20T08:15:23.999530+0000","last_clean":"2024-03-20T08:15:23.999530+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999530+0000","last_undegraded":"2024-03-20T08:15:23.999530+0000","last_fullsized":"2024-03-20T08:15:23.999530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992225+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992225+0000","last_peered":"2024-03-20T08:15:23.992225+0000","last_clean":"2024-03-20T08:15:23.992225+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992225+0000","last_undegraded":"2024-03-20T08:15:23.992225+0000","last_fullsized":"2024-03-20T08:15:23.992225+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082450+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082450+0000","last_peered":"2024-03-20T08:15:24.082450+0000","last_clean":"2024-03-20T08:15:24.082450+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082450+0000","last_undegraded":"2024-03-20T08:15:24.082450+0000","last_fullsized":"2024-03-20T08:15:24.082450+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1609,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998096+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998096+0000","last_peered":"2024-03-20T08:15:23.998096+0000","last_clean":"2024-03-20T08:15:23.998096+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998096+0000","last_undegraded":"2024-03-20T08:15:23.998096+0000","last_fullsized":"2024-03-20T08:15:23.998096+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9295,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083467+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083467+0000","last_peered":"2024-03-20T08:15:24.083467+0000","last_clean":"2024-03-20T08:15:24.083467+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083467+0000","last_undegraded":"2024-03-20T08:15:24.083467+0000","last_fullsized":"2024-03-20T08:15:24.083467+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083126+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083126+0000","last_peered":"2024-03-20T08:15:24.083126+0000","last_clean":"2024-03-20T08:15:24.083126+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083126+0000","last_undegraded":"2024-03-20T08:15:24.083126+0000","last_fullsized":"2024-03-20T08:15:24.083126+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082143+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082143+0000","last_peered":"2024-03-20T08:15:24.082143+0000","last_clean":"2024-03-20T08:15:24.082143+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082143+0000","last_undegraded":"2024-03-20T08:15:24.082143+0000","last_fullsized":"2024-03-20T08:15:24.082143+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998090+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998090+0000","last_peered":"2024-03-20T08:15:23.998090+0000","last_clean":"2024-03-20T08:15:23.998090+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998090+0000","last_undegraded":"2024-03-20T08:15:23.998090+0000","last_fullsized":"2024-03-20T08:15:23.998090+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082629+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082629+0000","last_peered":"2024-03-20T08:15:24.082629+0000","last_clean":"2024-03-20T08:15:24.082629+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082629+0000","last_undegraded":"2024-03-20T08:15:24.082629+0000","last_fullsized":"2024-03-20T08:15:24.082629+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999198+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999198+0000","last_peered":"2024-03-20T08:15:23.999198+0000","last_clean":"2024-03-20T08:15:23.999198+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999198+0000","last_undegraded":"2024-03-20T08:15:23.999198+0000","last_fullsized":"2024-03-20T08:15:23.999198+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002171+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002171+0000","last_peered":"2024-03-20T08:15:24.002171+0000","last_clean":"2024-03-20T08:15:24.002171+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002171+0000","last_undegraded":"2024-03-20T08:15:24.002171+0000","last_fullsized":"2024-03-20T08:15:24.002171+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150313+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150313+0000","last_peered":"2024-03-20T08:15:24.150313+0000","last_clean":"2024-03-20T08:15:24.150313+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150313+0000","last_undegraded":"2024-03-20T08:15:24.150313+0000","last_fullsized":"2024-03-20T08:15:24.150313+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002146+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002146+0000","last_peered":"2024-03-20T08:15:24.002146+0000","last_clean":"2024-03-20T08:15:24.002146+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002146+0000","last_undegraded":"2024-03-20T08:15:24.002146+0000","last_fullsized":"2024-03-20T08:15:24.002146+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992579+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992579+0000","last_peered":"2024-03-20T08:15:23.992579+0000","last_clean":"2024-03-20T08:15:23.992579+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992579+0000","last_undegraded":"2024-03-20T08:15:23.992579+0000","last_fullsized":"2024-03-20T08:15:23.992579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1214,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149496+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149496+0000","last_peered":"2024-03-20T08:15:24.149496+0000","last_clean":"2024-03-20T08:15:24.149496+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149496+0000","last_undegraded":"2024-03-20T08:15:24.149496+0000","last_fullsized":"2024-03-20T08:15:24.149496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1113,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998326+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998326+0000","last_peered":"2024-03-20T08:15:23.998326+0000","last_clean":"2024-03-20T08:15:23.998326+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998326+0000","last_undegraded":"2024-03-20T08:15:23.998326+0000","last_fullsized":"2024-03-20T08:15:23.998326+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1249,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149492+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149492+0000","last_peered":"2024-03-20T08:15:24.149492+0000","last_clean":"2024-03-20T08:15:24.149492+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149492+0000","last_undegraded":"2024-03-20T08:15:24.149492+0000","last_fullsized":"2024-03-20T08:15:24.149492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68664,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150417+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150417+0000","last_peered":"2024-03-20T08:15:24.150417+0000","last_clean":"2024-03-20T08:15:24.150417+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150417+0000","last_undegraded":"2024-03-20T08:15:24.150417+0000","last_fullsized":"2024-03-20T08:15:24.150417+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001603+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001603+0000","last_peered":"2024-03-20T08:15:24.001603+0000","last_clean":"2024-03-20T08:15:24.001603+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001603+0000","last_undegraded":"2024-03-20T08:15:24.001603+0000","last_fullsized":"2024-03-20T08:15:24.001603+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991658+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991658+0000","last_peered":"2024-03-20T08:15:23.991658+0000","last_clean":"2024-03-20T08:15:23.991658+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991658+0000","last_undegraded":"2024-03-20T08:15:23.991658+0000","last_fullsized":"2024-03-20T08:15:23.991658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31730,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992676+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992676+0000","last_peered":"2024-03-20T08:15:23.992676+0000","last_clean":"2024-03-20T08:15:23.992676+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992676+0000","last_undegraded":"2024-03-20T08:15:23.992676+0000","last_fullsized":"2024-03-20T08:15:23.992676+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001649+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001649+0000","last_peered":"2024-03-20T08:15:24.001649+0000","last_clean":"2024-03-20T08:15:24.001649+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001649+0000","last_undegraded":"2024-03-20T08:15:24.001649+0000","last_fullsized":"2024-03-20T08:15:24.001649+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992344+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992344+0000","last_peered":"2024-03-20T08:15:23.992344+0000","last_clean":"2024-03-20T08:15:23.992344+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992344+0000","last_undegraded":"2024-03-20T08:15:23.992344+0000","last_fullsized":"2024-03-20T08:15:23.992344+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991173+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991173+0000","last_peered":"2024-03-20T08:15:23.991173+0000","last_clean":"2024-03-20T08:15:23.991173+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991173+0000","last_undegraded":"2024-03-20T08:15:23.991173+0000","last_fullsized":"2024-03-20T08:15:23.991173+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3375,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990881+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990881+0000","last_peered":"2024-03-20T08:15:23.990881+0000","last_clean":"2024-03-20T08:15:23.990881+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990881+0000","last_undegraded":"2024-03-20T08:15:23.990881+0000","last_fullsized":"2024-03-20T08:15:23.990881+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1299,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991178+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991178+0000","last_peered":"2024-03-20T08:15:23.991178+0000","last_clean":"2024-03-20T08:15:23.991178+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991178+0000","last_undegraded":"2024-03-20T08:15:23.991178+0000","last_fullsized":"2024-03-20T08:15:23.991178+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999147+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999147+0000","last_peered":"2024-03-20T08:15:23.999147+0000","last_clean":"2024-03-20T08:15:23.999147+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999147+0000","last_undegraded":"2024-03-20T08:15:23.999147+0000","last_fullsized":"2024-03-20T08:15:23.999147+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4956,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005035+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005035+0000","last_peered":"2024-03-20T08:15:24.005035+0000","last_clean":"2024-03-20T08:15:24.005035+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005035+0000","last_undegraded":"2024-03-20T08:15:24.005035+0000","last_fullsized":"2024-03-20T08:15:24.005035+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991516+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991516+0000","last_peered":"2024-03-20T08:15:23.991516+0000","last_clean":"2024-03-20T08:15:23.991516+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991516+0000","last_undegraded":"2024-03-20T08:15:23.991516+0000","last_fullsized":"2024-03-20T08:15:23.991516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999265+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999265+0000","last_peered":"2024-03-20T08:15:23.999265+0000","last_clean":"2024-03-20T08:15:23.999265+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999265+0000","last_undegraded":"2024-03-20T08:15:23.999265+0000","last_fullsized":"2024-03-20T08:15:23.999265+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990328+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990328+0000","last_peered":"2024-03-20T08:15:23.990328+0000","last_clean":"2024-03-20T08:15:23.990328+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990328+0000","last_undegraded":"2024-03-20T08:15:23.990328+0000","last_fullsized":"2024-03-20T08:15:23.990328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086388+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086388+0000","last_peered":"2024-03-20T08:15:24.086388+0000","last_clean":"2024-03-20T08:15:24.086388+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086388+0000","last_undegraded":"2024-03-20T08:15:24.086388+0000","last_fullsized":"2024-03-20T08:15:24.086388+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082357+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082357+0000","last_peered":"2024-03-20T08:15:24.082357+0000","last_clean":"2024-03-20T08:15:24.082357+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082357+0000","last_undegraded":"2024-03-20T08:15:24.082357+0000","last_fullsized":"2024-03-20T08:15:24.082357+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991132+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991132+0000","last_peered":"2024-03-20T08:15:23.991132+0000","last_clean":"2024-03-20T08:15:23.991132+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991132+0000","last_undegraded":"2024-03-20T08:15:23.991132+0000","last_fullsized":"2024-03-20T08:15:23.991132+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002102+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002102+0000","last_peered":"2024-03-20T08:15:24.002102+0000","last_clean":"2024-03-20T08:15:24.002102+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002102+0000","last_undegraded":"2024-03-20T08:15:24.002102+0000","last_fullsized":"2024-03-20T08:15:24.002102+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1275,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992674+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992674+0000","last_peered":"2024-03-20T08:15:23.992674+0000","last_clean":"2024-03-20T08:15:23.992674+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992674+0000","last_undegraded":"2024-03-20T08:15:23.992674+0000","last_fullsized":"2024-03-20T08:15:23.992674+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082363+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082363+0000","last_peered":"2024-03-20T08:15:24.082363+0000","last_clean":"2024-03-20T08:15:24.082363+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082363+0000","last_undegraded":"2024-03-20T08:15:24.082363+0000","last_fullsized":"2024-03-20T08:15:24.082363+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991867+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991867+0000","last_peered":"2024-03-20T08:15:23.991867+0000","last_clean":"2024-03-20T08:15:23.991867+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991867+0000","last_undegraded":"2024-03-20T08:15:23.991867+0000","last_fullsized":"2024-03-20T08:15:23.991867+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082336+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082336+0000","last_peered":"2024-03-20T08:15:24.082336+0000","last_clean":"2024-03-20T08:15:24.082336+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082336+0000","last_undegraded":"2024-03-20T08:15:24.082336+0000","last_fullsized":"2024-03-20T08:15:24.082336+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980165,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484559,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011570,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188921,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703402,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656867,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446811,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629206,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.875 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:29.875 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:15:30.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:30 smithi118 ceph-mon[93121]: pgmap v994: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 9 op/s 2024-03-20T08:15:30.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:30 smithi118 ceph-mon[93121]: from='client.44447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:30.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:30 smithi118 ceph-mon[95692]: pgmap v994: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 9 op/s 2024-03-20T08:15:30.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:30 smithi118 ceph-mon[95692]: from='client.44447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:30 smithi196 ceph-mon[87080]: pgmap v994: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 5.5 MiB/s rd, 1.6 MiB/s wr, 9 op/s 2024-03-20T08:15:30.514 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:30 smithi196 ceph-mon[87080]: from='client.44447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:32.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:32 smithi196 ceph-mon[87080]: pgmap v995: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:15:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:32 smithi118 ceph-mon[93121]: pgmap v995: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:15:32.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:32 smithi118 ceph-mon[95692]: pgmap v995: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 7 op/s 2024-03-20T08:15:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:15:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:15:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:15:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:15:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:15:33] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:15:34.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:34 smithi196 ceph-mon[87080]: pgmap v996: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.8 MiB/s rd, 316 KiB/s wr, 6 op/s 2024-03-20T08:15:34.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:34 smithi118 ceph-mon[93121]: pgmap v996: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.8 MiB/s rd, 316 KiB/s wr, 6 op/s 2024-03-20T08:15:34.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:34 smithi118 ceph-mon[95692]: pgmap v996: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.8 MiB/s rd, 316 KiB/s wr, 6 op/s 2024-03-20T08:15:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:36 smithi196 ceph-mon[87080]: pgmap v997: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.4 MiB/s rd, 287 KiB/s wr, 5 op/s 2024-03-20T08:15:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:36 smithi118 ceph-mon[93121]: pgmap v997: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.4 MiB/s rd, 287 KiB/s wr, 5 op/s 2024-03-20T08:15:36.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:36 smithi118 ceph-mon[95692]: pgmap v997: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.4 MiB/s rd, 287 KiB/s wr, 5 op/s 2024-03-20T08:15:38.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:38 smithi196 ceph-mon[87080]: pgmap v998: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 254 KiB/s wr, 4 op/s 2024-03-20T08:15:38.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:38 smithi118 ceph-mon[93121]: pgmap v998: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 254 KiB/s wr, 4 op/s 2024-03-20T08:15:38.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:38 smithi118 ceph-mon[95692]: pgmap v998: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 3.0 MiB/s rd, 254 KiB/s wr, 4 op/s 2024-03-20T08:15:39.876 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:15:40.089 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:15:40.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:40 smithi118 ceph-mon[93121]: pgmap v999: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 85 B/s rd, 86 KiB/s wr, 0 op/s 2024-03-20T08:15:40.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:40 smithi118 ceph-mon[95692]: pgmap v999: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 85 B/s rd, 86 KiB/s wr, 0 op/s 2024-03-20T08:15:40.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:40 smithi196 ceph-mon[87080]: pgmap v999: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 85 B/s rd, 86 KiB/s wr, 0 op/s 2024-03-20T08:15:42.137 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:15:42.139 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:15:42.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:42 smithi196 ceph-mon[87080]: pgmap v1000: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:42.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:42 smithi118 ceph-mon[93121]: pgmap v1000: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:42.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:42 smithi118 ceph-mon[95692]: pgmap v1000: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:15:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:15:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:15:42.745 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1000,"stamp":"2024-03-20T08:15:40.626374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003338"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999130+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999130+0000","last_peered":"2024-03-20T08:15:23.999130+0000","last_clean":"2024-03-20T08:15:23.999130+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999130+0000","last_undegraded":"2024-03-20T08:15:23.999130+0000","last_fullsized":"2024-03-20T08:15:23.999130+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1339,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002139+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002139+0000","last_peered":"2024-03-20T08:15:24.002139+0000","last_clean":"2024-03-20T08:15:24.002139+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002139+0000","last_undegraded":"2024-03-20T08:15:24.002139+0000","last_fullsized":"2024-03-20T08:15:24.002139+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082706+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082706+0000","last_peered":"2024-03-20T08:15:24.082706+0000","last_clean":"2024-03-20T08:15:24.082706+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082706+0000","last_undegraded":"2024-03-20T08:15:24.082706+0000","last_fullsized":"2024-03-20T08:15:24.082706+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086423+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086423+0000","last_peered":"2024-03-20T08:15:24.086423+0000","last_clean":"2024-03-20T08:15:24.086423+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086423+0000","last_undegraded":"2024-03-20T08:15:24.086423+0000","last_fullsized":"2024-03-20T08:15:24.086423+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998738+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998738+0000","last_peered":"2024-03-20T08:15:23.998738+0000","last_clean":"2024-03-20T08:15:23.998738+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998738+0000","last_undegraded":"2024-03-20T08:15:23.998738+0000","last_fullsized":"2024-03-20T08:15:23.998738+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991415+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991415+0000","last_peered":"2024-03-20T08:15:23.991415+0000","last_clean":"2024-03-20T08:15:23.991415+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991415+0000","last_undegraded":"2024-03-20T08:15:23.991415+0000","last_fullsized":"2024-03-20T08:15:23.991415+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991194+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991194+0000","last_peered":"2024-03-20T08:15:23.991194+0000","last_clean":"2024-03-20T08:15:23.991194+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991194+0000","last_undegraded":"2024-03-20T08:15:23.991194+0000","last_fullsized":"2024-03-20T08:15:23.991194+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992438+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992438+0000","last_peered":"2024-03-20T08:15:23.992438+0000","last_clean":"2024-03-20T08:15:23.992438+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992438+0000","last_undegraded":"2024-03-20T08:15:23.992438+0000","last_fullsized":"2024-03-20T08:15:23.992438+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082177+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082177+0000","last_peered":"2024-03-20T08:15:24.082177+0000","last_clean":"2024-03-20T08:15:24.082177+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082177+0000","last_undegraded":"2024-03-20T08:15:24.082177+0000","last_fullsized":"2024-03-20T08:15:24.082177+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1299,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991364+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991364+0000","last_peered":"2024-03-20T08:15:23.991364+0000","last_clean":"2024-03-20T08:15:23.991364+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991364+0000","last_undegraded":"2024-03-20T08:15:23.991364+0000","last_fullsized":"2024-03-20T08:15:23.991364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999389+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999389+0000","last_peered":"2024-03-20T08:15:23.999389+0000","last_clean":"2024-03-20T08:15:23.999389+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999389+0000","last_undegraded":"2024-03-20T08:15:23.999389+0000","last_fullsized":"2024-03-20T08:15:23.999389+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1391,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083281+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083281+0000","last_peered":"2024-03-20T08:15:24.083281+0000","last_clean":"2024-03-20T08:15:24.083281+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083281+0000","last_undegraded":"2024-03-20T08:15:24.083281+0000","last_fullsized":"2024-03-20T08:15:24.083281+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082846+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082846+0000","last_peered":"2024-03-20T08:15:24.082846+0000","last_clean":"2024-03-20T08:15:24.082846+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082846+0000","last_undegraded":"2024-03-20T08:15:24.082846+0000","last_fullsized":"2024-03-20T08:15:24.082846+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1225,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082834+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082834+0000","last_peered":"2024-03-20T08:15:24.082834+0000","last_clean":"2024-03-20T08:15:24.082834+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082834+0000","last_undegraded":"2024-03-20T08:15:24.082834+0000","last_fullsized":"2024-03-20T08:15:24.082834+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991223+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991223+0000","last_peered":"2024-03-20T08:15:23.991223+0000","last_clean":"2024-03-20T08:15:23.991223+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991223+0000","last_undegraded":"2024-03-20T08:15:23.991223+0000","last_fullsized":"2024-03-20T08:15:23.991223+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991397+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991397+0000","last_peered":"2024-03-20T08:15:23.991397+0000","last_clean":"2024-03-20T08:15:23.991397+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991397+0000","last_undegraded":"2024-03-20T08:15:23.991397+0000","last_fullsized":"2024-03-20T08:15:23.991397+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991466+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991466+0000","last_peered":"2024-03-20T08:15:23.991466+0000","last_clean":"2024-03-20T08:15:23.991466+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991466+0000","last_undegraded":"2024-03-20T08:15:23.991466+0000","last_fullsized":"2024-03-20T08:15:23.991466+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1339,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002365+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002365+0000","last_peered":"2024-03-20T08:15:24.002365+0000","last_clean":"2024-03-20T08:15:24.002365+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002365+0000","last_undegraded":"2024-03-20T08:15:24.002365+0000","last_fullsized":"2024-03-20T08:15:24.002365+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991749+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991749+0000","last_peered":"2024-03-20T08:15:23.991749+0000","last_clean":"2024-03-20T08:15:23.991749+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991749+0000","last_undegraded":"2024-03-20T08:15:23.991749+0000","last_fullsized":"2024-03-20T08:15:23.991749+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002126+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002126+0000","last_peered":"2024-03-20T08:15:24.002126+0000","last_clean":"2024-03-20T08:15:24.002126+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002126+0000","last_undegraded":"2024-03-20T08:15:24.002126+0000","last_fullsized":"2024-03-20T08:15:24.002126+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1219,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991508+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991508+0000","last_peered":"2024-03-20T08:15:23.991508+0000","last_clean":"2024-03-20T08:15:23.991508+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991508+0000","last_undegraded":"2024-03-20T08:15:23.991508+0000","last_fullsized":"2024-03-20T08:15:23.991508+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998902+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998902+0000","last_peered":"2024-03-20T08:15:23.998902+0000","last_clean":"2024-03-20T08:15:23.998902+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998902+0000","last_undegraded":"2024-03-20T08:15:23.998902+0000","last_fullsized":"2024-03-20T08:15:23.998902+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082824+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082824+0000","last_peered":"2024-03-20T08:15:24.082824+0000","last_clean":"2024-03-20T08:15:24.082824+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082824+0000","last_undegraded":"2024-03-20T08:15:24.082824+0000","last_fullsized":"2024-03-20T08:15:24.082824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085437+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085437+0000","last_peered":"2024-03-20T08:15:24.085437+0000","last_clean":"2024-03-20T08:15:24.085437+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085437+0000","last_undegraded":"2024-03-20T08:15:24.085437+0000","last_fullsized":"2024-03-20T08:15:24.085437+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998772+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998772+0000","last_peered":"2024-03-20T08:15:23.998772+0000","last_clean":"2024-03-20T08:15:23.998772+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998772+0000","last_undegraded":"2024-03-20T08:15:23.998772+0000","last_fullsized":"2024-03-20T08:15:23.998772+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990901+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990901+0000","last_peered":"2024-03-20T08:15:23.990901+0000","last_clean":"2024-03-20T08:15:23.990901+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990901+0000","last_undegraded":"2024-03-20T08:15:23.990901+0000","last_fullsized":"2024-03-20T08:15:23.990901+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990851+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990851+0000","last_peered":"2024-03-20T08:15:23.990851+0000","last_clean":"2024-03-20T08:15:23.990851+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990851+0000","last_undegraded":"2024-03-20T08:15:23.990851+0000","last_fullsized":"2024-03-20T08:15:23.990851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1315,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990647+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990647+0000","last_peered":"2024-03-20T08:15:23.990647+0000","last_clean":"2024-03-20T08:15:23.990647+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990647+0000","last_undegraded":"2024-03-20T08:15:23.990647+0000","last_fullsized":"2024-03-20T08:15:23.990647+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998183+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998183+0000","last_peered":"2024-03-20T08:15:23.998183+0000","last_clean":"2024-03-20T08:15:23.998183+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998183+0000","last_undegraded":"2024-03-20T08:15:23.998183+0000","last_fullsized":"2024-03-20T08:15:23.998183+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150274+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150274+0000","last_peered":"2024-03-20T08:15:24.150274+0000","last_clean":"2024-03-20T08:15:24.150274+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150274+0000","last_undegraded":"2024-03-20T08:15:24.150274+0000","last_fullsized":"2024-03-20T08:15:24.150274+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998895+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998895+0000","last_peered":"2024-03-20T08:15:23.998895+0000","last_clean":"2024-03-20T08:15:23.998895+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998895+0000","last_undegraded":"2024-03-20T08:15:23.998895+0000","last_fullsized":"2024-03-20T08:15:23.998895+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11687,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.082829+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.082829+0000","last_peered":"2024-03-20T08:15:24.082829+0000","last_clean":"2024-03-20T08:15:24.082829+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.082829+0000","last_undegraded":"2024-03-20T08:15:24.082829+0000","last_fullsized":"2024-03-20T08:15:24.082829+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001728+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001728+0000","last_peered":"2024-03-20T08:15:24.001728+0000","last_clean":"2024-03-20T08:15:24.001728+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001728+0000","last_undegraded":"2024-03-20T08:15:24.001728+0000","last_fullsized":"2024-03-20T08:15:24.001728+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001977+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.001977+0000","last_peered":"2024-03-20T08:15:24.001977+0000","last_clean":"2024-03-20T08:15:24.001977+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.001977+0000","last_undegraded":"2024-03-20T08:15:24.001977+0000","last_fullsized":"2024-03-20T08:15:24.001977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082803+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082803+0000","last_peered":"2024-03-20T08:15:24.082803+0000","last_clean":"2024-03-20T08:15:24.082803+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082803+0000","last_undegraded":"2024-03-20T08:15:24.082803+0000","last_fullsized":"2024-03-20T08:15:24.082803+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5949,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083173+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083173+0000","last_peered":"2024-03-20T08:15:24.083173+0000","last_clean":"2024-03-20T08:15:24.083173+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083173+0000","last_undegraded":"2024-03-20T08:15:24.083173+0000","last_fullsized":"2024-03-20T08:15:24.083173+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002077+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002077+0000","last_peered":"2024-03-20T08:15:24.002077+0000","last_clean":"2024-03-20T08:15:24.002077+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002077+0000","last_undegraded":"2024-03-20T08:15:24.002077+0000","last_fullsized":"2024-03-20T08:15:24.002077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082497+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082497+0000","last_peered":"2024-03-20T08:15:24.082497+0000","last_clean":"2024-03-20T08:15:24.082497+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082497+0000","last_undegraded":"2024-03-20T08:15:24.082497+0000","last_fullsized":"2024-03-20T08:15:24.082497+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149780+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149780+0000","last_peered":"2024-03-20T08:15:24.149780+0000","last_clean":"2024-03-20T08:15:24.149780+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149780+0000","last_undegraded":"2024-03-20T08:15:24.149780+0000","last_fullsized":"2024-03-20T08:15:24.149780+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1332,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001769+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001769+0000","last_peered":"2024-03-20T08:15:24.001769+0000","last_clean":"2024-03-20T08:15:24.001769+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001769+0000","last_undegraded":"2024-03-20T08:15:24.001769+0000","last_fullsized":"2024-03-20T08:15:24.001769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001346+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001346+0000","last_peered":"2024-03-20T08:15:24.001346+0000","last_clean":"2024-03-20T08:15:24.001346+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001346+0000","last_undegraded":"2024-03-20T08:15:24.001346+0000","last_fullsized":"2024-03-20T08:15:24.001346+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990310+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990310+0000","last_peered":"2024-03-20T08:15:23.990310+0000","last_clean":"2024-03-20T08:15:23.990310+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990310+0000","last_undegraded":"2024-03-20T08:15:23.990310+0000","last_fullsized":"2024-03-20T08:15:23.990310+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3977,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999530+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999530+0000","last_peered":"2024-03-20T08:15:23.999530+0000","last_clean":"2024-03-20T08:15:23.999530+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999530+0000","last_undegraded":"2024-03-20T08:15:23.999530+0000","last_fullsized":"2024-03-20T08:15:23.999530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082450+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082450+0000","last_peered":"2024-03-20T08:15:24.082450+0000","last_clean":"2024-03-20T08:15:24.082450+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082450+0000","last_undegraded":"2024-03-20T08:15:24.082450+0000","last_fullsized":"2024-03-20T08:15:24.082450+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1609,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998096+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998096+0000","last_peered":"2024-03-20T08:15:23.998096+0000","last_clean":"2024-03-20T08:15:23.998096+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998096+0000","last_undegraded":"2024-03-20T08:15:23.998096+0000","last_fullsized":"2024-03-20T08:15:23.998096+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9295,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083467+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083467+0000","last_peered":"2024-03-20T08:15:24.083467+0000","last_clean":"2024-03-20T08:15:24.083467+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083467+0000","last_undegraded":"2024-03-20T08:15:24.083467+0000","last_fullsized":"2024-03-20T08:15:24.083467+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083126+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083126+0000","last_peered":"2024-03-20T08:15:24.083126+0000","last_clean":"2024-03-20T08:15:24.083126+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083126+0000","last_undegraded":"2024-03-20T08:15:24.083126+0000","last_fullsized":"2024-03-20T08:15:24.083126+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082143+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082143+0000","last_peered":"2024-03-20T08:15:24.082143+0000","last_clean":"2024-03-20T08:15:24.082143+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082143+0000","last_undegraded":"2024-03-20T08:15:24.082143+0000","last_fullsized":"2024-03-20T08:15:24.082143+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998090+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998090+0000","last_peered":"2024-03-20T08:15:23.998090+0000","last_clean":"2024-03-20T08:15:23.998090+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998090+0000","last_undegraded":"2024-03-20T08:15:23.998090+0000","last_fullsized":"2024-03-20T08:15:23.998090+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082629+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082629+0000","last_peered":"2024-03-20T08:15:24.082629+0000","last_clean":"2024-03-20T08:15:24.082629+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082629+0000","last_undegraded":"2024-03-20T08:15:24.082629+0000","last_fullsized":"2024-03-20T08:15:24.082629+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999198+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999198+0000","last_peered":"2024-03-20T08:15:23.999198+0000","last_clean":"2024-03-20T08:15:23.999198+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999198+0000","last_undegraded":"2024-03-20T08:15:23.999198+0000","last_fullsized":"2024-03-20T08:15:23.999198+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002171+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002171+0000","last_peered":"2024-03-20T08:15:24.002171+0000","last_clean":"2024-03-20T08:15:24.002171+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002171+0000","last_undegraded":"2024-03-20T08:15:24.002171+0000","last_fullsized":"2024-03-20T08:15:24.002171+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150313+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150313+0000","last_peered":"2024-03-20T08:15:24.150313+0000","last_clean":"2024-03-20T08:15:24.150313+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150313+0000","last_undegraded":"2024-03-20T08:15:24.150313+0000","last_fullsized":"2024-03-20T08:15:24.150313+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002146+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002146+0000","last_peered":"2024-03-20T08:15:24.002146+0000","last_clean":"2024-03-20T08:15:24.002146+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002146+0000","last_undegraded":"2024-03-20T08:15:24.002146+0000","last_fullsized":"2024-03-20T08:15:24.002146+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992579+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992579+0000","last_peered":"2024-03-20T08:15:23.992579+0000","last_clean":"2024-03-20T08:15:23.992579+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992579+0000","last_undegraded":"2024-03-20T08:15:23.992579+0000","last_fullsized":"2024-03-20T08:15:23.992579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1214,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149496+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149496+0000","last_peered":"2024-03-20T08:15:24.149496+0000","last_clean":"2024-03-20T08:15:24.149496+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149496+0000","last_undegraded":"2024-03-20T08:15:24.149496+0000","last_fullsized":"2024-03-20T08:15:24.149496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1113,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998326+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998326+0000","last_peered":"2024-03-20T08:15:23.998326+0000","last_clean":"2024-03-20T08:15:23.998326+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998326+0000","last_undegraded":"2024-03-20T08:15:23.998326+0000","last_fullsized":"2024-03-20T08:15:23.998326+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1249,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149492+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149492+0000","last_peered":"2024-03-20T08:15:24.149492+0000","last_clean":"2024-03-20T08:15:24.149492+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149492+0000","last_undegraded":"2024-03-20T08:15:24.149492+0000","last_fullsized":"2024-03-20T08:15:24.149492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68664,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150417+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150417+0000","last_peered":"2024-03-20T08:15:24.150417+0000","last_clean":"2024-03-20T08:15:24.150417+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150417+0000","last_undegraded":"2024-03-20T08:15:24.150417+0000","last_fullsized":"2024-03-20T08:15:24.150417+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001603+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001603+0000","last_peered":"2024-03-20T08:15:24.001603+0000","last_clean":"2024-03-20T08:15:24.001603+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001603+0000","last_undegraded":"2024-03-20T08:15:24.001603+0000","last_fullsized":"2024-03-20T08:15:24.001603+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991658+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991658+0000","last_peered":"2024-03-20T08:15:23.991658+0000","last_clean":"2024-03-20T08:15:23.991658+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991658+0000","last_undegraded":"2024-03-20T08:15:23.991658+0000","last_fullsized":"2024-03-20T08:15:23.991658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31730,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992676+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992676+0000","last_peered":"2024-03-20T08:15:23.992676+0000","last_clean":"2024-03-20T08:15:23.992676+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992676+0000","last_undegraded":"2024-03-20T08:15:23.992676+0000","last_fullsized":"2024-03-20T08:15:23.992676+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001649+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001649+0000","last_peered":"2024-03-20T08:15:24.001649+0000","last_clean":"2024-03-20T08:15:24.001649+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001649+0000","last_undegraded":"2024-03-20T08:15:24.001649+0000","last_fullsized":"2024-03-20T08:15:24.001649+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992344+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992344+0000","last_peered":"2024-03-20T08:15:23.992344+0000","last_clean":"2024-03-20T08:15:23.992344+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992344+0000","last_undegraded":"2024-03-20T08:15:23.992344+0000","last_fullsized":"2024-03-20T08:15:23.992344+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991173+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991173+0000","last_peered":"2024-03-20T08:15:23.991173+0000","last_clean":"2024-03-20T08:15:23.991173+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991173+0000","last_undegraded":"2024-03-20T08:15:23.991173+0000","last_fullsized":"2024-03-20T08:15:23.991173+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3375,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990881+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990881+0000","last_peered":"2024-03-20T08:15:23.990881+0000","last_clean":"2024-03-20T08:15:23.990881+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990881+0000","last_undegraded":"2024-03-20T08:15:23.990881+0000","last_fullsized":"2024-03-20T08:15:23.990881+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1299,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991178+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991178+0000","last_peered":"2024-03-20T08:15:23.991178+0000","last_clean":"2024-03-20T08:15:23.991178+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991178+0000","last_undegraded":"2024-03-20T08:15:23.991178+0000","last_fullsized":"2024-03-20T08:15:23.991178+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999147+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999147+0000","last_peered":"2024-03-20T08:15:23.999147+0000","last_clean":"2024-03-20T08:15:23.999147+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999147+0000","last_undegraded":"2024-03-20T08:15:23.999147+0000","last_fullsized":"2024-03-20T08:15:23.999147+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4956,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005035+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005035+0000","last_peered":"2024-03-20T08:15:24.005035+0000","last_clean":"2024-03-20T08:15:24.005035+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005035+0000","last_undegraded":"2024-03-20T08:15:24.005035+0000","last_fullsized":"2024-03-20T08:15:24.005035+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991516+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991516+0000","last_peered":"2024-03-20T08:15:23.991516+0000","last_clean":"2024-03-20T08:15:23.991516+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991516+0000","last_undegraded":"2024-03-20T08:15:23.991516+0000","last_fullsized":"2024-03-20T08:15:23.991516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999265+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999265+0000","last_peered":"2024-03-20T08:15:23.999265+0000","last_clean":"2024-03-20T08:15:23.999265+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999265+0000","last_undegraded":"2024-03-20T08:15:23.999265+0000","last_fullsized":"2024-03-20T08:15:23.999265+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990328+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990328+0000","last_peered":"2024-03-20T08:15:23.990328+0000","last_clean":"2024-03-20T08:15:23.990328+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990328+0000","last_undegraded":"2024-03-20T08:15:23.990328+0000","last_fullsized":"2024-03-20T08:15:23.990328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086388+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086388+0000","last_peered":"2024-03-20T08:15:24.086388+0000","last_clean":"2024-03-20T08:15:24.086388+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086388+0000","last_undegraded":"2024-03-20T08:15:24.086388+0000","last_fullsized":"2024-03-20T08:15:24.086388+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082357+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082357+0000","last_peered":"2024-03-20T08:15:24.082357+0000","last_clean":"2024-03-20T08:15:24.082357+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082357+0000","last_undegraded":"2024-03-20T08:15:24.082357+0000","last_fullsized":"2024-03-20T08:15:24.082357+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991132+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991132+0000","last_peered":"2024-03-20T08:15:23.991132+0000","last_clean":"2024-03-20T08:15:23.991132+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991132+0000","last_undegraded":"2024-03-20T08:15:23.991132+0000","last_fullsized":"2024-03-20T08:15:23.991132+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002102+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002102+0000","last_peered":"2024-03-20T08:15:24.002102+0000","last_clean":"2024-03-20T08:15:24.002102+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002102+0000","last_undegraded":"2024-03-20T08:15:24.002102+0000","last_fullsized":"2024-03-20T08:15:24.002102+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1275,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992674+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992674+0000","last_peered":"2024-03-20T08:15:23.992674+0000","last_clean":"2024-03-20T08:15:23.992674+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992674+0000","last_undegraded":"2024-03-20T08:15:23.992674+0000","last_fullsized":"2024-03-20T08:15:23.992674+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082363+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082363+0000","last_peered":"2024-03-20T08:15:24.082363+0000","last_clean":"2024-03-20T08:15:24.082363+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082363+0000","last_undegraded":"2024-03-20T08:15:24.082363+0000","last_fullsized":"2024-03-20T08:15:24.082363+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991867+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991867+0000","last_peered":"2024-03-20T08:15:23.991867+0000","last_clean":"2024-03-20T08:15:23.991867+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991867+0000","last_undegraded":"2024-03-20T08:15:23.991867+0000","last_fullsized":"2024-03-20T08:15:23.991867+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082336+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082336+0000","last_peered":"2024-03-20T08:15:24.082336+0000","last_clean":"2024-03-20T08:15:24.082336+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082336+0000","last_undegraded":"2024-03-20T08:15:24.082336+0000","last_fullsized":"2024-03-20T08:15:24.082336+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980168,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484561,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011572,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188923,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703404,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656870,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446814,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629209,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.751 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.752 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.752 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.752 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.752 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:42.752 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:15:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:43 smithi196 ceph-mon[87080]: from='client.44453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:43.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:43 smithi118 ceph-mon[93121]: from='client.44453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:43.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:43 smithi118 ceph-mon[95692]: from='client.44453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:15:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:15:43] "GET /metrics HTTP/1.1" 200 36407 "" "Prometheus/2.43.0" 2024-03-20T08:15:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:44 smithi196 ceph-mon[87080]: pgmap v1001: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:44 smithi118 ceph-mon[93121]: pgmap v1001: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:44 smithi118 ceph-mon[95692]: pgmap v1001: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:44.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:46.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:46 smithi196 ceph-mon[87080]: pgmap v1002: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:46 smithi118 ceph-mon[93121]: pgmap v1002: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:46.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:46 smithi118 ceph-mon[95692]: pgmap v1002: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:48 smithi196 ceph-mon[87080]: pgmap v1003: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:48 smithi118 ceph-mon[93121]: pgmap v1003: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:48.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:48 smithi118 ceph-mon[95692]: pgmap v1003: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:50.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:50 smithi196 ceph-mon[87080]: pgmap v1004: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:50 smithi118 ceph-mon[93121]: pgmap v1004: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:50.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:50 smithi118 ceph-mon[95692]: pgmap v1004: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:52.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:52 smithi196 ceph-mon[87080]: pgmap v1005: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:52 smithi118 ceph-mon[93121]: pgmap v1005: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:52.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:52 smithi118 ceph-mon[95692]: pgmap v1005: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:15:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:15:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:15:52.753 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:15:52.968 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:15:53.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:15:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:15:53] "GET /metrics HTTP/1.1" 200 36407 "" "Prometheus/2.43.0" 2024-03-20T08:15:54.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:54 smithi118 ceph-mon[93121]: pgmap v1006: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:54.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:54 smithi118 ceph-mon[95692]: pgmap v1006: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:54 smithi196 ceph-mon[87080]: pgmap v1006: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:54.946 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:15:54.948 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:15:55.593 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1007,"stamp":"2024-03-20T08:15:54.630283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003340"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999130+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999130+0000","last_peered":"2024-03-20T08:15:23.999130+0000","last_clean":"2024-03-20T08:15:23.999130+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999130+0000","last_undegraded":"2024-03-20T08:15:23.999130+0000","last_fullsized":"2024-03-20T08:15:23.999130+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1339,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002139+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002139+0000","last_peered":"2024-03-20T08:15:24.002139+0000","last_clean":"2024-03-20T08:15:24.002139+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002139+0000","last_undegraded":"2024-03-20T08:15:24.002139+0000","last_fullsized":"2024-03-20T08:15:24.002139+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998738+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998738+0000","last_peered":"2024-03-20T08:15:23.998738+0000","last_clean":"2024-03-20T08:15:23.998738+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998738+0000","last_undegraded":"2024-03-20T08:15:23.998738+0000","last_fullsized":"2024-03-20T08:15:23.998738+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991415+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991415+0000","last_peered":"2024-03-20T08:15:23.991415+0000","last_clean":"2024-03-20T08:15:23.991415+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991415+0000","last_undegraded":"2024-03-20T08:15:23.991415+0000","last_fullsized":"2024-03-20T08:15:23.991415+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991194+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991194+0000","last_peered":"2024-03-20T08:15:23.991194+0000","last_clean":"2024-03-20T08:15:23.991194+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991194+0000","last_undegraded":"2024-03-20T08:15:23.991194+0000","last_fullsized":"2024-03-20T08:15:23.991194+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992438+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992438+0000","last_peered":"2024-03-20T08:15:23.992438+0000","last_clean":"2024-03-20T08:15:23.992438+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992438+0000","last_undegraded":"2024-03-20T08:15:23.992438+0000","last_fullsized":"2024-03-20T08:15:23.992438+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082177+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082177+0000","last_peered":"2024-03-20T08:15:24.082177+0000","last_clean":"2024-03-20T08:15:24.082177+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082177+0000","last_undegraded":"2024-03-20T08:15:24.082177+0000","last_fullsized":"2024-03-20T08:15:24.082177+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1299,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991364+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991364+0000","last_peered":"2024-03-20T08:15:23.991364+0000","last_clean":"2024-03-20T08:15:23.991364+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991364+0000","last_undegraded":"2024-03-20T08:15:23.991364+0000","last_fullsized":"2024-03-20T08:15:23.991364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999389+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999389+0000","last_peered":"2024-03-20T08:15:23.999389+0000","last_clean":"2024-03-20T08:15:23.999389+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999389+0000","last_undegraded":"2024-03-20T08:15:23.999389+0000","last_fullsized":"2024-03-20T08:15:23.999389+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991223+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991223+0000","last_peered":"2024-03-20T08:15:23.991223+0000","last_clean":"2024-03-20T08:15:23.991223+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991223+0000","last_undegraded":"2024-03-20T08:15:23.991223+0000","last_fullsized":"2024-03-20T08:15:23.991223+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991397+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991397+0000","last_peered":"2024-03-20T08:15:23.991397+0000","last_clean":"2024-03-20T08:15:23.991397+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991397+0000","last_undegraded":"2024-03-20T08:15:23.991397+0000","last_fullsized":"2024-03-20T08:15:23.991397+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991466+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991466+0000","last_peered":"2024-03-20T08:15:23.991466+0000","last_clean":"2024-03-20T08:15:23.991466+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991466+0000","last_undegraded":"2024-03-20T08:15:23.991466+0000","last_fullsized":"2024-03-20T08:15:23.991466+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991749+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991749+0000","last_peered":"2024-03-20T08:15:23.991749+0000","last_clean":"2024-03-20T08:15:23.991749+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991749+0000","last_undegraded":"2024-03-20T08:15:23.991749+0000","last_fullsized":"2024-03-20T08:15:23.991749+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1219,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991508+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991508+0000","last_peered":"2024-03-20T08:15:23.991508+0000","last_clean":"2024-03-20T08:15:23.991508+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991508+0000","last_undegraded":"2024-03-20T08:15:23.991508+0000","last_fullsized":"2024-03-20T08:15:23.991508+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998902+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998902+0000","last_peered":"2024-03-20T08:15:23.998902+0000","last_clean":"2024-03-20T08:15:23.998902+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998902+0000","last_undegraded":"2024-03-20T08:15:23.998902+0000","last_fullsized":"2024-03-20T08:15:23.998902+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082824+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082824+0000","last_peered":"2024-03-20T08:15:24.082824+0000","last_clean":"2024-03-20T08:15:24.082824+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082824+0000","last_undegraded":"2024-03-20T08:15:24.082824+0000","last_fullsized":"2024-03-20T08:15:24.082824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085437+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085437+0000","last_peered":"2024-03-20T08:15:24.085437+0000","last_clean":"2024-03-20T08:15:24.085437+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085437+0000","last_undegraded":"2024-03-20T08:15:24.085437+0000","last_fullsized":"2024-03-20T08:15:24.085437+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998772+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998772+0000","last_peered":"2024-03-20T08:15:23.998772+0000","last_clean":"2024-03-20T08:15:23.998772+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998772+0000","last_undegraded":"2024-03-20T08:15:23.998772+0000","last_fullsized":"2024-03-20T08:15:23.998772+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990901+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990901+0000","last_peered":"2024-03-20T08:15:23.990901+0000","last_clean":"2024-03-20T08:15:23.990901+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990901+0000","last_undegraded":"2024-03-20T08:15:23.990901+0000","last_fullsized":"2024-03-20T08:15:23.990901+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990851+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990851+0000","last_peered":"2024-03-20T08:15:23.990851+0000","last_clean":"2024-03-20T08:15:23.990851+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990851+0000","last_undegraded":"2024-03-20T08:15:23.990851+0000","last_fullsized":"2024-03-20T08:15:23.990851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1315,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990647+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990647+0000","last_peered":"2024-03-20T08:15:23.990647+0000","last_clean":"2024-03-20T08:15:23.990647+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990647+0000","last_undegraded":"2024-03-20T08:15:23.990647+0000","last_fullsized":"2024-03-20T08:15:23.990647+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998183+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998183+0000","last_peered":"2024-03-20T08:15:23.998183+0000","last_clean":"2024-03-20T08:15:23.998183+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998183+0000","last_undegraded":"2024-03-20T08:15:23.998183+0000","last_fullsized":"2024-03-20T08:15:23.998183+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150274+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150274+0000","last_peered":"2024-03-20T08:15:24.150274+0000","last_clean":"2024-03-20T08:15:24.150274+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150274+0000","last_undegraded":"2024-03-20T08:15:24.150274+0000","last_fullsized":"2024-03-20T08:15:24.150274+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998895+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998895+0000","last_peered":"2024-03-20T08:15:23.998895+0000","last_clean":"2024-03-20T08:15:23.998895+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998895+0000","last_undegraded":"2024-03-20T08:15:23.998895+0000","last_fullsized":"2024-03-20T08:15:23.998895+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001728+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001728+0000","last_peered":"2024-03-20T08:15:24.001728+0000","last_clean":"2024-03-20T08:15:24.001728+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001728+0000","last_undegraded":"2024-03-20T08:15:24.001728+0000","last_fullsized":"2024-03-20T08:15:24.001728+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001977+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.001977+0000","last_peered":"2024-03-20T08:15:24.001977+0000","last_clean":"2024-03-20T08:15:24.001977+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.001977+0000","last_undegraded":"2024-03-20T08:15:24.001977+0000","last_fullsized":"2024-03-20T08:15:24.001977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082803+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082803+0000","last_peered":"2024-03-20T08:15:24.082803+0000","last_clean":"2024-03-20T08:15:24.082803+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082803+0000","last_undegraded":"2024-03-20T08:15:24.082803+0000","last_fullsized":"2024-03-20T08:15:24.082803+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002077+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002077+0000","last_peered":"2024-03-20T08:15:24.002077+0000","last_clean":"2024-03-20T08:15:24.002077+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002077+0000","last_undegraded":"2024-03-20T08:15:24.002077+0000","last_fullsized":"2024-03-20T08:15:24.002077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082497+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082497+0000","last_peered":"2024-03-20T08:15:24.082497+0000","last_clean":"2024-03-20T08:15:24.082497+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082497+0000","last_undegraded":"2024-03-20T08:15:24.082497+0000","last_fullsized":"2024-03-20T08:15:24.082497+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149780+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149780+0000","last_peered":"2024-03-20T08:15:24.149780+0000","last_clean":"2024-03-20T08:15:24.149780+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149780+0000","last_undegraded":"2024-03-20T08:15:24.149780+0000","last_fullsized":"2024-03-20T08:15:24.149780+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1332,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001769+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001769+0000","last_peered":"2024-03-20T08:15:24.001769+0000","last_clean":"2024-03-20T08:15:24.001769+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001769+0000","last_undegraded":"2024-03-20T08:15:24.001769+0000","last_fullsized":"2024-03-20T08:15:24.001769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001346+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001346+0000","last_peered":"2024-03-20T08:15:24.001346+0000","last_clean":"2024-03-20T08:15:24.001346+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001346+0000","last_undegraded":"2024-03-20T08:15:24.001346+0000","last_fullsized":"2024-03-20T08:15:24.001346+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990310+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990310+0000","last_peered":"2024-03-20T08:15:23.990310+0000","last_clean":"2024-03-20T08:15:23.990310+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990310+0000","last_undegraded":"2024-03-20T08:15:23.990310+0000","last_fullsized":"2024-03-20T08:15:23.990310+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3977,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999530+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999530+0000","last_peered":"2024-03-20T08:15:23.999530+0000","last_clean":"2024-03-20T08:15:23.999530+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999530+0000","last_undegraded":"2024-03-20T08:15:23.999530+0000","last_fullsized":"2024-03-20T08:15:23.999530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082450+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082450+0000","last_peered":"2024-03-20T08:15:24.082450+0000","last_clean":"2024-03-20T08:15:24.082450+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082450+0000","last_undegraded":"2024-03-20T08:15:24.082450+0000","last_fullsized":"2024-03-20T08:15:24.082450+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9295,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083467+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083467+0000","last_peered":"2024-03-20T08:15:24.083467+0000","last_clean":"2024-03-20T08:15:24.083467+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083467+0000","last_undegraded":"2024-03-20T08:15:24.083467+0000","last_fullsized":"2024-03-20T08:15:24.083467+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083126+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083126+0000","last_peered":"2024-03-20T08:15:24.083126+0000","last_clean":"2024-03-20T08:15:24.083126+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083126+0000","last_undegraded":"2024-03-20T08:15:24.083126+0000","last_fullsized":"2024-03-20T08:15:24.083126+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999198+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999198+0000","last_peered":"2024-03-20T08:15:23.999198+0000","last_clean":"2024-03-20T08:15:23.999198+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999198+0000","last_undegraded":"2024-03-20T08:15:23.999198+0000","last_fullsized":"2024-03-20T08:15:23.999198+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002171+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002171+0000","last_peered":"2024-03-20T08:15:24.002171+0000","last_clean":"2024-03-20T08:15:24.002171+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002171+0000","last_undegraded":"2024-03-20T08:15:24.002171+0000","last_fullsized":"2024-03-20T08:15:24.002171+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150313+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150313+0000","last_peered":"2024-03-20T08:15:24.150313+0000","last_clean":"2024-03-20T08:15:24.150313+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150313+0000","last_undegraded":"2024-03-20T08:15:24.150313+0000","last_fullsized":"2024-03-20T08:15:24.150313+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002146+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002146+0000","last_peered":"2024-03-20T08:15:24.002146+0000","last_clean":"2024-03-20T08:15:24.002146+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002146+0000","last_undegraded":"2024-03-20T08:15:24.002146+0000","last_fullsized":"2024-03-20T08:15:24.002146+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992579+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992579+0000","last_peered":"2024-03-20T08:15:23.992579+0000","last_clean":"2024-03-20T08:15:23.992579+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992579+0000","last_undegraded":"2024-03-20T08:15:23.992579+0000","last_fullsized":"2024-03-20T08:15:23.992579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1214,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149496+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149496+0000","last_peered":"2024-03-20T08:15:24.149496+0000","last_clean":"2024-03-20T08:15:24.149496+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149496+0000","last_undegraded":"2024-03-20T08:15:24.149496+0000","last_fullsized":"2024-03-20T08:15:24.149496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1113,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998326+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998326+0000","last_peered":"2024-03-20T08:15:23.998326+0000","last_clean":"2024-03-20T08:15:23.998326+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998326+0000","last_undegraded":"2024-03-20T08:15:23.998326+0000","last_fullsized":"2024-03-20T08:15:23.998326+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1249,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149492+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149492+0000","last_peered":"2024-03-20T08:15:24.149492+0000","last_clean":"2024-03-20T08:15:24.149492+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149492+0000","last_undegraded":"2024-03-20T08:15:24.149492+0000","last_fullsized":"2024-03-20T08:15:24.149492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68664,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150417+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150417+0000","last_peered":"2024-03-20T08:15:24.150417+0000","last_clean":"2024-03-20T08:15:24.150417+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150417+0000","last_undegraded":"2024-03-20T08:15:24.150417+0000","last_fullsized":"2024-03-20T08:15:24.150417+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001603+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001603+0000","last_peered":"2024-03-20T08:15:24.001603+0000","last_clean":"2024-03-20T08:15:24.001603+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001603+0000","last_undegraded":"2024-03-20T08:15:24.001603+0000","last_fullsized":"2024-03-20T08:15:24.001603+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991658+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991658+0000","last_peered":"2024-03-20T08:15:23.991658+0000","last_clean":"2024-03-20T08:15:23.991658+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991658+0000","last_undegraded":"2024-03-20T08:15:23.991658+0000","last_fullsized":"2024-03-20T08:15:23.991658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001649+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001649+0000","last_peered":"2024-03-20T08:15:24.001649+0000","last_clean":"2024-03-20T08:15:24.001649+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001649+0000","last_undegraded":"2024-03-20T08:15:24.001649+0000","last_fullsized":"2024-03-20T08:15:24.001649+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992344+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992344+0000","last_peered":"2024-03-20T08:15:23.992344+0000","last_clean":"2024-03-20T08:15:23.992344+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992344+0000","last_undegraded":"2024-03-20T08:15:23.992344+0000","last_fullsized":"2024-03-20T08:15:23.992344+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999147+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999147+0000","last_peered":"2024-03-20T08:15:23.999147+0000","last_clean":"2024-03-20T08:15:23.999147+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999147+0000","last_undegraded":"2024-03-20T08:15:23.999147+0000","last_fullsized":"2024-03-20T08:15:23.999147+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4956,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005035+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005035+0000","last_peered":"2024-03-20T08:15:24.005035+0000","last_clean":"2024-03-20T08:15:24.005035+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005035+0000","last_undegraded":"2024-03-20T08:15:24.005035+0000","last_fullsized":"2024-03-20T08:15:24.005035+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999265+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999265+0000","last_peered":"2024-03-20T08:15:23.999265+0000","last_clean":"2024-03-20T08:15:23.999265+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999265+0000","last_undegraded":"2024-03-20T08:15:23.999265+0000","last_fullsized":"2024-03-20T08:15:23.999265+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990328+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990328+0000","last_peered":"2024-03-20T08:15:23.990328+0000","last_clean":"2024-03-20T08:15:23.990328+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990328+0000","last_undegraded":"2024-03-20T08:15:23.990328+0000","last_fullsized":"2024-03-20T08:15:23.990328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086388+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086388+0000","last_peered":"2024-03-20T08:15:24.086388+0000","last_clean":"2024-03-20T08:15:24.086388+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086388+0000","last_undegraded":"2024-03-20T08:15:24.086388+0000","last_fullsized":"2024-03-20T08:15:24.086388+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991132+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991132+0000","last_peered":"2024-03-20T08:15:23.991132+0000","last_clean":"2024-03-20T08:15:23.991132+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991132+0000","last_undegraded":"2024-03-20T08:15:23.991132+0000","last_fullsized":"2024-03-20T08:15:23.991132+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002102+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002102+0000","last_peered":"2024-03-20T08:15:24.002102+0000","last_clean":"2024-03-20T08:15:24.002102+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002102+0000","last_undegraded":"2024-03-20T08:15:24.002102+0000","last_fullsized":"2024-03-20T08:15:24.002102+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1275,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992674+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992674+0000","last_peered":"2024-03-20T08:15:23.992674+0000","last_clean":"2024-03-20T08:15:23.992674+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992674+0000","last_undegraded":"2024-03-20T08:15:23.992674+0000","last_fullsized":"2024-03-20T08:15:23.992674+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082363+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082363+0000","last_peered":"2024-03-20T08:15:24.082363+0000","last_clean":"2024-03-20T08:15:24.082363+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082363+0000","last_undegraded":"2024-03-20T08:15:24.082363+0000","last_fullsized":"2024-03-20T08:15:24.082363+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991867+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991867+0000","last_peered":"2024-03-20T08:15:23.991867+0000","last_clean":"2024-03-20T08:15:23.991867+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991867+0000","last_undegraded":"2024-03-20T08:15:23.991867+0000","last_fullsized":"2024-03-20T08:15:23.991867+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082336+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082336+0000","last_peered":"2024-03-20T08:15:24.082336+0000","last_clean":"2024-03-20T08:15:24.082336+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082336+0000","last_undegraded":"2024-03-20T08:15:24.082336+0000","last_fullsized":"2024-03-20T08:15:24.082336+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980170,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484564,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011575,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188926,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656872,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446816,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629211,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:15:55.601 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.601 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.601 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.601 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.601 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.601 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.601 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:15:55.602 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:15:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:56 smithi196 ceph-mon[87080]: pgmap v1007: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:56 smithi196 ceph-mon[87080]: from='client.34539 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:56 smithi118 ceph-mon[93121]: pgmap v1007: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:56 smithi118 ceph-mon[93121]: from='client.34539 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:56.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:56 smithi118 ceph-mon[95692]: pgmap v1007: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:56 smithi118 ceph-mon[95692]: from='client.34539 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:15:58.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:58 smithi196 ceph-mon[87080]: pgmap v1008: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:58.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:58 smithi118 ceph-mon[93121]: pgmap v1008: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:58.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:58 smithi118 ceph-mon[95692]: pgmap v1008: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:15:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:15:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:15:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:15:59.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:15:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:00 smithi196 ceph-mon[87080]: pgmap v1009: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:00 smithi118 ceph-mon[93121]: pgmap v1009: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:00.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:00 smithi118 ceph-mon[95692]: pgmap v1009: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:02.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:02 smithi196 ceph-mon[87080]: pgmap v1010: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:02 smithi118 ceph-mon[93121]: pgmap v1010: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:02.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:02 smithi118 ceph-mon[95692]: pgmap v1010: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:16:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:16:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:16:03.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:16:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:16:03] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:16:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:04 smithi196 ceph-mon[87080]: pgmap v1011: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:04.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:04 smithi118 ceph-mon[93121]: pgmap v1011: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:04.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:04 smithi118 ceph-mon[95692]: pgmap v1011: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:05.604 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:16:05.829 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:16:06.259 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:06 smithi118 ceph-mon[93121]: pgmap v1012: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:06.259 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:06 smithi118 ceph-mon[95692]: pgmap v1012: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:06.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:06 smithi196 ceph-mon[87080]: pgmap v1012: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:07.788 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:16:07.790 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:16:08.388 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:08 smithi118 ceph-mon[93121]: pgmap v1013: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:08.388 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:08 smithi118 ceph-mon[95692]: pgmap v1013: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:08 smithi196 ceph-mon[87080]: pgmap v1013: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:08.408 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1013,"stamp":"2024-03-20T08:16:06.633130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002846"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1339,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002139+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002139+0000","last_peered":"2024-03-20T08:15:24.002139+0000","last_clean":"2024-03-20T08:15:24.002139+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002139+0000","last_undegraded":"2024-03-20T08:15:24.002139+0000","last_fullsized":"2024-03-20T08:15:24.002139+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998738+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998738+0000","last_peered":"2024-03-20T08:15:23.998738+0000","last_clean":"2024-03-20T08:15:23.998738+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998738+0000","last_undegraded":"2024-03-20T08:15:23.998738+0000","last_fullsized":"2024-03-20T08:15:23.998738+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991415+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991415+0000","last_peered":"2024-03-20T08:15:23.991415+0000","last_clean":"2024-03-20T08:15:23.991415+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991415+0000","last_undegraded":"2024-03-20T08:15:23.991415+0000","last_fullsized":"2024-03-20T08:15:23.991415+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1183,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991194+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991194+0000","last_peered":"2024-03-20T08:15:23.991194+0000","last_clean":"2024-03-20T08:15:23.991194+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991194+0000","last_undegraded":"2024-03-20T08:15:23.991194+0000","last_fullsized":"2024-03-20T08:15:23.991194+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992438+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992438+0000","last_peered":"2024-03-20T08:15:23.992438+0000","last_clean":"2024-03-20T08:15:23.992438+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992438+0000","last_undegraded":"2024-03-20T08:15:23.992438+0000","last_fullsized":"2024-03-20T08:15:23.992438+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082177+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082177+0000","last_peered":"2024-03-20T08:15:24.082177+0000","last_clean":"2024-03-20T08:15:24.082177+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082177+0000","last_undegraded":"2024-03-20T08:15:24.082177+0000","last_fullsized":"2024-03-20T08:15:24.082177+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999389+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999389+0000","last_peered":"2024-03-20T08:15:23.999389+0000","last_clean":"2024-03-20T08:15:23.999389+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999389+0000","last_undegraded":"2024-03-20T08:15:23.999389+0000","last_fullsized":"2024-03-20T08:15:23.999389+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991223+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991223+0000","last_peered":"2024-03-20T08:15:23.991223+0000","last_clean":"2024-03-20T08:15:23.991223+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991223+0000","last_undegraded":"2024-03-20T08:15:23.991223+0000","last_fullsized":"2024-03-20T08:15:23.991223+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991397+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991397+0000","last_peered":"2024-03-20T08:15:23.991397+0000","last_clean":"2024-03-20T08:15:23.991397+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991397+0000","last_undegraded":"2024-03-20T08:15:23.991397+0000","last_fullsized":"2024-03-20T08:15:23.991397+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991466+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991466+0000","last_peered":"2024-03-20T08:15:23.991466+0000","last_clean":"2024-03-20T08:15:23.991466+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991466+0000","last_undegraded":"2024-03-20T08:15:23.991466+0000","last_fullsized":"2024-03-20T08:15:23.991466+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991749+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991749+0000","last_peered":"2024-03-20T08:15:23.991749+0000","last_clean":"2024-03-20T08:15:23.991749+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991749+0000","last_undegraded":"2024-03-20T08:15:23.991749+0000","last_fullsized":"2024-03-20T08:15:23.991749+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998902+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998902+0000","last_peered":"2024-03-20T08:15:23.998902+0000","last_clean":"2024-03-20T08:15:23.998902+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998902+0000","last_undegraded":"2024-03-20T08:15:23.998902+0000","last_fullsized":"2024-03-20T08:15:23.998902+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082824+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082824+0000","last_peered":"2024-03-20T08:15:24.082824+0000","last_clean":"2024-03-20T08:15:24.082824+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082824+0000","last_undegraded":"2024-03-20T08:15:24.082824+0000","last_fullsized":"2024-03-20T08:15:24.082824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085437+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085437+0000","last_peered":"2024-03-20T08:15:24.085437+0000","last_clean":"2024-03-20T08:15:24.085437+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085437+0000","last_undegraded":"2024-03-20T08:15:24.085437+0000","last_fullsized":"2024-03-20T08:15:24.085437+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998772+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998772+0000","last_peered":"2024-03-20T08:15:23.998772+0000","last_clean":"2024-03-20T08:15:23.998772+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998772+0000","last_undegraded":"2024-03-20T08:15:23.998772+0000","last_fullsized":"2024-03-20T08:15:23.998772+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998183+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998183+0000","last_peered":"2024-03-20T08:15:23.998183+0000","last_clean":"2024-03-20T08:15:23.998183+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998183+0000","last_undegraded":"2024-03-20T08:15:23.998183+0000","last_fullsized":"2024-03-20T08:15:23.998183+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150274+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150274+0000","last_peered":"2024-03-20T08:15:24.150274+0000","last_clean":"2024-03-20T08:15:24.150274+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150274+0000","last_undegraded":"2024-03-20T08:15:24.150274+0000","last_fullsized":"2024-03-20T08:15:24.150274+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998895+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998895+0000","last_peered":"2024-03-20T08:15:23.998895+0000","last_clean":"2024-03-20T08:15:23.998895+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998895+0000","last_undegraded":"2024-03-20T08:15:23.998895+0000","last_fullsized":"2024-03-20T08:15:23.998895+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001728+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001728+0000","last_peered":"2024-03-20T08:15:24.001728+0000","last_clean":"2024-03-20T08:15:24.001728+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001728+0000","last_undegraded":"2024-03-20T08:15:24.001728+0000","last_fullsized":"2024-03-20T08:15:24.001728+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001977+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.001977+0000","last_peered":"2024-03-20T08:15:24.001977+0000","last_clean":"2024-03-20T08:15:24.001977+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.001977+0000","last_undegraded":"2024-03-20T08:15:24.001977+0000","last_fullsized":"2024-03-20T08:15:24.001977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082803+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082803+0000","last_peered":"2024-03-20T08:15:24.082803+0000","last_clean":"2024-03-20T08:15:24.082803+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082803+0000","last_undegraded":"2024-03-20T08:15:24.082803+0000","last_fullsized":"2024-03-20T08:15:24.082803+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002077+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002077+0000","last_peered":"2024-03-20T08:15:24.002077+0000","last_clean":"2024-03-20T08:15:24.002077+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002077+0000","last_undegraded":"2024-03-20T08:15:24.002077+0000","last_fullsized":"2024-03-20T08:15:24.002077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082497+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082497+0000","last_peered":"2024-03-20T08:15:24.082497+0000","last_clean":"2024-03-20T08:15:24.082497+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082497+0000","last_undegraded":"2024-03-20T08:15:24.082497+0000","last_fullsized":"2024-03-20T08:15:24.082497+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149780+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149780+0000","last_peered":"2024-03-20T08:15:24.149780+0000","last_clean":"2024-03-20T08:15:24.149780+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149780+0000","last_undegraded":"2024-03-20T08:15:24.149780+0000","last_fullsized":"2024-03-20T08:15:24.149780+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1332,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001769+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001769+0000","last_peered":"2024-03-20T08:15:24.001769+0000","last_clean":"2024-03-20T08:15:24.001769+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001769+0000","last_undegraded":"2024-03-20T08:15:24.001769+0000","last_fullsized":"2024-03-20T08:15:24.001769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990310+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990310+0000","last_peered":"2024-03-20T08:15:23.990310+0000","last_clean":"2024-03-20T08:15:23.990310+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990310+0000","last_undegraded":"2024-03-20T08:15:23.990310+0000","last_fullsized":"2024-03-20T08:15:23.990310+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082450+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082450+0000","last_peered":"2024-03-20T08:15:24.082450+0000","last_clean":"2024-03-20T08:15:24.082450+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082450+0000","last_undegraded":"2024-03-20T08:15:24.082450+0000","last_fullsized":"2024-03-20T08:15:24.082450+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9295,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083467+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083467+0000","last_peered":"2024-03-20T08:15:24.083467+0000","last_clean":"2024-03-20T08:15:24.083467+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083467+0000","last_undegraded":"2024-03-20T08:15:24.083467+0000","last_fullsized":"2024-03-20T08:15:24.083467+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083126+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083126+0000","last_peered":"2024-03-20T08:15:24.083126+0000","last_clean":"2024-03-20T08:15:24.083126+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083126+0000","last_undegraded":"2024-03-20T08:15:24.083126+0000","last_fullsized":"2024-03-20T08:15:24.083126+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002171+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002171+0000","last_peered":"2024-03-20T08:15:24.002171+0000","last_clean":"2024-03-20T08:15:24.002171+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002171+0000","last_undegraded":"2024-03-20T08:15:24.002171+0000","last_fullsized":"2024-03-20T08:15:24.002171+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150313+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150313+0000","last_peered":"2024-03-20T08:15:24.150313+0000","last_clean":"2024-03-20T08:15:24.150313+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150313+0000","last_undegraded":"2024-03-20T08:15:24.150313+0000","last_fullsized":"2024-03-20T08:15:24.150313+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002146+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002146+0000","last_peered":"2024-03-20T08:15:24.002146+0000","last_clean":"2024-03-20T08:15:24.002146+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002146+0000","last_undegraded":"2024-03-20T08:15:24.002146+0000","last_fullsized":"2024-03-20T08:15:24.002146+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1214,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149496+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149496+0000","last_peered":"2024-03-20T08:15:24.149496+0000","last_clean":"2024-03-20T08:15:24.149496+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149496+0000","last_undegraded":"2024-03-20T08:15:24.149496+0000","last_fullsized":"2024-03-20T08:15:24.149496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1113,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998326+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998326+0000","last_peered":"2024-03-20T08:15:23.998326+0000","last_clean":"2024-03-20T08:15:23.998326+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998326+0000","last_undegraded":"2024-03-20T08:15:23.998326+0000","last_fullsized":"2024-03-20T08:15:23.998326+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1249,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149492+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149492+0000","last_peered":"2024-03-20T08:15:24.149492+0000","last_clean":"2024-03-20T08:15:24.149492+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149492+0000","last_undegraded":"2024-03-20T08:15:24.149492+0000","last_fullsized":"2024-03-20T08:15:24.149492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68664,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150417+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150417+0000","last_peered":"2024-03-20T08:15:24.150417+0000","last_clean":"2024-03-20T08:15:24.150417+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150417+0000","last_undegraded":"2024-03-20T08:15:24.150417+0000","last_fullsized":"2024-03-20T08:15:24.150417+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001603+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001603+0000","last_peered":"2024-03-20T08:15:24.001603+0000","last_clean":"2024-03-20T08:15:24.001603+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001603+0000","last_undegraded":"2024-03-20T08:15:24.001603+0000","last_fullsized":"2024-03-20T08:15:24.001603+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001649+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001649+0000","last_peered":"2024-03-20T08:15:24.001649+0000","last_clean":"2024-03-20T08:15:24.001649+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001649+0000","last_undegraded":"2024-03-20T08:15:24.001649+0000","last_fullsized":"2024-03-20T08:15:24.001649+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992344+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992344+0000","last_peered":"2024-03-20T08:15:23.992344+0000","last_clean":"2024-03-20T08:15:23.992344+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992344+0000","last_undegraded":"2024-03-20T08:15:23.992344+0000","last_fullsized":"2024-03-20T08:15:23.992344+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999147+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999147+0000","last_peered":"2024-03-20T08:15:23.999147+0000","last_clean":"2024-03-20T08:15:23.999147+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999147+0000","last_undegraded":"2024-03-20T08:15:23.999147+0000","last_fullsized":"2024-03-20T08:15:23.999147+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4956,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005035+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005035+0000","last_peered":"2024-03-20T08:15:24.005035+0000","last_clean":"2024-03-20T08:15:24.005035+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005035+0000","last_undegraded":"2024-03-20T08:15:24.005035+0000","last_fullsized":"2024-03-20T08:15:24.005035+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999265+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999265+0000","last_peered":"2024-03-20T08:15:23.999265+0000","last_clean":"2024-03-20T08:15:23.999265+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999265+0000","last_undegraded":"2024-03-20T08:15:23.999265+0000","last_fullsized":"2024-03-20T08:15:23.999265+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990328+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990328+0000","last_peered":"2024-03-20T08:15:23.990328+0000","last_clean":"2024-03-20T08:15:23.990328+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990328+0000","last_undegraded":"2024-03-20T08:15:23.990328+0000","last_fullsized":"2024-03-20T08:15:23.990328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086388+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086388+0000","last_peered":"2024-03-20T08:15:24.086388+0000","last_clean":"2024-03-20T08:15:24.086388+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086388+0000","last_undegraded":"2024-03-20T08:15:24.086388+0000","last_fullsized":"2024-03-20T08:15:24.086388+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002102+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002102+0000","last_peered":"2024-03-20T08:15:24.002102+0000","last_clean":"2024-03-20T08:15:24.002102+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002102+0000","last_undegraded":"2024-03-20T08:15:24.002102+0000","last_fullsized":"2024-03-20T08:15:24.002102+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082363+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082363+0000","last_peered":"2024-03-20T08:15:24.082363+0000","last_clean":"2024-03-20T08:15:24.082363+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082363+0000","last_undegraded":"2024-03-20T08:15:24.082363+0000","last_fullsized":"2024-03-20T08:15:24.082363+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991867+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991867+0000","last_peered":"2024-03-20T08:15:23.991867+0000","last_clean":"2024-03-20T08:15:23.991867+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991867+0000","last_undegraded":"2024-03-20T08:15:23.991867+0000","last_fullsized":"2024-03-20T08:15:23.991867+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082336+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082336+0000","last_peered":"2024-03-20T08:15:24.082336+0000","last_clean":"2024-03-20T08:15:24.082336+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082336+0000","last_undegraded":"2024-03-20T08:15:24.082336+0000","last_fullsized":"2024-03-20T08:15:24.082336+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980173,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484566,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011577,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188929,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703409,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656875,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446819,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629214,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.414 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.415 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.415 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.415 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:08.415 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:16:09.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:09 smithi196 ceph-mon[87080]: from='client.34545 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:09.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:09 smithi118 ceph-mon[93121]: from='client.34545 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:09.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:09 smithi118 ceph-mon[95692]: from='client.34545 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:10.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:10 smithi196 ceph-mon[87080]: pgmap v1014: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:10.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:10 smithi118 ceph-mon[93121]: pgmap v1014: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:10.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:10 smithi118 ceph-mon[95692]: pgmap v1014: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:12.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:12 smithi196 ceph-mon[87080]: pgmap v1015: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:12.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:16:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:16:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:16:12.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:12 smithi118 ceph-mon[93121]: pgmap v1015: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:12.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:12 smithi118 ceph-mon[95692]: pgmap v1015: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:16:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:16:13] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:16:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:14 smithi196 ceph-mon[87080]: pgmap v1016: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:16:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:14 smithi118 ceph-mon[93121]: pgmap v1016: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:16:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:14 smithi118 ceph-mon[95692]: pgmap v1016: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:16:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:15 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:16:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:15 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:16:15.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:15 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:16:15.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:15 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:16:15.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:15 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:16:15.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:15 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:16:15.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:15 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:16:15.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:15 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:16:15.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:15 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:16:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:16 smithi196 ceph-mon[87080]: pgmap v1017: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:16 smithi118 ceph-mon[93121]: pgmap v1017: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:16.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:16 smithi118 ceph-mon[95692]: pgmap v1017: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:18 smithi196 ceph-mon[87080]: pgmap v1018: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:18.416 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:16:18.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:18 smithi118 ceph-mon[93121]: pgmap v1018: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:18.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:18 smithi118 ceph-mon[95692]: pgmap v1018: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:18.629 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:16:20.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:20 smithi196 ceph-mon[87080]: pgmap v1019: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:20 smithi118 ceph-mon[93121]: pgmap v1019: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:20 smithi118 ceph-mon[95692]: pgmap v1019: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:20.549 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:16:20.552 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:16:21.200 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1019,"stamp":"2024-03-20T08:16:18.636222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003092"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1339,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002139+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002139+0000","last_peered":"2024-03-20T08:15:24.002139+0000","last_clean":"2024-03-20T08:15:24.002139+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002139+0000","last_undegraded":"2024-03-20T08:15:24.002139+0000","last_fullsized":"2024-03-20T08:15:24.002139+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998738+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998738+0000","last_peered":"2024-03-20T08:15:23.998738+0000","last_clean":"2024-03-20T08:15:23.998738+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998738+0000","last_undegraded":"2024-03-20T08:15:23.998738+0000","last_fullsized":"2024-03-20T08:15:23.998738+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992438+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992438+0000","last_peered":"2024-03-20T08:15:23.992438+0000","last_clean":"2024-03-20T08:15:23.992438+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992438+0000","last_undegraded":"2024-03-20T08:15:23.992438+0000","last_fullsized":"2024-03-20T08:15:23.992438+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082177+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082177+0000","last_peered":"2024-03-20T08:15:24.082177+0000","last_clean":"2024-03-20T08:15:24.082177+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082177+0000","last_undegraded":"2024-03-20T08:15:24.082177+0000","last_fullsized":"2024-03-20T08:15:24.082177+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999389+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999389+0000","last_peered":"2024-03-20T08:15:23.999389+0000","last_clean":"2024-03-20T08:15:23.999389+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999389+0000","last_undegraded":"2024-03-20T08:15:23.999389+0000","last_fullsized":"2024-03-20T08:15:23.999389+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991223+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991223+0000","last_peered":"2024-03-20T08:15:23.991223+0000","last_clean":"2024-03-20T08:15:23.991223+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991223+0000","last_undegraded":"2024-03-20T08:15:23.991223+0000","last_fullsized":"2024-03-20T08:15:23.991223+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991397+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991397+0000","last_peered":"2024-03-20T08:15:23.991397+0000","last_clean":"2024-03-20T08:15:23.991397+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991397+0000","last_undegraded":"2024-03-20T08:15:23.991397+0000","last_fullsized":"2024-03-20T08:15:23.991397+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991466+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991466+0000","last_peered":"2024-03-20T08:15:23.991466+0000","last_clean":"2024-03-20T08:15:23.991466+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991466+0000","last_undegraded":"2024-03-20T08:15:23.991466+0000","last_fullsized":"2024-03-20T08:15:23.991466+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998902+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998902+0000","last_peered":"2024-03-20T08:15:23.998902+0000","last_clean":"2024-03-20T08:15:23.998902+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998902+0000","last_undegraded":"2024-03-20T08:15:23.998902+0000","last_fullsized":"2024-03-20T08:15:23.998902+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082824+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082824+0000","last_peered":"2024-03-20T08:15:24.082824+0000","last_clean":"2024-03-20T08:15:24.082824+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082824+0000","last_undegraded":"2024-03-20T08:15:24.082824+0000","last_fullsized":"2024-03-20T08:15:24.082824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085437+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085437+0000","last_peered":"2024-03-20T08:15:24.085437+0000","last_clean":"2024-03-20T08:15:24.085437+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085437+0000","last_undegraded":"2024-03-20T08:15:24.085437+0000","last_fullsized":"2024-03-20T08:15:24.085437+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998772+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998772+0000","last_peered":"2024-03-20T08:15:23.998772+0000","last_clean":"2024-03-20T08:15:23.998772+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998772+0000","last_undegraded":"2024-03-20T08:15:23.998772+0000","last_fullsized":"2024-03-20T08:15:23.998772+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998183+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998183+0000","last_peered":"2024-03-20T08:15:23.998183+0000","last_clean":"2024-03-20T08:15:23.998183+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998183+0000","last_undegraded":"2024-03-20T08:15:23.998183+0000","last_fullsized":"2024-03-20T08:15:23.998183+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150274+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150274+0000","last_peered":"2024-03-20T08:15:24.150274+0000","last_clean":"2024-03-20T08:15:24.150274+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150274+0000","last_undegraded":"2024-03-20T08:15:24.150274+0000","last_fullsized":"2024-03-20T08:15:24.150274+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998895+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998895+0000","last_peered":"2024-03-20T08:15:23.998895+0000","last_clean":"2024-03-20T08:15:23.998895+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998895+0000","last_undegraded":"2024-03-20T08:15:23.998895+0000","last_fullsized":"2024-03-20T08:15:23.998895+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001728+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001728+0000","last_peered":"2024-03-20T08:15:24.001728+0000","last_clean":"2024-03-20T08:15:24.001728+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001728+0000","last_undegraded":"2024-03-20T08:15:24.001728+0000","last_fullsized":"2024-03-20T08:15:24.001728+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001977+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.001977+0000","last_peered":"2024-03-20T08:15:24.001977+0000","last_clean":"2024-03-20T08:15:24.001977+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.001977+0000","last_undegraded":"2024-03-20T08:15:24.001977+0000","last_fullsized":"2024-03-20T08:15:24.001977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082803+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082803+0000","last_peered":"2024-03-20T08:15:24.082803+0000","last_clean":"2024-03-20T08:15:24.082803+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082803+0000","last_undegraded":"2024-03-20T08:15:24.082803+0000","last_fullsized":"2024-03-20T08:15:24.082803+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002077+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002077+0000","last_peered":"2024-03-20T08:15:24.002077+0000","last_clean":"2024-03-20T08:15:24.002077+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002077+0000","last_undegraded":"2024-03-20T08:15:24.002077+0000","last_fullsized":"2024-03-20T08:15:24.002077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082497+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082497+0000","last_peered":"2024-03-20T08:15:24.082497+0000","last_clean":"2024-03-20T08:15:24.082497+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082497+0000","last_undegraded":"2024-03-20T08:15:24.082497+0000","last_fullsized":"2024-03-20T08:15:24.082497+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1332,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001769+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001769+0000","last_peered":"2024-03-20T08:15:24.001769+0000","last_clean":"2024-03-20T08:15:24.001769+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001769+0000","last_undegraded":"2024-03-20T08:15:24.001769+0000","last_fullsized":"2024-03-20T08:15:24.001769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990310+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990310+0000","last_peered":"2024-03-20T08:15:23.990310+0000","last_clean":"2024-03-20T08:15:23.990310+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990310+0000","last_undegraded":"2024-03-20T08:15:23.990310+0000","last_fullsized":"2024-03-20T08:15:23.990310+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082450+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082450+0000","last_peered":"2024-03-20T08:15:24.082450+0000","last_clean":"2024-03-20T08:15:24.082450+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082450+0000","last_undegraded":"2024-03-20T08:15:24.082450+0000","last_fullsized":"2024-03-20T08:15:24.082450+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9295,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083467+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083467+0000","last_peered":"2024-03-20T08:15:24.083467+0000","last_clean":"2024-03-20T08:15:24.083467+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083467+0000","last_undegraded":"2024-03-20T08:15:24.083467+0000","last_fullsized":"2024-03-20T08:15:24.083467+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002171+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002171+0000","last_peered":"2024-03-20T08:15:24.002171+0000","last_clean":"2024-03-20T08:15:24.002171+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002171+0000","last_undegraded":"2024-03-20T08:15:24.002171+0000","last_fullsized":"2024-03-20T08:15:24.002171+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002146+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002146+0000","last_peered":"2024-03-20T08:15:24.002146+0000","last_clean":"2024-03-20T08:15:24.002146+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002146+0000","last_undegraded":"2024-03-20T08:15:24.002146+0000","last_fullsized":"2024-03-20T08:15:24.002146+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1249,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149492+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149492+0000","last_peered":"2024-03-20T08:15:24.149492+0000","last_clean":"2024-03-20T08:15:24.149492+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149492+0000","last_undegraded":"2024-03-20T08:15:24.149492+0000","last_fullsized":"2024-03-20T08:15:24.149492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68664,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150417+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150417+0000","last_peered":"2024-03-20T08:15:24.150417+0000","last_clean":"2024-03-20T08:15:24.150417+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150417+0000","last_undegraded":"2024-03-20T08:15:24.150417+0000","last_fullsized":"2024-03-20T08:15:24.150417+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001603+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001603+0000","last_peered":"2024-03-20T08:15:24.001603+0000","last_clean":"2024-03-20T08:15:24.001603+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001603+0000","last_undegraded":"2024-03-20T08:15:24.001603+0000","last_fullsized":"2024-03-20T08:15:24.001603+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001649+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001649+0000","last_peered":"2024-03-20T08:15:24.001649+0000","last_clean":"2024-03-20T08:15:24.001649+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001649+0000","last_undegraded":"2024-03-20T08:15:24.001649+0000","last_fullsized":"2024-03-20T08:15:24.001649+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999147+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999147+0000","last_peered":"2024-03-20T08:15:23.999147+0000","last_clean":"2024-03-20T08:15:23.999147+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999147+0000","last_undegraded":"2024-03-20T08:15:23.999147+0000","last_fullsized":"2024-03-20T08:15:23.999147+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999265+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999265+0000","last_peered":"2024-03-20T08:15:23.999265+0000","last_clean":"2024-03-20T08:15:23.999265+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999265+0000","last_undegraded":"2024-03-20T08:15:23.999265+0000","last_fullsized":"2024-03-20T08:15:23.999265+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990328+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990328+0000","last_peered":"2024-03-20T08:15:23.990328+0000","last_clean":"2024-03-20T08:15:23.990328+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990328+0000","last_undegraded":"2024-03-20T08:15:23.990328+0000","last_fullsized":"2024-03-20T08:15:23.990328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086388+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086388+0000","last_peered":"2024-03-20T08:15:24.086388+0000","last_clean":"2024-03-20T08:15:24.086388+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086388+0000","last_undegraded":"2024-03-20T08:15:24.086388+0000","last_fullsized":"2024-03-20T08:15:24.086388+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002102+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002102+0000","last_peered":"2024-03-20T08:15:24.002102+0000","last_clean":"2024-03-20T08:15:24.002102+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002102+0000","last_undegraded":"2024-03-20T08:15:24.002102+0000","last_fullsized":"2024-03-20T08:15:24.002102+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082363+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082363+0000","last_peered":"2024-03-20T08:15:24.082363+0000","last_clean":"2024-03-20T08:15:24.082363+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082363+0000","last_undegraded":"2024-03-20T08:15:24.082363+0000","last_fullsized":"2024-03-20T08:15:24.082363+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082336+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082336+0000","last_peered":"2024-03-20T08:15:24.082336+0000","last_clean":"2024-03-20T08:15:24.082336+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082336+0000","last_undegraded":"2024-03-20T08:15:24.082336+0000","last_fullsized":"2024-03-20T08:15:24.082336+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980175,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484569,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011580,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188931,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703412,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656877,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446821,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629216,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.209 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.210 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.210 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:21.210 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:16:22.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:22 smithi196 ceph-mon[87080]: from='client.34551 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:22.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:22 smithi196 ceph-mon[87080]: pgmap v1020: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:22.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:16:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:16:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:16:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:22 smithi118 ceph-mon[93121]: from='client.34551 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:22 smithi118 ceph-mon[93121]: pgmap v1020: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:22.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:22 smithi118 ceph-mon[95692]: from='client.34551 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:22.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:22 smithi118 ceph-mon[95692]: pgmap v1020: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:16:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:16:23] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:16:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:24 smithi196 ceph-mon[87080]: pgmap v1021: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:24 smithi118 ceph-mon[93121]: pgmap v1021: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:24.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:24 smithi118 ceph-mon[95692]: pgmap v1021: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:26.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:26 smithi196 ceph-mon[87080]: pgmap v1022: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:26 smithi118 ceph-mon[93121]: pgmap v1022: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:26.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:26 smithi118 ceph-mon[95692]: pgmap v1022: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:28.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:28 smithi196 ceph-mon[87080]: pgmap v1023: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:28 smithi118 ceph-mon[93121]: pgmap v1023: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:28 smithi118 ceph-mon[95692]: pgmap v1023: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:29.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:29 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:29.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:30 smithi196 ceph-mon[87080]: pgmap v1024: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:30 smithi118 ceph-mon[93121]: pgmap v1024: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:30.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:30 smithi118 ceph-mon[95692]: pgmap v1024: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:31.210 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:16:31.429 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:16:32.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:32 smithi196 ceph-mon[87080]: pgmap v1025: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:32.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:16:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:16:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:16:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:32 smithi118 ceph-mon[93121]: pgmap v1025: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:32.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:32 smithi118 ceph-mon[95692]: pgmap v1025: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:33.445 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:16:33.448 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:16:33.743 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:16:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:16:33] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:16:34.048 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1026,"stamp":"2024-03-20T08:16:32.639726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002963"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998738+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998738+0000","last_peered":"2024-03-20T08:15:23.998738+0000","last_clean":"2024-03-20T08:15:23.998738+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998738+0000","last_undegraded":"2024-03-20T08:15:23.998738+0000","last_fullsized":"2024-03-20T08:15:23.998738+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992438+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992438+0000","last_peered":"2024-03-20T08:15:23.992438+0000","last_clean":"2024-03-20T08:15:23.992438+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992438+0000","last_undegraded":"2024-03-20T08:15:23.992438+0000","last_fullsized":"2024-03-20T08:15:23.992438+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082177+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082177+0000","last_peered":"2024-03-20T08:15:24.082177+0000","last_clean":"2024-03-20T08:15:24.082177+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082177+0000","last_undegraded":"2024-03-20T08:15:24.082177+0000","last_fullsized":"2024-03-20T08:15:24.082177+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991223+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991223+0000","last_peered":"2024-03-20T08:15:23.991223+0000","last_clean":"2024-03-20T08:15:23.991223+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991223+0000","last_undegraded":"2024-03-20T08:15:23.991223+0000","last_fullsized":"2024-03-20T08:15:23.991223+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991466+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991466+0000","last_peered":"2024-03-20T08:15:23.991466+0000","last_clean":"2024-03-20T08:15:23.991466+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991466+0000","last_undegraded":"2024-03-20T08:15:23.991466+0000","last_fullsized":"2024-03-20T08:15:23.991466+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998902+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998902+0000","last_peered":"2024-03-20T08:15:23.998902+0000","last_clean":"2024-03-20T08:15:23.998902+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998902+0000","last_undegraded":"2024-03-20T08:15:23.998902+0000","last_fullsized":"2024-03-20T08:15:23.998902+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082824+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082824+0000","last_peered":"2024-03-20T08:15:24.082824+0000","last_clean":"2024-03-20T08:15:24.082824+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082824+0000","last_undegraded":"2024-03-20T08:15:24.082824+0000","last_fullsized":"2024-03-20T08:15:24.082824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085437+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085437+0000","last_peered":"2024-03-20T08:15:24.085437+0000","last_clean":"2024-03-20T08:15:24.085437+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085437+0000","last_undegraded":"2024-03-20T08:15:24.085437+0000","last_fullsized":"2024-03-20T08:15:24.085437+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998772+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998772+0000","last_peered":"2024-03-20T08:15:23.998772+0000","last_clean":"2024-03-20T08:15:23.998772+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998772+0000","last_undegraded":"2024-03-20T08:15:23.998772+0000","last_fullsized":"2024-03-20T08:15:23.998772+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998183+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998183+0000","last_peered":"2024-03-20T08:15:23.998183+0000","last_clean":"2024-03-20T08:15:23.998183+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998183+0000","last_undegraded":"2024-03-20T08:15:23.998183+0000","last_fullsized":"2024-03-20T08:15:23.998183+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150274+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150274+0000","last_peered":"2024-03-20T08:15:24.150274+0000","last_clean":"2024-03-20T08:15:24.150274+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150274+0000","last_undegraded":"2024-03-20T08:15:24.150274+0000","last_fullsized":"2024-03-20T08:15:24.150274+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082803+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082803+0000","last_peered":"2024-03-20T08:15:24.082803+0000","last_clean":"2024-03-20T08:15:24.082803+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082803+0000","last_undegraded":"2024-03-20T08:15:24.082803+0000","last_fullsized":"2024-03-20T08:15:24.082803+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082497+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082497+0000","last_peered":"2024-03-20T08:15:24.082497+0000","last_clean":"2024-03-20T08:15:24.082497+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082497+0000","last_undegraded":"2024-03-20T08:15:24.082497+0000","last_fullsized":"2024-03-20T08:15:24.082497+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1332,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001769+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001769+0000","last_peered":"2024-03-20T08:15:24.001769+0000","last_clean":"2024-03-20T08:15:24.001769+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001769+0000","last_undegraded":"2024-03-20T08:15:24.001769+0000","last_fullsized":"2024-03-20T08:15:24.001769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990310+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990310+0000","last_peered":"2024-03-20T08:15:23.990310+0000","last_clean":"2024-03-20T08:15:23.990310+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990310+0000","last_undegraded":"2024-03-20T08:15:23.990310+0000","last_fullsized":"2024-03-20T08:15:23.990310+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082450+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082450+0000","last_peered":"2024-03-20T08:15:24.082450+0000","last_clean":"2024-03-20T08:15:24.082450+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082450+0000","last_undegraded":"2024-03-20T08:15:24.082450+0000","last_fullsized":"2024-03-20T08:15:24.082450+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9295,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083467+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083467+0000","last_peered":"2024-03-20T08:15:24.083467+0000","last_clean":"2024-03-20T08:15:24.083467+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083467+0000","last_undegraded":"2024-03-20T08:15:24.083467+0000","last_fullsized":"2024-03-20T08:15:24.083467+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002146+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002146+0000","last_peered":"2024-03-20T08:15:24.002146+0000","last_clean":"2024-03-20T08:15:24.002146+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002146+0000","last_undegraded":"2024-03-20T08:15:24.002146+0000","last_fullsized":"2024-03-20T08:15:24.002146+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1249,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149492+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149492+0000","last_peered":"2024-03-20T08:15:24.149492+0000","last_clean":"2024-03-20T08:15:24.149492+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149492+0000","last_undegraded":"2024-03-20T08:15:24.149492+0000","last_fullsized":"2024-03-20T08:15:24.149492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68664,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150417+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150417+0000","last_peered":"2024-03-20T08:15:24.150417+0000","last_clean":"2024-03-20T08:15:24.150417+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150417+0000","last_undegraded":"2024-03-20T08:15:24.150417+0000","last_fullsized":"2024-03-20T08:15:24.150417+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001603+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001603+0000","last_peered":"2024-03-20T08:15:24.001603+0000","last_clean":"2024-03-20T08:15:24.001603+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001603+0000","last_undegraded":"2024-03-20T08:15:24.001603+0000","last_fullsized":"2024-03-20T08:15:24.001603+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001649+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001649+0000","last_peered":"2024-03-20T08:15:24.001649+0000","last_clean":"2024-03-20T08:15:24.001649+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001649+0000","last_undegraded":"2024-03-20T08:15:24.001649+0000","last_fullsized":"2024-03-20T08:15:24.001649+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990328+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990328+0000","last_peered":"2024-03-20T08:15:23.990328+0000","last_clean":"2024-03-20T08:15:23.990328+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990328+0000","last_undegraded":"2024-03-20T08:15:23.990328+0000","last_fullsized":"2024-03-20T08:15:23.990328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086388+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086388+0000","last_peered":"2024-03-20T08:15:24.086388+0000","last_clean":"2024-03-20T08:15:24.086388+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086388+0000","last_undegraded":"2024-03-20T08:15:24.086388+0000","last_fullsized":"2024-03-20T08:15:24.086388+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002102+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002102+0000","last_peered":"2024-03-20T08:15:24.002102+0000","last_clean":"2024-03-20T08:15:24.002102+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002102+0000","last_undegraded":"2024-03-20T08:15:24.002102+0000","last_fullsized":"2024-03-20T08:15:24.002102+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082363+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082363+0000","last_peered":"2024-03-20T08:15:24.082363+0000","last_clean":"2024-03-20T08:15:24.082363+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082363+0000","last_undegraded":"2024-03-20T08:15:24.082363+0000","last_fullsized":"2024-03-20T08:15:24.082363+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082336+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082336+0000","last_peered":"2024-03-20T08:15:24.082336+0000","last_clean":"2024-03-20T08:15:24.082336+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082336+0000","last_undegraded":"2024-03-20T08:15:24.082336+0000","last_fullsized":"2024-03-20T08:15:24.082336+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980178,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484572,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011583,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188934,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703414,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656880,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446824,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629219,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:16:34.054 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:34.055 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:16:34.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:34 smithi196 ceph-mon[87080]: pgmap v1026: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:34.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:34 smithi118 ceph-mon[93121]: pgmap v1026: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:34.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:34 smithi118 ceph-mon[95692]: pgmap v1026: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:35.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:35 smithi196 ceph-mon[87080]: from='client.34557 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:35.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:35 smithi118 ceph-mon[93121]: from='client.34557 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:35.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:35 smithi118 ceph-mon[95692]: from='client.34557 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:36 smithi196 ceph-mon[87080]: pgmap v1027: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:36 smithi118 ceph-mon[93121]: pgmap v1027: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:36.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:36 smithi118 ceph-mon[95692]: pgmap v1027: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:38.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:38 smithi196 ceph-mon[87080]: pgmap v1028: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:38.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:38 smithi118 ceph-mon[93121]: pgmap v1028: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:38.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:38 smithi118 ceph-mon[95692]: pgmap v1028: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:40.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:40 smithi196 ceph-mon[87080]: pgmap v1029: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:40.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:40 smithi118 ceph-mon[93121]: pgmap v1029: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:40.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:40 smithi118 ceph-mon[95692]: pgmap v1029: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:42.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:42 smithi196 ceph-mon[87080]: pgmap v1030: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:42.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:16:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:16:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:16:42.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:42 smithi118 ceph-mon[93121]: pgmap v1030: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:42.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:42 smithi118 ceph-mon[95692]: pgmap v1030: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:16:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:16:43] "GET /metrics HTTP/1.1" 200 36413 "" "Prometheus/2.43.0" 2024-03-20T08:16:44.057 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:16:44.267 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:16:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:44 smithi196 ceph-mon[87080]: pgmap v1031: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:44.389 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:44 smithi118 ceph-mon[93121]: pgmap v1031: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:44.390 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:44.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:44 smithi118 ceph-mon[95692]: pgmap v1031: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:44.390 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:46.266 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:16:46.268 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:16:46.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:46 smithi196 ceph-mon[87080]: pgmap v1032: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:46.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:46 smithi118 ceph-mon[93121]: pgmap v1032: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:46.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:46 smithi118 ceph-mon[95692]: pgmap v1032: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:46.963 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1032,"stamp":"2024-03-20T08:16:44.642789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003063"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998738+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998738+0000","last_peered":"2024-03-20T08:15:23.998738+0000","last_clean":"2024-03-20T08:15:23.998738+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998738+0000","last_undegraded":"2024-03-20T08:15:23.998738+0000","last_fullsized":"2024-03-20T08:15:23.998738+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992438+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992438+0000","last_peered":"2024-03-20T08:15:23.992438+0000","last_clean":"2024-03-20T08:15:23.992438+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992438+0000","last_undegraded":"2024-03-20T08:15:23.992438+0000","last_fullsized":"2024-03-20T08:15:23.992438+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082177+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082177+0000","last_peered":"2024-03-20T08:15:24.082177+0000","last_clean":"2024-03-20T08:15:24.082177+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082177+0000","last_undegraded":"2024-03-20T08:15:24.082177+0000","last_fullsized":"2024-03-20T08:15:24.082177+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991466+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991466+0000","last_peered":"2024-03-20T08:15:23.991466+0000","last_clean":"2024-03-20T08:15:23.991466+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991466+0000","last_undegraded":"2024-03-20T08:15:23.991466+0000","last_fullsized":"2024-03-20T08:15:23.991466+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991143+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991143+0000","last_peered":"2024-03-20T08:15:23.991143+0000","last_clean":"2024-03-20T08:15:23.991143+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991143+0000","last_undegraded":"2024-03-20T08:15:23.991143+0000","last_fullsized":"2024-03-20T08:15:23.991143+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998902+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998902+0000","last_peered":"2024-03-20T08:15:23.998902+0000","last_clean":"2024-03-20T08:15:23.998902+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998902+0000","last_undegraded":"2024-03-20T08:15:23.998902+0000","last_fullsized":"2024-03-20T08:15:23.998902+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082824+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082824+0000","last_peered":"2024-03-20T08:15:24.082824+0000","last_clean":"2024-03-20T08:15:24.082824+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082824+0000","last_undegraded":"2024-03-20T08:15:24.082824+0000","last_fullsized":"2024-03-20T08:15:24.082824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085437+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085437+0000","last_peered":"2024-03-20T08:15:24.085437+0000","last_clean":"2024-03-20T08:15:24.085437+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085437+0000","last_undegraded":"2024-03-20T08:15:24.085437+0000","last_fullsized":"2024-03-20T08:15:24.085437+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998772+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998772+0000","last_peered":"2024-03-20T08:15:23.998772+0000","last_clean":"2024-03-20T08:15:23.998772+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998772+0000","last_undegraded":"2024-03-20T08:15:23.998772+0000","last_fullsized":"2024-03-20T08:15:23.998772+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998183+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998183+0000","last_peered":"2024-03-20T08:15:23.998183+0000","last_clean":"2024-03-20T08:15:23.998183+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998183+0000","last_undegraded":"2024-03-20T08:15:23.998183+0000","last_fullsized":"2024-03-20T08:15:23.998183+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150274+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150274+0000","last_peered":"2024-03-20T08:15:24.150274+0000","last_clean":"2024-03-20T08:15:24.150274+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150274+0000","last_undegraded":"2024-03-20T08:15:24.150274+0000","last_fullsized":"2024-03-20T08:15:24.150274+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1332,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001769+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001769+0000","last_peered":"2024-03-20T08:15:24.001769+0000","last_clean":"2024-03-20T08:15:24.001769+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001769+0000","last_undegraded":"2024-03-20T08:15:24.001769+0000","last_fullsized":"2024-03-20T08:15:24.001769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990310+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990310+0000","last_peered":"2024-03-20T08:15:23.990310+0000","last_clean":"2024-03-20T08:15:23.990310+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990310+0000","last_undegraded":"2024-03-20T08:15:23.990310+0000","last_fullsized":"2024-03-20T08:15:23.990310+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9295,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083467+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083467+0000","last_peered":"2024-03-20T08:15:24.083467+0000","last_clean":"2024-03-20T08:15:24.083467+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083467+0000","last_undegraded":"2024-03-20T08:15:24.083467+0000","last_fullsized":"2024-03-20T08:15:24.083467+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002146+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002146+0000","last_peered":"2024-03-20T08:15:24.002146+0000","last_clean":"2024-03-20T08:15:24.002146+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002146+0000","last_undegraded":"2024-03-20T08:15:24.002146+0000","last_fullsized":"2024-03-20T08:15:24.002146+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68664,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150417+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150417+0000","last_peered":"2024-03-20T08:15:24.150417+0000","last_clean":"2024-03-20T08:15:24.150417+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150417+0000","last_undegraded":"2024-03-20T08:15:24.150417+0000","last_fullsized":"2024-03-20T08:15:24.150417+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001603+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001603+0000","last_peered":"2024-03-20T08:15:24.001603+0000","last_clean":"2024-03-20T08:15:24.001603+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001603+0000","last_undegraded":"2024-03-20T08:15:24.001603+0000","last_fullsized":"2024-03-20T08:15:24.001603+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001649+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001649+0000","last_peered":"2024-03-20T08:15:24.001649+0000","last_clean":"2024-03-20T08:15:24.001649+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001649+0000","last_undegraded":"2024-03-20T08:15:24.001649+0000","last_fullsized":"2024-03-20T08:15:24.001649+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990328+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990328+0000","last_peered":"2024-03-20T08:15:23.990328+0000","last_clean":"2024-03-20T08:15:23.990328+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990328+0000","last_undegraded":"2024-03-20T08:15:23.990328+0000","last_fullsized":"2024-03-20T08:15:23.990328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002102+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002102+0000","last_peered":"2024-03-20T08:15:24.002102+0000","last_clean":"2024-03-20T08:15:24.002102+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002102+0000","last_undegraded":"2024-03-20T08:15:24.002102+0000","last_fullsized":"2024-03-20T08:15:24.002102+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082363+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082363+0000","last_peered":"2024-03-20T08:15:24.082363+0000","last_clean":"2024-03-20T08:15:24.082363+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082363+0000","last_undegraded":"2024-03-20T08:15:24.082363+0000","last_fullsized":"2024-03-20T08:15:24.082363+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980180,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484574,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011585,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188936,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703417,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656882,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446826,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629221,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:16:46.971 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.971 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.971 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.971 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.971 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.971 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:46.972 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:16:47.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:47 smithi196 ceph-mon[87080]: from='client.34563 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:47.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:47 smithi118 ceph-mon[93121]: from='client.34563 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:47.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:47 smithi118 ceph-mon[95692]: from='client.34563 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:16:48.278 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:48 smithi118 ceph-mon[93121]: pgmap v1033: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:48.278 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:48 smithi118 ceph-mon[95692]: pgmap v1033: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:48 smithi196 ceph-mon[87080]: pgmap v1033: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:50.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:50 smithi196 ceph-mon[87080]: pgmap v1034: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:50.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:50 smithi118 ceph-mon[93121]: pgmap v1034: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:50.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:50 smithi118 ceph-mon[95692]: pgmap v1034: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:52.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:52 smithi196 ceph-mon[87080]: pgmap v1035: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:52.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:16:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:16:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:16:52.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:52 smithi118 ceph-mon[93121]: pgmap v1035: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:52.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:52 smithi118 ceph-mon[95692]: pgmap v1035: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:16:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:16:53] "GET /metrics HTTP/1.1" 200 36413 "" "Prometheus/2.43.0" 2024-03-20T08:16:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:54 smithi196 ceph-mon[87080]: pgmap v1036: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:54 smithi118 ceph-mon[93121]: pgmap v1036: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:54.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:54 smithi118 ceph-mon[95692]: pgmap v1036: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:56 smithi196 ceph-mon[87080]: pgmap v1037: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:56 smithi118 ceph-mon[93121]: pgmap v1037: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:56.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:56 smithi118 ceph-mon[95692]: pgmap v1037: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:56.973 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:16:57.187 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:16:58.283 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:58 smithi118 ceph-mon[93121]: pgmap v1038: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:58.284 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:58 smithi118 ceph-mon[95692]: pgmap v1038: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:58.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:58 smithi196 ceph-mon[87080]: pgmap v1038: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:16:59.073 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:16:59.074 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:16:59.322 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:16:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:59.322 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:16:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:16:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:16:59.708 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1039,"stamp":"2024-03-20T08:16:58.646482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003287"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998738+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998738+0000","last_peered":"2024-03-20T08:15:23.998738+0000","last_clean":"2024-03-20T08:15:23.998738+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998738+0000","last_undegraded":"2024-03-20T08:15:23.998738+0000","last_fullsized":"2024-03-20T08:15:23.998738+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082177+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082177+0000","last_peered":"2024-03-20T08:15:24.082177+0000","last_clean":"2024-03-20T08:15:24.082177+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082177+0000","last_undegraded":"2024-03-20T08:15:24.082177+0000","last_fullsized":"2024-03-20T08:15:24.082177+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1138,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998902+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998902+0000","last_peered":"2024-03-20T08:15:23.998902+0000","last_clean":"2024-03-20T08:15:23.998902+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998902+0000","last_undegraded":"2024-03-20T08:15:23.998902+0000","last_fullsized":"2024-03-20T08:15:23.998902+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082824+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082824+0000","last_peered":"2024-03-20T08:15:24.082824+0000","last_clean":"2024-03-20T08:15:24.082824+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082824+0000","last_undegraded":"2024-03-20T08:15:24.082824+0000","last_fullsized":"2024-03-20T08:15:24.082824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085437+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085437+0000","last_peered":"2024-03-20T08:15:24.085437+0000","last_clean":"2024-03-20T08:15:24.085437+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085437+0000","last_undegraded":"2024-03-20T08:15:24.085437+0000","last_fullsized":"2024-03-20T08:15:24.085437+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990310+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990310+0000","last_peered":"2024-03-20T08:15:23.990310+0000","last_clean":"2024-03-20T08:15:23.990310+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990310+0000","last_undegraded":"2024-03-20T08:15:23.990310+0000","last_fullsized":"2024-03-20T08:15:23.990310+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9295,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083467+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083467+0000","last_peered":"2024-03-20T08:15:24.083467+0000","last_clean":"2024-03-20T08:15:24.083467+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083467+0000","last_undegraded":"2024-03-20T08:15:24.083467+0000","last_fullsized":"2024-03-20T08:15:24.083467+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990328+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990328+0000","last_peered":"2024-03-20T08:15:23.990328+0000","last_clean":"2024-03-20T08:15:23.990328+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990328+0000","last_undegraded":"2024-03-20T08:15:23.990328+0000","last_fullsized":"2024-03-20T08:15:23.990328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1232,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082363+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082363+0000","last_peered":"2024-03-20T08:15:24.082363+0000","last_clean":"2024-03-20T08:15:24.082363+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082363+0000","last_undegraded":"2024-03-20T08:15:24.082363+0000","last_fullsized":"2024-03-20T08:15:24.082363+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980183,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484577,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011588,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188939,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703420,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656885,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446829,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629224,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.715 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:16:59.716 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:17:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:00 smithi118 ceph-mon[93121]: pgmap v1039: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:00 smithi118 ceph-mon[93121]: from='client.44489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:00 smithi118 ceph-mon[95692]: pgmap v1039: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:00.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:00 smithi118 ceph-mon[95692]: from='client.44489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:00.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:00 smithi196 ceph-mon[87080]: pgmap v1039: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:00.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:00 smithi196 ceph-mon[87080]: from='client.44489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:02.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:02 smithi118 ceph-mon[93121]: pgmap v1040: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:02.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:02 smithi118 ceph-mon[95692]: pgmap v1040: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:02 smithi196 ceph-mon[87080]: pgmap v1040: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:17:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:17:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:17:03.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:17:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:17:03] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:17:04.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:04 smithi118 ceph-mon[93121]: pgmap v1041: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:04.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:04 smithi118 ceph-mon[95692]: pgmap v1041: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:04 smithi196 ceph-mon[87080]: pgmap v1041: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:06.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:06 smithi118 ceph-mon[93121]: pgmap v1042: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:06.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:06 smithi118 ceph-mon[95692]: pgmap v1042: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:06 smithi196 ceph-mon[87080]: pgmap v1042: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:08.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:08 smithi118 ceph-mon[93121]: pgmap v1043: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:08.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:08 smithi118 ceph-mon[95692]: pgmap v1043: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:08 smithi196 ceph-mon[87080]: pgmap v1043: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:09.717 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:17:09.930 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:17:10.336 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:10 smithi118 ceph-mon[93121]: pgmap v1044: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:10.337 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:10 smithi118 ceph-mon[95692]: pgmap v1044: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:10 smithi196 ceph-mon[87080]: pgmap v1044: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:11.735 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:17:11.737 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:17:12.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:12 smithi118 ceph-mon[93121]: pgmap v1045: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:12.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:12 smithi118 ceph-mon[95692]: pgmap v1045: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:12.451 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1045,"stamp":"2024-03-20T08:17:10.649833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003351"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998590+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998590+0000","last_peered":"2024-03-20T08:15:23.998590+0000","last_clean":"2024-03-20T08:15:23.998590+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998590+0000","last_undegraded":"2024-03-20T08:15:23.998590+0000","last_fullsized":"2024-03-20T08:15:23.998590+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990994+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.990994+0000","last_peered":"2024-03-20T08:15:23.990994+0000","last_clean":"2024-03-20T08:15:23.990994+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.990994+0000","last_undegraded":"2024-03-20T08:15:23.990994+0000","last_fullsized":"2024-03-20T08:15:23.990994+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990869+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990869+0000","last_peered":"2024-03-20T08:15:23.990869+0000","last_clean":"2024-03-20T08:15:23.990869+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990869+0000","last_undegraded":"2024-03-20T08:15:23.990869+0000","last_fullsized":"2024-03-20T08:15:23.990869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992476+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992476+0000","last_peered":"2024-03-20T08:15:23.992476+0000","last_clean":"2024-03-20T08:15:23.992476+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992476+0000","last_undegraded":"2024-03-20T08:15:23.992476+0000","last_fullsized":"2024-03-20T08:15:23.992476+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999353+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999353+0000","last_peered":"2024-03-20T08:15:23.999353+0000","last_clean":"2024-03-20T08:15:23.999353+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999353+0000","last_undegraded":"2024-03-20T08:15:23.999353+0000","last_fullsized":"2024-03-20T08:15:23.999353+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1128,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998736+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.998736+0000","last_peered":"2024-03-20T08:15:23.998736+0000","last_clean":"2024-03-20T08:15:23.998736+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.998736+0000","last_undegraded":"2024-03-20T08:15:23.998736+0000","last_fullsized":"2024-03-20T08:15:23.998736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992736+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992736+0000","last_peered":"2024-03-20T08:15:23.992736+0000","last_clean":"2024-03-20T08:15:23.992736+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992736+0000","last_undegraded":"2024-03-20T08:15:23.992736+0000","last_fullsized":"2024-03-20T08:15:23.992736+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150136+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150136+0000","last_peered":"2024-03-20T08:15:24.150136+0000","last_clean":"2024-03-20T08:15:24.150136+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150136+0000","last_undegraded":"2024-03-20T08:15:24.150136+0000","last_fullsized":"2024-03-20T08:15:24.150136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1189,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991487+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991487+0000","last_peered":"2024-03-20T08:15:23.991487+0000","last_clean":"2024-03-20T08:15:23.991487+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991487+0000","last_undegraded":"2024-03-20T08:15:23.991487+0000","last_fullsized":"2024-03-20T08:15:23.991487+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49868,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.095461+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.095461+0000","last_peered":"2024-03-20T08:15:24.095461+0000","last_clean":"2024-03-20T08:15:24.095461+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.095461+0000","last_undegraded":"2024-03-20T08:15:24.095461+0000","last_fullsized":"2024-03-20T08:15:24.095461+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991563+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991563+0000","last_peered":"2024-03-20T08:15:23.991563+0000","last_clean":"2024-03-20T08:15:23.991563+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991563+0000","last_undegraded":"2024-03-20T08:15:23.991563+0000","last_fullsized":"2024-03-20T08:15:23.991563+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5627,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991583+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991583+0000","last_peered":"2024-03-20T08:15:23.991583+0000","last_clean":"2024-03-20T08:15:23.991583+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991583+0000","last_undegraded":"2024-03-20T08:15:23.991583+0000","last_fullsized":"2024-03-20T08:15:23.991583+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5190,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083530+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083530+0000","last_peered":"2024-03-20T08:15:24.083530+0000","last_clean":"2024-03-20T08:15:24.083530+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083530+0000","last_undegraded":"2024-03-20T08:15:24.083530+0000","last_fullsized":"2024-03-20T08:15:24.083530+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29908,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002125+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002125+0000","last_peered":"2024-03-20T08:15:24.002125+0000","last_clean":"2024-03-20T08:15:24.002125+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002125+0000","last_undegraded":"2024-03-20T08:15:24.002125+0000","last_fullsized":"2024-03-20T08:15:24.002125+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001824+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001824+0000","last_peered":"2024-03-20T08:15:24.001824+0000","last_clean":"2024-03-20T08:15:24.001824+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001824+0000","last_undegraded":"2024-03-20T08:15:24.001824+0000","last_fullsized":"2024-03-20T08:15:24.001824+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082717+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082717+0000","last_peered":"2024-03-20T08:15:24.082717+0000","last_clean":"2024-03-20T08:15:24.082717+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082717+0000","last_undegraded":"2024-03-20T08:15:24.082717+0000","last_fullsized":"2024-03-20T08:15:24.082717+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990815+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990815+0000","last_peered":"2024-03-20T08:15:23.990815+0000","last_clean":"2024-03-20T08:15:23.990815+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990815+0000","last_undegraded":"2024-03-20T08:15:23.990815+0000","last_fullsized":"2024-03-20T08:15:23.990815+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082378+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082378+0000","last_peered":"2024-03-20T08:15:24.082378+0000","last_clean":"2024-03-20T08:15:24.082378+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082378+0000","last_undegraded":"2024-03-20T08:15:24.082378+0000","last_fullsized":"2024-03-20T08:15:24.082378+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980186,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484579,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011590,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188941,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703422,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656888,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446832,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629227,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:17:12.457 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.457 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.457 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.457 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:12.458 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:17:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:12 smithi196 ceph-mon[87080]: pgmap v1045: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:17:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:17:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:17:13.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:13 smithi118 ceph-mon[93121]: from='client.44495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:13.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:13 smithi118 ceph-mon[95692]: from='client.44495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:13.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:13 smithi196 ceph-mon[87080]: from='client.44495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:17:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:17:13] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:17:14.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:14 smithi118 ceph-mon[95692]: pgmap v1046: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:14.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:14 smithi118 ceph-mon[93121]: pgmap v1046: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:14 smithi196 ceph-mon[87080]: pgmap v1046: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:15 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:17:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:15 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:17:15.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:15 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:17:15.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:15 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:17:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:15 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:17:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:15 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:17:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:15 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:17:15.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:15 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:17:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:15 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:17:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:15 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:17:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:15 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:17:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:15 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:17:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:16 smithi118 ceph-mon[93121]: pgmap v1047: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:16.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:16 smithi118 ceph-mon[95692]: pgmap v1047: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:16.540 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:16 smithi196 ceph-mon[87080]: pgmap v1047: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:18.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:18 smithi118 ceph-mon[93121]: pgmap v1048: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:18.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:18 smithi118 ceph-mon[95692]: pgmap v1048: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:18 smithi196 ceph-mon[87080]: pgmap v1048: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:20 smithi118 ceph-mon[93121]: pgmap v1049: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:20 smithi118 ceph-mon[95692]: pgmap v1049: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:20 smithi196 ceph-mon[87080]: pgmap v1049: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:22 smithi118 ceph-mon[93121]: pgmap v1050: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:22.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:22 smithi118 ceph-mon[95692]: pgmap v1050: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:22.460 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:17:22.643 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:22 smithi196 ceph-mon[87080]: pgmap v1050: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:22.643 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:17:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:17:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:17:22.673 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:17:23.830 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:17:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:17:23] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:17:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:24 smithi118 ceph-mon[93121]: pgmap v1051: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:24 smithi118 ceph-mon[95692]: pgmap v1051: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:24.623 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:17:24.626 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:17:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:24 smithi196 ceph-mon[87080]: pgmap v1051: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:25.342 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1051,"stamp":"2024-03-20T08:17:22.653245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003411"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149781+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149781+0000","last_peered":"2024-03-20T08:15:24.149781+0000","last_clean":"2024-03-20T08:15:24.149781+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149781+0000","last_undegraded":"2024-03-20T08:15:24.149781+0000","last_fullsized":"2024-03-20T08:15:24.149781+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083062+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083062+0000","last_peered":"2024-03-20T08:15:24.083062+0000","last_clean":"2024-03-20T08:15:24.083062+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083062+0000","last_undegraded":"2024-03-20T08:15:24.083062+0000","last_fullsized":"2024-03-20T08:15:24.083062+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26670,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992282+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992282+0000","last_peered":"2024-03-20T08:15:23.992282+0000","last_clean":"2024-03-20T08:15:23.992282+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992282+0000","last_undegraded":"2024-03-20T08:15:23.992282+0000","last_fullsized":"2024-03-20T08:15:23.992282+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17599,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992762+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.992762+0000","last_peered":"2024-03-20T08:15:23.992762+0000","last_clean":"2024-03-20T08:15:23.992762+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.992762+0000","last_undegraded":"2024-03-20T08:15:23.992762+0000","last_fullsized":"2024-03-20T08:15:23.992762+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991613+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991613+0000","last_peered":"2024-03-20T08:15:23.991613+0000","last_clean":"2024-03-20T08:15:23.991613+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991613+0000","last_undegraded":"2024-03-20T08:15:23.991613+0000","last_fullsized":"2024-03-20T08:15:23.991613+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5697,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992145+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992145+0000","last_peered":"2024-03-20T08:15:23.992145+0000","last_clean":"2024-03-20T08:15:23.992145+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992145+0000","last_undegraded":"2024-03-20T08:15:23.992145+0000","last_fullsized":"2024-03-20T08:15:23.992145+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980188,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484582,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011593,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188944,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703424,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656890,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446834,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629229,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:17:25.348 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.348 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.348 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.348 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.348 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.348 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:25.349 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:17:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:26 smithi196 ceph-mon[87080]: from='client.34581 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:26 smithi196 ceph-mon[87080]: pgmap v1052: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:26.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:26 smithi118 ceph-mon[95692]: from='client.34581 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:26.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:26 smithi118 ceph-mon[95692]: pgmap v1052: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:26.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:26 smithi118 ceph-mon[93121]: from='client.34581 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:26.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:26 smithi118 ceph-mon[93121]: pgmap v1052: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:28 smithi196 ceph-mon[87080]: pgmap v1053: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:28.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:28 smithi118 ceph-mon[93121]: pgmap v1053: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:28.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:28 smithi118 ceph-mon[95692]: pgmap v1053: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:29.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:29 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:29.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:29.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:30 smithi196 ceph-mon[87080]: pgmap v1054: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:30 smithi118 ceph-mon[93121]: pgmap v1054: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:30.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:30 smithi118 ceph-mon[95692]: pgmap v1054: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:32 smithi196 ceph-mon[87080]: pgmap v1055: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:17:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:17:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:17:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:32 smithi118 ceph-mon[93121]: pgmap v1055: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:32.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:32 smithi118 ceph-mon[95692]: pgmap v1055: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:17:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:17:33] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:17:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:34 smithi196 ceph-mon[87080]: pgmap v1056: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:34 smithi118 ceph-mon[93121]: pgmap v1056: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:34.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:34 smithi118 ceph-mon[95692]: pgmap v1056: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:35.350 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:17:35.564 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:17:36.403 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:36 smithi118 ceph-mon[93121]: pgmap v1057: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:36.404 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:36 smithi118 ceph-mon[95692]: pgmap v1057: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:36 smithi196 ceph-mon[87080]: pgmap v1057: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:37.583 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:17:37.585 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:17:38.221 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:38 smithi118 ceph-mon[93121]: pgmap v1058: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:38.221 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:38 smithi118 ceph-mon[95692]: pgmap v1058: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:38.241 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1058,"stamp":"2024-03-20T08:17:36.656793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002870"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001449+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001449+0000","last_peered":"2024-03-20T08:15:24.001449+0000","last_clean":"2024-03-20T08:15:24.001449+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001449+0000","last_undegraded":"2024-03-20T08:15:24.001449+0000","last_fullsized":"2024-03-20T08:15:24.001449+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082235+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082235+0000","last_peered":"2024-03-20T08:15:24.082235+0000","last_clean":"2024-03-20T08:15:24.082235+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082235+0000","last_undegraded":"2024-03-20T08:15:24.082235+0000","last_fullsized":"2024-03-20T08:15:24.082235+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083538+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083538+0000","last_peered":"2024-03-20T08:15:24.083538+0000","last_clean":"2024-03-20T08:15:24.083538+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083538+0000","last_undegraded":"2024-03-20T08:15:24.083538+0000","last_fullsized":"2024-03-20T08:15:24.083538+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086272+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086272+0000","last_peered":"2024-03-20T08:15:24.086272+0000","last_clean":"2024-03-20T08:15:24.086272+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086272+0000","last_undegraded":"2024-03-20T08:15:24.086272+0000","last_fullsized":"2024-03-20T08:15:24.086272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992268+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:23.992268+0000","last_peered":"2024-03-20T08:15:23.992268+0000","last_clean":"2024-03-20T08:15:23.992268+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:23.992268+0000","last_undegraded":"2024-03-20T08:15:23.992268+0000","last_fullsized":"2024-03-20T08:15:23.992268+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1610,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998164+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:23.998164+0000","last_peered":"2024-03-20T08:15:23.998164+0000","last_clean":"2024-03-20T08:15:23.998164+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:23.998164+0000","last_undegraded":"2024-03-20T08:15:23.998164+0000","last_fullsized":"2024-03-20T08:15:23.998164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998173+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:23.998173+0000","last_peered":"2024-03-20T08:15:23.998173+0000","last_clean":"2024-03-20T08:15:23.998173+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:23.998173+0000","last_undegraded":"2024-03-20T08:15:23.998173+0000","last_fullsized":"2024-03-20T08:15:23.998173+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1172,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991796+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:23.991796+0000","last_peered":"2024-03-20T08:15:23.991796+0000","last_clean":"2024-03-20T08:15:23.991796+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:23.991796+0000","last_undegraded":"2024-03-20T08:15:23.991796+0000","last_fullsized":"2024-03-20T08:15:23.991796+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991229+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:23.991229+0000","last_peered":"2024-03-20T08:15:23.991229+0000","last_clean":"2024-03-20T08:15:23.991229+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:23.991229+0000","last_undegraded":"2024-03-20T08:15:23.991229+0000","last_fullsized":"2024-03-20T08:15:23.991229+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991729+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:23.991729+0000","last_peered":"2024-03-20T08:15:23.991729+0000","last_clean":"2024-03-20T08:15:23.991729+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:23.991729+0000","last_undegraded":"2024-03-20T08:15:23.991729+0000","last_fullsized":"2024-03-20T08:15:23.991729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980191,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484584,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011595,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188947,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656893,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446837,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629232,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:17:38.247 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.247 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.247 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.247 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.247 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.247 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.247 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:38.248 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:17:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:38 smithi196 ceph-mon[87080]: pgmap v1058: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:39.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:39 smithi196 ceph-mon[87080]: from='client.34587 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:39.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:39 smithi118 ceph-mon[93121]: from='client.34587 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:39.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:39 smithi118 ceph-mon[95692]: from='client.34587 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:40 smithi196 ceph-mon[87080]: pgmap v1059: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:40 smithi118 ceph-mon[93121]: pgmap v1059: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:40.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:40 smithi118 ceph-mon[95692]: pgmap v1059: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:42 smithi196 ceph-mon[87080]: pgmap v1060: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:17:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:17:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:17:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:42 smithi118 ceph-mon[93121]: pgmap v1060: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:42.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:42 smithi118 ceph-mon[95692]: pgmap v1060: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:17:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:17:43] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:17:44.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:44 smithi196 ceph-mon[87080]: pgmap v1061: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:44.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:44.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:44 smithi118 ceph-mon[93121]: pgmap v1061: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:44.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:44.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:44 smithi118 ceph-mon[95692]: pgmap v1061: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:44.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:46.475 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:46 smithi118 ceph-mon[93121]: pgmap v1062: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:46.475 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:46 smithi118 ceph-mon[95692]: pgmap v1062: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:46 smithi196 ceph-mon[87080]: pgmap v1062: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:48.249 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:17:48.470 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:17:48.598 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:48 smithi118 ceph-mon[93121]: pgmap v1063: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:48.598 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:48 smithi118 ceph-mon[95692]: pgmap v1063: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:48.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:48 smithi196 ceph-mon[87080]: pgmap v1063: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:50.433 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:17:50.435 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:17:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:50 smithi196 ceph-mon[87080]: pgmap v1064: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:50 smithi118 ceph-mon[93121]: pgmap v1064: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:50.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:50 smithi118 ceph-mon[95692]: pgmap v1064: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:51.080 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1064,"stamp":"2024-03-20T08:17:48.659810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003016"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082752+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:24.082752+0000","last_peered":"2024-03-20T08:15:24.082752+0000","last_clean":"2024-03-20T08:15:24.082752+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:24.082752+0000","last_undegraded":"2024-03-20T08:15:24.082752+0000","last_fullsized":"2024-03-20T08:15:24.082752+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1392,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083330+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:24.083330+0000","last_peered":"2024-03-20T08:15:24.083330+0000","last_clean":"2024-03-20T08:15:24.083330+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:24.083330+0000","last_undegraded":"2024-03-20T08:15:24.083330+0000","last_fullsized":"2024-03-20T08:15:24.083330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082910+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:24.082910+0000","last_peered":"2024-03-20T08:15:24.082910+0000","last_clean":"2024-03-20T08:15:24.082910+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:24.082910+0000","last_undegraded":"2024-03-20T08:15:24.082910+0000","last_fullsized":"2024-03-20T08:15:24.082910+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149670+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.149670+0000","last_peered":"2024-03-20T08:15:24.149670+0000","last_clean":"2024-03-20T08:15:24.149670+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.149670+0000","last_undegraded":"2024-03-20T08:15:24.149670+0000","last_fullsized":"2024-03-20T08:15:24.149670+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1382,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082785+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:24.082785+0000","last_peered":"2024-03-20T08:15:24.082785+0000","last_clean":"2024-03-20T08:15:24.082785+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:24.082785+0000","last_undegraded":"2024-03-20T08:15:24.082785+0000","last_fullsized":"2024-03-20T08:15:24.082785+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083111+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:24.083111+0000","last_peered":"2024-03-20T08:15:24.083111+0000","last_clean":"2024-03-20T08:15:24.083111+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:24.083111+0000","last_undegraded":"2024-03-20T08:15:24.083111+0000","last_fullsized":"2024-03-20T08:15:24.083111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1194,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082932+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:24.082932+0000","last_peered":"2024-03-20T08:15:24.082932+0000","last_clean":"2024-03-20T08:15:24.082932+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:24.082932+0000","last_undegraded":"2024-03-20T08:15:24.082932+0000","last_fullsized":"2024-03-20T08:15:24.082932+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11688,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083144+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:24.083144+0000","last_peered":"2024-03-20T08:15:24.083144+0000","last_clean":"2024-03-20T08:15:24.083144+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:24.083144+0000","last_undegraded":"2024-03-20T08:15:24.083144+0000","last_fullsized":"2024-03-20T08:15:24.083144+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5950,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083340+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:24.083340+0000","last_peered":"2024-03-20T08:15:24.083340+0000","last_clean":"2024-03-20T08:15:24.083340+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:24.083340+0000","last_undegraded":"2024-03-20T08:15:24.083340+0000","last_fullsized":"2024-03-20T08:15:24.083340+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001409+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:15:24.001409+0000","last_peered":"2024-03-20T08:15:24.001409+0000","last_clean":"2024-03-20T08:15:24.001409+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:15:24.001409+0000","last_undegraded":"2024-03-20T08:15:24.001409+0000","last_fullsized":"2024-03-20T08:15:24.001409+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082200+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:24.082200+0000","last_peered":"2024-03-20T08:15:24.082200+0000","last_clean":"2024-03-20T08:15:24.082200+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:24.082200+0000","last_undegraded":"2024-03-20T08:15:24.082200+0000","last_fullsized":"2024-03-20T08:15:24.082200+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082671+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:24.082671+0000","last_peered":"2024-03-20T08:15:24.082671+0000","last_clean":"2024-03-20T08:15:24.082671+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:24.082671+0000","last_undegraded":"2024-03-20T08:15:24.082671+0000","last_fullsized":"2024-03-20T08:15:24.082671+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31731,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992886+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:23.992886+0000","last_peered":"2024-03-20T08:15:23.992886+0000","last_clean":"2024-03-20T08:15:23.992886+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:23.992886+0000","last_undegraded":"2024-03-20T08:15:23.992886+0000","last_fullsized":"2024-03-20T08:15:23.992886+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991201+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:23.991201+0000","last_peered":"2024-03-20T08:15:23.991201+0000","last_clean":"2024-03-20T08:15:23.991201+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:23.991201+0000","last_undegraded":"2024-03-20T08:15:23.991201+0000","last_fullsized":"2024-03-20T08:15:23.991201+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3376,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990949+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:23.990949+0000","last_peered":"2024-03-20T08:15:23.990949+0000","last_clean":"2024-03-20T08:15:23.990949+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:23.990949+0000","last_undegraded":"2024-03-20T08:15:23.990949+0000","last_fullsized":"2024-03-20T08:15:23.990949+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991224+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:23.991224+0000","last_peered":"2024-03-20T08:15:23.991224+0000","last_clean":"2024-03-20T08:15:23.991224+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:23.991224+0000","last_undegraded":"2024-03-20T08:15:23.991224+0000","last_fullsized":"2024-03-20T08:15:23.991224+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1305,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991562+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:23.991562+0000","last_peered":"2024-03-20T08:15:23.991562+0000","last_clean":"2024-03-20T08:15:23.991562+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:23.991562+0000","last_undegraded":"2024-03-20T08:15:23.991562+0000","last_fullsized":"2024-03-20T08:15:23.991562+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082408+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:24.082408+0000","last_peered":"2024-03-20T08:15:24.082408+0000","last_clean":"2024-03-20T08:15:24.082408+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:24.082408+0000","last_undegraded":"2024-03-20T08:15:24.082408+0000","last_fullsized":"2024-03-20T08:15:24.082408+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991197+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:23.991197+0000","last_peered":"2024-03-20T08:15:23.991197+0000","last_clean":"2024-03-20T08:15:23.991197+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:23.991197+0000","last_undegraded":"2024-03-20T08:15:23.991197+0000","last_fullsized":"2024-03-20T08:15:23.991197+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980193,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484587,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011598,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188949,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656895,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446839,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629234,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:17:51.086 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:17:51.087 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:17:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:52 smithi196 ceph-mon[87080]: from='client.34593 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:52 smithi196 ceph-mon[87080]: pgmap v1065: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:52.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:17:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:17:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:17:52.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:52 smithi118 ceph-mon[93121]: from='client.34593 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:52.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:52 smithi118 ceph-mon[93121]: pgmap v1065: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:52.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:52 smithi118 ceph-mon[95692]: from='client.34593 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:17:52.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:52 smithi118 ceph-mon[95692]: pgmap v1065: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:17:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:17:53] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:17:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:54 smithi196 ceph-mon[87080]: pgmap v1066: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:54 smithi118 ceph-mon[93121]: pgmap v1066: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:54.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:54 smithi118 ceph-mon[95692]: pgmap v1066: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:56 smithi196 ceph-mon[87080]: pgmap v1067: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:56.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:56 smithi118 ceph-mon[93121]: pgmap v1067: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:56.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:56 smithi118 ceph-mon[95692]: pgmap v1067: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:58 smithi196 ceph-mon[87080]: pgmap v1068: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:58.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:58 smithi118 ceph-mon[93121]: pgmap v1068: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:58.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:58 smithi118 ceph-mon[95692]: pgmap v1068: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:17:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:17:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:59.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:17:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:17:59.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:17:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:00 smithi196 ceph-mon[87080]: pgmap v1069: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:00.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:00 smithi118 ceph-mon[93121]: pgmap v1069: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:00.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:00 smithi118 ceph-mon[95692]: pgmap v1069: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:01.089 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:18:01.299 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:18:02.509 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:02 smithi118 ceph-mon[93121]: pgmap v1070: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:02.510 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:02 smithi118 ceph-mon[95692]: pgmap v1070: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:02 smithi196 ceph-mon[87080]: pgmap v1070: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:18:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:18:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:18:03.253 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:18:03.255 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:18:03.867 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:18:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:18:03] "GET /metrics HTTP/1.1" 200 36414 "" "Prometheus/2.43.0" 2024-03-20T08:18:03.888 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1071,"stamp":"2024-03-20T08:18:02.663770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003158"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999182+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:23.999182+0000","last_peered":"2024-03-20T08:15:23.999182+0000","last_clean":"2024-03-20T08:15:23.999182+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:23.999182+0000","last_undegraded":"2024-03-20T08:15:23.999182+0000","last_fullsized":"2024-03-20T08:15:23.999182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002389+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:24.002389+0000","last_peered":"2024-03-20T08:15:24.002389+0000","last_clean":"2024-03-20T08:15:24.002389+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:24.002389+0000","last_undegraded":"2024-03-20T08:15:24.002389+0000","last_fullsized":"2024-03-20T08:15:24.002389+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002174+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:24.002174+0000","last_peered":"2024-03-20T08:15:24.002174+0000","last_clean":"2024-03-20T08:15:24.002174+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:24.002174+0000","last_undegraded":"2024-03-20T08:15:24.002174+0000","last_fullsized":"2024-03-20T08:15:24.002174+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3978,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.999759+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:23.999759+0000","last_peered":"2024-03-20T08:15:23.999759+0000","last_clean":"2024-03-20T08:15:23.999759+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:23.999759+0000","last_undegraded":"2024-03-20T08:15:23.999759+0000","last_fullsized":"2024-03-20T08:15:23.999759+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999266+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:23.999266+0000","last_peered":"2024-03-20T08:15:23.999266+0000","last_clean":"2024-03-20T08:15:23.999266+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:23.999266+0000","last_undegraded":"2024-03-20T08:15:23.999266+0000","last_fullsized":"2024-03-20T08:15:23.999266+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992621+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:23.992621+0000","last_peered":"2024-03-20T08:15:23.992621+0000","last_clean":"2024-03-20T08:15:23.992621+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:23.992621+0000","last_undegraded":"2024-03-20T08:15:23.992621+0000","last_fullsized":"2024-03-20T08:15:23.992621+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991700+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:23.991700+0000","last_peered":"2024-03-20T08:15:23.991700+0000","last_clean":"2024-03-20T08:15:23.991700+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:23.991700+0000","last_undegraded":"2024-03-20T08:15:23.991700+0000","last_fullsized":"2024-03-20T08:15:23.991700+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991177+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:23.991177+0000","last_peered":"2024-03-20T08:15:23.991177+0000","last_clean":"2024-03-20T08:15:23.991177+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:23.991177+0000","last_undegraded":"2024-03-20T08:15:23.991177+0000","last_fullsized":"2024-03-20T08:15:23.991177+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1276,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992730+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:23.992730+0000","last_peered":"2024-03-20T08:15:23.992730+0000","last_clean":"2024-03-20T08:15:23.992730+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:23.992730+0000","last_undegraded":"2024-03-20T08:15:23.992730+0000","last_fullsized":"2024-03-20T08:15:23.992730+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980196,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484590,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011601,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188952,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703432,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656898,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446842,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629237,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:18:03.893 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.893 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.893 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.893 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:03.894 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:18:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:04 smithi196 ceph-mon[87080]: pgmap v1071: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:04 smithi196 ceph-mon[87080]: from='client.34599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:04 smithi118 ceph-mon[93121]: pgmap v1071: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:04 smithi118 ceph-mon[93121]: from='client.34599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:04 smithi118 ceph-mon[95692]: pgmap v1071: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:04 smithi118 ceph-mon[95692]: from='client.34599 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:06 smithi196 ceph-mon[87080]: pgmap v1072: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:06 smithi118 ceph-mon[93121]: pgmap v1072: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:06.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:06 smithi118 ceph-mon[95692]: pgmap v1072: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:08 smithi196 ceph-mon[87080]: pgmap v1073: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:08 smithi118 ceph-mon[93121]: pgmap v1073: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:08.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:08 smithi118 ceph-mon[95692]: pgmap v1073: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:10 smithi196 ceph-mon[87080]: pgmap v1074: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:10.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:10 smithi118 ceph-mon[93121]: pgmap v1074: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:10 smithi118 ceph-mon[95692]: pgmap v1074: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:12 smithi196 ceph-mon[87080]: pgmap v1075: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:18:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:18:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:18:12.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:12 smithi118 ceph-mon[93121]: pgmap v1075: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:12.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:12 smithi118 ceph-mon[95692]: pgmap v1075: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:13.896 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:18:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:18:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:18:13] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:18:14.111 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:18:14.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:14 smithi118 ceph-mon[93121]: pgmap v1076: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:14.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:14.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:14 smithi118 ceph-mon[95692]: pgmap v1076: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:14.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:14 smithi196 ceph-mon[87080]: pgmap v1076: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:16.117 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:18:16.119 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:18:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[93121]: pgmap v1077: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:18:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:18:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:18:16.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:18:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[95692]: pgmap v1077: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:18:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:18:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:18:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:18:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:16 smithi196 ceph-mon[87080]: pgmap v1077: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:18:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:18:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:18:16.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:18:16.786 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1077,"stamp":"2024-03-20T08:18:14.667101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003330"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086466+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:24.086466+0000","last_peered":"2024-03-20T08:15:24.086466+0000","last_clean":"2024-03-20T08:15:24.086466+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:24.086466+0000","last_undegraded":"2024-03-20T08:15:24.086466+0000","last_fullsized":"2024-03-20T08:15:24.086466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1300,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991407+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:15:23.991407+0000","last_peered":"2024-03-20T08:15:23.991407+0000","last_clean":"2024-03-20T08:15:23.991407+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:15:23.991407+0000","last_undegraded":"2024-03-20T08:15:23.991407+0000","last_fullsized":"2024-03-20T08:15:23.991407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1226,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082896+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:24.082896+0000","last_peered":"2024-03-20T08:15:24.082896+0000","last_clean":"2024-03-20T08:15:24.082896+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:24.082896+0000","last_undegraded":"2024-03-20T08:15:24.082896+0000","last_fullsized":"2024-03-20T08:15:24.082896+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1220,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991599+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:23.991599+0000","last_peered":"2024-03-20T08:15:23.991599+0000","last_clean":"2024-03-20T08:15:23.991599+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:23.991599+0000","last_undegraded":"2024-03-20T08:15:23.991599+0000","last_fullsized":"2024-03-20T08:15:23.991599+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990963+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:23.990963+0000","last_peered":"2024-03-20T08:15:23.990963+0000","last_clean":"2024-03-20T08:15:23.990963+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:23.990963+0000","last_undegraded":"2024-03-20T08:15:23.990963+0000","last_fullsized":"2024-03-20T08:15:23.990963+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990923+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:23.990923+0000","last_peered":"2024-03-20T08:15:23.990923+0000","last_clean":"2024-03-20T08:15:23.990923+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:23.990923+0000","last_undegraded":"2024-03-20T08:15:23.990923+0000","last_fullsized":"2024-03-20T08:15:23.990923+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1316,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990700+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:15:23.990700+0000","last_peered":"2024-03-20T08:15:23.990700+0000","last_clean":"2024-03-20T08:15:23.990700+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:15:23.990700+0000","last_undegraded":"2024-03-20T08:15:23.990700+0000","last_fullsized":"2024-03-20T08:15:23.990700+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149839+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:15:24.149839+0000","last_peered":"2024-03-20T08:15:24.149839+0000","last_clean":"2024-03-20T08:15:24.149839+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:15:24.149839+0000","last_undegraded":"2024-03-20T08:15:24.149839+0000","last_fullsized":"2024-03-20T08:15:24.149839+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083168+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:15:24.083168+0000","last_peered":"2024-03-20T08:15:24.083168+0000","last_clean":"2024-03-20T08:15:24.083168+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:15:24.083168+0000","last_undegraded":"2024-03-20T08:15:24.083168+0000","last_fullsized":"2024-03-20T08:15:24.083168+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4604,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150485+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:15:24.150485+0000","last_peered":"2024-03-20T08:15:24.150485+0000","last_clean":"2024-03-20T08:15:24.150485+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:15:24.150485+0000","last_undegraded":"2024-03-20T08:15:24.150485+0000","last_fullsized":"2024-03-20T08:15:24.150485+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1215,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1114,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998376+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:15:23.998376+0000","last_peered":"2024-03-20T08:15:23.998376+0000","last_clean":"2024-03-20T08:15:23.998376+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:15:23.998376+0000","last_undegraded":"2024-03-20T08:15:23.998376+0000","last_fullsized":"2024-03-20T08:15:23.998376+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4957,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.005221+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:15:24.005221+0000","last_peered":"2024-03-20T08:15:24.005221+0000","last_clean":"2024-03-20T08:15:24.005221+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:15:24.005221+0000","last_undegraded":"2024-03-20T08:15:24.005221+0000","last_fullsized":"2024-03-20T08:15:24.005221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980198,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484592,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011603,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188954,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703435,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656900,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446844,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629239,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:18:16.793 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.794 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:16.795 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:18:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:17 smithi196 ceph-mon[87080]: from='client.44525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:17 smithi118 ceph-mon[93121]: from='client.44525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:17.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:17 smithi118 ceph-mon[95692]: from='client.44525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:18 smithi196 ceph-mon[87080]: pgmap v1078: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:18.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:18 smithi118 ceph-mon[93121]: pgmap v1078: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:18.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:18 smithi118 ceph-mon[95692]: pgmap v1078: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:20 smithi118 ceph-mon[93121]: pgmap v1079: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:20 smithi118 ceph-mon[95692]: pgmap v1079: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:20 smithi196 ceph-mon[87080]: pgmap v1079: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:22 smithi196 ceph-mon[87080]: pgmap v1080: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:22.759 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:18:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:18:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:18:22.759 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:22 smithi118 ceph-mon[93121]: pgmap v1080: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:22.759 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:22 smithi118 ceph-mon[95692]: pgmap v1080: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:18:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:18:23] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:18:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:24 smithi196 ceph-mon[87080]: pgmap v1081: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:24 smithi118 ceph-mon[93121]: pgmap v1081: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:24.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:24 smithi118 ceph-mon[95692]: pgmap v1081: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:26 smithi196 ceph-mon[87080]: pgmap v1082: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:26.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:26 smithi118 ceph-mon[93121]: pgmap v1082: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:26.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:26 smithi118 ceph-mon[95692]: pgmap v1082: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:26.795 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:18:27.055 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:18:28.264 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:28 smithi118 ceph-mon[93121]: pgmap v1083: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:28.264 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:28 smithi118 ceph-mon[95692]: pgmap v1083: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:28 smithi196 ceph-mon[87080]: pgmap v1083: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:29.032 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:18:29.034 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:18:29.413 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:29.413 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:29.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:29 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:29.734 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1084,"stamp":"2024-03-20T08:18:28.670798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003263"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1340,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002189+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:15:24.002189+0000","last_peered":"2024-03-20T08:15:24.002189+0000","last_clean":"2024-03-20T08:15:24.002189+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:15:24.002189+0000","last_undegraded":"2024-03-20T08:15:24.002189+0000","last_fullsized":"2024-03-20T08:15:24.002189+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991459+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:15:23.991459+0000","last_peered":"2024-03-20T08:15:23.991459+0000","last_clean":"2024-03-20T08:15:23.991459+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:15:23.991459+0000","last_undegraded":"2024-03-20T08:15:23.991459+0000","last_fullsized":"2024-03-20T08:15:23.991459+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1184,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991267+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:15:23.991267+0000","last_peered":"2024-03-20T08:15:23.991267+0000","last_clean":"2024-03-20T08:15:23.991267+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:15:23.991267+0000","last_undegraded":"2024-03-20T08:15:23.991267+0000","last_fullsized":"2024-03-20T08:15:23.991267+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991807+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:15:23.991807+0000","last_peered":"2024-03-20T08:15:23.991807+0000","last_clean":"2024-03-20T08:15:23.991807+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:15:23.991807+0000","last_undegraded":"2024-03-20T08:15:23.991807+0000","last_fullsized":"2024-03-20T08:15:23.991807+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001776+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:15:24.001776+0000","last_peered":"2024-03-20T08:15:24.001776+0000","last_clean":"2024-03-20T08:15:24.001776+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:15:24.001776+0000","last_undegraded":"2024-03-20T08:15:24.001776+0000","last_fullsized":"2024-03-20T08:15:24.001776+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002024+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:15:24.002024+0000","last_peered":"2024-03-20T08:15:24.002024+0000","last_clean":"2024-03-20T08:15:24.002024+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:15:24.002024+0000","last_undegraded":"2024-03-20T08:15:24.002024+0000","last_fullsized":"2024-03-20T08:15:24.002024+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002124+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:15:24.002124+0000","last_peered":"2024-03-20T08:15:24.002124+0000","last_clean":"2024-03-20T08:15:24.002124+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:15:24.002124+0000","last_undegraded":"2024-03-20T08:15:24.002124+0000","last_fullsized":"2024-03-20T08:15:24.002124+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002210+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:15:24.002210+0000","last_peered":"2024-03-20T08:15:24.002210+0000","last_clean":"2024-03-20T08:15:24.002210+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:15:24.002210+0000","last_undegraded":"2024-03-20T08:15:24.002210+0000","last_fullsized":"2024-03-20T08:15:24.002210+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1283,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992386+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:15:23.992386+0000","last_peered":"2024-03-20T08:15:23.992386+0000","last_clean":"2024-03-20T08:15:23.992386+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:15:23.992386+0000","last_undegraded":"2024-03-20T08:15:23.992386+0000","last_fullsized":"2024-03-20T08:15:23.992386+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991923+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:15:23.991923+0000","last_peered":"2024-03-20T08:15:23.991923+0000","last_clean":"2024-03-20T08:15:23.991923+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:15:23.991923+0000","last_undegraded":"2024-03-20T08:15:23.991923+0000","last_fullsized":"2024-03-20T08:15:23.991923+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980201,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484595,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011606,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188957,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703438,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656903,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446847,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629242,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:18:29.740 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.740 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.740 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.740 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.740 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.740 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:29.741 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:18:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:30 smithi118 ceph-mon[93121]: pgmap v1084: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:30 smithi118 ceph-mon[93121]: from='client.44531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:30 smithi118 ceph-mon[95692]: pgmap v1084: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:30 smithi118 ceph-mon[95692]: from='client.44531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:30 smithi196 ceph-mon[87080]: pgmap v1084: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:30 smithi196 ceph-mon[87080]: from='client.44531 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:32 smithi196 ceph-mon[87080]: pgmap v1085: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:18:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:18:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:18:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:32 smithi118 ceph-mon[93121]: pgmap v1085: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:32 smithi118 ceph-mon[95692]: pgmap v1085: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:18:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:18:33] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:18:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:34 smithi196 ceph-mon[87080]: pgmap v1086: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:34 smithi118 ceph-mon[93121]: pgmap v1086: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:34.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:34 smithi118 ceph-mon[95692]: pgmap v1086: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:36 smithi196 ceph-mon[87080]: pgmap v1087: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:36 smithi118 ceph-mon[93121]: pgmap v1087: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:36 smithi118 ceph-mon[95692]: pgmap v1087: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:38 smithi196 ceph-mon[87080]: pgmap v1088: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:38 smithi118 ceph-mon[93121]: pgmap v1088: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:38.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:38 smithi118 ceph-mon[95692]: pgmap v1088: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:39.742 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:18:39.953 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:18:40.392 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:40 smithi118 ceph-mon[93121]: pgmap v1089: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:40.392 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:40 smithi118 ceph-mon[95692]: pgmap v1089: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:40 smithi196 ceph-mon[87080]: pgmap v1089: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:41.946 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:18:41.951 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:18:42.579 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:42 smithi118 ceph-mon[93121]: pgmap v1090: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:42.579 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:42 smithi118 ceph-mon[95692]: pgmap v1090: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:42.599 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1090,"stamp":"2024-03-20T08:18:40.674294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003495"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999422+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:15:23.999422+0000","last_peered":"2024-03-20T08:15:23.999422+0000","last_clean":"2024-03-20T08:15:23.999422+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:15:23.999422+0000","last_undegraded":"2024-03-20T08:15:23.999422+0000","last_fullsized":"2024-03-20T08:15:23.999422+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082922+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:15:24.082922+0000","last_peered":"2024-03-20T08:15:24.082922+0000","last_clean":"2024-03-20T08:15:24.082922+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:15:24.082922+0000","last_undegraded":"2024-03-20T08:15:24.082922+0000","last_fullsized":"2024-03-20T08:15:24.082922+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082545+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:15:24.082545+0000","last_peered":"2024-03-20T08:15:24.082545+0000","last_clean":"2024-03-20T08:15:24.082545+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:15:24.082545+0000","last_undegraded":"2024-03-20T08:15:24.082545+0000","last_fullsized":"2024-03-20T08:15:24.082545+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082521+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:15:24.082521+0000","last_peered":"2024-03-20T08:15:24.082521+0000","last_clean":"2024-03-20T08:15:24.082521+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:15:24.082521+0000","last_undegraded":"2024-03-20T08:15:24.082521+0000","last_fullsized":"2024-03-20T08:15:24.082521+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1250,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149545+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:15:24.149545+0000","last_peered":"2024-03-20T08:15:24.149545+0000","last_clean":"2024-03-20T08:15:24.149545+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:15:24.149545+0000","last_undegraded":"2024-03-20T08:15:24.149545+0000","last_fullsized":"2024-03-20T08:15:24.149545+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999319+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:15:23.999319+0000","last_peered":"2024-03-20T08:15:23.999319+0000","last_clean":"2024-03-20T08:15:23.999319+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:15:23.999319+0000","last_undegraded":"2024-03-20T08:15:23.999319+0000","last_fullsized":"2024-03-20T08:15:23.999319+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999325+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:15:23.999325+0000","last_peered":"2024-03-20T08:15:23.999325+0000","last_clean":"2024-03-20T08:15:23.999325+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:15:23.999325+0000","last_undegraded":"2024-03-20T08:15:23.999325+0000","last_fullsized":"2024-03-20T08:15:23.999325+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086438+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:15:24.086438+0000","last_peered":"2024-03-20T08:15:24.086438+0000","last_clean":"2024-03-20T08:15:24.086438+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:15:24.086438+0000","last_undegraded":"2024-03-20T08:15:24.086438+0000","last_fullsized":"2024-03-20T08:15:24.086438+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980204,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484597,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188959,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703440,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656906,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446850,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629245,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:18:42.664 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:42.665 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:18:42.666 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:42 smithi196 ceph-mon[87080]: pgmap v1090: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:42.666 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:18:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:18:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:18:43.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:43 smithi118 ceph-mon[93121]: from='client.44537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:43.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:43 smithi118 ceph-mon[95692]: from='client.44537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:43.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:43 smithi196 ceph-mon[87080]: from='client.44537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:18:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:18:43] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:18:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:44 smithi196 ceph-mon[87080]: pgmap v1091: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:44.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:44 smithi118 ceph-mon[95692]: pgmap v1091: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:44.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:44.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:44 smithi118 ceph-mon[93121]: pgmap v1091: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:44.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:46 smithi196 ceph-mon[87080]: pgmap v1092: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:46 smithi118 ceph-mon[93121]: pgmap v1092: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:46 smithi118 ceph-mon[95692]: pgmap v1092: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:48.641 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:48 smithi196 ceph-mon[87080]: pgmap v1093: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:48 smithi118 ceph-mon[93121]: pgmap v1093: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:48.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:48 smithi118 ceph-mon[95692]: pgmap v1093: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:50 smithi196 ceph-mon[87080]: pgmap v1094: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:50 smithi118 ceph-mon[93121]: pgmap v1094: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:50.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:50 smithi118 ceph-mon[95692]: pgmap v1094: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:52.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:52 smithi196 ceph-mon[87080]: pgmap v1095: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:52.642 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:18:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:18:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:18:52.668 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:18:52.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:52 smithi118 ceph-mon[93121]: pgmap v1095: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:52.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:52 smithi118 ceph-mon[95692]: pgmap v1095: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:52.888 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:18:53.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:18:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:18:53] "GET /metrics HTTP/1.1" 200 36412 "" "Prometheus/2.43.0" 2024-03-20T08:18:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:54 smithi196 ceph-mon[87080]: pgmap v1096: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:54 smithi118 ceph-mon[93121]: pgmap v1096: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:54.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:54 smithi118 ceph-mon[95692]: pgmap v1096: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:55.265 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:18:55.268 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:18:55.916 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1097,"stamp":"2024-03-20T08:18:54.678059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003241"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991469+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:15:23.991469+0000","last_peered":"2024-03-20T08:15:23.991469+0000","last_clean":"2024-03-20T08:15:23.991469+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:15:23.991469+0000","last_undegraded":"2024-03-20T08:15:23.991469+0000","last_fullsized":"2024-03-20T08:15:23.991469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998829+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:15:23.998829+0000","last_peered":"2024-03-20T08:15:23.998829+0000","last_clean":"2024-03-20T08:15:23.998829+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:15:23.998829+0000","last_undegraded":"2024-03-20T08:15:23.998829+0000","last_fullsized":"2024-03-20T08:15:23.998829+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998242+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:15:23.998242+0000","last_peered":"2024-03-20T08:15:23.998242+0000","last_clean":"2024-03-20T08:15:23.998242+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:15:23.998242+0000","last_undegraded":"2024-03-20T08:15:23.998242+0000","last_fullsized":"2024-03-20T08:15:23.998242+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1333,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001811+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:15:24.001811+0000","last_peered":"2024-03-20T08:15:24.001811+0000","last_clean":"2024-03-20T08:15:24.001811+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:15:24.001811+0000","last_undegraded":"2024-03-20T08:15:24.001811+0000","last_fullsized":"2024-03-20T08:15:24.001811+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002195+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:15:24.002195+0000","last_peered":"2024-03-20T08:15:24.002195+0000","last_clean":"2024-03-20T08:15:24.002195+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:15:24.002195+0000","last_undegraded":"2024-03-20T08:15:24.002195+0000","last_fullsized":"2024-03-20T08:15:24.002195+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001646+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:15:24.001646+0000","last_peered":"2024-03-20T08:15:24.001646+0000","last_clean":"2024-03-20T08:15:24.001646+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:15:24.001646+0000","last_undegraded":"2024-03-20T08:15:24.001646+0000","last_fullsized":"2024-03-20T08:15:24.001646+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001699+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:15:24.001699+0000","last_peered":"2024-03-20T08:15:24.001699+0000","last_clean":"2024-03-20T08:15:24.001699+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:15:24.001699+0000","last_undegraded":"2024-03-20T08:15:24.001699+0000","last_fullsized":"2024-03-20T08:15:24.001699+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.002145+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:15:24.002145+0000","last_peered":"2024-03-20T08:15:24.002145+0000","last_clean":"2024-03-20T08:15:24.002145+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:15:24.002145+0000","last_undegraded":"2024-03-20T08:15:24.002145+0000","last_fullsized":"2024-03-20T08:15:24.002145+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082382+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:15:24.082382+0000","last_peered":"2024-03-20T08:15:24.082382+0000","last_clean":"2024-03-20T08:15:24.082382+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:15:24.082382+0000","last_undegraded":"2024-03-20T08:15:24.082382+0000","last_fullsized":"2024-03-20T08:15:24.082382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980206,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484600,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011611,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188962,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703443,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656908,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446852,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629247,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:18:55.927 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.927 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:18:55.929 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:18:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:56 smithi196 ceph-mon[87080]: pgmap v1097: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:56 smithi196 ceph-mon[87080]: from='client.34623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:56.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:56 smithi118 ceph-mon[93121]: pgmap v1097: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:56.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:56 smithi118 ceph-mon[93121]: from='client.34623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:56.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:56 smithi118 ceph-mon[95692]: pgmap v1097: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:56.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:56 smithi118 ceph-mon[95692]: from='client.34623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:18:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:58 smithi196 ceph-mon[87080]: pgmap v1098: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:18:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:58.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:58 smithi118 ceph-mon[93121]: pgmap v1098: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:58.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:18:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:18:58.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:58 smithi118 ceph-mon[95692]: pgmap v1098: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:18:58.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:18:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:00 smithi196 ceph-mon[87080]: pgmap v1099: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:00.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:00 smithi118 ceph-mon[93121]: pgmap v1099: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:00.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:00 smithi118 ceph-mon[95692]: pgmap v1099: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:02 smithi196 ceph-mon[87080]: pgmap v1100: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:19:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:19:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:19:02.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:02 smithi118 ceph-mon[93121]: pgmap v1100: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:02.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:02 smithi118 ceph-mon[95692]: pgmap v1100: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:03.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:19:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:19:03] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:19:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:04 smithi196 ceph-mon[87080]: pgmap v1101: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:04 smithi118 ceph-mon[93121]: pgmap v1101: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:04 smithi118 ceph-mon[95692]: pgmap v1101: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:05.930 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:19:06.142 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:19:06.574 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:06 smithi118 ceph-mon[93121]: pgmap v1102: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:06.575 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:06 smithi118 ceph-mon[95692]: pgmap v1102: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:06 smithi196 ceph-mon[87080]: pgmap v1102: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:08.098 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:19:08.101 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:19:08.411 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:08 smithi118 ceph-mon[95692]: pgmap v1103: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:08.412 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:08 smithi118 ceph-mon[93121]: pgmap v1103: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:08.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:08 smithi196 ceph-mon[87080]: pgmap v1103: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:08.783 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1103,"stamp":"2024-03-20T08:19:06.680796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002737"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992480+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:15:23.992480+0000","last_peered":"2024-03-20T08:15:23.992480+0000","last_clean":"2024-03-20T08:15:23.992480+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:15:23.992480+0000","last_undegraded":"2024-03-20T08:15:23.992480+0000","last_fullsized":"2024-03-20T08:15:23.992480+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990926+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:15:23.990926+0000","last_peered":"2024-03-20T08:15:23.990926+0000","last_clean":"2024-03-20T08:15:23.990926+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:15:23.990926+0000","last_undegraded":"2024-03-20T08:15:23.990926+0000","last_fullsized":"2024-03-20T08:15:23.990926+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992521+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:15:23.992521+0000","last_peered":"2024-03-20T08:15:23.992521+0000","last_clean":"2024-03-20T08:15:23.992521+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:15:23.992521+0000","last_undegraded":"2024-03-20T08:15:23.992521+0000","last_fullsized":"2024-03-20T08:15:23.992521+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991265+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:15:23.991265+0000","last_peered":"2024-03-20T08:15:23.991265+0000","last_clean":"2024-03-20T08:15:23.991265+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:15:23.991265+0000","last_undegraded":"2024-03-20T08:15:23.991265+0000","last_fullsized":"2024-03-20T08:15:23.991265+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991510+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:15:23.991510+0000","last_peered":"2024-03-20T08:15:23.991510+0000","last_clean":"2024-03-20T08:15:23.991510+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:15:23.991510+0000","last_undegraded":"2024-03-20T08:15:23.991510+0000","last_fullsized":"2024-03-20T08:15:23.991510+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991186+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:15:23.991186+0000","last_peered":"2024-03-20T08:15:23.991186+0000","last_clean":"2024-03-20T08:15:23.991186+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:15:23.991186+0000","last_undegraded":"2024-03-20T08:15:23.991186+0000","last_fullsized":"2024-03-20T08:15:23.991186+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150333+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:15:24.150333+0000","last_peered":"2024-03-20T08:15:24.150333+0000","last_clean":"2024-03-20T08:15:24.150333+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:15:24.150333+0000","last_undegraded":"2024-03-20T08:15:24.150333+0000","last_fullsized":"2024-03-20T08:15:24.150333+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990367+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:15:23.990367+0000","last_peered":"2024-03-20T08:15:23.990367+0000","last_clean":"2024-03-20T08:15:23.990367+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:15:23.990367+0000","last_undegraded":"2024-03-20T08:15:23.990367+0000","last_fullsized":"2024-03-20T08:15:23.990367+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9296,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083720+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:15:24.083720+0000","last_peered":"2024-03-20T08:15:24.083720+0000","last_clean":"2024-03-20T08:15:24.083720+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:15:24.083720+0000","last_undegraded":"2024-03-20T08:15:24.083720+0000","last_fullsized":"2024-03-20T08:15:24.083720+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991604+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:15:23.991604+0000","last_peered":"2024-03-20T08:15:23.991604+0000","last_clean":"2024-03-20T08:15:23.991604+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:15:23.991604+0000","last_undegraded":"2024-03-20T08:15:23.991604+0000","last_fullsized":"2024-03-20T08:15:23.991604+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5628,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.991990+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:15:23.991990+0000","last_peered":"2024-03-20T08:15:23.991990+0000","last_clean":"2024-03-20T08:15:23.991990+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:15:23.991990+0000","last_undegraded":"2024-03-20T08:15:23.991990+0000","last_fullsized":"2024-03-20T08:15:23.991990+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68665,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.150592+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:15:24.150592+0000","last_peered":"2024-03-20T08:15:24.150592+0000","last_clean":"2024-03-20T08:15:24.150592+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:15:24.150592+0000","last_undegraded":"2024-03-20T08:15:24.150592+0000","last_fullsized":"2024-03-20T08:15:24.150592+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990374+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:15:23.990374+0000","last_peered":"2024-03-20T08:15:23.990374+0000","last_clean":"2024-03-20T08:15:23.990374+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:15:23.990374+0000","last_undegraded":"2024-03-20T08:15:23.990374+0000","last_fullsized":"2024-03-20T08:15:23.990374+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1233,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082410+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:15:24.082410+0000","last_peered":"2024-03-20T08:15:24.082410+0000","last_clean":"2024-03-20T08:15:24.082410+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:15:24.082410+0000","last_undegraded":"2024-03-20T08:15:24.082410+0000","last_fullsized":"2024-03-20T08:15:24.082410+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980209,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484602,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011613,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188965,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703445,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656911,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446855,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629250,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:19:08.789 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.790 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:08.791 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:19:09.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:09 smithi196 ceph-mon[87080]: from='client.44549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:09.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:09 smithi118 ceph-mon[93121]: from='client.44549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:09.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:09 smithi118 ceph-mon[95692]: from='client.44549 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:10 smithi196 ceph-mon[87080]: pgmap v1104: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:10.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:10 smithi118 ceph-mon[95692]: pgmap v1104: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:10.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:10 smithi118 ceph-mon[93121]: pgmap v1104: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:12 smithi196 ceph-mon[87080]: pgmap v1105: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:12.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:19:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:19:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:19:12.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:12 smithi118 ceph-mon[93121]: pgmap v1105: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:12.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:12 smithi118 ceph-mon[95692]: pgmap v1105: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:13.586 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:13.586 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:13.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:19:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:19:13] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:19:14.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:14 smithi196 ceph-mon[87080]: pgmap v1106: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:14 smithi118 ceph-mon[93121]: pgmap v1106: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:14.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:14 smithi118 ceph-mon[95692]: pgmap v1106: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:16 smithi196 ceph-mon[87080]: pgmap v1107: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:19:16.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:16 smithi118 ceph-mon[93121]: pgmap v1107: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:16.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:19:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:16 smithi118 ceph-mon[95692]: pgmap v1107: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:19:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:17 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:19:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:17 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:19:17.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:17 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:19:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:17 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:19:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:17 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:19:17.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:17 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:19:17.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:17 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:19:17.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:17 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:19:17.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:17 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:19:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:18 smithi196 ceph-mon[87080]: pgmap v1108: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:18.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:18 smithi118 ceph-mon[93121]: pgmap v1108: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:18.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:18 smithi118 ceph-mon[95692]: pgmap v1108: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:18.792 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:19:19.001 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:19:20.487 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:20 smithi118 ceph-mon[93121]: pgmap v1109: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:20.487 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:20 smithi118 ceph-mon[95692]: pgmap v1109: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:20.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:20 smithi196 ceph-mon[87080]: pgmap v1109: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:20.922 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:19:20.924 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:19:21.732 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1110,"stamp":"2024-03-20T08:19:20.684237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002871"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998808+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:15:23.998808+0000","last_peered":"2024-03-20T08:15:23.998808+0000","last_clean":"2024-03-20T08:15:23.998808+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:15:23.998808+0000","last_undegraded":"2024-03-20T08:15:23.998808+0000","last_fullsized":"2024-03-20T08:15:23.998808+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082234+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:15:24.082234+0000","last_peered":"2024-03-20T08:15:24.082234+0000","last_clean":"2024-03-20T08:15:24.082234+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:15:24.082234+0000","last_undegraded":"2024-03-20T08:15:24.082234+0000","last_fullsized":"2024-03-20T08:15:24.082234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1139,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998968+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:15:23.998968+0000","last_peered":"2024-03-20T08:15:23.998968+0000","last_clean":"2024-03-20T08:15:23.998968+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:15:23.998968+0000","last_undegraded":"2024-03-20T08:15:23.998968+0000","last_fullsized":"2024-03-20T08:15:23.998968+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082868+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:15:24.082868+0000","last_peered":"2024-03-20T08:15:24.082868+0000","last_clean":"2024-03-20T08:15:24.082868+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:15:24.082868+0000","last_undegraded":"2024-03-20T08:15:24.082868+0000","last_fullsized":"2024-03-20T08:15:24.082868+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.085500+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:15:24.085500+0000","last_peered":"2024-03-20T08:15:24.085500+0000","last_clean":"2024-03-20T08:15:24.085500+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:15:24.085500+0000","last_undegraded":"2024-03-20T08:15:24.085500+0000","last_fullsized":"2024-03-20T08:15:24.085500+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1190,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991537+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:15:23.991537+0000","last_peered":"2024-03-20T08:15:23.991537+0000","last_clean":"2024-03-20T08:15:23.991537+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:15:23.991537+0000","last_undegraded":"2024-03-20T08:15:23.991537+0000","last_fullsized":"2024-03-20T08:15:23.991537+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49869,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.099298+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:15:24.099298+0000","last_peered":"2024-03-20T08:15:24.099298+0000","last_clean":"2024-03-20T08:15:24.099298+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:15:24.099298+0000","last_undegraded":"2024-03-20T08:15:24.099298+0000","last_fullsized":"2024-03-20T08:15:24.099298+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29909,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.002686+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:15:24.002686+0000","last_peered":"2024-03-20T08:15:24.002686+0000","last_clean":"2024-03-20T08:15:24.002686+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:15:24.002686+0000","last_undegraded":"2024-03-20T08:15:24.002686+0000","last_fullsized":"2024-03-20T08:15:24.002686+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001898+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:15:24.001898+0000","last_peered":"2024-03-20T08:15:24.001898+0000","last_clean":"2024-03-20T08:15:24.001898+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:15:24.001898+0000","last_undegraded":"2024-03-20T08:15:24.001898+0000","last_fullsized":"2024-03-20T08:15:24.001898+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.990867+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:15:23.990867+0000","last_peered":"2024-03-20T08:15:23.990867+0000","last_clean":"2024-03-20T08:15:23.990867+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:15:23.990867+0000","last_undegraded":"2024-03-20T08:15:23.990867+0000","last_fullsized":"2024-03-20T08:15:23.990867+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980212,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484605,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011616,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188967,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656914,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629253,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:19:21.738 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:21.739 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:19:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:22 smithi196 ceph-mon[87080]: pgmap v1110: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:22 smithi196 ceph-mon[87080]: from='client.34635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:19:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:19:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:19:22.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:22 smithi118 ceph-mon[93121]: pgmap v1110: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:22.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:22 smithi118 ceph-mon[93121]: from='client.34635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:22.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:22 smithi118 ceph-mon[95692]: pgmap v1110: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:22.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:22 smithi118 ceph-mon[95692]: from='client.34635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:19:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:19:23] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:19:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:24 smithi196 ceph-mon[87080]: pgmap v1111: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:24 smithi118 ceph-mon[93121]: pgmap v1111: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:24.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:24 smithi118 ceph-mon[95692]: pgmap v1111: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:26.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:26 smithi196 ceph-mon[87080]: pgmap v1112: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:26.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:26 smithi118 ceph-mon[93121]: pgmap v1112: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:26 smithi118 ceph-mon[95692]: pgmap v1112: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:28 smithi196 ceph-mon[87080]: pgmap v1113: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:28.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:28 smithi118 ceph-mon[93121]: pgmap v1113: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:28.674 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:28 smithi118 ceph-mon[95692]: pgmap v1113: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:28.674 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:30 smithi196 ceph-mon[87080]: pgmap v1114: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:30 smithi118 ceph-mon[93121]: pgmap v1114: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:30 smithi118 ceph-mon[95692]: pgmap v1114: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:31.741 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:19:31.954 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:19:32.599 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:32 smithi118 ceph-mon[93121]: pgmap v1115: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:32.599 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:32 smithi118 ceph-mon[95692]: pgmap v1115: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:32 smithi196 ceph-mon[87080]: pgmap v1115: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:19:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:19:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:19:33.672 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:19:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:19:33] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:19:34.172 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:19:34.174 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:19:34.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:34 smithi118 ceph-mon[93121]: pgmap v1116: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:34.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:34 smithi118 ceph-mon[95692]: pgmap v1116: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:34 smithi196 ceph-mon[87080]: pgmap v1116: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:34.850 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1116,"stamp":"2024-03-20T08:19:32.687649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003412"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998706+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:15:23.998706+0000","last_peered":"2024-03-20T08:15:23.998706+0000","last_clean":"2024-03-20T08:15:23.998706+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:15:23.998706+0000","last_undegraded":"2024-03-20T08:15:23.998706+0000","last_fullsized":"2024-03-20T08:15:23.998706+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.998231+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:15:23.998231+0000","last_peered":"2024-03-20T08:15:23.998231+0000","last_clean":"2024-03-20T08:15:23.998231+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:15:23.998231+0000","last_undegraded":"2024-03-20T08:15:23.998231+0000","last_fullsized":"2024-03-20T08:15:23.998231+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991037+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:15:23.991037+0000","last_peered":"2024-03-20T08:15:23.991037+0000","last_clean":"2024-03-20T08:15:23.991037+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:15:23.991037+0000","last_undegraded":"2024-03-20T08:15:23.991037+0000","last_fullsized":"2024-03-20T08:15:23.991037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999388+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:15:23.999388+0000","last_peered":"2024-03-20T08:15:23.999388+0000","last_clean":"2024-03-20T08:15:23.999388+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:15:23.999388+0000","last_undegraded":"2024-03-20T08:15:23.999388+0000","last_fullsized":"2024-03-20T08:15:23.999388+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1129,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.999046+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:15:23.999046+0000","last_peered":"2024-03-20T08:15:23.999046+0000","last_clean":"2024-03-20T08:15:23.999046+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:15:23.999046+0000","last_undegraded":"2024-03-20T08:15:23.999046+0000","last_fullsized":"2024-03-20T08:15:23.999046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.992784+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:15:23.992784+0000","last_peered":"2024-03-20T08:15:23.992784+0000","last_clean":"2024-03-20T08:15:23.992784+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:15:23.992784+0000","last_undegraded":"2024-03-20T08:15:23.992784+0000","last_fullsized":"2024-03-20T08:15:23.992784+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.150181+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:15:24.150181+0000","last_peered":"2024-03-20T08:15:24.150181+0000","last_clean":"2024-03-20T08:15:24.150181+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:15:24.150181+0000","last_undegraded":"2024-03-20T08:15:24.150181+0000","last_fullsized":"2024-03-20T08:15:24.150181+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26671,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992533+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:15:23.992533+0000","last_peered":"2024-03-20T08:15:23.992533+0000","last_clean":"2024-03-20T08:15:23.992533+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:15:23.992533+0000","last_undegraded":"2024-03-20T08:15:23.992533+0000","last_fullsized":"2024-03-20T08:15:23.992533+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17600,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.993106+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:15:23.993106+0000","last_peered":"2024-03-20T08:15:23.993106+0000","last_clean":"2024-03-20T08:15:23.993106+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:15:23.993106+0000","last_undegraded":"2024-03-20T08:15:23.993106+0000","last_fullsized":"2024-03-20T08:15:23.993106+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.838040+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:15:34.838040+0000","last_peered":"2024-03-20T08:15:34.838040+0000","last_clean":"2024-03-20T08:15:34.838040+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:15:34.838040+0000","last_undegraded":"2024-03-20T08:15:34.838040+0000","last_fullsized":"2024-03-20T08:15:34.838040+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1611,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870494+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:15:44.870494+0000","last_peered":"2024-03-20T08:15:44.870494+0000","last_clean":"2024-03-20T08:15:44.870494+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:15:44.870494+0000","last_undegraded":"2024-03-20T08:15:44.870494+0000","last_fullsized":"2024-03-20T08:15:44.870494+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:44.870471+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:15:44.870471+0000","last_peered":"2024-03-20T08:15:44.870471+0000","last_clean":"2024-03-20T08:15:44.870471+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:15:44.870471+0000","last_undegraded":"2024-03-20T08:15:44.870471+0000","last_fullsized":"2024-03-20T08:15:44.870471+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:23.991654+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:15:23.991654+0000","last_peered":"2024-03-20T08:15:23.991654+0000","last_clean":"2024-03-20T08:15:23.991654+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:15:23.991654+0000","last_undegraded":"2024-03-20T08:15:23.991654+0000","last_fullsized":"2024-03-20T08:15:23.991654+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5191,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:24.083849+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:15:24.083849+0000","last_peered":"2024-03-20T08:15:24.083849+0000","last_clean":"2024-03-20T08:15:24.083849+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:15:24.083849+0000","last_undegraded":"2024-03-20T08:15:24.083849+0000","last_fullsized":"2024-03-20T08:15:24.083849+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5698,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:23.992471+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:15:23.992471+0000","last_peered":"2024-03-20T08:15:23.992471+0000","last_clean":"2024-03-20T08:15:23.992471+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:15:23.992471+0000","last_undegraded":"2024-03-20T08:15:23.992471+0000","last_fullsized":"2024-03-20T08:15:23.992471+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082774+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:15:24.082774+0000","last_peered":"2024-03-20T08:15:24.082774+0000","last_clean":"2024-03-20T08:15:24.082774+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:15:24.082774+0000","last_undegraded":"2024-03-20T08:15:24.082774+0000","last_fullsized":"2024-03-20T08:15:24.082774+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082438+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:15:24.082438+0000","last_peered":"2024-03-20T08:15:24.082438+0000","last_clean":"2024-03-20T08:15:24.082438+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:15:24.082438+0000","last_undegraded":"2024-03-20T08:15:24.082438+0000","last_fullsized":"2024-03-20T08:15:24.082438+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980214,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484608,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011619,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188970,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656916,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629255,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:19:34.856 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.856 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.856 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:34.857 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:19:35.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:35 smithi196 ceph-mon[87080]: from='client.34641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:35.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:35 smithi118 ceph-mon[93121]: from='client.34641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:35.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:35 smithi118 ceph-mon[95692]: from='client.34641 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:36 smithi196 ceph-mon[87080]: pgmap v1117: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:36 smithi118 ceph-mon[93121]: pgmap v1117: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:36 smithi118 ceph-mon[95692]: pgmap v1117: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:37.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:37 smithi196 ceph-mon[87080]: pgmap v1118: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:37.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:37 smithi118 ceph-mon[93121]: pgmap v1118: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:37.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:37 smithi118 ceph-mon[95692]: pgmap v1118: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:40.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:39 smithi196 ceph-mon[87080]: pgmap v1119: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:40.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:39 smithi118 ceph-mon[93121]: pgmap v1119: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:40.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:39 smithi118 ceph-mon[95692]: pgmap v1119: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:42.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:41 smithi196 ceph-mon[87080]: pgmap v1120: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:42.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:41 smithi118 ceph-mon[93121]: pgmap v1120: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:42.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:41 smithi118 ceph-mon[95692]: pgmap v1120: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:19:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:19:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:19:43.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:43 smithi118 ceph-mon[93121]: pgmap v1121: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:43.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:43.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:19:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:19:43] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:19:43.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:43 smithi118 ceph-mon[95692]: pgmap v1121: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:43.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:43 smithi196 ceph-mon[87080]: pgmap v1121: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:44.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:44.858 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:19:45.077 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:19:46.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:45 smithi196 ceph-mon[87080]: pgmap v1122: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:46.160 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:45 smithi118 ceph-mon[93121]: pgmap v1122: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:46.161 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:45 smithi118 ceph-mon[95692]: pgmap v1122: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:46.901 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:19:46.903 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:19:47.556 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1123,"stamp":"2024-03-20T08:19:46.691189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002991"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083110+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:15:24.083110+0000","last_peered":"2024-03-20T08:15:24.083110+0000","last_clean":"2024-03-20T08:15:24.083110+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:15:24.083110+0000","last_undegraded":"2024-03-20T08:15:24.083110+0000","last_fullsized":"2024-03-20T08:15:24.083110+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1173,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403103+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:15:27.403103+0000","last_peered":"2024-03-20T08:15:27.403103+0000","last_clean":"2024-03-20T08:15:27.403103+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:15:27.403103+0000","last_undegraded":"2024-03-20T08:15:27.403103+0000","last_fullsized":"2024-03-20T08:15:27.403103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403046+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:15:27.403046+0000","last_peered":"2024-03-20T08:15:27.403046+0000","last_clean":"2024-03-20T08:15:27.403046+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:15:27.403046+0000","last_undegraded":"2024-03-20T08:15:27.403046+0000","last_fullsized":"2024-03-20T08:15:27.403046+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:27.403092+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:15:27.403092+0000","last_peered":"2024-03-20T08:15:27.403092+0000","last_clean":"2024-03-20T08:15:27.403092+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:15:27.403092+0000","last_undegraded":"2024-03-20T08:15:27.403092+0000","last_fullsized":"2024-03-20T08:15:27.403092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980217,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484610,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011621,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188973,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703453,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656919,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446863,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629258,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:19:47.566 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.566 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.566 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.566 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.566 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.566 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.566 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.566 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.567 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.567 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.567 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.567 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.567 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.567 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.567 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:19:47.567 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:19:47.842 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:47 smithi118 ceph-mon[93121]: pgmap v1123: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:47.842 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:47 smithi118 ceph-mon[93121]: from='client.34647 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:47.843 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:47 smithi118 ceph-mon[95692]: pgmap v1123: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:47.843 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:47 smithi118 ceph-mon[95692]: from='client.34647 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:48.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:47 smithi196 ceph-mon[87080]: pgmap v1123: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:48.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:47 smithi196 ceph-mon[87080]: from='client.34647 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:19:50.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:49 smithi196 ceph-mon[87080]: pgmap v1124: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:50.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:49 smithi118 ceph-mon[93121]: pgmap v1124: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:50.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:49 smithi118 ceph-mon[95692]: pgmap v1124: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:51 smithi118 ceph-mon[93121]: pgmap v1125: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:51 smithi118 ceph-mon[95692]: pgmap v1125: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:52.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:51 smithi196 ceph-mon[87080]: pgmap v1125: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:19:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:19:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:19:53.911 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:19:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:19:53] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:19:54.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:53 smithi118 ceph-mon[93121]: pgmap v1126: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:54.172 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:53 smithi118 ceph-mon[95692]: pgmap v1126: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:53 smithi196 ceph-mon[87080]: pgmap v1126: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:56.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:55 smithi118 ceph-mon[93121]: pgmap v1127: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:56.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:55 smithi118 ceph-mon[95692]: pgmap v1127: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:55 smithi196 ceph-mon[87080]: pgmap v1127: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:57.568 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:19:57.788 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:19:58.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:57 smithi118 ceph-mon[93121]: pgmap v1128: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:58.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:57 smithi118 ceph-mon[95692]: pgmap v1128: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:58.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:57 smithi196 ceph-mon[87080]: pgmap v1128: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:19:59.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:59.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:19:59.760 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:19:59.762 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:20:00.077 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:19:59 smithi118 ceph-mon[95692]: pgmap v1129: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:00.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:19:59 smithi118 ceph-mon[93121]: pgmap v1129: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:00.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR Possible data damage: 15 pgs snaptrim_error 2024-03-20T08:20:00.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : [ERR] PG_DAMAGED: Possible data damage: 15 pgs snaptrim_error 2024-03-20T08:20:00.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.0 is active+clean+snaptrim_error, acting [3,1,4] 2024-03-20T08:20:00.077 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.1 is active+clean+snaptrim_error, acting [4,3,7] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.2 is active+clean+snaptrim_error, acting [6,0,5] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.3 is active+clean+snaptrim_error, acting [0,3,6] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.4 is active+clean+snaptrim_error, acting [7,2,5] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.5 is active+clean+snaptrim_error, acting [0,1,4] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.6 is active+clean+snaptrim_error, acting [2,5,1] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.7 is active+clean+snaptrim_error, acting [5,1,0] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.8 is active+clean+snaptrim_error, acting [2,0,1] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.9 is active+clean+snaptrim_error, acting [7,6,1] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.a is active+clean+snaptrim_error, acting [2,4,3] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.c is active+clean+snaptrim_error, acting [1,4,0] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.d is active+clean+snaptrim_error, acting [2,7,5] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.e is active+clean+snaptrim_error, acting [4,5,0] 2024-03-20T08:20:00.078 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:19:59.999+0000 7f75139bd700 -1 log_channel(cluster) log [ERR] : pg 5.f is active+clean+snaptrim_error, acting [5,4,6] 2024-03-20T08:20:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:19:59 smithi196 ceph-mon[87080]: pgmap v1129: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:00.486 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1129,"stamp":"2024-03-20T08:19:58.694280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003090"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872349+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:15:59.872349+0000","last_peered":"2024-03-20T08:15:59.872349+0000","last_clean":"2024-03-20T08:15:59.872349+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:15:59.872349+0000","last_undegraded":"2024-03-20T08:15:59.872349+0000","last_fullsized":"2024-03-20T08:15:59.872349+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964697+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:15:50.964697+0000","last_peered":"2024-03-20T08:15:50.964697+0000","last_clean":"2024-03-20T08:15:50.964697+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:15:50.964697+0000","last_undegraded":"2024-03-20T08:15:50.964697+0000","last_fullsized":"2024-03-20T08:15:50.964697+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.149896+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:15:24.149896+0000","last_peered":"2024-03-20T08:15:24.149896+0000","last_clean":"2024-03-20T08:15:24.149896+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:15:24.149896+0000","last_undegraded":"2024-03-20T08:15:24.149896+0000","last_fullsized":"2024-03-20T08:15:24.149896+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.001497+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:15:24.001497+0000","last_peered":"2024-03-20T08:15:24.001497+0000","last_clean":"2024-03-20T08:15:24.001497+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:15:24.001497+0000","last_undegraded":"2024-03-20T08:15:24.001497+0000","last_fullsized":"2024-03-20T08:15:24.001497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.082306+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:15:24.082306+0000","last_peered":"2024-03-20T08:15:24.082306+0000","last_clean":"2024-03-20T08:15:24.082306+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:15:24.082306+0000","last_undegraded":"2024-03-20T08:15:24.082306+0000","last_fullsized":"2024-03-20T08:15:24.082306+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.083581+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:15:24.083581+0000","last_peered":"2024-03-20T08:15:24.083581+0000","last_clean":"2024-03-20T08:15:24.083581+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:15:24.083581+0000","last_undegraded":"2024-03-20T08:15:24.083581+0000","last_fullsized":"2024-03-20T08:15:24.083581+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1393,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964739+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:15:50.964739+0000","last_peered":"2024-03-20T08:15:50.964739+0000","last_clean":"2024-03-20T08:15:50.964739+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:15:50.964739+0000","last_undegraded":"2024-03-20T08:15:50.964739+0000","last_fullsized":"2024-03-20T08:15:50.964739+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964723+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:15:50.964723+0000","last_peered":"2024-03-20T08:15:50.964723+0000","last_clean":"2024-03-20T08:15:50.964723+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:15:50.964723+0000","last_undegraded":"2024-03-20T08:15:50.964723+0000","last_fullsized":"2024-03-20T08:15:50.964723+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.086364+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:15:24.086364+0000","last_peered":"2024-03-20T08:15:24.086364+0000","last_clean":"2024-03-20T08:15:24.086364+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:15:24.086364+0000","last_undegraded":"2024-03-20T08:15:24.086364+0000","last_fullsized":"2024-03-20T08:15:24.086364+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11689,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964670+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:15:50.964670+0000","last_peered":"2024-03-20T08:15:50.964670+0000","last_clean":"2024-03-20T08:15:50.964670+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:15:50.964670+0000","last_undegraded":"2024-03-20T08:15:50.964670+0000","last_fullsized":"2024-03-20T08:15:50.964670+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5951,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:50.964705+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:15:50.964705+0000","last_peered":"2024-03-20T08:15:50.964705+0000","last_clean":"2024-03-20T08:15:50.964705+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:15:50.964705+0000","last_undegraded":"2024-03-20T08:15:50.964705+0000","last_fullsized":"2024-03-20T08:15:50.964705+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.278155+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:16:02.278155+0000","last_peered":"2024-03-20T08:16:02.278155+0000","last_clean":"2024-03-20T08:16:02.278155+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:16:02.278155+0000","last_undegraded":"2024-03-20T08:16:02.278155+0000","last_fullsized":"2024-03-20T08:16:02.278155+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3979,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:59.872278+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:15:59.872278+0000","last_peered":"2024-03-20T08:15:59.872278+0000","last_clean":"2024-03-20T08:15:59.872278+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:15:59.872278+0000","last_undegraded":"2024-03-20T08:15:59.872278+0000","last_fullsized":"2024-03-20T08:15:59.872278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964638+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:15:50.964638+0000","last_peered":"2024-03-20T08:15:50.964638+0000","last_clean":"2024-03-20T08:15:50.964638+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:15:50.964638+0000","last_undegraded":"2024-03-20T08:15:50.964638+0000","last_fullsized":"2024-03-20T08:15:50.964638+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964687+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:15:50.964687+0000","last_peered":"2024-03-20T08:15:50.964687+0000","last_clean":"2024-03-20T08:15:50.964687+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:15:50.964687+0000","last_undegraded":"2024-03-20T08:15:50.964687+0000","last_fullsized":"2024-03-20T08:15:50.964687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:59.872330+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:15:59.872330+0000","last_peered":"2024-03-20T08:15:59.872330+0000","last_clean":"2024-03-20T08:15:59.872330+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:15:59.872330+0000","last_undegraded":"2024-03-20T08:15:59.872330+0000","last_fullsized":"2024-03-20T08:15:59.872330+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:50.964661+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:15:50.964661+0000","last_peered":"2024-03-20T08:15:50.964661+0000","last_clean":"2024-03-20T08:15:50.964661+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:15:50.964661+0000","last_undegraded":"2024-03-20T08:15:50.964661+0000","last_fullsized":"2024-03-20T08:15:50.964661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980219,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484613,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011624,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188975,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656921,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446865,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629260,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:20:00.492 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.492 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.492 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.492 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.492 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.492 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.492 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.492 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.493 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.493 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.493 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.493 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.493 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.493 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.493 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:00.493 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:20:01.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: from='client.44573 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: Health detail: HEALTH_ERR Possible data damage: 15 pgs snaptrim_error 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: [ERR] PG_DAMAGED: Possible data damage: 15 pgs snaptrim_error 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.0 is active+clean+snaptrim_error, acting [3,1,4] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.1 is active+clean+snaptrim_error, acting [4,3,7] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.2 is active+clean+snaptrim_error, acting [6,0,5] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.3 is active+clean+snaptrim_error, acting [0,3,6] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.4 is active+clean+snaptrim_error, acting [7,2,5] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.5 is active+clean+snaptrim_error, acting [0,1,4] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.6 is active+clean+snaptrim_error, acting [2,5,1] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.7 is active+clean+snaptrim_error, acting [5,1,0] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.8 is active+clean+snaptrim_error, acting [2,0,1] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.9 is active+clean+snaptrim_error, acting [7,6,1] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.a is active+clean+snaptrim_error, acting [2,4,3] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.c is active+clean+snaptrim_error, acting [1,4,0] 2024-03-20T08:20:01.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.d is active+clean+snaptrim_error, acting [2,7,5] 2024-03-20T08:20:01.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.e is active+clean+snaptrim_error, acting [4,5,0] 2024-03-20T08:20:01.391 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:00 smithi196 ceph-mon[87080]: pg 5.f is active+clean+snaptrim_error, acting [5,4,6] 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: from='client.44573 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: Health detail: HEALTH_ERR Possible data damage: 15 pgs snaptrim_error 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: [ERR] PG_DAMAGED: Possible data damage: 15 pgs snaptrim_error 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.0 is active+clean+snaptrim_error, acting [3,1,4] 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.1 is active+clean+snaptrim_error, acting [4,3,7] 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.2 is active+clean+snaptrim_error, acting [6,0,5] 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.3 is active+clean+snaptrim_error, acting [0,3,6] 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.4 is active+clean+snaptrim_error, acting [7,2,5] 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.5 is active+clean+snaptrim_error, acting [0,1,4] 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.6 is active+clean+snaptrim_error, acting [2,5,1] 2024-03-20T08:20:01.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.7 is active+clean+snaptrim_error, acting [5,1,0] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.8 is active+clean+snaptrim_error, acting [2,0,1] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.9 is active+clean+snaptrim_error, acting [7,6,1] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.a is active+clean+snaptrim_error, acting [2,4,3] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.c is active+clean+snaptrim_error, acting [1,4,0] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.d is active+clean+snaptrim_error, acting [2,7,5] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.e is active+clean+snaptrim_error, acting [4,5,0] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[93121]: pg 5.f is active+clean+snaptrim_error, acting [5,4,6] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: from='client.44573 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: Health detail: HEALTH_ERR Possible data damage: 15 pgs snaptrim_error 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: [ERR] PG_DAMAGED: Possible data damage: 15 pgs snaptrim_error 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.0 is active+clean+snaptrim_error, acting [3,1,4] 2024-03-20T08:20:01.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.1 is active+clean+snaptrim_error, acting [4,3,7] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.2 is active+clean+snaptrim_error, acting [6,0,5] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.3 is active+clean+snaptrim_error, acting [0,3,6] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.4 is active+clean+snaptrim_error, acting [7,2,5] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.5 is active+clean+snaptrim_error, acting [0,1,4] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.6 is active+clean+snaptrim_error, acting [2,5,1] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.7 is active+clean+snaptrim_error, acting [5,1,0] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.8 is active+clean+snaptrim_error, acting [2,0,1] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.9 is active+clean+snaptrim_error, acting [7,6,1] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.a is active+clean+snaptrim_error, acting [2,4,3] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.c is active+clean+snaptrim_error, acting [1,4,0] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.d is active+clean+snaptrim_error, acting [2,7,5] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.e is active+clean+snaptrim_error, acting [4,5,0] 2024-03-20T08:20:01.425 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:00 smithi118 ceph-mon[95692]: pg 5.f is active+clean+snaptrim_error, acting [5,4,6] 2024-03-20T08:20:02.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:01 smithi196 ceph-mon[87080]: pgmap v1130: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:02.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:01 smithi118 ceph-mon[95692]: pgmap v1130: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:02.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:01 smithi118 ceph-mon[93121]: pgmap v1130: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:20:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:20:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:20:03.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:20:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:20:03] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:20:04.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:03 smithi196 ceph-mon[87080]: pgmap v1131: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:04.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:03 smithi118 ceph-mon[93121]: pgmap v1131: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:04.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:03 smithi118 ceph-mon[95692]: pgmap v1131: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:06.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:05 smithi196 ceph-mon[87080]: pgmap v1132: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:06.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:05 smithi118 ceph-mon[93121]: pgmap v1132: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:06.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:05 smithi118 ceph-mon[95692]: pgmap v1132: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:07 smithi196 ceph-mon[87080]: pgmap v1133: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:08.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:07 smithi118 ceph-mon[95692]: pgmap v1133: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:08.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:07 smithi118 ceph-mon[93121]: pgmap v1133: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:10.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:09 smithi196 ceph-mon[87080]: pgmap v1134: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:10.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:09 smithi118 ceph-mon[95692]: pgmap v1134: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:10.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:09 smithi118 ceph-mon[93121]: pgmap v1134: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:10.493 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:20:10.708 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:20:11.973 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:11 smithi118 ceph-mon[95692]: pgmap v1135: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:11.973 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:11 smithi118 ceph-mon[93121]: pgmap v1135: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:12.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:11 smithi196 ceph-mon[87080]: pgmap v1135: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:12.432 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:20:12.434 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:20:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:20:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:20:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:20:13.163 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1135,"stamp":"2024-03-20T08:20:10.697299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003018"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:19.820350+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:16:19.820350+0000","last_peered":"2024-03-20T08:16:19.820350+0000","last_clean":"2024-03-20T08:16:19.820350+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:16:19.820350+0000","last_undegraded":"2024-03-20T08:16:19.820350+0000","last_fullsized":"2024-03-20T08:16:19.820350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839608+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:15:54.839608+0000","last_peered":"2024-03-20T08:15:54.839608+0000","last_clean":"2024-03-20T08:15:54.839608+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:15:54.839608+0000","last_undegraded":"2024-03-20T08:15:54.839608+0000","last_fullsized":"2024-03-20T08:15:54.839608+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839569+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:15:54.839569+0000","last_peered":"2024-03-20T08:15:54.839569+0000","last_clean":"2024-03-20T08:15:54.839569+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:15:54.839569+0000","last_undegraded":"2024-03-20T08:15:54.839569+0000","last_fullsized":"2024-03-20T08:15:54.839569+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31732,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:15:42.855325+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:15:42.855325+0000","last_peered":"2024-03-20T08:15:42.855325+0000","last_clean":"2024-03-20T08:15:42.855325+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:15:42.855325+0000","last_undegraded":"2024-03-20T08:15:42.855325+0000","last_fullsized":"2024-03-20T08:15:42.855325+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855297+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:15:42.855297+0000","last_peered":"2024-03-20T08:15:42.855297+0000","last_clean":"2024-03-20T08:15:42.855297+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:15:42.855297+0000","last_undegraded":"2024-03-20T08:15:42.855297+0000","last_fullsized":"2024-03-20T08:15:42.855297+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3377,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855264+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:15:42.855264+0000","last_peered":"2024-03-20T08:15:42.855264+0000","last_clean":"2024-03-20T08:15:42.855264+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:15:42.855264+0000","last_undegraded":"2024-03-20T08:15:42.855264+0000","last_fullsized":"2024-03-20T08:15:42.855264+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855313+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:15:42.855313+0000","last_peered":"2024-03-20T08:15:42.855313+0000","last_clean":"2024-03-20T08:15:42.855313+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:15:42.855313+0000","last_undegraded":"2024-03-20T08:15:42.855313+0000","last_fullsized":"2024-03-20T08:15:42.855313+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1306,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.855362+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:15:42.855362+0000","last_peered":"2024-03-20T08:15:42.855362+0000","last_clean":"2024-03-20T08:15:42.855362+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:15:42.855362+0000","last_undegraded":"2024-03-20T08:15:42.855362+0000","last_fullsized":"2024-03-20T08:15:42.855362+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839596+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:15:54.839596+0000","last_peered":"2024-03-20T08:15:54.839596+0000","last_clean":"2024-03-20T08:15:54.839596+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:15:54.839596+0000","last_undegraded":"2024-03-20T08:15:54.839596+0000","last_fullsized":"2024-03-20T08:15:54.839596+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1277,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:54.839615+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:15:54.839615+0000","last_peered":"2024-03-20T08:15:54.839615+0000","last_clean":"2024-03-20T08:15:54.839615+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:15:54.839615+0000","last_undegraded":"2024-03-20T08:15:54.839615+0000","last_fullsized":"2024-03-20T08:15:54.839615+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.404686+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:15:42.404686+0000","last_peered":"2024-03-20T08:15:42.404686+0000","last_clean":"2024-03-20T08:15:42.404686+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:15:42.404686+0000","last_undegraded":"2024-03-20T08:15:42.404686+0000","last_fullsized":"2024-03-20T08:15:42.404686+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980222,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484615,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011626,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188977,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703458,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656924,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446868,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629263,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:20:13.169 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.169 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:13.170 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:20:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:20:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:20:13] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:20:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:13 smithi196 ceph-mon[87080]: from='client.44579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:13 smithi196 ceph-mon[87080]: pgmap v1136: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:14.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:13 smithi118 ceph-mon[93121]: from='client.44579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:13 smithi118 ceph-mon[93121]: pgmap v1136: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:13 smithi118 ceph-mon[95692]: from='client.44579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:13 smithi118 ceph-mon[95692]: pgmap v1136: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:16.224 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:15 smithi118 ceph-mon[93121]: pgmap v1137: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:16.224 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:15 smithi118 ceph-mon[95692]: pgmap v1137: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:15 smithi196 ceph-mon[87080]: pgmap v1137: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:17.140 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:16 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:20:17.141 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:16 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:20:17.290 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:16 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:20:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:17 smithi196 ceph-mon[87080]: pgmap v1138: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:17 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:20:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:17 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:20:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:17 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:20:18.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:17 smithi118 ceph-mon[93121]: pgmap v1138: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:18.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:17 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:20:18.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:17 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:20:18.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:17 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:20:18.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:17 smithi118 ceph-mon[95692]: pgmap v1138: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:18.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:17 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:20:18.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:17 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:20:18.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:17 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:20:20.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:19 smithi196 ceph-mon[87080]: pgmap v1139: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:20.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:19 smithi118 ceph-mon[93121]: pgmap v1139: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:20.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:19 smithi118 ceph-mon[95692]: pgmap v1139: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:22.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:21 smithi196 ceph-mon[87080]: pgmap v1140: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:22.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:21 smithi118 ceph-mon[93121]: pgmap v1140: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:22.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:21 smithi118 ceph-mon[95692]: pgmap v1140: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:20:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:20:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:20:23.172 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:20:23.383 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:20:23.830 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:20:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:20:23] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:20:24.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:23 smithi118 ceph-mon[93121]: pgmap v1141: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:24.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:23 smithi118 ceph-mon[95692]: pgmap v1141: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:23 smithi196 ceph-mon[87080]: pgmap v1141: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:25.368 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:20:25.372 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:20:26.022 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1142,"stamp":"2024-03-20T08:20:24.701581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003561"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1255,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:24.689257+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:15:24.689257+0000","last_peered":"2024-03-20T08:15:24.689257+0000","last_clean":"2024-03-20T08:15:24.689257+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:15:24.689257+0000","last_undegraded":"2024-03-20T08:15:24.689257+0000","last_fullsized":"2024-03-20T08:15:24.689257+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1383,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:25.962855+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:15:25.962855+0000","last_peered":"2024-03-20T08:15:25.962855+0000","last_clean":"2024-03-20T08:15:25.962855+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:15:25.962855+0000","last_undegraded":"2024-03-20T08:15:25.962855+0000","last_fullsized":"2024-03-20T08:15:25.962855+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815294+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:15:34.815294+0000","last_peered":"2024-03-20T08:15:34.815294+0000","last_clean":"2024-03-20T08:15:34.815294+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:15:34.815294+0000","last_undegraded":"2024-03-20T08:15:34.815294+0000","last_fullsized":"2024-03-20T08:15:34.815294+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276506+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:15:42.276506+0000","last_peered":"2024-03-20T08:15:42.276506+0000","last_clean":"2024-03-20T08:15:42.276506+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:15:42.276506+0000","last_undegraded":"2024-03-20T08:15:42.276506+0000","last_fullsized":"2024-03-20T08:15:42.276506+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815350+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:15:34.815350+0000","last_peered":"2024-03-20T08:15:34.815350+0000","last_clean":"2024-03-20T08:15:34.815350+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:15:34.815350+0000","last_undegraded":"2024-03-20T08:15:34.815350+0000","last_fullsized":"2024-03-20T08:15:34.815350+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1195,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:34.815338+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:15:34.815338+0000","last_peered":"2024-03-20T08:15:34.815338+0000","last_clean":"2024-03-20T08:15:34.815338+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:15:34.815338+0000","last_undegraded":"2024-03-20T08:15:34.815338+0000","last_fullsized":"2024-03-20T08:15:34.815338+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:42.276540+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:15:42.276540+0000","last_peered":"2024-03-20T08:15:42.276540+0000","last_clean":"2024-03-20T08:15:42.276540+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:15:42.276540+0000","last_undegraded":"2024-03-20T08:15:42.276540+0000","last_fullsized":"2024-03-20T08:15:42.276540+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694643+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:16:14.694643+0000","last_peered":"2024-03-20T08:16:14.694643+0000","last_clean":"2024-03-20T08:16:14.694643+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:16:14.694643+0000","last_undegraded":"2024-03-20T08:16:14.694643+0000","last_fullsized":"2024-03-20T08:16:14.694643+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:15.966624+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:16:15.966624+0000","last_peered":"2024-03-20T08:16:15.966624+0000","last_clean":"2024-03-20T08:16:15.966624+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:16:15.966624+0000","last_undegraded":"2024-03-20T08:16:15.966624+0000","last_fullsized":"2024-03-20T08:16:15.966624+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4605,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.694612+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:16:14.694612+0000","last_peered":"2024-03-20T08:16:14.694612+0000","last_clean":"2024-03-20T08:16:14.694612+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:16:14.694612+0000","last_undegraded":"2024-03-20T08:16:14.694612+0000","last_fullsized":"2024-03-20T08:16:14.694612+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1216,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.694591+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:16:14.694591+0000","last_peered":"2024-03-20T08:16:14.694591+0000","last_clean":"2024-03-20T08:16:14.694591+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:16:14.694591+0000","last_undegraded":"2024-03-20T08:16:14.694591+0000","last_fullsized":"2024-03-20T08:16:14.694591+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1115,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.874080+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:16:14.874080+0000","last_peered":"2024-03-20T08:16:14.874080+0000","last_clean":"2024-03-20T08:16:14.874080+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:16:14.874080+0000","last_undegraded":"2024-03-20T08:16:14.874080+0000","last_fullsized":"2024-03-20T08:16:14.874080+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4958,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:14.874107+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:16:14.874107+0000","last_peered":"2024-03-20T08:16:14.874107+0000","last_clean":"2024-03-20T08:16:14.874107+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:16:14.874107+0000","last_undegraded":"2024-03-20T08:16:14.874107+0000","last_fullsized":"2024-03-20T08:16:14.874107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980224,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484618,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011629,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188980,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703461,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656926,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446870,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629265,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.028 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.029 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.029 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.029 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.029 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.029 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:26.029 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:20:26.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:25 smithi196 ceph-mon[87080]: pgmap v1142: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:26.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:25 smithi196 ceph-mon[87080]: from='client.34665 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:26.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:25 smithi118 ceph-mon[93121]: pgmap v1142: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:26.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:25 smithi118 ceph-mon[93121]: from='client.34665 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:26.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:25 smithi118 ceph-mon[95692]: pgmap v1142: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:26.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:25 smithi118 ceph-mon[95692]: from='client.34665 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:28.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:27 smithi196 ceph-mon[87080]: pgmap v1143: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:28.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:27 smithi118 ceph-mon[93121]: pgmap v1143: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:28.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:27 smithi118 ceph-mon[95692]: pgmap v1143: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:29.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:29 smithi196 ceph-mon[87080]: pgmap v1144: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:29 smithi118 ceph-mon[93121]: pgmap v1144: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:29 smithi118 ceph-mon[95692]: pgmap v1144: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:32.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:31 smithi196 ceph-mon[87080]: pgmap v1145: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:32.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:31 smithi118 ceph-mon[93121]: pgmap v1145: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:32.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:31 smithi118 ceph-mon[95692]: pgmap v1145: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:32.641 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:20:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:20:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:20:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:20:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:20:33] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:20:34.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:33 smithi196 ceph-mon[87080]: pgmap v1146: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:34.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:33 smithi118 ceph-mon[93121]: pgmap v1146: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:34.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:33 smithi118 ceph-mon[95692]: pgmap v1146: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:36.030 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:20:36.243 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:20:36.364 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:35 smithi118 ceph-mon[93121]: pgmap v1147: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:36.364 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:35 smithi118 ceph-mon[95692]: pgmap v1147: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:36.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:35 smithi196 ceph-mon[87080]: pgmap v1147: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:38.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:37 smithi118 ceph-mon[93121]: pgmap v1148: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:37 smithi118 ceph-mon[95692]: pgmap v1148: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:38.238 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:20:38.240 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:20:38.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:37 smithi196 ceph-mon[87080]: pgmap v1148: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:38.946 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1148,"stamp":"2024-03-20T08:20:36.704622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1341,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280037+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:16:27.280037+0000","last_peered":"2024-03-20T08:16:27.280037+0000","last_clean":"2024-03-20T08:16:27.280037+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:16:27.280037+0000","last_undegraded":"2024-03-20T08:16:27.280037+0000","last_fullsized":"2024-03-20T08:16:27.280037+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816750+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:15:49.816750+0000","last_peered":"2024-03-20T08:15:49.816750+0000","last_clean":"2024-03-20T08:15:49.816750+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:15:49.816750+0000","last_undegraded":"2024-03-20T08:15:49.816750+0000","last_fullsized":"2024-03-20T08:15:49.816750+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1301,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856971+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:16:02.856971+0000","last_peered":"2024-03-20T08:16:02.856971+0000","last_clean":"2024-03-20T08:16:02.856971+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:16:02.856971+0000","last_undegraded":"2024-03-20T08:16:02.856971+0000","last_fullsized":"2024-03-20T08:16:02.856971+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1227,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:49.816774+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:15:49.816774+0000","last_peered":"2024-03-20T08:15:49.816774+0000","last_clean":"2024-03-20T08:15:49.816774+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:15:49.816774+0000","last_undegraded":"2024-03-20T08:15:49.816774+0000","last_fullsized":"2024-03-20T08:15:49.816774+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1221,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406457+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:15:57.406457+0000","last_peered":"2024-03-20T08:15:57.406457+0000","last_clean":"2024-03-20T08:15:57.406457+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:15:57.406457+0000","last_undegraded":"2024-03-20T08:15:57.406457+0000","last_fullsized":"2024-03-20T08:15:57.406457+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406429+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:15:57.406429+0000","last_peered":"2024-03-20T08:15:57.406429+0000","last_clean":"2024-03-20T08:15:57.406429+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:15:57.406429+0000","last_undegraded":"2024-03-20T08:15:57.406429+0000","last_fullsized":"2024-03-20T08:15:57.406429+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:15:57.406388+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:15:57.406388+0000","last_peered":"2024-03-20T08:15:57.406388+0000","last_clean":"2024-03-20T08:15:57.406388+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:15:57.406388+0000","last_undegraded":"2024-03-20T08:15:57.406388+0000","last_fullsized":"2024-03-20T08:15:57.406388+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1317,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:02.856925+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:16:02.856925+0000","last_peered":"2024-03-20T08:16:02.856925+0000","last_clean":"2024-03-20T08:16:02.856925+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:16:02.856925+0000","last_undegraded":"2024-03-20T08:16:02.856925+0000","last_fullsized":"2024-03-20T08:16:02.856925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279964+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:16:27.279964+0000","last_peered":"2024-03-20T08:16:27.279964+0000","last_clean":"2024-03-20T08:16:27.279964+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:16:27.279964+0000","last_undegraded":"2024-03-20T08:16:27.279964+0000","last_fullsized":"2024-03-20T08:16:27.279964+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.279988+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:16:27.279988+0000","last_peered":"2024-03-20T08:16:27.279988+0000","last_clean":"2024-03-20T08:16:27.279988+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:16:27.279988+0000","last_undegraded":"2024-03-20T08:16:27.279988+0000","last_fullsized":"2024-03-20T08:16:27.279988+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822554+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:16:39.822554+0000","last_peered":"2024-03-20T08:16:39.822554+0000","last_clean":"2024-03-20T08:16:39.822554+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:16:39.822554+0000","last_undegraded":"2024-03-20T08:16:39.822554+0000","last_fullsized":"2024-03-20T08:16:39.822554+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280002+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:16:27.280002+0000","last_peered":"2024-03-20T08:16:27.280002+0000","last_clean":"2024-03-20T08:16:27.280002+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:16:27.280002+0000","last_undegraded":"2024-03-20T08:16:27.280002+0000","last_fullsized":"2024-03-20T08:16:27.280002+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822478+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:16:39.822478+0000","last_peered":"2024-03-20T08:16:39.822478+0000","last_clean":"2024-03-20T08:16:39.822478+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:16:39.822478+0000","last_undegraded":"2024-03-20T08:16:39.822478+0000","last_fullsized":"2024-03-20T08:16:39.822478+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822530+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:16:39.822530+0000","last_peered":"2024-03-20T08:16:39.822530+0000","last_clean":"2024-03-20T08:16:39.822530+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:16:39.822530+0000","last_undegraded":"2024-03-20T08:16:39.822530+0000","last_fullsized":"2024-03-20T08:16:39.822530+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.280016+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:16:27.280016+0000","last_peered":"2024-03-20T08:16:27.280016+0000","last_clean":"2024-03-20T08:16:27.280016+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:16:27.280016+0000","last_undegraded":"2024-03-20T08:16:27.280016+0000","last_fullsized":"2024-03-20T08:16:27.280016+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1284,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841118+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:16:14.841118+0000","last_peered":"2024-03-20T08:16:14.841118+0000","last_clean":"2024-03-20T08:16:14.841118+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:16:14.841118+0000","last_undegraded":"2024-03-20T08:16:14.841118+0000","last_fullsized":"2024-03-20T08:16:14.841118+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:39.822511+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:16:39.822511+0000","last_peered":"2024-03-20T08:16:39.822511+0000","last_clean":"2024-03-20T08:16:39.822511+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:16:39.822511+0000","last_undegraded":"2024-03-20T08:16:39.822511+0000","last_fullsized":"2024-03-20T08:16:39.822511+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:14.841098+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:16:14.841098+0000","last_peered":"2024-03-20T08:16:14.841098+0000","last_clean":"2024-03-20T08:16:14.841098+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:16:14.841098+0000","last_undegraded":"2024-03-20T08:16:14.841098+0000","last_fullsized":"2024-03-20T08:16:14.841098+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980227,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484620,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011631,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188983,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703463,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656929,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446873,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629268,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:20:38.956 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.957 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:38.958 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:20:39.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:38 smithi196 ceph-mon[87080]: from='client.44591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:39.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:38 smithi118 ceph-mon[93121]: from='client.44591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:39.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:38 smithi118 ceph-mon[95692]: from='client.44591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:40.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:39 smithi196 ceph-mon[87080]: pgmap v1149: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:40.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:39 smithi118 ceph-mon[93121]: pgmap v1149: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:40.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:39 smithi118 ceph-mon[95692]: pgmap v1149: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:42.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:41 smithi196 ceph-mon[87080]: pgmap v1150: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:42.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:41 smithi118 ceph-mon[93121]: pgmap v1150: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:42.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:41 smithi118 ceph-mon[95692]: pgmap v1150: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:20:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:20:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:20:43.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:20:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:20:43] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:20:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:43 smithi196 ceph-mon[87080]: pgmap v1151: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:44.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:43 smithi118 ceph-mon[93121]: pgmap v1151: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:43 smithi118 ceph-mon[95692]: pgmap v1151: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:46.248 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:45 smithi118 ceph-mon[93121]: pgmap v1152: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:46.249 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:45 smithi118 ceph-mon[95692]: pgmap v1152: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:46.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:45 smithi196 ceph-mon[87080]: pgmap v1152: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:48.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:47 smithi196 ceph-mon[87080]: pgmap v1153: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:48.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:47 smithi118 ceph-mon[93121]: pgmap v1153: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:48.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:47 smithi118 ceph-mon[95692]: pgmap v1153: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:48.958 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:20:49.209 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:20:50.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:49 smithi196 ceph-mon[87080]: pgmap v1154: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:50.417 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:49 smithi118 ceph-mon[93121]: pgmap v1154: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:50.417 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:49 smithi118 ceph-mon[95692]: pgmap v1154: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:51.227 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:20:51.229 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:20:51.922 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1155,"stamp":"2024-03-20T08:20:50.708516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003329"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408237+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:16:12.408237+0000","last_peered":"2024-03-20T08:16:12.408237+0000","last_clean":"2024-03-20T08:16:12.408237+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:16:12.408237+0000","last_undegraded":"2024-03-20T08:16:12.408237+0000","last_fullsized":"2024-03-20T08:16:12.408237+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1185,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408210+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:16:12.408210+0000","last_peered":"2024-03-20T08:16:12.408210+0000","last_clean":"2024-03-20T08:16:12.408210+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:16:12.408210+0000","last_undegraded":"2024-03-20T08:16:12.408210+0000","last_fullsized":"2024-03-20T08:16:12.408210+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876013+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:16:29.876013+0000","last_peered":"2024-03-20T08:16:29.876013+0000","last_clean":"2024-03-20T08:16:29.876013+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:16:29.876013+0000","last_undegraded":"2024-03-20T08:16:29.876013+0000","last_fullsized":"2024-03-20T08:16:29.876013+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:12.408251+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:16:12.408251+0000","last_peered":"2024-03-20T08:16:12.408251+0000","last_clean":"2024-03-20T08:16:12.408251+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:16:12.408251+0000","last_undegraded":"2024-03-20T08:16:12.408251+0000","last_fullsized":"2024-03-20T08:16:12.408251+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876023+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:16:29.876023+0000","last_peered":"2024-03-20T08:16:29.876023+0000","last_clean":"2024-03-20T08:16:29.876023+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:16:29.876023+0000","last_undegraded":"2024-03-20T08:16:29.876023+0000","last_fullsized":"2024-03-20T08:16:29.876023+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1251,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:34.697031+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:16:34.697031+0000","last_peered":"2024-03-20T08:16:34.697031+0000","last_clean":"2024-03-20T08:16:34.697031+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:16:34.697031+0000","last_undegraded":"2024-03-20T08:16:34.697031+0000","last_fullsized":"2024-03-20T08:16:34.697031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.875989+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:16:29.875989+0000","last_peered":"2024-03-20T08:16:29.875989+0000","last_clean":"2024-03-20T08:16:29.875989+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:16:29.875989+0000","last_undegraded":"2024-03-20T08:16:29.875989+0000","last_fullsized":"2024-03-20T08:16:29.875989+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:29.876037+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:16:29.876037+0000","last_peered":"2024-03-20T08:16:29.876037+0000","last_clean":"2024-03-20T08:16:29.876037+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:16:29.876037+0000","last_undegraded":"2024-03-20T08:16:29.876037+0000","last_fullsized":"2024-03-20T08:16:29.876037+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980230,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484623,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011634,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188985,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703466,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656932,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446876,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629271,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:20:51.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.928 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:20:51.929 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:20:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:51 smithi118 ceph-mon[93121]: pgmap v1155: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:52.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:51 smithi118 ceph-mon[93121]: from='client.44597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:51 smithi118 ceph-mon[95692]: pgmap v1155: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:52.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:51 smithi118 ceph-mon[95692]: from='client.44597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:52.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:51 smithi196 ceph-mon[87080]: pgmap v1155: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:52.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:51 smithi196 ceph-mon[87080]: from='client.44597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:20:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:20:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:20:52.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:20:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:20:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:20:53] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:20:54.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:54 smithi196 ceph-mon[87080]: pgmap v1156: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:54 smithi118 ceph-mon[93121]: pgmap v1156: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:54 smithi118 ceph-mon[95692]: pgmap v1156: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:56.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:56 smithi196 ceph-mon[87080]: pgmap v1157: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:56 smithi118 ceph-mon[93121]: pgmap v1157: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:56 smithi118 ceph-mon[95692]: pgmap v1157: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:58.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:58 smithi196 ceph-mon[87080]: pgmap v1158: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:58.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:58 smithi118 ceph-mon[93121]: pgmap v1158: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:58.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:58 smithi118 ceph-mon[95692]: pgmap v1158: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:20:59.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:20:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:59.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:20:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:20:59.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:20:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:00.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:00 smithi196 ceph-mon[87080]: pgmap v1159: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:00.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:00 smithi118 ceph-mon[93121]: pgmap v1159: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:00.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:00 smithi118 ceph-mon[95692]: pgmap v1159: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:01.930 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:21:02.150 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:21:02.277 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:02 smithi118 ceph-mon[93121]: pgmap v1160: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:02.277 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:02 smithi118 ceph-mon[95692]: pgmap v1160: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:02.314 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:02 smithi196 ceph-mon[87080]: pgmap v1160: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:21:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:21:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:21:03.786 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:21:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:21:03] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:21:04.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:04 smithi118 ceph-mon[93121]: pgmap v1161: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:04.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:04 smithi118 ceph-mon[95692]: pgmap v1161: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:04.373 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:21:04.375 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:21:04.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:04 smithi196 ceph-mon[87080]: pgmap v1161: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:04.987 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1161,"stamp":"2024-03-20T08:21:02.711739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003222"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846102+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:17:14.846102+0000","last_peered":"2024-03-20T08:17:14.846102+0000","last_clean":"2024-03-20T08:17:14.846102+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:17:14.846102+0000","last_undegraded":"2024-03-20T08:17:14.846102+0000","last_fullsized":"2024-03-20T08:17:14.846102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846117+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:17:14.846117+0000","last_peered":"2024-03-20T08:17:14.846117+0000","last_clean":"2024-03-20T08:17:14.846117+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:17:14.846117+0000","last_undegraded":"2024-03-20T08:17:14.846117+0000","last_fullsized":"2024-03-20T08:17:14.846117+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1334,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281973+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:16:52.281973+0000","last_peered":"2024-03-20T08:16:52.281973+0000","last_clean":"2024-03-20T08:16:52.281973+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:16:52.281973+0000","last_undegraded":"2024-03-20T08:16:52.281973+0000","last_fullsized":"2024-03-20T08:16:52.281973+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846051+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:17:14.846051+0000","last_peered":"2024-03-20T08:17:14.846051+0000","last_clean":"2024-03-20T08:17:14.846051+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:17:14.846051+0000","last_undegraded":"2024-03-20T08:17:14.846051+0000","last_fullsized":"2024-03-20T08:17:14.846051+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281958+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:16:52.281958+0000","last_peered":"2024-03-20T08:16:52.281958+0000","last_clean":"2024-03-20T08:16:52.281958+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:16:52.281958+0000","last_undegraded":"2024-03-20T08:16:52.281958+0000","last_fullsized":"2024-03-20T08:16:52.281958+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5629,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:14.846008+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:17:14.846008+0000","last_peered":"2024-03-20T08:17:14.846008+0000","last_clean":"2024-03-20T08:17:14.846008+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:17:14.846008+0000","last_undegraded":"2024-03-20T08:17:14.846008+0000","last_fullsized":"2024-03-20T08:17:14.846008+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281883+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:16:52.281883+0000","last_peered":"2024-03-20T08:16:52.281883+0000","last_clean":"2024-03-20T08:16:52.281883+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:16:52.281883+0000","last_undegraded":"2024-03-20T08:16:52.281883+0000","last_fullsized":"2024-03-20T08:16:52.281883+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281910+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:16:52.281910+0000","last_peered":"2024-03-20T08:16:52.281910+0000","last_clean":"2024-03-20T08:16:52.281910+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:16:52.281910+0000","last_undegraded":"2024-03-20T08:16:52.281910+0000","last_fullsized":"2024-03-20T08:16:52.281910+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:52.281942+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:16:52.281942+0000","last_peered":"2024-03-20T08:16:52.281942+0000","last_clean":"2024-03-20T08:16:52.281942+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:16:52.281942+0000","last_undegraded":"2024-03-20T08:16:52.281942+0000","last_fullsized":"2024-03-20T08:16:52.281942+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:40.968512+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:16:40.968512+0000","last_peered":"2024-03-20T08:16:40.968512+0000","last_clean":"2024-03-20T08:16:40.968512+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:16:40.968512+0000","last_undegraded":"2024-03-20T08:16:40.968512+0000","last_fullsized":"2024-03-20T08:16:40.968512+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980232,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484626,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011637,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188988,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703468,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656934,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446878,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629273,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:21:04.994 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.995 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.995 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.995 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.995 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.995 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.995 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.995 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.996 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.996 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.996 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.996 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.996 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.996 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.996 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:04.996 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:21:05.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:05 smithi196 ceph-mon[87080]: from='client.34683 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:05.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:05 smithi118 ceph-mon[93121]: from='client.34683 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:05.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:05 smithi118 ceph-mon[95692]: from='client.34683 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:06.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:06 smithi196 ceph-mon[87080]: pgmap v1162: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:06.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:06 smithi118 ceph-mon[93121]: pgmap v1162: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:06.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:06 smithi118 ceph-mon[95692]: pgmap v1162: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:08.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:08 smithi196 ceph-mon[87080]: pgmap v1163: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:08.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:08 smithi118 ceph-mon[93121]: pgmap v1163: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:08.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:08 smithi118 ceph-mon[95692]: pgmap v1163: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:10.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:10 smithi196 ceph-mon[87080]: pgmap v1164: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:10.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:10 smithi118 ceph-mon[93121]: pgmap v1164: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:10.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:10 smithi118 ceph-mon[95692]: pgmap v1164: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:12.313 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:12 smithi196 ceph-mon[87080]: pgmap v1165: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:12.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:12 smithi118 ceph-mon[93121]: pgmap v1165: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:12.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:12 smithi118 ceph-mon[95692]: pgmap v1165: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:21:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:21:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:21:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:21:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:21:13] "GET /metrics HTTP/1.1" 200 36405 "" "Prometheus/2.43.0" 2024-03-20T08:21:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:14 smithi196 ceph-mon[87080]: pgmap v1166: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:14.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:14.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:14 smithi118 ceph-mon[93121]: pgmap v1166: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:14.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:14 smithi118 ceph-mon[95692]: pgmap v1166: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:14.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:14.997 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:21:15.242 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:21:16.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:16 smithi196 ceph-mon[87080]: pgmap v1167: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:16.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:16 smithi118 ceph-mon[93121]: pgmap v1167: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:16.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:16 smithi118 ceph-mon[95692]: pgmap v1167: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:17.212 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:21:17.214 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:21:17.856 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1168,"stamp":"2024-03-20T08:21:16.715111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002867"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:27.409776+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:16:27.409776+0000","last_peered":"2024-03-20T08:16:27.409776+0000","last_clean":"2024-03-20T08:16:27.409776+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:16:27.409776+0000","last_undegraded":"2024-03-20T08:16:27.409776+0000","last_fullsized":"2024-03-20T08:16:27.409776+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877644+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:16:44.877644+0000","last_peered":"2024-03-20T08:16:44.877644+0000","last_clean":"2024-03-20T08:16:44.877644+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:16:44.877644+0000","last_undegraded":"2024-03-20T08:16:44.877644+0000","last_fullsized":"2024-03-20T08:16:44.877644+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:44.877587+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:16:44.877587+0000","last_peered":"2024-03-20T08:16:44.877587+0000","last_clean":"2024-03-20T08:16:44.877587+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:16:44.877587+0000","last_undegraded":"2024-03-20T08:16:44.877587+0000","last_fullsized":"2024-03-20T08:16:44.877587+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.699342+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:16:54.699342+0000","last_peered":"2024-03-20T08:16:54.699342+0000","last_clean":"2024-03-20T08:16:54.699342+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:16:54.699342+0000","last_undegraded":"2024-03-20T08:16:54.699342+0000","last_fullsized":"2024-03-20T08:16:54.699342+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861907+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:17:07.861907+0000","last_peered":"2024-03-20T08:17:07.861907+0000","last_clean":"2024-03-20T08:17:07.861907+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:17:07.861907+0000","last_undegraded":"2024-03-20T08:17:07.861907+0000","last_fullsized":"2024-03-20T08:17:07.861907+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1191,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415833+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:17:17.415833+0000","last_peered":"2024-03-20T08:17:17.415833+0000","last_clean":"2024-03-20T08:17:17.415833+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:17:17.415833+0000","last_undegraded":"2024-03-20T08:17:17.415833+0000","last_fullsized":"2024-03-20T08:17:17.415833+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9297,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:59.824692+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:16:59.824692+0000","last_peered":"2024-03-20T08:16:59.824692+0000","last_clean":"2024-03-20T08:16:59.824692+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:16:59.824692+0000","last_undegraded":"2024-03-20T08:16:59.824692+0000","last_fullsized":"2024-03-20T08:16:59.824692+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49870,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.415790+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:17:17.415790+0000","last_peered":"2024-03-20T08:17:17.415790+0000","last_clean":"2024-03-20T08:17:17.415790+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:17:17.415790+0000","last_undegraded":"2024-03-20T08:17:17.415790+0000","last_fullsized":"2024-03-20T08:17:17.415790+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68666,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:16:54.699271+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:16:54.699271+0000","last_peered":"2024-03-20T08:16:54.699271+0000","last_clean":"2024-03-20T08:16:54.699271+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:16:54.699271+0000","last_undegraded":"2024-03-20T08:16:54.699271+0000","last_fullsized":"2024-03-20T08:16:54.699271+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.415752+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:17:17.415752+0000","last_peered":"2024-03-20T08:17:17.415752+0000","last_clean":"2024-03-20T08:17:17.415752+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:17:17.415752+0000","last_undegraded":"2024-03-20T08:17:17.415752+0000","last_fullsized":"2024-03-20T08:17:17.415752+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:07.861925+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:17:07.861925+0000","last_peered":"2024-03-20T08:17:07.861925+0000","last_clean":"2024-03-20T08:17:07.861925+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:17:07.861925+0000","last_undegraded":"2024-03-20T08:17:07.861925+0000","last_fullsized":"2024-03-20T08:17:07.861925+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1234,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.824634+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:16:59.824634+0000","last_peered":"2024-03-20T08:16:59.824634+0000","last_clean":"2024-03-20T08:16:59.824634+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:16:59.824634+0000","last_undegraded":"2024-03-20T08:16:59.824634+0000","last_fullsized":"2024-03-20T08:16:59.824634+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980235,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484628,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011639,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188991,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703471,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656937,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446881,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629276,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:21:17.862 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.862 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.862 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.862 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.862 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.862 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.862 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:17.863 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:21:18.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:18 smithi118 ceph-mon[93121]: pgmap v1168: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:18.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:18 smithi118 ceph-mon[93121]: from='client.34689 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:18.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:18 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:21:18.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:18 smithi118 ceph-mon[95692]: pgmap v1168: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:18.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:18 smithi118 ceph-mon[95692]: from='client.34689 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:18.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:18 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:21:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:18 smithi196 ceph-mon[87080]: pgmap v1168: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:18 smithi196 ceph-mon[87080]: from='client.34689 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:18.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:18 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:21:19.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:19 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:21:19.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:19 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:21:19.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:19 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:21:19.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:19 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:21:19.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:19 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:21:19.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:19 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:21:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:19 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:21:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:19 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:21:19.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:19 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:21:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:20 smithi196 ceph-mon[87080]: pgmap v1169: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:20 smithi118 ceph-mon[93121]: pgmap v1169: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:20 smithi118 ceph-mon[95692]: pgmap v1169: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:22 smithi196 ceph-mon[87080]: pgmap v1170: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:21:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:21:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:21:22.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:22 smithi118 ceph-mon[93121]: pgmap v1170: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:22.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:22 smithi118 ceph-mon[95692]: pgmap v1170: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:21:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:21:23] "GET /metrics HTTP/1.1" 200 36405 "" "Prometheus/2.43.0" 2024-03-20T08:21:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:24 smithi196 ceph-mon[87080]: pgmap v1171: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:24 smithi118 ceph-mon[93121]: pgmap v1171: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:24 smithi118 ceph-mon[95692]: pgmap v1171: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:26 smithi196 ceph-mon[87080]: pgmap v1172: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:26.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:26 smithi118 ceph-mon[93121]: pgmap v1172: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:26 smithi118 ceph-mon[95692]: pgmap v1172: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:27.864 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:21:28.077 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:21:28.461 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:28 smithi118 ceph-mon[93121]: pgmap v1173: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:28.461 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:28.461 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:28 smithi118 ceph-mon[95692]: pgmap v1173: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:28.461 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:28 smithi196 ceph-mon[87080]: pgmap v1173: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:30.084 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:21:30.087 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:21:30.414 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:30 smithi118 ceph-mon[93121]: pgmap v1174: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:30.415 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:30 smithi118 ceph-mon[95692]: pgmap v1174: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:30 smithi196 ceph-mon[87080]: pgmap v1174: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:30.736 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1174,"stamp":"2024-03-20T08:21:28.718193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003081"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980237,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484631,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011642,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188993,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703474,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656939,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446883,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629278,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:21:30.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.741 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:30.742 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:21:31.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:31 smithi196 ceph-mon[87080]: from='client.44615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:31.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:31 smithi118 ceph-mon[93121]: from='client.44615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:31.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:31 smithi118 ceph-mon[95692]: from='client.44615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:32 smithi196 ceph-mon[87080]: pgmap v1175: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:32.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:21:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:21:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:21:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:32 smithi118 ceph-mon[93121]: pgmap v1175: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:32 smithi118 ceph-mon[95692]: pgmap v1175: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:21:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:21:33] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:21:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:34 smithi196 ceph-mon[87080]: pgmap v1176: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:34 smithi118 ceph-mon[93121]: pgmap v1176: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:34 smithi118 ceph-mon[95692]: pgmap v1176: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:36 smithi196 ceph-mon[87080]: pgmap v1177: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:36 smithi118 ceph-mon[93121]: pgmap v1177: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:36 smithi118 ceph-mon[95692]: pgmap v1177: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:38 smithi196 ceph-mon[87080]: pgmap v1178: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:38 smithi118 ceph-mon[93121]: pgmap v1178: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:38 smithi118 ceph-mon[95692]: pgmap v1178: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:40 smithi196 ceph-mon[87080]: pgmap v1179: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:40 smithi118 ceph-mon[93121]: pgmap v1179: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:40 smithi118 ceph-mon[95692]: pgmap v1179: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:40.743 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:21:40.983 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:21:42.478 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:42 smithi118 ceph-mon[93121]: pgmap v1180: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:42.479 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:42 smithi118 ceph-mon[95692]: pgmap v1180: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:42 smithi196 ceph-mon[87080]: pgmap v1180: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:42.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:21:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:21:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:21:42.921 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:21:42.924 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:21:43.546 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:43.546 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:43.566 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1181,"stamp":"2024-03-20T08:21:42.722148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003139"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1125,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879547+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:16:59.879547+0000","last_peered":"2024-03-20T08:16:59.879547+0000","last_clean":"2024-03-20T08:16:59.879547+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:16:59.879547+0000","last_undegraded":"2024-03-20T08:16:59.879547+0000","last_fullsized":"2024-03-20T08:16:59.879547+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844233+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:16:54.844233+0000","last_peered":"2024-03-20T08:16:54.844233+0000","last_clean":"2024-03-20T08:16:54.844233+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:16:54.844233+0000","last_undegraded":"2024-03-20T08:16:54.844233+0000","last_fullsized":"2024-03-20T08:16:54.844233+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970453+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:17:05.970453+0000","last_peered":"2024-03-20T08:17:05.970453+0000","last_clean":"2024-03-20T08:17:05.970453+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:17:05.970453+0000","last_undegraded":"2024-03-20T08:17:05.970453+0000","last_fullsized":"2024-03-20T08:17:05.970453+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:42.860089+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:16:42.860089+0000","last_peered":"2024-03-20T08:16:42.860089+0000","last_clean":"2024-03-20T08:16:42.860089+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:16:42.860089+0000","last_undegraded":"2024-03-20T08:16:42.860089+0000","last_fullsized":"2024-03-20T08:16:42.860089+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844220+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:16:54.844220+0000","last_peered":"2024-03-20T08:16:54.844220+0000","last_clean":"2024-03-20T08:16:54.844220+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:16:54.844220+0000","last_undegraded":"2024-03-20T08:16:54.844220+0000","last_fullsized":"2024-03-20T08:16:54.844220+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:54.844192+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:16:54.844192+0000","last_peered":"2024-03-20T08:16:54.844192+0000","last_clean":"2024-03-20T08:16:54.844192+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:16:54.844192+0000","last_undegraded":"2024-03-20T08:16:54.844192+0000","last_fullsized":"2024-03-20T08:16:54.844192+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1140,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:16:59.879492+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:16:59.879492+0000","last_peered":"2024-03-20T08:16:59.879492+0000","last_clean":"2024-03-20T08:16:59.879492+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:16:59.879492+0000","last_undegraded":"2024-03-20T08:16:59.879492+0000","last_fullsized":"2024-03-20T08:16:59.879492+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1389,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970490+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:17:05.970490+0000","last_peered":"2024-03-20T08:17:05.970490+0000","last_clean":"2024-03-20T08:17:05.970490+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:17:05.970490+0000","last_undegraded":"2024-03-20T08:17:05.970490+0000","last_fullsized":"2024-03-20T08:17:05.970490+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:05.970473+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:17:05.970473+0000","last_peered":"2024-03-20T08:17:05.970473+0000","last_clean":"2024-03-20T08:17:05.970473+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:17:05.970473+0000","last_undegraded":"2024-03-20T08:17:05.970473+0000","last_fullsized":"2024-03-20T08:17:05.970473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26672,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863717+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:17:32.863717+0000","last_peered":"2024-03-20T08:17:32.863717+0000","last_clean":"2024-03-20T08:17:32.863717+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:17:32.863717+0000","last_undegraded":"2024-03-20T08:17:32.863717+0000","last_fullsized":"2024-03-20T08:17:32.863717+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17601,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863803+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:17:32.863803+0000","last_peered":"2024-03-20T08:17:32.863803+0000","last_clean":"2024-03-20T08:17:32.863803+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:17:32.863803+0000","last_undegraded":"2024-03-20T08:17:32.863803+0000","last_fullsized":"2024-03-20T08:17:32.863803+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.847869+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:17:39.847869+0000","last_peered":"2024-03-20T08:17:39.847869+0000","last_clean":"2024-03-20T08:17:39.847869+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:17:39.847869+0000","last_undegraded":"2024-03-20T08:17:39.847869+0000","last_fullsized":"2024-03-20T08:17:39.847869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1612,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884496+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:17:44.884496+0000","last_peered":"2024-03-20T08:17:44.884496+0000","last_clean":"2024-03-20T08:17:44.884496+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:17:44.884496+0000","last_undegraded":"2024-03-20T08:17:44.884496+0000","last_fullsized":"2024-03-20T08:17:44.884496+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:44.884473+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:17:44.884473+0000","last_peered":"2024-03-20T08:17:44.884473+0000","last_clean":"2024-03-20T08:17:44.884473+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:17:44.884473+0000","last_undegraded":"2024-03-20T08:17:44.884473+0000","last_fullsized":"2024-03-20T08:17:44.884473+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:32.863793+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:17:32.863793+0000","last_peered":"2024-03-20T08:17:32.863793+0000","last_clean":"2024-03-20T08:17:32.863793+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:17:32.863793+0000","last_undegraded":"2024-03-20T08:17:32.863793+0000","last_fullsized":"2024-03-20T08:17:32.863793+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5192,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:19.826822+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:17:19.826822+0000","last_peered":"2024-03-20T08:17:19.826822+0000","last_clean":"2024-03-20T08:17:19.826822+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:17:19.826822+0000","last_undegraded":"2024-03-20T08:17:19.826822+0000","last_fullsized":"2024-03-20T08:17:19.826822+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29910,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:17.283764+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:17:17.283764+0000","last_peered":"2024-03-20T08:17:17.283764+0000","last_clean":"2024-03-20T08:17:17.283764+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:17:17.283764+0000","last_undegraded":"2024-03-20T08:17:17.283764+0000","last_fullsized":"2024-03-20T08:17:17.283764+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5699,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:32.863755+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:17:32.863755+0000","last_peered":"2024-03-20T08:17:32.863755+0000","last_clean":"2024-03-20T08:17:32.863755+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:17:32.863755+0000","last_undegraded":"2024-03-20T08:17:32.863755+0000","last_fullsized":"2024-03-20T08:17:32.863755+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:17.283817+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:17:17.283817+0000","last_peered":"2024-03-20T08:17:17.283817+0000","last_clean":"2024-03-20T08:17:17.283817+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:17:17.283817+0000","last_undegraded":"2024-03-20T08:17:17.283817+0000","last_fullsized":"2024-03-20T08:17:17.283817+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826785+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:17:19.826785+0000","last_peered":"2024-03-20T08:17:19.826785+0000","last_clean":"2024-03-20T08:17:19.826785+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:17:19.826785+0000","last_undegraded":"2024-03-20T08:17:19.826785+0000","last_fullsized":"2024-03-20T08:17:19.826785+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:19.826752+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:17:19.826752+0000","last_peered":"2024-03-20T08:17:19.826752+0000","last_clean":"2024-03-20T08:17:19.826752+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:17:19.826752+0000","last_undegraded":"2024-03-20T08:17:19.826752+0000","last_fullsized":"2024-03-20T08:17:19.826752+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980240,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484634,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011645,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188996,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703476,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656942,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446886,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629281,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:21:43.572 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.572 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:43.573 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:21:43.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:21:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:21:43] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:21:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:44 smithi196 ceph-mon[87080]: pgmap v1181: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:44 smithi196 ceph-mon[87080]: from='client.44621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:44.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:44 smithi118 ceph-mon[93121]: pgmap v1181: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:44.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:44 smithi118 ceph-mon[93121]: from='client.44621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:44.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:44 smithi118 ceph-mon[95692]: pgmap v1181: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:44.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:44 smithi118 ceph-mon[95692]: from='client.44621 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:46 smithi196 ceph-mon[87080]: pgmap v1182: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:46 smithi118 ceph-mon[93121]: pgmap v1182: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:46 smithi118 ceph-mon[95692]: pgmap v1182: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:48.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:48 smithi196 ceph-mon[87080]: pgmap v1183: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:48.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:48 smithi118 ceph-mon[93121]: pgmap v1183: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:48 smithi118 ceph-mon[95692]: pgmap v1183: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:50 smithi196 ceph-mon[87080]: pgmap v1184: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:50 smithi118 ceph-mon[93121]: pgmap v1184: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:50.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:50 smithi118 ceph-mon[95692]: pgmap v1184: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:52 smithi196 ceph-mon[87080]: pgmap v1185: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:21:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:21:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:21:52.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:52 smithi118 ceph-mon[93121]: pgmap v1185: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:52.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:52 smithi118 ceph-mon[95692]: pgmap v1185: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:53.575 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:21:53.820 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:21:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:21:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:21:53] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:21:54.612 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:54 smithi118 ceph-mon[93121]: pgmap v1186: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:54.612 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:54 smithi118 ceph-mon[95692]: pgmap v1186: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:54 smithi196 ceph-mon[87080]: pgmap v1186: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:55.842 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:21:55.844 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:21:56.504 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:56 smithi118 ceph-mon[93121]: pgmap v1187: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:56.505 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:56 smithi118 ceph-mon[95692]: pgmap v1187: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:56.525 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1187,"stamp":"2024-03-20T08:21:54.725460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003311"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974123+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:17:55.974123+0000","last_peered":"2024-03-20T08:17:55.974123+0000","last_clean":"2024-03-20T08:17:55.974123+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:17:55.974123+0000","last_undegraded":"2024-03-20T08:17:55.974123+0000","last_fullsized":"2024-03-20T08:17:55.974123+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1394,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974170+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:17:55.974170+0000","last_peered":"2024-03-20T08:17:55.974170+0000","last_clean":"2024-03-20T08:17:55.974170+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:17:55.974170+0000","last_undegraded":"2024-03-20T08:17:55.974170+0000","last_fullsized":"2024-03-20T08:17:55.974170+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974153+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:17:55.974153+0000","last_peered":"2024-03-20T08:17:55.974153+0000","last_clean":"2024-03-20T08:17:55.974153+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:17:55.974153+0000","last_undegraded":"2024-03-20T08:17:55.974153+0000","last_fullsized":"2024-03-20T08:17:55.974153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.701190+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:17:14.701190+0000","last_peered":"2024-03-20T08:17:14.701190+0000","last_clean":"2024-03-20T08:17:14.701190+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:17:14.701190+0000","last_undegraded":"2024-03-20T08:17:14.701190+0000","last_fullsized":"2024-03-20T08:17:14.701190+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11690,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974090+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:17:55.974090+0000","last_peered":"2024-03-20T08:17:55.974090+0000","last_clean":"2024-03-20T08:17:55.974090+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:17:55.974090+0000","last_undegraded":"2024-03-20T08:17:55.974090+0000","last_fullsized":"2024-03-20T08:17:55.974090+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5952,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:55.974130+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:17:55.974130+0000","last_peered":"2024-03-20T08:17:55.974130+0000","last_clean":"2024-03-20T08:17:55.974130+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:17:55.974130+0000","last_undegraded":"2024-03-20T08:17:55.974130+0000","last_fullsized":"2024-03-20T08:17:55.974130+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:02.287344+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:18:02.287344+0000","last_peered":"2024-03-20T08:18:02.287344+0000","last_clean":"2024-03-20T08:18:02.287344+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:18:02.287344+0000","last_undegraded":"2024-03-20T08:18:02.287344+0000","last_fullsized":"2024-03-20T08:18:02.287344+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974059+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:17:55.974059+0000","last_peered":"2024-03-20T08:17:55.974059+0000","last_clean":"2024-03-20T08:17:55.974059+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:17:55.974059+0000","last_undegraded":"2024-03-20T08:17:55.974059+0000","last_fullsized":"2024-03-20T08:17:55.974059+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974106+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:17:55.974106+0000","last_peered":"2024-03-20T08:17:55.974106+0000","last_clean":"2024-03-20T08:17:55.974106+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:17:55.974106+0000","last_undegraded":"2024-03-20T08:17:55.974106+0000","last_fullsized":"2024-03-20T08:17:55.974106+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1174,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418121+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:17:37.418121+0000","last_peered":"2024-03-20T08:17:37.418121+0000","last_clean":"2024-03-20T08:17:37.418121+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:17:37.418121+0000","last_undegraded":"2024-03-20T08:17:37.418121+0000","last_fullsized":"2024-03-20T08:17:37.418121+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418069+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:17:37.418069+0000","last_peered":"2024-03-20T08:17:37.418069+0000","last_clean":"2024-03-20T08:17:37.418069+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:17:37.418069+0000","last_undegraded":"2024-03-20T08:17:37.418069+0000","last_fullsized":"2024-03-20T08:17:37.418069+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974081+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:17:55.974081+0000","last_peered":"2024-03-20T08:17:55.974081+0000","last_clean":"2024-03-20T08:17:55.974081+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:17:55.974081+0000","last_undegraded":"2024-03-20T08:17:55.974081+0000","last_fullsized":"2024-03-20T08:17:55.974081+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:37.418111+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:17:37.418111+0000","last_peered":"2024-03-20T08:17:37.418111+0000","last_clean":"2024-03-20T08:17:37.418111+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:17:37.418111+0000","last_undegraded":"2024-03-20T08:17:37.418111+0000","last_fullsized":"2024-03-20T08:17:37.418111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980242,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484636,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011647,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490188998,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703479,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656944,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446888,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629283,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.531 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.532 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.532 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.532 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.532 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:21:56.532 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:21:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:56 smithi196 ceph-mon[87080]: pgmap v1187: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:57.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:57 smithi118 ceph-mon[93121]: from='client.44627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:57.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:57 smithi118 ceph-mon[95692]: from='client.44627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:57.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:57 smithi196 ceph-mon[87080]: from='client.44627 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:21:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:58 smithi196 ceph-mon[87080]: pgmap v1188: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:58.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:58 smithi118 ceph-mon[93121]: pgmap v1188: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:58.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:58.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:58 smithi118 ceph-mon[95692]: pgmap v1188: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:58.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:21:59.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:21:59 smithi118 ceph-mon[93121]: pgmap v1189: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:59.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:21:59 smithi118 ceph-mon[95692]: pgmap v1189: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:21:59.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:21:59 smithi196 ceph-mon[87080]: pgmap v1189: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:02.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:01 smithi196 ceph-mon[87080]: pgmap v1190: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:02.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:01 smithi118 ceph-mon[93121]: pgmap v1190: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:02.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:01 smithi118 ceph-mon[95692]: pgmap v1190: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:02.639 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:22:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=cleanup t=2024-03-20T08:22:02.497554548Z level=info msg="Completed cleanup jobs" duration=3.943696ms 2024-03-20T08:22:02.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:22:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:22:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:22:03.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:03 smithi118 ceph-mon[93121]: pgmap v1191: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:03.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:03 smithi118 ceph-mon[95692]: pgmap v1191: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:03.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:22:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:22:03] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:22:04.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:03 smithi196 ceph-mon[87080]: pgmap v1191: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:06.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:05 smithi196 ceph-mon[87080]: pgmap v1192: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:06.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:05 smithi118 ceph-mon[93121]: pgmap v1192: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:06.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:05 smithi118 ceph-mon[95692]: pgmap v1192: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:06.532 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:22:06.748 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:22:08.079 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:07 smithi118 ceph-mon[93121]: pgmap v1193: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:08.079 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:07 smithi118 ceph-mon[95692]: pgmap v1193: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:08.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:07 smithi196 ceph-mon[87080]: pgmap v1193: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:08.899 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:22:08.901 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:22:09.532 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1194,"stamp":"2024-03-20T08:22:08.729179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003273"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887112+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:18:04.887112+0000","last_peered":"2024-03-20T08:18:04.887112+0000","last_clean":"2024-03-20T08:18:04.887112+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:18:04.887112+0000","last_undegraded":"2024-03-20T08:18:04.887112+0000","last_fullsized":"2024-03-20T08:18:04.887112+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833192+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:18:19.833192+0000","last_peered":"2024-03-20T08:18:19.833192+0000","last_clean":"2024-03-20T08:18:19.833192+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:18:19.833192+0000","last_undegraded":"2024-03-20T08:18:19.833192+0000","last_fullsized":"2024-03-20T08:18:19.833192+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881148+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:17:14.881148+0000","last_peered":"2024-03-20T08:17:14.881148+0000","last_clean":"2024-03-20T08:17:14.881148+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:17:14.881148+0000","last_undegraded":"2024-03-20T08:17:14.881148+0000","last_fullsized":"2024-03-20T08:17:14.881148+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881114+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:17:14.881114+0000","last_peered":"2024-03-20T08:17:14.881114+0000","last_clean":"2024-03-20T08:17:14.881114+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:17:14.881114+0000","last_undegraded":"2024-03-20T08:17:14.881114+0000","last_fullsized":"2024-03-20T08:17:14.881114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.845976+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:17:14.845976+0000","last_peered":"2024-03-20T08:17:14.845976+0000","last_clean":"2024-03-20T08:17:14.845976+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:17:14.845976+0000","last_undegraded":"2024-03-20T08:17:14.845976+0000","last_fullsized":"2024-03-20T08:17:14.845976+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828871+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:17:39.828871+0000","last_peered":"2024-03-20T08:17:39.828871+0000","last_clean":"2024-03-20T08:17:39.828871+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:17:39.828871+0000","last_undegraded":"2024-03-20T08:17:39.828871+0000","last_fullsized":"2024-03-20T08:17:39.828871+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828987+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:17:39.828987+0000","last_peered":"2024-03-20T08:17:39.828987+0000","last_clean":"2024-03-20T08:17:39.828987+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:17:39.828987+0000","last_undegraded":"2024-03-20T08:17:39.828987+0000","last_fullsized":"2024-03-20T08:17:39.828987+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1122,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881179+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:17:14.881179+0000","last_peered":"2024-03-20T08:17:14.881179+0000","last_clean":"2024-03-20T08:17:14.881179+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:17:14.881179+0000","last_undegraded":"2024-03-20T08:17:14.881179+0000","last_fullsized":"2024-03-20T08:17:14.881179+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1130,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.881161+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:17:14.881161+0000","last_peered":"2024-03-20T08:17:14.881161+0000","last_clean":"2024-03-20T08:17:14.881161+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:17:14.881161+0000","last_undegraded":"2024-03-20T08:17:14.881161+0000","last_fullsized":"2024-03-20T08:17:14.881161+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:14.846083+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:17:14.846083+0000","last_peered":"2024-03-20T08:17:14.846083+0000","last_clean":"2024-03-20T08:17:14.846083+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:17:14.846083+0000","last_undegraded":"2024-03-20T08:17:14.846083+0000","last_fullsized":"2024-03-20T08:17:14.846083+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:39.828930+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:17:39.828930+0000","last_peered":"2024-03-20T08:17:39.828930+0000","last_clean":"2024-03-20T08:17:39.828930+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:17:39.828930+0000","last_undegraded":"2024-03-20T08:17:39.828930+0000","last_fullsized":"2024-03-20T08:17:39.828930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3980,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:04.887031+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:18:04.887031+0000","last_peered":"2024-03-20T08:18:04.887031+0000","last_clean":"2024-03-20T08:18:04.887031+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:18:04.887031+0000","last_undegraded":"2024-03-20T08:18:04.887031+0000","last_fullsized":"2024-03-20T08:18:04.887031+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.887092+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:18:04.887092+0000","last_peered":"2024-03-20T08:18:04.887092+0000","last_clean":"2024-03-20T08:18:04.887092+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:18:04.887092+0000","last_undegraded":"2024-03-20T08:18:04.887092+0000","last_fullsized":"2024-03-20T08:18:04.887092+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980245,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484639,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011650,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189001,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703482,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656947,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446891,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629286,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.538 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.539 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.539 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.539 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.539 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:09.539 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:22:10.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:09 smithi196 ceph-mon[87080]: pgmap v1194: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:10.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:09 smithi196 ceph-mon[87080]: from='client.34713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:10.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:09 smithi118 ceph-mon[93121]: pgmap v1194: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:10.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:09 smithi118 ceph-mon[93121]: from='client.34713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:09 smithi118 ceph-mon[95692]: pgmap v1194: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:10.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:09 smithi118 ceph-mon[95692]: from='client.34713 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:12.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:11 smithi196 ceph-mon[87080]: pgmap v1195: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:12.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:11 smithi118 ceph-mon[93121]: pgmap v1195: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:12.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:11 smithi118 ceph-mon[95692]: pgmap v1195: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:22:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:22:12.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:22:13.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:13 smithi118 ceph-mon[93121]: pgmap v1196: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:13.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:13.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:22:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:22:13] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:22:13.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:13 smithi118 ceph-mon[95692]: pgmap v1196: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:13.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:14.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:13 smithi196 ceph-mon[87080]: pgmap v1196: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:14.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:16.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:15 smithi196 ceph-mon[87080]: pgmap v1197: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:16.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:15 smithi118 ceph-mon[93121]: pgmap v1197: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:16.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:15 smithi118 ceph-mon[95692]: pgmap v1197: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:18.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:17 smithi196 ceph-mon[87080]: pgmap v1198: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:18.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:17 smithi118 ceph-mon[93121]: pgmap v1198: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:18.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:17 smithi118 ceph-mon[95692]: pgmap v1198: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:19.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:18 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:22:19.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:18 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:22:19.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:18 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:22:19.540 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:22:19.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:19 smithi118 ceph-mon[93121]: pgmap v1199: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:19.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:19 smithi118 ceph-mon[95692]: pgmap v1199: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:19.990 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:22:20.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:19 smithi196 ceph-mon[87080]: pgmap v1199: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:21.919 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:22:21.921 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:22:22.064 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:21 smithi118 ceph-mon[93121]: pgmap v1200: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:22.065 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:21 smithi118 ceph-mon[95692]: pgmap v1200: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:22.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:21 smithi196 ceph-mon[87080]: pgmap v1200: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:22.579 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1200,"stamp":"2024-03-20T08:22:20.732082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002903"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:34.703383+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:17:34.703383+0000","last_peered":"2024-03-20T08:17:34.703383+0000","last_clean":"2024-03-20T08:17:34.703383+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:17:34.703383+0000","last_undegraded":"2024-03-20T08:17:34.703383+0000","last_fullsized":"2024-03-20T08:17:34.703383+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:42.285837+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:17:42.285837+0000","last_peered":"2024-03-20T08:17:42.285837+0000","last_clean":"2024-03-20T08:17:42.285837+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:17:42.285837+0000","last_undegraded":"2024-03-20T08:17:42.285837+0000","last_fullsized":"2024-03-20T08:17:42.285837+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:30.972333+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:17:30.972333+0000","last_peered":"2024-03-20T08:17:30.972333+0000","last_clean":"2024-03-20T08:17:30.972333+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:17:30.972333+0000","last_undegraded":"2024-03-20T08:17:30.972333+0000","last_fullsized":"2024-03-20T08:17:30.972333+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708612+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:18:19.708612+0000","last_peered":"2024-03-20T08:18:19.708612+0000","last_clean":"2024-03-20T08:18:19.708612+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:18:19.708612+0000","last_undegraded":"2024-03-20T08:18:19.708612+0000","last_fullsized":"2024-03-20T08:18:19.708612+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4606,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:19.708577+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:18:19.708577+0000","last_peered":"2024-03-20T08:18:19.708577+0000","last_clean":"2024-03-20T08:18:19.708577+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:18:19.708577+0000","last_undegraded":"2024-03-20T08:18:19.708577+0000","last_fullsized":"2024-03-20T08:18:19.708577+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849977+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:18:04.849977+0000","last_peered":"2024-03-20T08:18:04.849977+0000","last_clean":"2024-03-20T08:18:04.849977+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:18:04.849977+0000","last_undegraded":"2024-03-20T08:18:04.849977+0000","last_fullsized":"2024-03-20T08:18:04.849977+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1217,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.708556+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:18:19.708556+0000","last_peered":"2024-03-20T08:18:19.708556+0000","last_clean":"2024-03-20T08:18:19.708556+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:18:19.708556+0000","last_undegraded":"2024-03-20T08:18:19.708556+0000","last_fullsized":"2024-03-20T08:18:19.708556+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849917+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:18:04.849917+0000","last_peered":"2024-03-20T08:18:04.849917+0000","last_clean":"2024-03-20T08:18:04.849917+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:18:04.849917+0000","last_undegraded":"2024-03-20T08:18:04.849917+0000","last_fullsized":"2024-03-20T08:18:04.849917+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31733,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:17:57.865838+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:17:57.865838+0000","last_peered":"2024-03-20T08:17:57.865838+0000","last_clean":"2024-03-20T08:17:57.865838+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:17:57.865838+0000","last_undegraded":"2024-03-20T08:17:57.865838+0000","last_fullsized":"2024-03-20T08:17:57.865838+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865820+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:17:57.865820+0000","last_peered":"2024-03-20T08:17:57.865820+0000","last_clean":"2024-03-20T08:17:57.865820+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:17:57.865820+0000","last_undegraded":"2024-03-20T08:17:57.865820+0000","last_fullsized":"2024-03-20T08:17:57.865820+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3378,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865790+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:17:57.865790+0000","last_peered":"2024-03-20T08:17:57.865790+0000","last_clean":"2024-03-20T08:17:57.865790+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:17:57.865790+0000","last_undegraded":"2024-03-20T08:17:57.865790+0000","last_fullsized":"2024-03-20T08:17:57.865790+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865829+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:17:57.865829+0000","last_peered":"2024-03-20T08:17:57.865829+0000","last_clean":"2024-03-20T08:17:57.865829+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:17:57.865829+0000","last_undegraded":"2024-03-20T08:17:57.865829+0000","last_fullsized":"2024-03-20T08:17:57.865829+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1307,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.865864+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:17:57.865864+0000","last_peered":"2024-03-20T08:17:57.865864+0000","last_clean":"2024-03-20T08:17:57.865864+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:17:57.865864+0000","last_undegraded":"2024-03-20T08:17:57.865864+0000","last_fullsized":"2024-03-20T08:17:57.865864+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849957+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:18:04.849957+0000","last_peered":"2024-03-20T08:18:04.849957+0000","last_clean":"2024-03-20T08:18:04.849957+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:18:04.849957+0000","last_undegraded":"2024-03-20T08:18:04.849957+0000","last_fullsized":"2024-03-20T08:18:04.849957+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1278,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:04.849991+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:18:04.849991+0000","last_peered":"2024-03-20T08:18:04.849991+0000","last_clean":"2024-03-20T08:18:04.849991+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:18:04.849991+0000","last_undegraded":"2024-03-20T08:18:04.849991+0000","last_fullsized":"2024-03-20T08:18:04.849991+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:57.420300+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:17:57.420300+0000","last_peered":"2024-03-20T08:17:57.420300+0000","last_clean":"2024-03-20T08:17:57.420300+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:17:57.420300+0000","last_undegraded":"2024-03-20T08:17:57.420300+0000","last_fullsized":"2024-03-20T08:17:57.420300+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980248,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484641,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011652,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189003,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703484,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656950,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446894,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629289,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.585 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:22.586 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:22:22.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:22:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:22:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:22:23.372 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:23 smithi118 ceph-mon[93121]: from='client.44639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:23.373 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:23.373 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:23 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:23.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:23 smithi118 ceph-mon[95692]: from='client.44639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:23.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:23.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:23 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:23 smithi196 ceph-mon[87080]: from='client.44639 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:23.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:23 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:23.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:22:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:22:23] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:22:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:24 smithi196 ceph-mon[87080]: pgmap v1201: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:24.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:24 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:24.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:24 smithi118 ceph-mon[93121]: pgmap v1201: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:24 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:24 smithi118 ceph-mon[95692]: pgmap v1201: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:24 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:22:25.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:22:25.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:25.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:22:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:22:25.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:25.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:22:25.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:22:25.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:22:26.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:26 smithi196 ceph-mon[87080]: pgmap v1202: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:26.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:26 smithi118 ceph-mon[93121]: pgmap v1202: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:26.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:26 smithi118 ceph-mon[95692]: pgmap v1202: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:28.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:28 smithi118 ceph-mon[93121]: pgmap v1203: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:28.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:28 smithi118 ceph-mon[95692]: pgmap v1203: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:28.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:28 smithi196 ceph-mon[87080]: pgmap v1203: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:29.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:29.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:29.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:29 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:30.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:30 smithi196 ceph-mon[87080]: pgmap v1204: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:30.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:30 smithi118 ceph-mon[93121]: pgmap v1204: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:30.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:30 smithi118 ceph-mon[95692]: pgmap v1204: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:32.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:32 smithi118 ceph-mon[93121]: pgmap v1205: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:32.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:32 smithi118 ceph-mon[95692]: pgmap v1205: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:32.586 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:22:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:32 smithi196 ceph-mon[87080]: pgmap v1205: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:32.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:22:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:22:32.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:22:32.797 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:22:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:22:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:22:33] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:22:34.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:34 smithi118 ceph-mon[93121]: pgmap v1206: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:34.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:34 smithi118 ceph-mon[95692]: pgmap v1206: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:34 smithi196 ceph-mon[87080]: pgmap v1206: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:34.895 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:22:34.897 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:22:35.562 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1207,"stamp":"2024-03-20T08:22:34.735492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002855"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289827+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:18:32.289827+0000","last_peered":"2024-03-20T08:18:32.289827+0000","last_clean":"2024-03-20T08:18:32.289827+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:18:32.289827+0000","last_undegraded":"2024-03-20T08:18:32.289827+0000","last_fullsized":"2024-03-20T08:18:32.289827+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289769+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:18:32.289769+0000","last_peered":"2024-03-20T08:18:32.289769+0000","last_clean":"2024-03-20T08:18:32.289769+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:18:32.289769+0000","last_undegraded":"2024-03-20T08:18:32.289769+0000","last_fullsized":"2024-03-20T08:18:32.289769+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289800+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:18:32.289800+0000","last_peered":"2024-03-20T08:18:32.289800+0000","last_clean":"2024-03-20T08:18:32.289800+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:18:32.289800+0000","last_undegraded":"2024-03-20T08:18:32.289800+0000","last_fullsized":"2024-03-20T08:18:32.289800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289809+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:18:32.289809+0000","last_peered":"2024-03-20T08:18:32.289809+0000","last_clean":"2024-03-20T08:18:32.289809+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:18:32.289809+0000","last_undegraded":"2024-03-20T08:18:32.289809+0000","last_fullsized":"2024-03-20T08:18:32.289809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:25.976329+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:18:25.976329+0000","last_peered":"2024-03-20T08:18:25.976329+0000","last_clean":"2024-03-20T08:18:25.976329+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:18:25.976329+0000","last_undegraded":"2024-03-20T08:18:25.976329+0000","last_fullsized":"2024-03-20T08:18:25.976329+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:32.289816+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:18:32.289816+0000","last_peered":"2024-03-20T08:18:32.289816+0000","last_clean":"2024-03-20T08:18:32.289816+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:18:32.289816+0000","last_undegraded":"2024-03-20T08:18:32.289816+0000","last_fullsized":"2024-03-20T08:18:32.289816+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1116,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:24.889167+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:18:24.889167+0000","last_peered":"2024-03-20T08:18:24.889167+0000","last_clean":"2024-03-20T08:18:24.889167+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:18:24.889167+0000","last_undegraded":"2024-03-20T08:18:24.889167+0000","last_fullsized":"2024-03-20T08:18:24.889167+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4959,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:18:24.889216+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:18:24.889216+0000","last_peered":"2024-03-20T08:18:24.889216+0000","last_clean":"2024-03-20T08:18:24.889216+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:18:24.889216+0000","last_undegraded":"2024-03-20T08:18:24.889216+0000","last_fullsized":"2024-03-20T08:18:24.889216+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980250,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484644,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011655,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189006,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703487,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656952,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446896,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629291,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.568 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.569 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.569 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.569 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.569 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.569 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.569 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:35.569 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:22:36.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:36 smithi118 ceph-mon[93121]: pgmap v1207: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:36.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:36 smithi118 ceph-mon[93121]: from='client.44645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:36.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:36 smithi118 ceph-mon[95692]: pgmap v1207: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:36.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:36 smithi118 ceph-mon[95692]: from='client.44645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:36 smithi196 ceph-mon[87080]: pgmap v1207: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:36 smithi196 ceph-mon[87080]: from='client.44645 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:38.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:38 smithi118 ceph-mon[93121]: pgmap v1208: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:38.424 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:38 smithi118 ceph-mon[95692]: pgmap v1208: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:38 smithi196 ceph-mon[87080]: pgmap v1208: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:40.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:40 smithi118 ceph-mon[93121]: pgmap v1209: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:40.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:40 smithi118 ceph-mon[95692]: pgmap v1209: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:40 smithi196 ceph-mon[87080]: pgmap v1209: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:42.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:42 smithi118 ceph-mon[93121]: pgmap v1210: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:42.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:42 smithi118 ceph-mon[95692]: pgmap v1210: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:42 smithi196 ceph-mon[87080]: pgmap v1210: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:42.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:22:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:22:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:22:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:22:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:22:43] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:22:44.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:44 smithi118 ceph-mon[93121]: pgmap v1211: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:44.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:44 smithi118 ceph-mon[95692]: pgmap v1211: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:44.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:44 smithi196 ceph-mon[87080]: pgmap v1211: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:45.571 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:22:45.795 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:22:46.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:46 smithi118 ceph-mon[93121]: pgmap v1212: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:46.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:46 smithi118 ceph-mon[95692]: pgmap v1212: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:46 smithi196 ceph-mon[87080]: pgmap v1212: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:47.778 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:22:47.781 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:22:48.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:48 smithi118 ceph-mon[93121]: pgmap v1213: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:48.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:48 smithi118 ceph-mon[95692]: pgmap v1213: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:48.494 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1213,"stamp":"2024-03-20T08:22:46.738324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002832"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1256,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:54.705550+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:17:54.705550+0000","last_peered":"2024-03-20T08:17:54.705550+0000","last_clean":"2024-03-20T08:17:54.705550+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:17:54.705550+0000","last_undegraded":"2024-03-20T08:17:54.705550+0000","last_fullsized":"2024-03-20T08:17:54.705550+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1384,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:55.974114+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:17:55.974114+0000","last_peered":"2024-03-20T08:17:55.974114+0000","last_clean":"2024-03-20T08:17:55.974114+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:17:55.974114+0000","last_undegraded":"2024-03-20T08:17:55.974114+0000","last_fullsized":"2024-03-20T08:17:55.974114+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831046+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:17:59.831046+0000","last_peered":"2024-03-20T08:17:59.831046+0000","last_clean":"2024-03-20T08:17:59.831046+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:17:59.831046+0000","last_undegraded":"2024-03-20T08:17:59.831046+0000","last_fullsized":"2024-03-20T08:17:59.831046+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1342,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287805+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:18:07.287805+0000","last_peered":"2024-03-20T08:18:07.287805+0000","last_clean":"2024-03-20T08:18:07.287805+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:18:07.287805+0000","last_undegraded":"2024-03-20T08:18:07.287805+0000","last_fullsized":"2024-03-20T08:18:07.287805+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831103+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:17:59.831103+0000","last_peered":"2024-03-20T08:17:59.831103+0000","last_clean":"2024-03-20T08:17:59.831103+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:17:59.831103+0000","last_undegraded":"2024-03-20T08:17:59.831103+0000","last_fullsized":"2024-03-20T08:17:59.831103+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1196,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:17:59.831093+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:17:59.831093+0000","last_peered":"2024-03-20T08:17:59.831093+0000","last_clean":"2024-03-20T08:17:59.831093+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:17:59.831093+0000","last_undegraded":"2024-03-20T08:17:59.831093+0000","last_fullsized":"2024-03-20T08:17:59.831093+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:07.287870+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:18:07.287870+0000","last_peered":"2024-03-20T08:18:07.287870+0000","last_clean":"2024-03-20T08:18:07.287870+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:18:07.287870+0000","last_undegraded":"2024-03-20T08:18:07.287870+0000","last_fullsized":"2024-03-20T08:18:07.287870+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1222,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422362+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:18:17.422362+0000","last_peered":"2024-03-20T08:18:17.422362+0000","last_clean":"2024-03-20T08:18:17.422362+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:18:17.422362+0000","last_undegraded":"2024-03-20T08:18:17.422362+0000","last_fullsized":"2024-03-20T08:18:17.422362+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422311+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:18:17.422311+0000","last_peered":"2024-03-20T08:18:17.422311+0000","last_clean":"2024-03-20T08:18:17.422311+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:18:17.422311+0000","last_undegraded":"2024-03-20T08:18:17.422311+0000","last_fullsized":"2024-03-20T08:18:17.422311+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:17.422278+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:18:17.422278+0000","last_peered":"2024-03-20T08:18:17.422278+0000","last_clean":"2024-03-20T08:18:17.422278+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:18:17.422278+0000","last_undegraded":"2024-03-20T08:18:17.422278+0000","last_fullsized":"2024-03-20T08:18:17.422278+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836153+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:18:44.836153+0000","last_peered":"2024-03-20T08:18:44.836153+0000","last_clean":"2024-03-20T08:18:44.836153+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:18:44.836153+0000","last_undegraded":"2024-03-20T08:18:44.836153+0000","last_fullsized":"2024-03-20T08:18:44.836153+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836102+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:18:44.836102+0000","last_peered":"2024-03-20T08:18:44.836102+0000","last_clean":"2024-03-20T08:18:44.836102+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:18:44.836102+0000","last_undegraded":"2024-03-20T08:18:44.836102+0000","last_fullsized":"2024-03-20T08:18:44.836102+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836136+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:18:44.836136+0000","last_peered":"2024-03-20T08:18:44.836136+0000","last_clean":"2024-03-20T08:18:44.836136+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:18:44.836136+0000","last_undegraded":"2024-03-20T08:18:44.836136+0000","last_fullsized":"2024-03-20T08:18:44.836136+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1285,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851894+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:18:29.851894+0000","last_peered":"2024-03-20T08:18:29.851894+0000","last_clean":"2024-03-20T08:18:29.851894+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:18:29.851894+0000","last_undegraded":"2024-03-20T08:18:29.851894+0000","last_fullsized":"2024-03-20T08:18:29.851894+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.836125+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:18:44.836125+0000","last_peered":"2024-03-20T08:18:44.836125+0000","last_clean":"2024-03-20T08:18:44.836125+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:18:44.836125+0000","last_undegraded":"2024-03-20T08:18:44.836125+0000","last_fullsized":"2024-03-20T08:18:44.836125+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:29.851860+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:18:29.851860+0000","last_peered":"2024-03-20T08:18:29.851860+0000","last_clean":"2024-03-20T08:18:29.851860+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:18:29.851860+0000","last_undegraded":"2024-03-20T08:18:29.851860+0000","last_fullsized":"2024-03-20T08:18:29.851860+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980253,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484646,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011657,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189009,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703489,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656955,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446899,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629294,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:22:48.500 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.501 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.502 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.502 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.502 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:22:48.502 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:22:48.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:48 smithi196 ceph-mon[87080]: pgmap v1213: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:49.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:49 smithi118 ceph-mon[93121]: from='client.34731 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:49.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:49 smithi118 ceph-mon[95692]: from='client.34731 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:49.541 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:49 smithi196 ceph-mon[87080]: from='client.34731 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:22:50.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:50 smithi118 ceph-mon[93121]: pgmap v1214: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:50.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:50 smithi118 ceph-mon[95692]: pgmap v1214: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:50.554 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:50 smithi196 ceph-mon[87080]: pgmap v1214: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:52.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:52 smithi118 ceph-mon[93121]: pgmap v1215: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:52.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:52 smithi118 ceph-mon[95692]: pgmap v1215: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:52 smithi196 ceph-mon[87080]: pgmap v1215: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:52.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:22:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:22:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:22:53.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:22:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:22:53] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:22:54.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:54 smithi118 ceph-mon[93121]: pgmap v1216: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:54.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:54 smithi118 ceph-mon[95692]: pgmap v1216: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:54 smithi196 ceph-mon[87080]: pgmap v1216: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:56.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:56 smithi118 ceph-mon[93121]: pgmap v1217: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:56.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:56 smithi118 ceph-mon[95692]: pgmap v1217: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:56 smithi196 ceph-mon[87080]: pgmap v1217: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:58.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:58 smithi118 ceph-mon[93121]: pgmap v1218: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:58.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:58 smithi118 ceph-mon[95692]: pgmap v1218: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:58.503 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:22:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:58 smithi196 ceph-mon[87080]: pgmap v1218: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:22:58.732 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:22:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:22:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:59.665 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:22:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:22:59.665 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:22:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:00.373 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:00 smithi118 ceph-mon[95692]: pgmap v1219: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:00.373 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:00 smithi118 ceph-mon[93121]: pgmap v1219: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:00 smithi196 ceph-mon[87080]: pgmap v1219: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:00.844 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:23:00.847 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:23:01.580 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1220,"stamp":"2024-03-20T08:23:00.742009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003092"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1302,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867658+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:18:22.867658+0000","last_peered":"2024-03-20T08:18:22.867658+0000","last_clean":"2024-03-20T08:18:22.867658+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:18:22.867658+0000","last_undegraded":"2024-03-20T08:18:22.867658+0000","last_fullsized":"2024-03-20T08:18:22.867658+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1318,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:22.867615+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:18:22.867615+0000","last_peered":"2024-03-20T08:18:22.867615+0000","last_clean":"2024-03-20T08:18:22.867615+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:18:22.867615+0000","last_undegraded":"2024-03-20T08:18:22.867615+0000","last_fullsized":"2024-03-20T08:18:22.867615+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1252,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.711327+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:18:44.711327+0000","last_peered":"2024-03-20T08:18:44.711327+0000","last_clean":"2024-03-20T08:18:44.711327+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:18:44.711327+0000","last_undegraded":"2024-03-20T08:18:44.711327+0000","last_fullsized":"2024-03-20T08:18:44.711327+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980256,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484649,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011660,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189011,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703492,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656958,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446902,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629297,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.586 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.587 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.587 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:01.587 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:23:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:02 smithi196 ceph-mon[87080]: pgmap v1220: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:02 smithi196 ceph-mon[87080]: from='client.34737 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:02.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:23:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:23:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:23:02.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:02 smithi118 ceph-mon[93121]: pgmap v1220: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:02.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:02 smithi118 ceph-mon[93121]: from='client.34737 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:02.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:02 smithi118 ceph-mon[95692]: pgmap v1220: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:02.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:02 smithi118 ceph-mon[95692]: from='client.34737 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:03.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:23:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:23:03] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:23:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:04 smithi196 ceph-mon[87080]: pgmap v1221: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:04 smithi118 ceph-mon[93121]: pgmap v1221: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:04 smithi118 ceph-mon[95692]: pgmap v1221: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:06 smithi196 ceph-mon[87080]: pgmap v1222: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:06.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:06 smithi118 ceph-mon[95692]: pgmap v1222: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:06 smithi118 ceph-mon[93121]: pgmap v1222: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:08 smithi196 ceph-mon[87080]: pgmap v1223: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:08 smithi118 ceph-mon[93121]: pgmap v1223: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:08 smithi118 ceph-mon[95692]: pgmap v1223: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:10 smithi196 ceph-mon[87080]: pgmap v1224: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:10.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:10 smithi118 ceph-mon[93121]: pgmap v1224: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:10.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:10 smithi118 ceph-mon[95692]: pgmap v1224: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:11.587 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:23:11.796 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:23:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:12 smithi196 ceph-mon[87080]: pgmap v1225: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:12.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:23:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:23:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:23:12.640 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:12 smithi118 ceph-mon[93121]: pgmap v1225: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:12.640 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:12 smithi118 ceph-mon[95692]: pgmap v1225: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:13.822 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:23:13.824 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:23:13.836 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:23:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:23:13] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:23:14.591 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:14 smithi118 ceph-mon[93121]: pgmap v1226: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:14.591 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:14.591 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:14 smithi118 ceph-mon[95692]: pgmap v1226: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:14.591 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:14.614 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1226,"stamp":"2024-03-20T08:23:12.745191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003181"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833162+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:18:19.833162+0000","last_peered":"2024-03-20T08:18:19.833162+0000","last_clean":"2024-03-20T08:18:19.833162+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:18:19.833162+0000","last_undegraded":"2024-03-20T08:18:19.833162+0000","last_fullsized":"2024-03-20T08:18:19.833162+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891597+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:18:44.891597+0000","last_peered":"2024-03-20T08:18:44.891597+0000","last_clean":"2024-03-20T08:18:44.891597+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:18:44.891597+0000","last_undegraded":"2024-03-20T08:18:44.891597+0000","last_fullsized":"2024-03-20T08:18:44.891597+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855435+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:19:14.855435+0000","last_peered":"2024-03-20T08:19:14.855435+0000","last_clean":"2024-03-20T08:19:14.855435+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:19:14.855435+0000","last_undegraded":"2024-03-20T08:19:14.855435+0000","last_fullsized":"2024-03-20T08:19:14.855435+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855443+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:19:14.855443+0000","last_peered":"2024-03-20T08:19:14.855443+0000","last_clean":"2024-03-20T08:19:14.855443+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:19:14.855443+0000","last_undegraded":"2024-03-20T08:19:14.855443+0000","last_fullsized":"2024-03-20T08:19:14.855443+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1228,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:19.833183+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:18:19.833183+0000","last_peered":"2024-03-20T08:18:19.833183+0000","last_clean":"2024-03-20T08:18:19.833183+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:18:19.833183+0000","last_undegraded":"2024-03-20T08:18:19.833183+0000","last_fullsized":"2024-03-20T08:18:19.833183+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891616+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:18:44.891616+0000","last_peered":"2024-03-20T08:18:44.891616+0000","last_clean":"2024-03-20T08:18:44.891616+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:18:44.891616+0000","last_undegraded":"2024-03-20T08:18:44.891616+0000","last_fullsized":"2024-03-20T08:18:44.891616+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1335,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292109+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:19:02.292109+0000","last_peered":"2024-03-20T08:19:02.292109+0000","last_clean":"2024-03-20T08:19:02.292109+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:19:02.292109+0000","last_undegraded":"2024-03-20T08:19:02.292109+0000","last_fullsized":"2024-03-20T08:19:02.292109+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871516+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:19:12.871516+0000","last_peered":"2024-03-20T08:19:12.871516+0000","last_clean":"2024-03-20T08:19:12.871516+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:19:12.871516+0000","last_undegraded":"2024-03-20T08:19:12.871516+0000","last_fullsized":"2024-03-20T08:19:12.871516+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:14.855413+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:19:14.855413+0000","last_peered":"2024-03-20T08:19:14.855413+0000","last_clean":"2024-03-20T08:19:14.855413+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:19:14.855413+0000","last_undegraded":"2024-03-20T08:19:14.855413+0000","last_fullsized":"2024-03-20T08:19:14.855413+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292095+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:19:02.292095+0000","last_peered":"2024-03-20T08:19:02.292095+0000","last_clean":"2024-03-20T08:19:02.292095+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:19:02.292095+0000","last_undegraded":"2024-03-20T08:19:02.292095+0000","last_fullsized":"2024-03-20T08:19:02.292095+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5630,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:14.855383+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:19:14.855383+0000","last_peered":"2024-03-20T08:19:14.855383+0000","last_clean":"2024-03-20T08:19:14.855383+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:19:14.855383+0000","last_undegraded":"2024-03-20T08:19:14.855383+0000","last_fullsized":"2024-03-20T08:19:14.855383+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292033+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:19:02.292033+0000","last_peered":"2024-03-20T08:19:02.292033+0000","last_clean":"2024-03-20T08:19:02.292033+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:19:02.292033+0000","last_undegraded":"2024-03-20T08:19:02.292033+0000","last_fullsized":"2024-03-20T08:19:02.292033+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292053+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:19:02.292053+0000","last_peered":"2024-03-20T08:19:02.292053+0000","last_clean":"2024-03-20T08:19:02.292053+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:19:02.292053+0000","last_undegraded":"2024-03-20T08:19:02.292053+0000","last_fullsized":"2024-03-20T08:19:02.292053+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891570+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:18:44.891570+0000","last_peered":"2024-03-20T08:18:44.891570+0000","last_clean":"2024-03-20T08:18:44.891570+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:18:44.891570+0000","last_undegraded":"2024-03-20T08:18:44.891570+0000","last_fullsized":"2024-03-20T08:18:44.891570+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:44.891637+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:18:44.891637+0000","last_peered":"2024-03-20T08:18:44.891637+0000","last_clean":"2024-03-20T08:18:44.891637+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:18:44.891637+0000","last_undegraded":"2024-03-20T08:18:44.891637+0000","last_fullsized":"2024-03-20T08:18:44.891637+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871538+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:19:12.871538+0000","last_peered":"2024-03-20T08:19:12.871538+0000","last_clean":"2024-03-20T08:19:12.871538+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:19:12.871538+0000","last_undegraded":"2024-03-20T08:19:12.871538+0000","last_fullsized":"2024-03-20T08:19:12.871538+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:02.292077+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:19:02.292077+0000","last_peered":"2024-03-20T08:19:02.292077+0000","last_clean":"2024-03-20T08:19:02.292077+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:19:02.292077+0000","last_undegraded":"2024-03-20T08:19:02.292077+0000","last_fullsized":"2024-03-20T08:19:02.292077+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:55.978633+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:18:55.978633+0000","last_peered":"2024-03-20T08:18:55.978633+0000","last_clean":"2024-03-20T08:18:55.978633+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:18:55.978633+0000","last_undegraded":"2024-03-20T08:18:55.978633+0000","last_fullsized":"2024-03-20T08:18:55.978633+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980258,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484652,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011663,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189014,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703494,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656960,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446904,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629299,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.620 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.621 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.621 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.621 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.621 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.621 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.621 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:14.621 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:23:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:14 smithi196 ceph-mon[87080]: pgmap v1226: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:15.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:15 smithi196 ceph-mon[87080]: from='client.44663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:15.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:15 smithi118 ceph-mon[93121]: from='client.44663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:15.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:15 smithi118 ceph-mon[95692]: from='client.44663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:16.573 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:16 smithi118 ceph-mon[93121]: pgmap v1227: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:16.574 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:16 smithi118 ceph-mon[95692]: pgmap v1227: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:16.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:16 smithi196 ceph-mon[87080]: pgmap v1227: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:18 smithi196 ceph-mon[87080]: pgmap v1228: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:18.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:18 smithi118 ceph-mon[93121]: pgmap v1228: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:18.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:18 smithi118 ceph-mon[95692]: pgmap v1228: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:20 smithi196 ceph-mon[87080]: pgmap v1229: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:20 smithi118 ceph-mon[93121]: pgmap v1229: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:20 smithi118 ceph-mon[95692]: pgmap v1229: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:22 smithi196 ceph-mon[87080]: pgmap v1230: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:22.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:23:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:23:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:23:22.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:22 smithi118 ceph-mon[93121]: pgmap v1230: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:22.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:22 smithi118 ceph-mon[95692]: pgmap v1230: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:23.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:23:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:23:23] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:23:24.622 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:23:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:24 smithi196 ceph-mon[87080]: pgmap v1231: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:24.650 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:24 smithi118 ceph-mon[93121]: pgmap v1231: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:24.650 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:24 smithi118 ceph-mon[95692]: pgmap v1231: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:24.859 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:23:25.236 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:25 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:23:25.237 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:25 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:23:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:25 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:23:26.317 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:26 smithi118 ceph-mon[95692]: pgmap v1232: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:26.317 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:23:26.317 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:23:26.317 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:23:26.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:26 smithi118 ceph-mon[93121]: pgmap v1232: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:26.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:23:26.317 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:23:26.318 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:23:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:26 smithi196 ceph-mon[87080]: pgmap v1232: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:23:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:23:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:23:26.797 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:23:26.799 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:23:27.569 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1233,"stamp":"2024-03-20T08:23:26.748671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002994"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424603+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:18:37.424603+0000","last_peered":"2024-03-20T08:18:37.424603+0000","last_clean":"2024-03-20T08:18:37.424603+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:18:37.424603+0000","last_undegraded":"2024-03-20T08:18:37.424603+0000","last_fullsized":"2024-03-20T08:18:37.424603+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1186,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424554+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:18:37.424554+0000","last_peered":"2024-03-20T08:18:37.424554+0000","last_clean":"2024-03-20T08:18:37.424554+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:18:37.424554+0000","last_undegraded":"2024-03-20T08:18:37.424554+0000","last_fullsized":"2024-03-20T08:18:37.424554+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:37.424623+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:18:37.424623+0000","last_peered":"2024-03-20T08:18:37.424623+0000","last_clean":"2024-03-20T08:18:37.424623+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:18:37.424623+0000","last_undegraded":"2024-03-20T08:18:37.424623+0000","last_fullsized":"2024-03-20T08:18:37.424623+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1192,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430107+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:19:22.430107+0000","last_peered":"2024-03-20T08:19:22.430107+0000","last_clean":"2024-03-20T08:19:22.430107+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:19:22.430107+0000","last_undegraded":"2024-03-20T08:19:22.430107+0000","last_fullsized":"2024-03-20T08:19:22.430107+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9298,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.839110+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:19:09.839110+0000","last_peered":"2024-03-20T08:19:09.839110+0000","last_clean":"2024-03-20T08:19:09.839110+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:19:09.839110+0000","last_undegraded":"2024-03-20T08:19:09.839110+0000","last_fullsized":"2024-03-20T08:19:09.839110+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49871,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:22.430068+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:19:22.430068+0000","last_peered":"2024-03-20T08:19:22.430068+0000","last_clean":"2024-03-20T08:19:22.430068+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:19:22.430068+0000","last_undegraded":"2024-03-20T08:19:22.430068+0000","last_fullsized":"2024-03-20T08:19:22.430068+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:22.430030+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:19:22.430030+0000","last_peered":"2024-03-20T08:19:22.430030+0000","last_clean":"2024-03-20T08:19:22.430030+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:19:22.430030+0000","last_undegraded":"2024-03-20T08:19:22.430030+0000","last_fullsized":"2024-03-20T08:19:22.430030+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1235,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.839064+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:19:09.839064+0000","last_peered":"2024-03-20T08:19:09.839064+0000","last_clean":"2024-03-20T08:19:09.839064+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:19:09.839064+0000","last_undegraded":"2024-03-20T08:19:09.839064+0000","last_fullsized":"2024-03-20T08:19:09.839064+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980261,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484654,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011665,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189017,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703497,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656963,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446907,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629302,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:23:27.575 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.575 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.575 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.575 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.575 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.575 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.575 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.575 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.576 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.576 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.576 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.576 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.576 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.576 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.576 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:27.576 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:23:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:28 smithi196 ceph-mon[87080]: pgmap v1233: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:28 smithi196 ceph-mon[87080]: from='client.44669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:28.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:28 smithi118 ceph-mon[93121]: pgmap v1233: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:28.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:28 smithi118 ceph-mon[93121]: from='client.44669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:28 smithi118 ceph-mon[95692]: pgmap v1233: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:28 smithi118 ceph-mon[95692]: from='client.44669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:29.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:29 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:29.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:29 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:29.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:29 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:30 smithi196 ceph-mon[87080]: pgmap v1234: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:30.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:30 smithi118 ceph-mon[93121]: pgmap v1234: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:30 smithi118 ceph-mon[95692]: pgmap v1234: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:32 smithi196 ceph-mon[87080]: pgmap v1235: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:32.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:23:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:23:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:23:32.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:32 smithi118 ceph-mon[93121]: pgmap v1235: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:32.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:32 smithi118 ceph-mon[95692]: pgmap v1235: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:23:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:23:33] "GET /metrics HTTP/1.1" 200 36411 "" "Prometheus/2.43.0" 2024-03-20T08:23:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:34 smithi196 ceph-mon[87080]: pgmap v1236: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:34 smithi118 ceph-mon[93121]: pgmap v1236: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:34 smithi118 ceph-mon[95692]: pgmap v1236: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:36 smithi196 ceph-mon[87080]: pgmap v1237: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:36 smithi118 ceph-mon[93121]: pgmap v1237: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:36.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:36 smithi118 ceph-mon[95692]: pgmap v1237: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:37.577 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:23:37.791 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:23:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:38 smithi196 ceph-mon[87080]: pgmap v1238: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:38.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:38 smithi118 ceph-mon[93121]: pgmap v1238: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:38 smithi118 ceph-mon[95692]: pgmap v1238: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:39.958 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:23:39.960 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:23:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:40 smithi196 ceph-mon[87080]: pgmap v1239: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:40.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:40 smithi118 ceph-mon[93121]: pgmap v1239: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:40 smithi118 ceph-mon[95692]: pgmap v1239: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:40.720 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1239,"stamp":"2024-03-20T08:23:38.752278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003607"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893940+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:19:04.893940+0000","last_peered":"2024-03-20T08:19:04.893940+0000","last_clean":"2024-03-20T08:19:04.893940+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:19:04.893940+0000","last_undegraded":"2024-03-20T08:19:04.893940+0000","last_fullsized":"2024-03-20T08:19:04.893940+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:04.893882+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:19:04.893882+0000","last_peered":"2024-03-20T08:19:04.893882+0000","last_clean":"2024-03-20T08:19:04.893882+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:19:04.893882+0000","last_undegraded":"2024-03-20T08:19:04.893882+0000","last_fullsized":"2024-03-20T08:19:04.893882+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:09.714182+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:19:09.714182+0000","last_peered":"2024-03-20T08:19:09.714182+0000","last_clean":"2024-03-20T08:19:09.714182+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:19:09.714182+0000","last_undegraded":"2024-03-20T08:19:09.714182+0000","last_fullsized":"2024-03-20T08:19:09.714182+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1613,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898308+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:19:44.898308+0000","last_peered":"2024-03-20T08:19:44.898308+0000","last_clean":"2024-03-20T08:19:44.898308+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:19:44.898308+0000","last_undegraded":"2024-03-20T08:19:44.898308+0000","last_fullsized":"2024-03-20T08:19:44.898308+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898272+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:19:44.898272+0000","last_peered":"2024-03-20T08:19:44.898272+0000","last_clean":"2024-03-20T08:19:44.898272+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:19:44.898272+0000","last_undegraded":"2024-03-20T08:19:44.898272+0000","last_fullsized":"2024-03-20T08:19:44.898272+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68667,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:09.714143+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:19:09.714143+0000","last_peered":"2024-03-20T08:19:09.714143+0000","last_clean":"2024-03-20T08:19:09.714143+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:19:09.714143+0000","last_undegraded":"2024-03-20T08:19:09.714143+0000","last_fullsized":"2024-03-20T08:19:09.714143+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980263,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484657,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011668,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189019,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703500,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656965,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446909,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629304,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.727 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.728 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:40.728 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:23:41.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:41 smithi196 ceph-mon[87080]: from='client.34755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:41.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:41 smithi118 ceph-mon[93121]: from='client.34755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:41.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:41 smithi118 ceph-mon[95692]: from='client.34755 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:42 smithi196 ceph-mon[87080]: pgmap v1240: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:42.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:23:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:23:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:23:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:42 smithi118 ceph-mon[93121]: pgmap v1240: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:42.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:42 smithi118 ceph-mon[95692]: pgmap v1240: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:43.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:23:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:23:43] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:23:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:44 smithi196 ceph-mon[87080]: pgmap v1241: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:44 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:44.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:44 smithi118 ceph-mon[93121]: pgmap v1241: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:44.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:44 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:44.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:44 smithi118 ceph-mon[95692]: pgmap v1241: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:44.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:44 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:46 smithi196 ceph-mon[87080]: pgmap v1242: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:46 smithi118 ceph-mon[93121]: pgmap v1242: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:46 smithi118 ceph-mon[95692]: pgmap v1242: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:48.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:48 smithi196 ceph-mon[87080]: pgmap v1243: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:48.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:48 smithi118 ceph-mon[93121]: pgmap v1243: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:48 smithi118 ceph-mon[95692]: pgmap v1243: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:50 smithi196 ceph-mon[87080]: pgmap v1244: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:50 smithi118 ceph-mon[93121]: pgmap v1244: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:50.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:50 smithi118 ceph-mon[95692]: pgmap v1244: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:50.728 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:23:50.946 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:23:52.601 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:52 smithi118 ceph-mon[93121]: pgmap v1245: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:52.601 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:52 smithi118 ceph-mon[95692]: pgmap v1245: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:52 smithi196 ceph-mon[87080]: pgmap v1245: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:23:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:23:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:23:53.150 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:23:53.152 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:23:53.863 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:23:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:23:53] "GET /metrics HTTP/1.1" 200 36410 "" "Prometheus/2.43.0" 2024-03-20T08:23:53.883 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1246,"stamp":"2024-03-20T08:23:52.755979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003226"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:18:57.427395+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:18:57.427395+0000","last_peered":"2024-03-20T08:18:57.427395+0000","last_clean":"2024-03-20T08:18:57.427395+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:18:57.427395+0000","last_undegraded":"2024-03-20T08:18:57.427395+0000","last_fullsized":"2024-03-20T08:18:57.427395+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909806+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:21:24.909806+0000","last_peered":"2024-03-20T08:21:24.909806+0000","last_clean":"2024-03-20T08:21:24.909806+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:21:24.909806+0000","last_undegraded":"2024-03-20T08:21:24.909806+0000","last_fullsized":"2024-03-20T08:21:24.909806+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909777+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:21:24.909777+0000","last_peered":"2024-03-20T08:21:24.909777+0000","last_clean":"2024-03-20T08:21:24.909777+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:21:24.909777+0000","last_undegraded":"2024-03-20T08:21:24.909777+0000","last_fullsized":"2024-03-20T08:21:24.909777+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26673,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873738+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:19:42.873738+0000","last_peered":"2024-03-20T08:19:42.873738+0000","last_clean":"2024-03-20T08:19:42.873738+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:19:42.873738+0000","last_undegraded":"2024-03-20T08:19:42.873738+0000","last_fullsized":"2024-03-20T08:19:42.873738+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1261,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.728906+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:21:24.728906+0000","last_peered":"2024-03-20T08:21:24.728906+0000","last_clean":"2024-03-20T08:21:24.728906+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:21:24.728906+0000","last_undegraded":"2024-03-20T08:21:24.728906+0000","last_fullsized":"2024-03-20T08:21:24.728906+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17602,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873850+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:19:42.873850+0000","last_peered":"2024-03-20T08:19:42.873850+0000","last_clean":"2024-03-20T08:19:42.873850+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:19:42.873850+0000","last_undegraded":"2024-03-20T08:19:42.873850+0000","last_fullsized":"2024-03-20T08:19:42.873850+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296909+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:20:02.296909+0000","last_peered":"2024-03-20T08:20:02.296909+0000","last_clean":"2024-03-20T08:20:02.296909+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:20:02.296909+0000","last_undegraded":"2024-03-20T08:20:02.296909+0000","last_fullsized":"2024-03-20T08:20:02.296909+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1288,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857809+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:19:44.857809+0000","last_peered":"2024-03-20T08:19:44.857809+0000","last_clean":"2024-03-20T08:19:44.857809+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:19:44.857809+0000","last_undegraded":"2024-03-20T08:19:44.857809+0000","last_fullsized":"2024-03-20T08:19:44.857809+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1614,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912064+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:21:44.912064+0000","last_peered":"2024-03-20T08:21:44.912064+0000","last_clean":"2024-03-20T08:21:44.912064+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:21:44.912064+0000","last_undegraded":"2024-03-20T08:21:44.912064+0000","last_fullsized":"2024-03-20T08:21:44.912064+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912041+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:21:44.912041+0000","last_peered":"2024-03-20T08:21:44.912041+0000","last_clean":"2024-03-20T08:21:44.912041+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:21:44.912041+0000","last_undegraded":"2024-03-20T08:21:44.912041+0000","last_fullsized":"2024-03-20T08:21:44.912041+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:42.873840+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:19:42.873840+0000","last_peered":"2024-03-20T08:19:42.873840+0000","last_clean":"2024-03-20T08:19:42.873840+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:19:42.873840+0000","last_undegraded":"2024-03-20T08:19:42.873840+0000","last_fullsized":"2024-03-20T08:19:42.873840+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68668,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:24.728852+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:21:24.728852+0000","last_peered":"2024-03-20T08:21:24.728852+0000","last_clean":"2024-03-20T08:21:24.728852+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:21:24.728852+0000","last_undegraded":"2024-03-20T08:21:24.728852+0000","last_fullsized":"2024-03-20T08:21:24.728852+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29911,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:32.294538+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:19:32.294538+0000","last_peered":"2024-03-20T08:19:32.294538+0000","last_clean":"2024-03-20T08:19:32.294538+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:19:32.294538+0000","last_undegraded":"2024-03-20T08:19:32.294538+0000","last_fullsized":"2024-03-20T08:19:32.294538+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5700,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:42.873798+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:19:42.873798+0000","last_peered":"2024-03-20T08:19:42.873798+0000","last_clean":"2024-03-20T08:19:42.873798+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:19:42.873798+0000","last_undegraded":"2024-03-20T08:19:42.873798+0000","last_fullsized":"2024-03-20T08:19:42.873798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1347,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:32.294591+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:19:32.294591+0000","last_peered":"2024-03-20T08:19:32.294591+0000","last_clean":"2024-03-20T08:19:32.294591+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:19:32.294591+0000","last_undegraded":"2024-03-20T08:19:32.294591+0000","last_fullsized":"2024-03-20T08:19:32.294591+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980266,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484660,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011671,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189022,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703502,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656968,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446912,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629307,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.889 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.890 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.890 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.890 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.890 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:23:53.890 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:23:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:54 smithi196 ceph-mon[87080]: pgmap v1246: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:54 smithi196 ceph-mon[87080]: from='client.34761 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:54 smithi118 ceph-mon[93121]: pgmap v1246: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:54 smithi118 ceph-mon[93121]: from='client.34761 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:54.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:54 smithi118 ceph-mon[95692]: pgmap v1246: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:54.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:54 smithi118 ceph-mon[95692]: from='client.34761 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:23:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:56 smithi196 ceph-mon[87080]: pgmap v1247: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:56.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:56 smithi118 ceph-mon[93121]: pgmap v1247: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:56.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:56 smithi118 ceph-mon[95692]: pgmap v1247: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:58 smithi196 ceph-mon[87080]: pgmap v1248: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:58.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:58 smithi118 ceph-mon[93121]: pgmap v1248: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:58.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:58 smithi118 ceph-mon[95692]: pgmap v1248: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:23:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:23:59 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:59.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:23:59 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:23:59.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:23:59 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:00 smithi196 ceph-mon[87080]: pgmap v1249: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:00.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:00 smithi118 ceph-mon[93121]: pgmap v1249: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:00.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:00 smithi118 ceph-mon[95692]: pgmap v1249: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:02 smithi196 ceph-mon[87080]: pgmap v1250: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:02.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:24:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:24:02.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:24:02.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:02 smithi118 ceph-mon[93121]: pgmap v1250: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:02.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:02 smithi118 ceph-mon[95692]: pgmap v1250: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:03.891 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:24:03.916 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:24:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:24:03] "GET /metrics HTTP/1.1" 200 36407 "" "Prometheus/2.43.0" 2024-03-20T08:24:04.108 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:24:04.499 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:04 smithi118 ceph-mon[93121]: pgmap v1251: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:04.500 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:04 smithi118 ceph-mon[95692]: pgmap v1251: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:04 smithi196 ceph-mon[87080]: pgmap v1251: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:06.149 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:24:06.151 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:24:06.422 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:06 smithi118 ceph-mon[93121]: pgmap v1252: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:06.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:06 smithi118 ceph-mon[95692]: pgmap v1252: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:06 smithi196 ceph-mon[87080]: pgmap v1252: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:06.867 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1252,"stamp":"2024-03-20T08:24:04.759577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003597"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983848+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:20:00.983848+0000","last_peered":"2024-03-20T08:20:00.983848+0000","last_clean":"2024-03-20T08:20:00.983848+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:20:00.983848+0000","last_undegraded":"2024-03-20T08:20:00.983848+0000","last_fullsized":"2024-03-20T08:20:00.983848+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1126,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896105+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:19:24.896105+0000","last_peered":"2024-03-20T08:19:24.896105+0000","last_clean":"2024-03-20T08:19:24.896105+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:19:24.896105+0000","last_undegraded":"2024-03-20T08:19:24.896105+0000","last_fullsized":"2024-03-20T08:19:24.896105+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855851+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:19:19.855851+0000","last_peered":"2024-03-20T08:19:19.855851+0000","last_clean":"2024-03-20T08:19:19.855851+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:19:19.855851+0000","last_undegraded":"2024-03-20T08:19:19.855851+0000","last_fullsized":"2024-03-20T08:19:19.855851+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980939+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:19:25.980939+0000","last_peered":"2024-03-20T08:19:25.980939+0000","last_clean":"2024-03-20T08:19:25.980939+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:19:25.980939+0000","last_undegraded":"2024-03-20T08:19:25.980939+0000","last_fullsized":"2024-03-20T08:19:25.980939+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1395,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983887+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:20:00.983887+0000","last_peered":"2024-03-20T08:20:00.983887+0000","last_clean":"2024-03-20T08:20:00.983887+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:20:00.983887+0000","last_undegraded":"2024-03-20T08:20:00.983887+0000","last_fullsized":"2024-03-20T08:20:00.983887+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983877+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:20:00.983877+0000","last_peered":"2024-03-20T08:20:00.983877+0000","last_clean":"2024-03-20T08:20:00.983877+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:20:00.983877+0000","last_undegraded":"2024-03-20T08:20:00.983877+0000","last_fullsized":"2024-03-20T08:20:00.983877+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:12.871573+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:19:12.871573+0000","last_peered":"2024-03-20T08:19:12.871573+0000","last_clean":"2024-03-20T08:19:12.871573+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:19:12.871573+0000","last_undegraded":"2024-03-20T08:19:12.871573+0000","last_fullsized":"2024-03-20T08:19:12.871573+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444464+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:21:27.444464+0000","last_peered":"2024-03-20T08:21:27.444464+0000","last_clean":"2024-03-20T08:21:27.444464+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:21:27.444464+0000","last_undegraded":"2024-03-20T08:21:27.444464+0000","last_fullsized":"2024-03-20T08:21:27.444464+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855837+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:19:19.855837+0000","last_peered":"2024-03-20T08:19:19.855837+0000","last_clean":"2024-03-20T08:19:19.855837+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:19:19.855837+0000","last_undegraded":"2024-03-20T08:19:19.855837+0000","last_fullsized":"2024-03-20T08:19:19.855837+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:19.855809+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:19:19.855809+0000","last_peered":"2024-03-20T08:19:19.855809+0000","last_clean":"2024-03-20T08:19:19.855809+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:19:19.855809+0000","last_undegraded":"2024-03-20T08:19:19.855809+0000","last_fullsized":"2024-03-20T08:19:19.855809+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.716687+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:19:34.716687+0000","last_peered":"2024-03-20T08:19:34.716687+0000","last_clean":"2024-03-20T08:19:34.716687+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:19:34.716687+0000","last_undegraded":"2024-03-20T08:19:34.716687+0000","last_fullsized":"2024-03-20T08:19:34.716687+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1141,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:24.896074+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:19:24.896074+0000","last_peered":"2024-03-20T08:19:24.896074+0000","last_clean":"2024-03-20T08:19:24.896074+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:19:24.896074+0000","last_undegraded":"2024-03-20T08:19:24.896074+0000","last_fullsized":"2024-03-20T08:19:24.896074+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1390,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980986+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:19:25.980986+0000","last_peered":"2024-03-20T08:19:25.980986+0000","last_clean":"2024-03-20T08:19:25.980986+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:19:25.980986+0000","last_undegraded":"2024-03-20T08:19:25.980986+0000","last_fullsized":"2024-03-20T08:19:25.980986+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:25.980964+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:19:25.980964+0000","last_peered":"2024-03-20T08:19:25.980964+0000","last_clean":"2024-03-20T08:19:25.980964+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:19:25.980964+0000","last_undegraded":"2024-03-20T08:19:25.980964+0000","last_fullsized":"2024-03-20T08:19:25.980964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909806+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:21:24.909806+0000","last_peered":"2024-03-20T08:21:24.909806+0000","last_clean":"2024-03-20T08:21:24.909806+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:21:24.909806+0000","last_undegraded":"2024-03-20T08:21:24.909806+0000","last_fullsized":"2024-03-20T08:21:24.909806+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909777+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:21:24.909777+0000","last_peered":"2024-03-20T08:21:24.909777+0000","last_clean":"2024-03-20T08:21:24.909777+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:21:24.909777+0000","last_undegraded":"2024-03-20T08:21:24.909777+0000","last_fullsized":"2024-03-20T08:21:24.909777+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26674,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884162+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:21:52.884162+0000","last_peered":"2024-03-20T08:21:52.884162+0000","last_clean":"2024-03-20T08:21:52.884162+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:21:52.884162+0000","last_undegraded":"2024-03-20T08:21:52.884162+0000","last_fullsized":"2024-03-20T08:21:52.884162+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1261,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.728906+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:21:24.728906+0000","last_peered":"2024-03-20T08:21:24.728906+0000","last_clean":"2024-03-20T08:21:24.728906+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:21:24.728906+0000","last_undegraded":"2024-03-20T08:21:24.728906+0000","last_fullsized":"2024-03-20T08:21:24.728906+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11691,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983822+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:20:00.983822+0000","last_peered":"2024-03-20T08:20:00.983822+0000","last_clean":"2024-03-20T08:20:00.983822+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:20:00.983822+0000","last_undegraded":"2024-03-20T08:20:00.983822+0000","last_fullsized":"2024-03-20T08:20:00.983822+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5953,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:00.983855+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:20:00.983855+0000","last_peered":"2024-03-20T08:20:00.983855+0000","last_clean":"2024-03-20T08:20:00.983855+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:20:00.983855+0000","last_undegraded":"2024-03-20T08:20:00.983855+0000","last_fullsized":"2024-03-20T08:20:00.983855+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884275+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:21:52.884275+0000","last_peered":"2024-03-20T08:21:52.884275+0000","last_clean":"2024-03-20T08:21:52.884275+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:21:52.884275+0000","last_undegraded":"2024-03-20T08:21:52.884275+0000","last_fullsized":"2024-03-20T08:21:52.884275+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:02.306280+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:22:02.306280+0000","last_peered":"2024-03-20T08:22:02.306280+0000","last_clean":"2024-03-20T08:22:02.306280+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:22:02.306280+0000","last_undegraded":"2024-03-20T08:22:02.306280+0000","last_fullsized":"2024-03-20T08:22:02.306280+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.867213+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:21:49.867213+0000","last_peered":"2024-03-20T08:21:49.867213+0000","last_clean":"2024-03-20T08:21:49.867213+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:21:49.867213+0000","last_undegraded":"2024-03-20T08:21:49.867213+0000","last_fullsized":"2024-03-20T08:21:49.867213+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1614,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912064+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:21:44.912064+0000","last_peered":"2024-03-20T08:21:44.912064+0000","last_clean":"2024-03-20T08:21:44.912064+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:21:44.912064+0000","last_undegraded":"2024-03-20T08:21:44.912064+0000","last_fullsized":"2024-03-20T08:21:44.912064+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983784+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:20:00.983784+0000","last_peered":"2024-03-20T08:20:00.983784+0000","last_clean":"2024-03-20T08:20:00.983784+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:20:00.983784+0000","last_undegraded":"2024-03-20T08:20:00.983784+0000","last_fullsized":"2024-03-20T08:20:00.983784+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912041+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:21:44.912041+0000","last_peered":"2024-03-20T08:21:44.912041+0000","last_clean":"2024-03-20T08:21:44.912041+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:21:44.912041+0000","last_undegraded":"2024-03-20T08:21:44.912041+0000","last_fullsized":"2024-03-20T08:21:44.912041+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983838+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:20:00.983838+0000","last_peered":"2024-03-20T08:20:00.983838+0000","last_clean":"2024-03-20T08:20:00.983838+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:20:00.983838+0000","last_undegraded":"2024-03-20T08:20:00.983838+0000","last_fullsized":"2024-03-20T08:20:00.983838+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1175,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432931+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:19:47.432931+0000","last_peered":"2024-03-20T08:19:47.432931+0000","last_clean":"2024-03-20T08:19:47.432931+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:19:47.432931+0000","last_undegraded":"2024-03-20T08:19:47.432931+0000","last_fullsized":"2024-03-20T08:19:47.432931+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:52.884264+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:21:52.884264+0000","last_peered":"2024-03-20T08:21:52.884264+0000","last_clean":"2024-03-20T08:21:52.884264+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:21:52.884264+0000","last_undegraded":"2024-03-20T08:21:52.884264+0000","last_fullsized":"2024-03-20T08:21:52.884264+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5193,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:19:34.842053+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:19:34.842053+0000","last_peered":"2024-03-20T08:19:34.842053+0000","last_clean":"2024-03-20T08:19:34.842053+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:19:34.842053+0000","last_undegraded":"2024-03-20T08:19:34.842053+0000","last_fullsized":"2024-03-20T08:19:34.842053+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68668,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:24.728852+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:21:24.728852+0000","last_peered":"2024-03-20T08:21:24.728852+0000","last_clean":"2024-03-20T08:21:24.728852+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:21:24.728852+0000","last_undegraded":"2024-03-20T08:21:24.728852+0000","last_fullsized":"2024-03-20T08:21:24.728852+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29912,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:47.305131+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:21:47.305131+0000","last_peered":"2024-03-20T08:21:47.305131+0000","last_clean":"2024-03-20T08:21:47.305131+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:21:47.305131+0000","last_undegraded":"2024-03-20T08:21:47.305131+0000","last_fullsized":"2024-03-20T08:21:47.305131+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432866+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:19:47.432866+0000","last_peered":"2024-03-20T08:19:47.432866+0000","last_clean":"2024-03-20T08:19:47.432866+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:19:47.432866+0000","last_undegraded":"2024-03-20T08:19:47.432866+0000","last_fullsized":"2024-03-20T08:19:47.432866+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5701,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884206+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:21:52.884206+0000","last_peered":"2024-03-20T08:21:52.884206+0000","last_clean":"2024-03-20T08:21:52.884206+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:21:52.884206+0000","last_undegraded":"2024-03-20T08:21:52.884206+0000","last_fullsized":"2024-03-20T08:21:52.884206+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:47.305170+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:21:47.305170+0000","last_peered":"2024-03-20T08:21:47.305170+0000","last_clean":"2024-03-20T08:21:47.305170+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:21:47.305170+0000","last_undegraded":"2024-03-20T08:21:47.305170+0000","last_fullsized":"2024-03-20T08:21:47.305170+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.842029+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:19:34.842029+0000","last_peered":"2024-03-20T08:19:34.842029+0000","last_clean":"2024-03-20T08:19:34.842029+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:19:34.842029+0000","last_undegraded":"2024-03-20T08:19:34.842029+0000","last_fullsized":"2024-03-20T08:19:34.842029+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:00.983809+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:20:00.983809+0000","last_peered":"2024-03-20T08:20:00.983809+0000","last_clean":"2024-03-20T08:20:00.983809+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:20:00.983809+0000","last_undegraded":"2024-03-20T08:20:00.983809+0000","last_fullsized":"2024-03-20T08:20:00.983809+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:47.432920+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:19:47.432920+0000","last_peered":"2024-03-20T08:19:47.432920+0000","last_clean":"2024-03-20T08:19:47.432920+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:19:47.432920+0000","last_undegraded":"2024-03-20T08:19:47.432920+0000","last_fullsized":"2024-03-20T08:19:47.432920+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:34.841999+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:19:34.841999+0000","last_peered":"2024-03-20T08:19:34.841999+0000","last_clean":"2024-03-20T08:19:34.841999+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:19:34.841999+0000","last_undegraded":"2024-03-20T08:19:34.841999+0000","last_fullsized":"2024-03-20T08:19:34.841999+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980268,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484662,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011673,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189024,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703505,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656970,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446915,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629309,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.873 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.874 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:06.874 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:24:07.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:07 smithi196 ceph-mon[87080]: from='client.34767 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:07.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:07 smithi118 ceph-mon[93121]: from='client.34767 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:07.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:07 smithi118 ceph-mon[95692]: from='client.34767 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:08 smithi196 ceph-mon[87080]: pgmap v1253: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:08 smithi118 ceph-mon[93121]: pgmap v1253: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:08 smithi118 ceph-mon[95692]: pgmap v1253: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:10 smithi196 ceph-mon[87080]: pgmap v1254: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:10.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:10 smithi118 ceph-mon[95692]: pgmap v1254: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:10.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:10 smithi118 ceph-mon[93121]: pgmap v1254: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:12 smithi196 ceph-mon[87080]: pgmap v1255: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:12.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:24:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:24:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:24:12.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:12 smithi118 ceph-mon[95692]: pgmap v1255: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:12.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:12 smithi118 ceph-mon[93121]: pgmap v1255: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:13.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:24:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:24:13] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:24:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:14 smithi196 ceph-mon[87080]: pgmap v1256: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:14.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:14 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:14 smithi118 ceph-mon[93121]: pgmap v1256: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:14 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:14.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:14 smithi118 ceph-mon[95692]: pgmap v1256: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:14.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:14 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:16 smithi196 ceph-mon[87080]: pgmap v1257: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:16 smithi118 ceph-mon[93121]: pgmap v1257: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:16 smithi118 ceph-mon[95692]: pgmap v1257: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:16.875 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:24:17.105 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:24:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:18 smithi196 ceph-mon[87080]: pgmap v1258: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:18.644 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:18 smithi118 ceph-mon[93121]: pgmap v1258: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:18.644 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:18 smithi118 ceph-mon[95692]: pgmap v1258: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:19.091 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:24:19.094 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:24:19.857 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1259,"stamp":"2024-03-20T08:24:18.763396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003355"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901293+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:20:09.901293+0000","last_peered":"2024-03-20T08:20:09.901293+0000","last_clean":"2024-03-20T08:20:09.901293+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:20:09.901293+0000","last_undegraded":"2024-03-20T08:20:09.901293+0000","last_fullsized":"2024-03-20T08:20:09.901293+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994305+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:22:05.994305+0000","last_peered":"2024-03-20T08:22:05.994305+0000","last_clean":"2024-03-20T08:22:05.994305+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:22:05.994305+0000","last_undegraded":"2024-03-20T08:22:05.994305+0000","last_fullsized":"2024-03-20T08:22:05.994305+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:19.846974+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:20:19.846974+0000","last_peered":"2024-03-20T08:20:19.846974+0000","last_clean":"2024-03-20T08:20:19.846974+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:20:19.846974+0000","last_undegraded":"2024-03-20T08:20:19.846974+0000","last_fullsized":"2024-03-20T08:20:19.846974+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1127,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912609+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:21:49.912609+0000","last_peered":"2024-03-20T08:21:49.912609+0000","last_clean":"2024-03-20T08:21:49.912609+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:21:49.912609+0000","last_undegraded":"2024-03-20T08:21:49.912609+0000","last_fullsized":"2024-03-20T08:21:49.912609+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866885+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:21:44.866885+0000","last_peered":"2024-03-20T08:21:44.866885+0000","last_clean":"2024-03-20T08:21:44.866885+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:21:44.866885+0000","last_undegraded":"2024-03-20T08:21:44.866885+0000","last_fullsized":"2024-03-20T08:21:44.866885+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992610+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:21:45.992610+0000","last_peered":"2024-03-20T08:21:45.992610+0000","last_clean":"2024-03-20T08:21:45.992610+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:21:45.992610+0000","last_undegraded":"2024-03-20T08:21:45.992610+0000","last_fullsized":"2024-03-20T08:21:45.992610+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1396,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994341+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:22:05.994341+0000","last_peered":"2024-03-20T08:22:05.994341+0000","last_clean":"2024-03-20T08:22:05.994341+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:22:05.994341+0000","last_undegraded":"2024-03-20T08:22:05.994341+0000","last_fullsized":"2024-03-20T08:22:05.994341+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994332+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:22:05.994332+0000","last_peered":"2024-03-20T08:22:05.994332+0000","last_clean":"2024-03-20T08:22:05.994332+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:22:05.994332+0000","last_undegraded":"2024-03-20T08:22:05.994332+0000","last_fullsized":"2024-03-20T08:22:05.994332+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:42.883328+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:21:42.883328+0000","last_peered":"2024-03-20T08:21:42.883328+0000","last_clean":"2024-03-20T08:21:42.883328+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:21:42.883328+0000","last_undegraded":"2024-03-20T08:21:42.883328+0000","last_fullsized":"2024-03-20T08:21:42.883328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444464+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:21:27.444464+0000","last_peered":"2024-03-20T08:21:27.444464+0000","last_clean":"2024-03-20T08:21:27.444464+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:21:27.444464+0000","last_undegraded":"2024-03-20T08:21:27.444464+0000","last_fullsized":"2024-03-20T08:21:27.444464+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866861+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:21:44.866861+0000","last_peered":"2024-03-20T08:21:44.866861+0000","last_clean":"2024-03-20T08:21:44.866861+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:21:44.866861+0000","last_undegraded":"2024-03-20T08:21:44.866861+0000","last_fullsized":"2024-03-20T08:21:44.866861+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866825+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:21:44.866825+0000","last_peered":"2024-03-20T08:21:44.866825+0000","last_clean":"2024-03-20T08:21:44.866825+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:21:44.866825+0000","last_undegraded":"2024-03-20T08:21:44.866825+0000","last_fullsized":"2024-03-20T08:21:44.866825+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:54.732439+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:21:54.732439+0000","last_peered":"2024-03-20T08:21:54.732439+0000","last_clean":"2024-03-20T08:21:54.732439+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:21:54.732439+0000","last_undegraded":"2024-03-20T08:21:54.732439+0000","last_fullsized":"2024-03-20T08:21:54.732439+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912579+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:21:49.912579+0000","last_peered":"2024-03-20T08:21:49.912579+0000","last_clean":"2024-03-20T08:21:49.912579+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:21:49.912579+0000","last_undegraded":"2024-03-20T08:21:49.912579+0000","last_fullsized":"2024-03-20T08:21:49.912579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1391,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992645+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:21:45.992645+0000","last_peered":"2024-03-20T08:21:45.992645+0000","last_clean":"2024-03-20T08:21:45.992645+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:21:45.992645+0000","last_undegraded":"2024-03-20T08:21:45.992645+0000","last_fullsized":"2024-03-20T08:21:45.992645+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992629+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:21:45.992629+0000","last_peered":"2024-03-20T08:21:45.992629+0000","last_clean":"2024-03-20T08:21:45.992629+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:21:45.992629+0000","last_undegraded":"2024-03-20T08:21:45.992629+0000","last_fullsized":"2024-03-20T08:21:45.992629+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909806+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:21:24.909806+0000","last_peered":"2024-03-20T08:21:24.909806+0000","last_clean":"2024-03-20T08:21:24.909806+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:21:24.909806+0000","last_undegraded":"2024-03-20T08:21:24.909806+0000","last_fullsized":"2024-03-20T08:21:24.909806+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909777+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:21:24.909777+0000","last_peered":"2024-03-20T08:21:24.909777+0000","last_clean":"2024-03-20T08:21:24.909777+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:21:24.909777+0000","last_undegraded":"2024-03-20T08:21:24.909777+0000","last_fullsized":"2024-03-20T08:21:24.909777+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26674,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884162+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:21:52.884162+0000","last_peered":"2024-03-20T08:21:52.884162+0000","last_clean":"2024-03-20T08:21:52.884162+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:21:52.884162+0000","last_undegraded":"2024-03-20T08:21:52.884162+0000","last_fullsized":"2024-03-20T08:21:52.884162+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1261,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.728906+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:21:24.728906+0000","last_peered":"2024-03-20T08:21:24.728906+0000","last_clean":"2024-03-20T08:21:24.728906+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:21:24.728906+0000","last_undegraded":"2024-03-20T08:21:24.728906+0000","last_fullsized":"2024-03-20T08:21:24.728906+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11692,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994261+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:22:05.994261+0000","last_peered":"2024-03-20T08:22:05.994261+0000","last_clean":"2024-03-20T08:22:05.994261+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:22:05.994261+0000","last_undegraded":"2024-03-20T08:22:05.994261+0000","last_fullsized":"2024-03-20T08:22:05.994261+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5954,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994315+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:22:05.994315+0000","last_peered":"2024-03-20T08:22:05.994315+0000","last_clean":"2024-03-20T08:22:05.994315+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:22:05.994315+0000","last_undegraded":"2024-03-20T08:22:05.994315+0000","last_fullsized":"2024-03-20T08:22:05.994315+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884275+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:21:52.884275+0000","last_peered":"2024-03-20T08:21:52.884275+0000","last_clean":"2024-03-20T08:21:52.884275+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:21:52.884275+0000","last_undegraded":"2024-03-20T08:21:52.884275+0000","last_fullsized":"2024-03-20T08:21:52.884275+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:02.306280+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:22:02.306280+0000","last_peered":"2024-03-20T08:22:02.306280+0000","last_clean":"2024-03-20T08:22:02.306280+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:22:02.306280+0000","last_undegraded":"2024-03-20T08:22:02.306280+0000","last_fullsized":"2024-03-20T08:22:02.306280+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3981,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:09.901221+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:20:09.901221+0000","last_peered":"2024-03-20T08:20:09.901221+0000","last_clean":"2024-03-20T08:20:09.901221+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:20:09.901221+0000","last_undegraded":"2024-03-20T08:20:09.901221+0000","last_fullsized":"2024-03-20T08:20:09.901221+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.867213+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:21:49.867213+0000","last_peered":"2024-03-20T08:21:49.867213+0000","last_clean":"2024-03-20T08:21:49.867213+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:21:49.867213+0000","last_undegraded":"2024-03-20T08:21:49.867213+0000","last_fullsized":"2024-03-20T08:21:49.867213+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1614,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912064+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:21:44.912064+0000","last_peered":"2024-03-20T08:21:44.912064+0000","last_clean":"2024-03-20T08:21:44.912064+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:21:44.912064+0000","last_undegraded":"2024-03-20T08:21:44.912064+0000","last_fullsized":"2024-03-20T08:21:44.912064+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994228+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:22:05.994228+0000","last_peered":"2024-03-20T08:22:05.994228+0000","last_clean":"2024-03-20T08:22:05.994228+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:22:05.994228+0000","last_undegraded":"2024-03-20T08:22:05.994228+0000","last_fullsized":"2024-03-20T08:22:05.994228+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912041+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:21:44.912041+0000","last_peered":"2024-03-20T08:21:44.912041+0000","last_clean":"2024-03-20T08:21:44.912041+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:21:44.912041+0000","last_undegraded":"2024-03-20T08:21:44.912041+0000","last_fullsized":"2024-03-20T08:21:44.912041+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994287+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:22:05.994287+0000","last_peered":"2024-03-20T08:22:05.994287+0000","last_clean":"2024-03-20T08:22:05.994287+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:22:05.994287+0000","last_undegraded":"2024-03-20T08:22:05.994287+0000","last_fullsized":"2024-03-20T08:22:05.994287+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1144,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:09.901273+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:20:09.901273+0000","last_peered":"2024-03-20T08:20:09.901273+0000","last_clean":"2024-03-20T08:20:09.901273+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:20:09.901273+0000","last_undegraded":"2024-03-20T08:20:09.901273+0000","last_fullsized":"2024-03-20T08:20:09.901273+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1176,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447941+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:21:57.447941+0000","last_peered":"2024-03-20T08:21:57.447941+0000","last_clean":"2024-03-20T08:21:57.447941+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:21:57.447941+0000","last_undegraded":"2024-03-20T08:21:57.447941+0000","last_fullsized":"2024-03-20T08:21:57.447941+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:52.884264+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:21:52.884264+0000","last_peered":"2024-03-20T08:21:52.884264+0000","last_clean":"2024-03-20T08:21:52.884264+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:21:52.884264+0000","last_undegraded":"2024-03-20T08:21:52.884264+0000","last_fullsized":"2024-03-20T08:21:52.884264+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5194,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:49.857496+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:21:49.857496+0000","last_peered":"2024-03-20T08:21:49.857496+0000","last_clean":"2024-03-20T08:21:49.857496+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:21:49.857496+0000","last_undegraded":"2024-03-20T08:21:49.857496+0000","last_fullsized":"2024-03-20T08:21:49.857496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68668,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:24.728852+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:21:24.728852+0000","last_peered":"2024-03-20T08:21:24.728852+0000","last_clean":"2024-03-20T08:21:24.728852+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:21:24.728852+0000","last_undegraded":"2024-03-20T08:21:24.728852+0000","last_fullsized":"2024-03-20T08:21:24.728852+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29912,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:47.305131+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:21:47.305131+0000","last_peered":"2024-03-20T08:21:47.305131+0000","last_clean":"2024-03-20T08:21:47.305131+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:21:47.305131+0000","last_undegraded":"2024-03-20T08:21:47.305131+0000","last_fullsized":"2024-03-20T08:21:47.305131+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447869+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:21:57.447869+0000","last_peered":"2024-03-20T08:21:57.447869+0000","last_clean":"2024-03-20T08:21:57.447869+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:21:57.447869+0000","last_undegraded":"2024-03-20T08:21:57.447869+0000","last_fullsized":"2024-03-20T08:21:57.447869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5701,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884206+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:21:52.884206+0000","last_peered":"2024-03-20T08:21:52.884206+0000","last_clean":"2024-03-20T08:21:52.884206+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:21:52.884206+0000","last_undegraded":"2024-03-20T08:21:52.884206+0000","last_fullsized":"2024-03-20T08:21:52.884206+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:47.305170+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:21:47.305170+0000","last_peered":"2024-03-20T08:21:47.305170+0000","last_clean":"2024-03-20T08:21:47.305170+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:21:47.305170+0000","last_undegraded":"2024-03-20T08:21:47.305170+0000","last_fullsized":"2024-03-20T08:21:47.305170+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857470+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:21:49.857470+0000","last_peered":"2024-03-20T08:21:49.857470+0000","last_clean":"2024-03-20T08:21:49.857470+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:21:49.857470+0000","last_undegraded":"2024-03-20T08:21:49.857470+0000","last_fullsized":"2024-03-20T08:21:49.857470+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994252+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:22:05.994252+0000","last_peered":"2024-03-20T08:22:05.994252+0000","last_clean":"2024-03-20T08:22:05.994252+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:22:05.994252+0000","last_undegraded":"2024-03-20T08:22:05.994252+0000","last_fullsized":"2024-03-20T08:22:05.994252+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447930+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:21:57.447930+0000","last_peered":"2024-03-20T08:21:57.447930+0000","last_clean":"2024-03-20T08:21:57.447930+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:21:57.447930+0000","last_undegraded":"2024-03-20T08:21:57.447930+0000","last_fullsized":"2024-03-20T08:21:57.447930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857436+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:21:49.857436+0000","last_peered":"2024-03-20T08:21:49.857436+0000","last_clean":"2024-03-20T08:21:49.857436+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:21:49.857436+0000","last_undegraded":"2024-03-20T08:21:49.857436+0000","last_fullsized":"2024-03-20T08:21:49.857436+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980271,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484665,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011676,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189027,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703508,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656973,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446917,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629312,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.863 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.864 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.864 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.864 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:19.864 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:24:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:20 smithi196 ceph-mon[87080]: pgmap v1259: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:20.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:20 smithi196 ceph-mon[87080]: from='client.44693 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:20 smithi118 ceph-mon[93121]: pgmap v1259: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:20.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:20 smithi118 ceph-mon[93121]: from='client.44693 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:20 smithi118 ceph-mon[95692]: pgmap v1259: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:20 smithi118 ceph-mon[95692]: from='client.44693 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:22 smithi196 ceph-mon[87080]: pgmap v1260: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:22.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:24:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:24:22.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:24:22.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:22 smithi118 ceph-mon[95692]: pgmap v1260: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:22.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:22 smithi118 ceph-mon[93121]: pgmap v1260: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:23.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:24:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:24:23] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:24:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:24 smithi196 ceph-mon[87080]: pgmap v1261: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:24.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:24 smithi118 ceph-mon[93121]: pgmap v1261: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:24.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:24 smithi118 ceph-mon[95692]: pgmap v1261: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:26 smithi196 ceph-mon[87080]: pgmap v1262: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:24:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:26.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:26 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:26 smithi118 ceph-mon[95692]: pgmap v1262: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:24:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:26 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:26.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:26 smithi118 ceph-mon[93121]: pgmap v1262: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:26.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:24:26.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:26.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:26 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:24:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:24:27.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:27.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:24:27.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:24:27.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:24:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:24:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:24:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:28 smithi196 ceph-mon[87080]: pgmap v1263: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:28.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:28 smithi118 ceph-mon[93121]: pgmap v1263: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:28.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:28 smithi118 ceph-mon[95692]: pgmap v1263: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:29.865 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:24:30.079 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:24:30.482 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:30 smithi118 ceph-mon[93121]: pgmap v1264: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:30.482 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:30 smithi118 ceph-mon[95692]: pgmap v1264: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:30 smithi196 ceph-mon[87080]: pgmap v1264: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:32.095 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:24:32.097 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:24:32.401 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:32 smithi118 ceph-mon[93121]: pgmap v1265: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:32.402 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:32 smithi118 ceph-mon[95692]: pgmap v1265: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:32.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:32 smithi196 ceph-mon[87080]: pgmap v1265: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:24:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:24:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:24:32.860 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1265,"stamp":"2024-03-20T08:24:30.766698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003301"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915382+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:22:14.915382+0000","last_peered":"2024-03-20T08:22:14.915382+0000","last_clean":"2024-03-20T08:22:14.915382+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:22:14.915382+0000","last_undegraded":"2024-03-20T08:22:14.915382+0000","last_fullsized":"2024-03-20T08:22:14.915382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994305+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:22:05.994305+0000","last_peered":"2024-03-20T08:22:05.994305+0000","last_clean":"2024-03-20T08:22:05.994305+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:22:05.994305+0000","last_undegraded":"2024-03-20T08:22:05.994305+0000","last_fullsized":"2024-03-20T08:22:05.994305+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860780+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:22:19.860780+0000","last_peered":"2024-03-20T08:22:19.860780+0000","last_clean":"2024-03-20T08:22:19.860780+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:22:19.860780+0000","last_undegraded":"2024-03-20T08:22:19.860780+0000","last_fullsized":"2024-03-20T08:22:19.860780+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1127,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912609+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:21:49.912609+0000","last_peered":"2024-03-20T08:21:49.912609+0000","last_clean":"2024-03-20T08:21:49.912609+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:21:49.912609+0000","last_undegraded":"2024-03-20T08:21:49.912609+0000","last_fullsized":"2024-03-20T08:21:49.912609+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866885+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:21:44.866885+0000","last_peered":"2024-03-20T08:21:44.866885+0000","last_clean":"2024-03-20T08:21:44.866885+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:21:44.866885+0000","last_undegraded":"2024-03-20T08:21:44.866885+0000","last_fullsized":"2024-03-20T08:21:44.866885+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992610+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:21:45.992610+0000","last_peered":"2024-03-20T08:21:45.992610+0000","last_clean":"2024-03-20T08:21:45.992610+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:21:45.992610+0000","last_undegraded":"2024-03-20T08:21:45.992610+0000","last_fullsized":"2024-03-20T08:21:45.992610+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:02.296931+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:20:02.296931+0000","last_peered":"2024-03-20T08:20:02.296931+0000","last_clean":"2024-03-20T08:20:02.296931+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:20:02.296931+0000","last_undegraded":"2024-03-20T08:20:02.296931+0000","last_fullsized":"2024-03-20T08:20:02.296931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1202,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844843+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:19:59.844843+0000","last_peered":"2024-03-20T08:19:59.844843+0000","last_clean":"2024-03-20T08:19:59.844843+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:19:59.844843+0000","last_undegraded":"2024-03-20T08:19:59.844843+0000","last_fullsized":"2024-03-20T08:19:59.844843+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844957+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:19:59.844957+0000","last_peered":"2024-03-20T08:19:59.844957+0000","last_clean":"2024-03-20T08:19:59.844957+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:19:59.844957+0000","last_undegraded":"2024-03-20T08:19:59.844957+0000","last_fullsized":"2024-03-20T08:19:59.844957+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1396,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994341+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:22:05.994341+0000","last_peered":"2024-03-20T08:22:05.994341+0000","last_clean":"2024-03-20T08:22:05.994341+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:22:05.994341+0000","last_undegraded":"2024-03-20T08:22:05.994341+0000","last_fullsized":"2024-03-20T08:22:05.994341+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994332+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:22:05.994332+0000","last_peered":"2024-03-20T08:22:05.994332+0000","last_clean":"2024-03-20T08:22:05.994332+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:22:05.994332+0000","last_undegraded":"2024-03-20T08:22:05.994332+0000","last_fullsized":"2024-03-20T08:22:05.994332+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:42.883328+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:21:42.883328+0000","last_peered":"2024-03-20T08:21:42.883328+0000","last_clean":"2024-03-20T08:21:42.883328+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:21:42.883328+0000","last_undegraded":"2024-03-20T08:21:42.883328+0000","last_fullsized":"2024-03-20T08:21:42.883328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444464+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:21:27.444464+0000","last_peered":"2024-03-20T08:21:27.444464+0000","last_clean":"2024-03-20T08:21:27.444464+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:21:27.444464+0000","last_undegraded":"2024-03-20T08:21:27.444464+0000","last_fullsized":"2024-03-20T08:21:27.444464+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866861+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:21:44.866861+0000","last_peered":"2024-03-20T08:21:44.866861+0000","last_clean":"2024-03-20T08:21:44.866861+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:21:44.866861+0000","last_undegraded":"2024-03-20T08:21:44.866861+0000","last_fullsized":"2024-03-20T08:21:44.866861+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866825+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:21:44.866825+0000","last_peered":"2024-03-20T08:21:44.866825+0000","last_clean":"2024-03-20T08:21:44.866825+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:21:44.866825+0000","last_undegraded":"2024-03-20T08:21:44.866825+0000","last_fullsized":"2024-03-20T08:21:44.866825+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:54.732439+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:21:54.732439+0000","last_peered":"2024-03-20T08:21:54.732439+0000","last_clean":"2024-03-20T08:21:54.732439+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:21:54.732439+0000","last_undegraded":"2024-03-20T08:21:54.732439+0000","last_fullsized":"2024-03-20T08:21:54.732439+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912579+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:21:49.912579+0000","last_peered":"2024-03-20T08:21:49.912579+0000","last_clean":"2024-03-20T08:21:49.912579+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:21:49.912579+0000","last_undegraded":"2024-03-20T08:21:49.912579+0000","last_fullsized":"2024-03-20T08:21:49.912579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1391,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992645+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:21:45.992645+0000","last_peered":"2024-03-20T08:21:45.992645+0000","last_clean":"2024-03-20T08:21:45.992645+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:21:45.992645+0000","last_undegraded":"2024-03-20T08:21:45.992645+0000","last_fullsized":"2024-03-20T08:21:45.992645+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992629+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:21:45.992629+0000","last_peered":"2024-03-20T08:21:45.992629+0000","last_clean":"2024-03-20T08:21:45.992629+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:21:45.992629+0000","last_undegraded":"2024-03-20T08:21:45.992629+0000","last_fullsized":"2024-03-20T08:21:45.992629+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.844909+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:19:59.844909+0000","last_peered":"2024-03-20T08:19:59.844909+0000","last_clean":"2024-03-20T08:19:59.844909+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:19:59.844909+0000","last_undegraded":"2024-03-20T08:19:59.844909+0000","last_fullsized":"2024-03-20T08:19:59.844909+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909806+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:21:24.909806+0000","last_peered":"2024-03-20T08:21:24.909806+0000","last_clean":"2024-03-20T08:21:24.909806+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:21:24.909806+0000","last_undegraded":"2024-03-20T08:21:24.909806+0000","last_fullsized":"2024-03-20T08:21:24.909806+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909777+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:21:24.909777+0000","last_peered":"2024-03-20T08:21:24.909777+0000","last_clean":"2024-03-20T08:21:24.909777+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:21:24.909777+0000","last_undegraded":"2024-03-20T08:21:24.909777+0000","last_fullsized":"2024-03-20T08:21:24.909777+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26674,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884162+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:21:52.884162+0000","last_peered":"2024-03-20T08:21:52.884162+0000","last_clean":"2024-03-20T08:21:52.884162+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:21:52.884162+0000","last_undegraded":"2024-03-20T08:21:52.884162+0000","last_fullsized":"2024-03-20T08:21:52.884162+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1261,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.728906+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:21:24.728906+0000","last_peered":"2024-03-20T08:21:24.728906+0000","last_clean":"2024-03-20T08:21:24.728906+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:21:24.728906+0000","last_undegraded":"2024-03-20T08:21:24.728906+0000","last_fullsized":"2024-03-20T08:21:24.728906+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11692,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994261+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:22:05.994261+0000","last_peered":"2024-03-20T08:22:05.994261+0000","last_clean":"2024-03-20T08:22:05.994261+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:22:05.994261+0000","last_undegraded":"2024-03-20T08:22:05.994261+0000","last_fullsized":"2024-03-20T08:22:05.994261+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5954,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994315+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:22:05.994315+0000","last_peered":"2024-03-20T08:22:05.994315+0000","last_clean":"2024-03-20T08:22:05.994315+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:22:05.994315+0000","last_undegraded":"2024-03-20T08:22:05.994315+0000","last_fullsized":"2024-03-20T08:22:05.994315+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722598+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:20:24.722598+0000","last_peered":"2024-03-20T08:20:24.722598+0000","last_clean":"2024-03-20T08:20:24.722598+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:20:24.722598+0000","last_undegraded":"2024-03-20T08:20:24.722598+0000","last_fullsized":"2024-03-20T08:20:24.722598+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884275+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:21:52.884275+0000","last_peered":"2024-03-20T08:21:52.884275+0000","last_clean":"2024-03-20T08:21:52.884275+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:21:52.884275+0000","last_undegraded":"2024-03-20T08:21:52.884275+0000","last_fullsized":"2024-03-20T08:21:52.884275+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:02.306280+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:22:02.306280+0000","last_peered":"2024-03-20T08:22:02.306280+0000","last_clean":"2024-03-20T08:22:02.306280+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:22:02.306280+0000","last_undegraded":"2024-03-20T08:22:02.306280+0000","last_fullsized":"2024-03-20T08:22:02.306280+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3982,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:14.915318+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:22:14.915318+0000","last_peered":"2024-03-20T08:22:14.915318+0000","last_clean":"2024-03-20T08:22:14.915318+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:22:14.915318+0000","last_undegraded":"2024-03-20T08:22:14.915318+0000","last_fullsized":"2024-03-20T08:22:14.915318+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.867213+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:21:49.867213+0000","last_peered":"2024-03-20T08:21:49.867213+0000","last_clean":"2024-03-20T08:21:49.867213+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:21:49.867213+0000","last_undegraded":"2024-03-20T08:21:49.867213+0000","last_fullsized":"2024-03-20T08:21:49.867213+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1614,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912064+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:21:44.912064+0000","last_peered":"2024-03-20T08:21:44.912064+0000","last_clean":"2024-03-20T08:21:44.912064+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:21:44.912064+0000","last_undegraded":"2024-03-20T08:21:44.912064+0000","last_fullsized":"2024-03-20T08:21:44.912064+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994228+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:22:05.994228+0000","last_peered":"2024-03-20T08:22:05.994228+0000","last_clean":"2024-03-20T08:22:05.994228+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:22:05.994228+0000","last_undegraded":"2024-03-20T08:22:05.994228+0000","last_fullsized":"2024-03-20T08:22:05.994228+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912041+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:21:44.912041+0000","last_peered":"2024-03-20T08:21:44.912041+0000","last_clean":"2024-03-20T08:21:44.912041+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:21:44.912041+0000","last_undegraded":"2024-03-20T08:21:44.912041+0000","last_fullsized":"2024-03-20T08:21:44.912041+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994287+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:22:05.994287+0000","last_peered":"2024-03-20T08:22:05.994287+0000","last_clean":"2024-03-20T08:22:05.994287+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:22:05.994287+0000","last_undegraded":"2024-03-20T08:22:05.994287+0000","last_fullsized":"2024-03-20T08:22:05.994287+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1145,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915352+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:22:14.915352+0000","last_peered":"2024-03-20T08:22:14.915352+0000","last_clean":"2024-03-20T08:22:14.915352+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:22:14.915352+0000","last_undegraded":"2024-03-20T08:22:14.915352+0000","last_fullsized":"2024-03-20T08:22:14.915352+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4607,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:24.722522+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:20:24.722522+0000","last_peered":"2024-03-20T08:20:24.722522+0000","last_clean":"2024-03-20T08:20:24.722522+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:20:24.722522+0000","last_undegraded":"2024-03-20T08:20:24.722522+0000","last_fullsized":"2024-03-20T08:20:24.722522+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860100+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:20:14.860100+0000","last_peered":"2024-03-20T08:20:14.860100+0000","last_clean":"2024-03-20T08:20:14.860100+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:20:14.860100+0000","last_undegraded":"2024-03-20T08:20:14.860100+0000","last_fullsized":"2024-03-20T08:20:14.860100+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1218,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722488+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:20:24.722488+0000","last_peered":"2024-03-20T08:20:24.722488+0000","last_clean":"2024-03-20T08:20:24.722488+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:20:24.722488+0000","last_undegraded":"2024-03-20T08:20:24.722488+0000","last_fullsized":"2024-03-20T08:20:24.722488+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1176,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447941+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:21:57.447941+0000","last_peered":"2024-03-20T08:21:57.447941+0000","last_clean":"2024-03-20T08:21:57.447941+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:21:57.447941+0000","last_undegraded":"2024-03-20T08:21:57.447941+0000","last_fullsized":"2024-03-20T08:21:57.447941+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:52.884264+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:21:52.884264+0000","last_peered":"2024-03-20T08:21:52.884264+0000","last_clean":"2024-03-20T08:21:52.884264+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:21:52.884264+0000","last_undegraded":"2024-03-20T08:21:52.884264+0000","last_fullsized":"2024-03-20T08:21:52.884264+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5194,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:49.857496+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:21:49.857496+0000","last_peered":"2024-03-20T08:21:49.857496+0000","last_clean":"2024-03-20T08:21:49.857496+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:21:49.857496+0000","last_undegraded":"2024-03-20T08:21:49.857496+0000","last_fullsized":"2024-03-20T08:21:49.857496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68668,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:24.728852+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:21:24.728852+0000","last_peered":"2024-03-20T08:21:24.728852+0000","last_clean":"2024-03-20T08:21:24.728852+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:21:24.728852+0000","last_undegraded":"2024-03-20T08:21:24.728852+0000","last_fullsized":"2024-03-20T08:21:24.728852+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860060+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:20:14.860060+0000","last_peered":"2024-03-20T08:20:14.860060+0000","last_clean":"2024-03-20T08:20:14.860060+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:20:14.860060+0000","last_undegraded":"2024-03-20T08:20:14.860060+0000","last_fullsized":"2024-03-20T08:20:14.860060+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31734,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:12.876340+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:20:12.876340+0000","last_peered":"2024-03-20T08:20:12.876340+0000","last_clean":"2024-03-20T08:20:12.876340+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:20:12.876340+0000","last_undegraded":"2024-03-20T08:20:12.876340+0000","last_fullsized":"2024-03-20T08:20:12.876340+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29912,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:47.305131+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:21:47.305131+0000","last_peered":"2024-03-20T08:21:47.305131+0000","last_clean":"2024-03-20T08:21:47.305131+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:21:47.305131+0000","last_undegraded":"2024-03-20T08:21:47.305131+0000","last_fullsized":"2024-03-20T08:21:47.305131+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1310,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876311+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:20:12.876311+0000","last_peered":"2024-03-20T08:20:12.876311+0000","last_clean":"2024-03-20T08:20:12.876311+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:20:12.876311+0000","last_undegraded":"2024-03-20T08:20:12.876311+0000","last_fullsized":"2024-03-20T08:20:12.876311+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447869+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:21:57.447869+0000","last_peered":"2024-03-20T08:21:57.447869+0000","last_clean":"2024-03-20T08:21:57.447869+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:21:57.447869+0000","last_undegraded":"2024-03-20T08:21:57.447869+0000","last_fullsized":"2024-03-20T08:21:57.447869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3379,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876267+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:20:12.876267+0000","last_peered":"2024-03-20T08:20:12.876267+0000","last_clean":"2024-03-20T08:20:12.876267+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:20:12.876267+0000","last_undegraded":"2024-03-20T08:20:12.876267+0000","last_fullsized":"2024-03-20T08:20:12.876267+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876326+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:20:12.876326+0000","last_peered":"2024-03-20T08:20:12.876326+0000","last_clean":"2024-03-20T08:20:12.876326+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:20:12.876326+0000","last_undegraded":"2024-03-20T08:20:12.876326+0000","last_fullsized":"2024-03-20T08:20:12.876326+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5701,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884206+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:21:52.884206+0000","last_peered":"2024-03-20T08:21:52.884206+0000","last_clean":"2024-03-20T08:21:52.884206+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:21:52.884206+0000","last_undegraded":"2024-03-20T08:21:52.884206+0000","last_fullsized":"2024-03-20T08:21:52.884206+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:47.305170+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:21:47.305170+0000","last_peered":"2024-03-20T08:21:47.305170+0000","last_clean":"2024-03-20T08:21:47.305170+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:21:47.305170+0000","last_undegraded":"2024-03-20T08:21:47.305170+0000","last_fullsized":"2024-03-20T08:21:47.305170+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857470+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:21:49.857470+0000","last_peered":"2024-03-20T08:21:49.857470+0000","last_clean":"2024-03-20T08:21:49.857470+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:21:49.857470+0000","last_undegraded":"2024-03-20T08:21:49.857470+0000","last_fullsized":"2024-03-20T08:21:49.857470+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1308,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.876384+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:20:12.876384+0000","last_peered":"2024-03-20T08:20:12.876384+0000","last_clean":"2024-03-20T08:20:12.876384+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:20:12.876384+0000","last_undegraded":"2024-03-20T08:20:12.876384+0000","last_fullsized":"2024-03-20T08:20:12.876384+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994252+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:22:05.994252+0000","last_peered":"2024-03-20T08:22:05.994252+0000","last_clean":"2024-03-20T08:22:05.994252+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:22:05.994252+0000","last_undegraded":"2024-03-20T08:22:05.994252+0000","last_fullsized":"2024-03-20T08:22:05.994252+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1294,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860088+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:20:14.860088+0000","last_peered":"2024-03-20T08:20:14.860088+0000","last_clean":"2024-03-20T08:20:14.860088+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:20:14.860088+0000","last_undegraded":"2024-03-20T08:20:14.860088+0000","last_fullsized":"2024-03-20T08:20:14.860088+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447930+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:21:57.447930+0000","last_peered":"2024-03-20T08:21:57.447930+0000","last_clean":"2024-03-20T08:21:57.447930+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:21:57.447930+0000","last_undegraded":"2024-03-20T08:21:57.447930+0000","last_fullsized":"2024-03-20T08:21:57.447930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1279,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:14.860108+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:20:14.860108+0000","last_peered":"2024-03-20T08:20:14.860108+0000","last_clean":"2024-03-20T08:20:14.860108+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:20:14.860108+0000","last_undegraded":"2024-03-20T08:20:14.860108+0000","last_fullsized":"2024-03-20T08:20:14.860108+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857436+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:21:49.857436+0000","last_peered":"2024-03-20T08:21:49.857436+0000","last_clean":"2024-03-20T08:21:49.857436+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:21:49.857436+0000","last_undegraded":"2024-03-20T08:21:49.857436+0000","last_fullsized":"2024-03-20T08:21:49.857436+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1204,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:12.435753+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:20:12.435753+0000","last_peered":"2024-03-20T08:20:12.435753+0000","last_clean":"2024-03-20T08:20:12.435753+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:20:12.435753+0000","last_undegraded":"2024-03-20T08:20:12.435753+0000","last_fullsized":"2024-03-20T08:20:12.435753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980274,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484667,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011678,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189029,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703510,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656976,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446920,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629315,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:24:32.866 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:32.867 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:24:33.587 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:33 smithi118 ceph-mon[93121]: from='client.44699 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:33.587 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:33 smithi118 ceph-mon[95692]: from='client.44699 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:33.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:33 smithi196 ceph-mon[87080]: from='client.44699 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:33.922 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:24:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:24:33] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:24:34.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:34 smithi196 ceph-mon[87080]: pgmap v1266: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:34.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:34 smithi118 ceph-mon[93121]: pgmap v1266: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:34.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:34 smithi118 ceph-mon[95692]: pgmap v1266: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:36.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:36 smithi196 ceph-mon[87080]: pgmap v1267: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:36.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:36 smithi118 ceph-mon[93121]: pgmap v1267: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:36.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:36 smithi118 ceph-mon[95692]: pgmap v1267: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:38.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:38 smithi196 ceph-mon[87080]: pgmap v1268: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:38.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:38 smithi118 ceph-mon[93121]: pgmap v1268: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:38.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:38 smithi118 ceph-mon[95692]: pgmap v1268: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:40.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:40 smithi196 ceph-mon[87080]: pgmap v1269: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:40.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:40 smithi118 ceph-mon[93121]: pgmap v1269: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:40.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:40 smithi118 ceph-mon[95692]: pgmap v1269: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:42.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:42 smithi196 ceph-mon[87080]: pgmap v1270: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:24:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:24:42.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:24:42.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:42 smithi118 ceph-mon[93121]: pgmap v1270: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:42.672 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:42 smithi118 ceph-mon[95692]: pgmap v1270: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:42.869 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:24:43.082 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:24:43.456 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:43 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:43.456 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:43 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:43.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:43 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:43.909 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:24:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:24:43] "GET /metrics HTTP/1.1" 200 36406 "" "Prometheus/2.43.0" 2024-03-20T08:24:44.623 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:44 smithi118 ceph-mon[93121]: pgmap v1271: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:44.623 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:44 smithi118 ceph-mon[95692]: pgmap v1271: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:44.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:44 smithi196 ceph-mon[87080]: pgmap v1271: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:45.098 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:24:45.100 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:24:45.819 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1272,"stamp":"2024-03-20T08:24:44.770738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003505"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915382+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:22:14.915382+0000","last_peered":"2024-03-20T08:22:14.915382+0000","last_clean":"2024-03-20T08:22:14.915382+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:22:14.915382+0000","last_undegraded":"2024-03-20T08:22:14.915382+0000","last_fullsized":"2024-03-20T08:22:14.915382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299838+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:20:37.299838+0000","last_peered":"2024-03-20T08:20:37.299838+0000","last_clean":"2024-03-20T08:20:37.299838+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:20:37.299838+0000","last_undegraded":"2024-03-20T08:20:37.299838+0000","last_fullsized":"2024-03-20T08:20:37.299838+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994305+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:22:05.994305+0000","last_peered":"2024-03-20T08:22:05.994305+0000","last_clean":"2024-03-20T08:22:05.994305+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:22:05.994305+0000","last_undegraded":"2024-03-20T08:22:05.994305+0000","last_fullsized":"2024-03-20T08:22:05.994305+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860780+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:22:19.860780+0000","last_peered":"2024-03-20T08:22:19.860780+0000","last_clean":"2024-03-20T08:22:19.860780+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:22:19.860780+0000","last_undegraded":"2024-03-20T08:22:19.860780+0000","last_fullsized":"2024-03-20T08:22:19.860780+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898358+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:19:44.898358+0000","last_peered":"2024-03-20T08:19:44.898358+0000","last_clean":"2024-03-20T08:19:44.898358+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:19:44.898358+0000","last_undegraded":"2024-03-20T08:19:44.898358+0000","last_fullsized":"2024-03-20T08:19:44.898358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898331+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:19:44.898331+0000","last_peered":"2024-03-20T08:19:44.898331+0000","last_clean":"2024-03-20T08:19:44.898331+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:19:44.898331+0000","last_undegraded":"2024-03-20T08:19:44.898331+0000","last_fullsized":"2024-03-20T08:19:44.898331+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857764+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:19:44.857764+0000","last_peered":"2024-03-20T08:19:44.857764+0000","last_clean":"2024-03-20T08:19:44.857764+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:19:44.857764+0000","last_undegraded":"2024-03-20T08:19:44.857764+0000","last_fullsized":"2024-03-20T08:19:44.857764+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1127,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912609+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:21:49.912609+0000","last_peered":"2024-03-20T08:21:49.912609+0000","last_clean":"2024-03-20T08:21:49.912609+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:21:49.912609+0000","last_undegraded":"2024-03-20T08:21:49.912609+0000","last_fullsized":"2024-03-20T08:21:49.912609+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:59.719522+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:19:59.719522+0000","last_peered":"2024-03-20T08:19:59.719522+0000","last_clean":"2024-03-20T08:19:59.719522+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:19:59.719522+0000","last_undegraded":"2024-03-20T08:19:59.719522+0000","last_fullsized":"2024-03-20T08:19:59.719522+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866885+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:21:44.866885+0000","last_peered":"2024-03-20T08:21:44.866885+0000","last_clean":"2024-03-20T08:21:44.866885+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:21:44.866885+0000","last_undegraded":"2024-03-20T08:21:44.866885+0000","last_fullsized":"2024-03-20T08:21:44.866885+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992610+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:21:45.992610+0000","last_peered":"2024-03-20T08:21:45.992610+0000","last_clean":"2024-03-20T08:21:45.992610+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:21:45.992610+0000","last_undegraded":"2024-03-20T08:21:45.992610+0000","last_fullsized":"2024-03-20T08:21:45.992610+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:22.307947+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:22:22.307947+0000","last_peered":"2024-03-20T08:22:22.307947+0000","last_clean":"2024-03-20T08:22:22.307947+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:22:22.307947+0000","last_undegraded":"2024-03-20T08:22:22.307947+0000","last_fullsized":"2024-03-20T08:22:22.307947+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860734+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:22:19.860734+0000","last_peered":"2024-03-20T08:22:19.860734+0000","last_clean":"2024-03-20T08:22:19.860734+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:22:19.860734+0000","last_undegraded":"2024-03-20T08:22:19.860734+0000","last_fullsized":"2024-03-20T08:22:19.860734+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1225,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860800+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:22:19.860800+0000","last_peered":"2024-03-20T08:22:19.860800+0000","last_clean":"2024-03-20T08:22:19.860800+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:22:19.860800+0000","last_undegraded":"2024-03-20T08:22:19.860800+0000","last_fullsized":"2024-03-20T08:22:19.860800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1396,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994341+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:22:05.994341+0000","last_peered":"2024-03-20T08:22:05.994341+0000","last_clean":"2024-03-20T08:22:05.994341+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:22:05.994341+0000","last_undegraded":"2024-03-20T08:22:05.994341+0000","last_fullsized":"2024-03-20T08:22:05.994341+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994332+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:22:05.994332+0000","last_peered":"2024-03-20T08:22:05.994332+0000","last_clean":"2024-03-20T08:22:05.994332+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:22:05.994332+0000","last_undegraded":"2024-03-20T08:22:05.994332+0000","last_fullsized":"2024-03-20T08:22:05.994332+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1123,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898407+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:19:44.898407+0000","last_peered":"2024-03-20T08:19:44.898407+0000","last_clean":"2024-03-20T08:19:44.898407+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:19:44.898407+0000","last_undegraded":"2024-03-20T08:19:44.898407+0000","last_fullsized":"2024-03-20T08:19:44.898407+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1131,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.898377+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:19:44.898377+0000","last_peered":"2024-03-20T08:19:44.898377+0000","last_clean":"2024-03-20T08:19:44.898377+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:19:44.898377+0000","last_undegraded":"2024-03-20T08:19:44.898377+0000","last_fullsized":"2024-03-20T08:19:44.898377+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:42.883328+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:21:42.883328+0000","last_peered":"2024-03-20T08:21:42.883328+0000","last_clean":"2024-03-20T08:21:42.883328+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:21:42.883328+0000","last_undegraded":"2024-03-20T08:21:42.883328+0000","last_fullsized":"2024-03-20T08:21:42.883328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:44.857826+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:19:44.857826+0000","last_peered":"2024-03-20T08:19:44.857826+0000","last_clean":"2024-03-20T08:19:44.857826+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:19:44.857826+0000","last_undegraded":"2024-03-20T08:19:44.857826+0000","last_fullsized":"2024-03-20T08:19:44.857826+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444464+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:21:27.444464+0000","last_peered":"2024-03-20T08:21:27.444464+0000","last_clean":"2024-03-20T08:21:27.444464+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:21:27.444464+0000","last_undegraded":"2024-03-20T08:21:27.444464+0000","last_fullsized":"2024-03-20T08:21:27.444464+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866861+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:21:44.866861+0000","last_peered":"2024-03-20T08:21:44.866861+0000","last_clean":"2024-03-20T08:21:44.866861+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:21:44.866861+0000","last_undegraded":"2024-03-20T08:21:44.866861+0000","last_fullsized":"2024-03-20T08:21:44.866861+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866825+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:21:44.866825+0000","last_peered":"2024-03-20T08:21:44.866825+0000","last_clean":"2024-03-20T08:21:44.866825+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:21:44.866825+0000","last_undegraded":"2024-03-20T08:21:44.866825+0000","last_fullsized":"2024-03-20T08:21:44.866825+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:19:55.983402+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:19:55.983402+0000","last_peered":"2024-03-20T08:19:55.983402+0000","last_clean":"2024-03-20T08:19:55.983402+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:19:55.983402+0000","last_undegraded":"2024-03-20T08:19:55.983402+0000","last_fullsized":"2024-03-20T08:19:55.983402+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:54.732439+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:21:54.732439+0000","last_peered":"2024-03-20T08:21:54.732439+0000","last_clean":"2024-03-20T08:21:54.732439+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:21:54.732439+0000","last_undegraded":"2024-03-20T08:21:54.732439+0000","last_fullsized":"2024-03-20T08:21:54.732439+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912579+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:21:49.912579+0000","last_peered":"2024-03-20T08:21:49.912579+0000","last_clean":"2024-03-20T08:21:49.912579+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:21:49.912579+0000","last_undegraded":"2024-03-20T08:21:49.912579+0000","last_fullsized":"2024-03-20T08:21:49.912579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1391,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992645+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:21:45.992645+0000","last_peered":"2024-03-20T08:21:45.992645+0000","last_clean":"2024-03-20T08:21:45.992645+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:21:45.992645+0000","last_undegraded":"2024-03-20T08:21:45.992645+0000","last_fullsized":"2024-03-20T08:21:45.992645+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992629+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:21:45.992629+0000","last_peered":"2024-03-20T08:21:45.992629+0000","last_clean":"2024-03-20T08:21:45.992629+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:21:45.992629+0000","last_undegraded":"2024-03-20T08:21:45.992629+0000","last_fullsized":"2024-03-20T08:21:45.992629+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860764+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:22:19.860764+0000","last_peered":"2024-03-20T08:22:19.860764+0000","last_clean":"2024-03-20T08:22:19.860764+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:22:19.860764+0000","last_undegraded":"2024-03-20T08:22:19.860764+0000","last_fullsized":"2024-03-20T08:22:19.860764+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909806+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:21:24.909806+0000","last_peered":"2024-03-20T08:21:24.909806+0000","last_clean":"2024-03-20T08:21:24.909806+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:21:24.909806+0000","last_undegraded":"2024-03-20T08:21:24.909806+0000","last_fullsized":"2024-03-20T08:21:24.909806+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909777+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:21:24.909777+0000","last_peered":"2024-03-20T08:21:24.909777+0000","last_clean":"2024-03-20T08:21:24.909777+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:21:24.909777+0000","last_undegraded":"2024-03-20T08:21:24.909777+0000","last_fullsized":"2024-03-20T08:21:24.909777+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26674,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884162+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:21:52.884162+0000","last_peered":"2024-03-20T08:21:52.884162+0000","last_clean":"2024-03-20T08:21:52.884162+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:21:52.884162+0000","last_undegraded":"2024-03-20T08:21:52.884162+0000","last_fullsized":"2024-03-20T08:21:52.884162+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1261,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.728906+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:21:24.728906+0000","last_peered":"2024-03-20T08:21:24.728906+0000","last_clean":"2024-03-20T08:21:24.728906+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:21:24.728906+0000","last_undegraded":"2024-03-20T08:21:24.728906+0000","last_fullsized":"2024-03-20T08:21:24.728906+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11692,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994261+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:22:05.994261+0000","last_peered":"2024-03-20T08:22:05.994261+0000","last_clean":"2024-03-20T08:22:05.994261+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:22:05.994261+0000","last_undegraded":"2024-03-20T08:22:05.994261+0000","last_fullsized":"2024-03-20T08:22:05.994261+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299771+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:20:37.299771+0000","last_peered":"2024-03-20T08:20:37.299771+0000","last_clean":"2024-03-20T08:20:37.299771+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:20:37.299771+0000","last_undegraded":"2024-03-20T08:20:37.299771+0000","last_fullsized":"2024-03-20T08:20:37.299771+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299794+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:20:37.299794+0000","last_peered":"2024-03-20T08:20:37.299794+0000","last_clean":"2024-03-20T08:20:37.299794+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:20:37.299794+0000","last_undegraded":"2024-03-20T08:20:37.299794+0000","last_fullsized":"2024-03-20T08:20:37.299794+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5954,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994315+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:22:05.994315+0000","last_peered":"2024-03-20T08:22:05.994315+0000","last_clean":"2024-03-20T08:22:05.994315+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:22:05.994315+0000","last_undegraded":"2024-03-20T08:22:05.994315+0000","last_fullsized":"2024-03-20T08:22:05.994315+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299808+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:20:37.299808+0000","last_peered":"2024-03-20T08:20:37.299808+0000","last_clean":"2024-03-20T08:20:37.299808+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:20:37.299808+0000","last_undegraded":"2024-03-20T08:20:37.299808+0000","last_fullsized":"2024-03-20T08:20:37.299808+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:29.735999+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:22:29.735999+0000","last_peered":"2024-03-20T08:22:29.735999+0000","last_clean":"2024-03-20T08:22:29.735999+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:22:29.735999+0000","last_undegraded":"2024-03-20T08:22:29.735999+0000","last_fullsized":"2024-03-20T08:22:29.735999+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884275+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:21:52.884275+0000","last_peered":"2024-03-20T08:21:52.884275+0000","last_clean":"2024-03-20T08:21:52.884275+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:21:52.884275+0000","last_undegraded":"2024-03-20T08:21:52.884275+0000","last_fullsized":"2024-03-20T08:21:52.884275+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:02.306280+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:22:02.306280+0000","last_peered":"2024-03-20T08:22:02.306280+0000","last_clean":"2024-03-20T08:22:02.306280+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:22:02.306280+0000","last_undegraded":"2024-03-20T08:22:02.306280+0000","last_fullsized":"2024-03-20T08:22:02.306280+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3982,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:14.915318+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:22:14.915318+0000","last_peered":"2024-03-20T08:22:14.915318+0000","last_clean":"2024-03-20T08:22:14.915318+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:22:14.915318+0000","last_undegraded":"2024-03-20T08:22:14.915318+0000","last_fullsized":"2024-03-20T08:22:14.915318+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.867213+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:21:49.867213+0000","last_peered":"2024-03-20T08:21:49.867213+0000","last_clean":"2024-03-20T08:21:49.867213+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:21:49.867213+0000","last_undegraded":"2024-03-20T08:21:49.867213+0000","last_fullsized":"2024-03-20T08:21:49.867213+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1614,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912064+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:21:44.912064+0000","last_peered":"2024-03-20T08:21:44.912064+0000","last_clean":"2024-03-20T08:21:44.912064+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:21:44.912064+0000","last_undegraded":"2024-03-20T08:21:44.912064+0000","last_fullsized":"2024-03-20T08:21:44.912064+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:35.987032+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:20:35.987032+0000","last_peered":"2024-03-20T08:20:35.987032+0000","last_clean":"2024-03-20T08:20:35.987032+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:20:35.987032+0000","last_undegraded":"2024-03-20T08:20:35.987032+0000","last_fullsized":"2024-03-20T08:20:35.987032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994228+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:22:05.994228+0000","last_peered":"2024-03-20T08:22:05.994228+0000","last_clean":"2024-03-20T08:22:05.994228+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:22:05.994228+0000","last_undegraded":"2024-03-20T08:22:05.994228+0000","last_fullsized":"2024-03-20T08:22:05.994228+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912041+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:21:44.912041+0000","last_peered":"2024-03-20T08:21:44.912041+0000","last_clean":"2024-03-20T08:21:44.912041+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:21:44.912041+0000","last_undegraded":"2024-03-20T08:21:44.912041+0000","last_fullsized":"2024-03-20T08:21:44.912041+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994287+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:22:05.994287+0000","last_peered":"2024-03-20T08:22:05.994287+0000","last_clean":"2024-03-20T08:22:05.994287+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:22:05.994287+0000","last_undegraded":"2024-03-20T08:22:05.994287+0000","last_fullsized":"2024-03-20T08:22:05.994287+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1145,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915352+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:22:14.915352+0000","last_peered":"2024-03-20T08:22:14.915352+0000","last_clean":"2024-03-20T08:22:14.915352+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:22:14.915352+0000","last_undegraded":"2024-03-20T08:22:14.915352+0000","last_fullsized":"2024-03-20T08:22:14.915352+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.299819+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:20:37.299819+0000","last_peered":"2024-03-20T08:20:37.299819+0000","last_clean":"2024-03-20T08:20:37.299819+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:20:37.299819+0000","last_undegraded":"2024-03-20T08:20:37.299819+0000","last_fullsized":"2024-03-20T08:20:37.299819+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4608,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:29.735964+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:22:29.735964+0000","last_peered":"2024-03-20T08:22:29.735964+0000","last_clean":"2024-03-20T08:22:29.735964+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:22:29.735964+0000","last_undegraded":"2024-03-20T08:22:29.735964+0000","last_fullsized":"2024-03-20T08:22:29.735964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779096+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:22:24.779096+0000","last_peered":"2024-03-20T08:22:24.779096+0000","last_clean":"2024-03-20T08:22:24.779096+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:22:24.779096+0000","last_undegraded":"2024-03-20T08:22:24.779096+0000","last_fullsized":"2024-03-20T08:22:24.779096+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1219,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:29.735941+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:22:29.735941+0000","last_peered":"2024-03-20T08:22:29.735941+0000","last_clean":"2024-03-20T08:22:29.735941+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:22:29.735941+0000","last_undegraded":"2024-03-20T08:22:29.735941+0000","last_fullsized":"2024-03-20T08:22:29.735941+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1176,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447941+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:21:57.447941+0000","last_peered":"2024-03-20T08:21:57.447941+0000","last_clean":"2024-03-20T08:21:57.447941+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:21:57.447941+0000","last_undegraded":"2024-03-20T08:21:57.447941+0000","last_fullsized":"2024-03-20T08:21:57.447941+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:52.884264+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:21:52.884264+0000","last_peered":"2024-03-20T08:21:52.884264+0000","last_clean":"2024-03-20T08:21:52.884264+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:21:52.884264+0000","last_undegraded":"2024-03-20T08:21:52.884264+0000","last_fullsized":"2024-03-20T08:21:52.884264+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5194,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:49.857496+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:21:49.857496+0000","last_peered":"2024-03-20T08:21:49.857496+0000","last_clean":"2024-03-20T08:21:49.857496+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:21:49.857496+0000","last_undegraded":"2024-03-20T08:21:49.857496+0000","last_fullsized":"2024-03-20T08:21:49.857496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1117,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:34.903866+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:20:34.903866+0000","last_peered":"2024-03-20T08:20:34.903866+0000","last_clean":"2024-03-20T08:20:34.903866+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:20:34.903866+0000","last_undegraded":"2024-03-20T08:20:34.903866+0000","last_fullsized":"2024-03-20T08:20:34.903866+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68668,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:24.728852+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:21:24.728852+0000","last_peered":"2024-03-20T08:21:24.728852+0000","last_clean":"2024-03-20T08:21:24.728852+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:21:24.728852+0000","last_undegraded":"2024-03-20T08:21:24.728852+0000","last_fullsized":"2024-03-20T08:21:24.728852+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779055+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:22:24.779055+0000","last_peered":"2024-03-20T08:22:24.779055+0000","last_clean":"2024-03-20T08:22:24.779055+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:22:24.779055+0000","last_undegraded":"2024-03-20T08:22:24.779055+0000","last_fullsized":"2024-03-20T08:22:24.779055+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31735,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:27.886950+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:22:27.886950+0000","last_peered":"2024-03-20T08:22:27.886950+0000","last_clean":"2024-03-20T08:22:27.886950+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:22:27.886950+0000","last_undegraded":"2024-03-20T08:22:27.886950+0000","last_fullsized":"2024-03-20T08:22:27.886950+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29912,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:47.305131+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:21:47.305131+0000","last_peered":"2024-03-20T08:21:47.305131+0000","last_clean":"2024-03-20T08:21:47.305131+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:21:47.305131+0000","last_undegraded":"2024-03-20T08:21:47.305131+0000","last_fullsized":"2024-03-20T08:21:47.305131+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1311,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886931+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:22:27.886931+0000","last_peered":"2024-03-20T08:22:27.886931+0000","last_clean":"2024-03-20T08:22:27.886931+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:22:27.886931+0000","last_undegraded":"2024-03-20T08:22:27.886931+0000","last_fullsized":"2024-03-20T08:22:27.886931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447869+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:21:57.447869+0000","last_peered":"2024-03-20T08:21:57.447869+0000","last_clean":"2024-03-20T08:21:57.447869+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:21:57.447869+0000","last_undegraded":"2024-03-20T08:21:57.447869+0000","last_fullsized":"2024-03-20T08:21:57.447869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3380,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886897+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:22:27.886897+0000","last_peered":"2024-03-20T08:22:27.886897+0000","last_clean":"2024-03-20T08:22:27.886897+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:22:27.886897+0000","last_undegraded":"2024-03-20T08:22:27.886897+0000","last_fullsized":"2024-03-20T08:22:27.886897+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886942+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:22:27.886942+0000","last_peered":"2024-03-20T08:22:27.886942+0000","last_clean":"2024-03-20T08:22:27.886942+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:22:27.886942+0000","last_undegraded":"2024-03-20T08:22:27.886942+0000","last_fullsized":"2024-03-20T08:22:27.886942+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5701,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884206+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:21:52.884206+0000","last_peered":"2024-03-20T08:21:52.884206+0000","last_clean":"2024-03-20T08:21:52.884206+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:21:52.884206+0000","last_undegraded":"2024-03-20T08:21:52.884206+0000","last_fullsized":"2024-03-20T08:21:52.884206+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:47.305170+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:21:47.305170+0000","last_peered":"2024-03-20T08:21:47.305170+0000","last_clean":"2024-03-20T08:21:47.305170+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:21:47.305170+0000","last_undegraded":"2024-03-20T08:21:47.305170+0000","last_fullsized":"2024-03-20T08:21:47.305170+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857470+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:21:49.857470+0000","last_peered":"2024-03-20T08:21:49.857470+0000","last_clean":"2024-03-20T08:21:49.857470+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:21:49.857470+0000","last_undegraded":"2024-03-20T08:21:49.857470+0000","last_fullsized":"2024-03-20T08:21:49.857470+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4960,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:20:34.903927+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:20:34.903927+0000","last_peered":"2024-03-20T08:20:34.903927+0000","last_clean":"2024-03-20T08:20:34.903927+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:20:34.903927+0000","last_undegraded":"2024-03-20T08:20:34.903927+0000","last_fullsized":"2024-03-20T08:20:34.903927+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886979+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:22:27.886979+0000","last_peered":"2024-03-20T08:22:27.886979+0000","last_clean":"2024-03-20T08:22:27.886979+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:22:27.886979+0000","last_undegraded":"2024-03-20T08:22:27.886979+0000","last_fullsized":"2024-03-20T08:22:27.886979+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994252+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:22:05.994252+0000","last_peered":"2024-03-20T08:22:05.994252+0000","last_clean":"2024-03-20T08:22:05.994252+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:22:05.994252+0000","last_undegraded":"2024-03-20T08:22:05.994252+0000","last_fullsized":"2024-03-20T08:22:05.994252+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779085+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:22:24.779085+0000","last_peered":"2024-03-20T08:22:24.779085+0000","last_clean":"2024-03-20T08:22:24.779085+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:22:24.779085+0000","last_undegraded":"2024-03-20T08:22:24.779085+0000","last_fullsized":"2024-03-20T08:22:24.779085+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447930+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:21:57.447930+0000","last_peered":"2024-03-20T08:21:57.447930+0000","last_clean":"2024-03-20T08:21:57.447930+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:21:57.447930+0000","last_undegraded":"2024-03-20T08:21:57.447930+0000","last_fullsized":"2024-03-20T08:21:57.447930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779104+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:22:24.779104+0000","last_peered":"2024-03-20T08:22:24.779104+0000","last_clean":"2024-03-20T08:22:24.779104+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:22:24.779104+0000","last_undegraded":"2024-03-20T08:22:24.779104+0000","last_fullsized":"2024-03-20T08:22:24.779104+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857436+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:21:49.857436+0000","last_peered":"2024-03-20T08:21:49.857436+0000","last_clean":"2024-03-20T08:21:49.857436+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:21:49.857436+0000","last_undegraded":"2024-03-20T08:21:49.857436+0000","last_fullsized":"2024-03-20T08:21:49.857436+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.451137+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:22:27.451137+0000","last_peered":"2024-03-20T08:22:27.451137+0000","last_clean":"2024-03-20T08:22:27.451137+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:22:27.451137+0000","last_undegraded":"2024-03-20T08:22:27.451137+0000","last_fullsized":"2024-03-20T08:22:27.451137+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980276,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484670,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011681,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189032,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703513,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656978,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446923,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629317,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:24:45.824 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.824 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:45.825 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:24:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:46 smithi196 ceph-mon[87080]: pgmap v1272: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:46.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:46 smithi196 ceph-mon[87080]: from='client.34785 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:46.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:46 smithi118 ceph-mon[93121]: pgmap v1272: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:46.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:46 smithi118 ceph-mon[93121]: from='client.34785 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:46 smithi118 ceph-mon[95692]: pgmap v1272: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:46.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:46 smithi118 ceph-mon[95692]: from='client.34785 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:48.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:48 smithi196 ceph-mon[87080]: pgmap v1273: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:48.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:48 smithi118 ceph-mon[93121]: pgmap v1273: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:48.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:48 smithi118 ceph-mon[95692]: pgmap v1273: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:50.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:50 smithi196 ceph-mon[87080]: pgmap v1274: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:50.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:50 smithi118 ceph-mon[93121]: pgmap v1274: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:50.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:50 smithi118 ceph-mon[95692]: pgmap v1274: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:52.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:52 smithi196 ceph-mon[87080]: pgmap v1275: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:52.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:24:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:24:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:24:52.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:52 smithi118 ceph-mon[93121]: pgmap v1275: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:52.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:52 smithi118 ceph-mon[95692]: pgmap v1275: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:53.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:24:53 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:24:53] "GET /metrics HTTP/1.1" 200 36406 "" "Prometheus/2.43.0" 2024-03-20T08:24:54.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:54 smithi196 ceph-mon[87080]: pgmap v1276: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:54.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:54 smithi118 ceph-mon[93121]: pgmap v1276: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:54.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:54 smithi118 ceph-mon[95692]: pgmap v1276: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:55.826 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:24:56.042 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:24:56.414 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:56 smithi118 ceph-mon[93121]: pgmap v1277: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:56.414 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:56 smithi118 ceph-mon[95692]: pgmap v1277: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:56.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:56 smithi196 ceph-mon[87080]: pgmap v1277: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:58.057 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:24:58.059 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:24:58.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:58 smithi118 ceph-mon[93121]: pgmap v1278: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:58.514 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:58 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:58.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:58 smithi118 ceph-mon[95692]: pgmap v1278: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:58.514 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:58 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:58 smithi196 ceph-mon[87080]: pgmap v1278: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:24:58.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:58 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:24:58.980 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1278,"stamp":"2024-03-20T08:24:56.773843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003104"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915382+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:22:14.915382+0000","last_peered":"2024-03-20T08:22:14.915382+0000","last_clean":"2024-03-20T08:22:14.915382+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:22:14.915382+0000","last_undegraded":"2024-03-20T08:22:14.915382+0000","last_fullsized":"2024-03-20T08:22:14.915382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309560+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:22:42.309560+0000","last_peered":"2024-03-20T08:22:42.309560+0000","last_clean":"2024-03-20T08:22:42.309560+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:22:42.309560+0000","last_undegraded":"2024-03-20T08:22:42.309560+0000","last_fullsized":"2024-03-20T08:22:42.309560+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994305+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:22:05.994305+0000","last_peered":"2024-03-20T08:22:05.994305+0000","last_clean":"2024-03-20T08:22:05.994305+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:22:05.994305+0000","last_undegraded":"2024-03-20T08:22:05.994305+0000","last_fullsized":"2024-03-20T08:22:05.994305+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860780+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:22:19.860780+0000","last_peered":"2024-03-20T08:22:19.860780+0000","last_clean":"2024-03-20T08:22:19.860780+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:22:19.860780+0000","last_undegraded":"2024-03-20T08:22:19.860780+0000","last_fullsized":"2024-03-20T08:22:19.860780+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915278+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:22:14.915278+0000","last_peered":"2024-03-20T08:22:14.915278+0000","last_clean":"2024-03-20T08:22:14.915278+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:22:14.915278+0000","last_undegraded":"2024-03-20T08:22:14.915278+0000","last_fullsized":"2024-03-20T08:22:14.915278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915248+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:22:14.915248+0000","last_peered":"2024-03-20T08:22:14.915248+0000","last_clean":"2024-03-20T08:22:14.915248+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:22:14.915248+0000","last_undegraded":"2024-03-20T08:22:14.915248+0000","last_fullsized":"2024-03-20T08:22:14.915248+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.778123+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:22:14.778123+0000","last_peered":"2024-03-20T08:22:14.778123+0000","last_clean":"2024-03-20T08:22:14.778123+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:22:14.778123+0000","last_undegraded":"2024-03-20T08:22:14.778123+0000","last_fullsized":"2024-03-20T08:22:14.778123+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1127,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912609+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:21:49.912609+0000","last_peered":"2024-03-20T08:21:49.912609+0000","last_clean":"2024-03-20T08:21:49.912609+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:21:49.912609+0000","last_undegraded":"2024-03-20T08:21:49.912609+0000","last_fullsized":"2024-03-20T08:21:49.912609+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.735508+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:22:24.735508+0000","last_peered":"2024-03-20T08:22:24.735508+0000","last_clean":"2024-03-20T08:22:24.735508+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:22:24.735508+0000","last_undegraded":"2024-03-20T08:22:24.735508+0000","last_fullsized":"2024-03-20T08:22:24.735508+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866885+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:21:44.866885+0000","last_peered":"2024-03-20T08:21:44.866885+0000","last_clean":"2024-03-20T08:21:44.866885+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:21:44.866885+0000","last_undegraded":"2024-03-20T08:21:44.866885+0000","last_fullsized":"2024-03-20T08:21:44.866885+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992610+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:21:45.992610+0000","last_peered":"2024-03-20T08:21:45.992610+0000","last_clean":"2024-03-20T08:21:45.992610+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:21:45.992610+0000","last_undegraded":"2024-03-20T08:21:45.992610+0000","last_fullsized":"2024-03-20T08:21:45.992610+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:22.307947+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:22:22.307947+0000","last_peered":"2024-03-20T08:22:22.307947+0000","last_clean":"2024-03-20T08:22:22.307947+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:22:22.307947+0000","last_undegraded":"2024-03-20T08:22:22.307947+0000","last_fullsized":"2024-03-20T08:22:22.307947+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860734+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:22:19.860734+0000","last_peered":"2024-03-20T08:22:19.860734+0000","last_clean":"2024-03-20T08:22:19.860734+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:22:19.860734+0000","last_undegraded":"2024-03-20T08:22:19.860734+0000","last_fullsized":"2024-03-20T08:22:19.860734+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1225,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860800+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:22:19.860800+0000","last_peered":"2024-03-20T08:22:19.860800+0000","last_clean":"2024-03-20T08:22:19.860800+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:22:19.860800+0000","last_undegraded":"2024-03-20T08:22:19.860800+0000","last_fullsized":"2024-03-20T08:22:19.860800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1396,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994341+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:22:05.994341+0000","last_peered":"2024-03-20T08:22:05.994341+0000","last_clean":"2024-03-20T08:22:05.994341+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:22:05.994341+0000","last_undegraded":"2024-03-20T08:22:05.994341+0000","last_fullsized":"2024-03-20T08:22:05.994341+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994332+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:22:05.994332+0000","last_peered":"2024-03-20T08:22:05.994332+0000","last_clean":"2024-03-20T08:22:05.994332+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:22:05.994332+0000","last_undegraded":"2024-03-20T08:22:05.994332+0000","last_fullsized":"2024-03-20T08:22:05.994332+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915365+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:22:14.915365+0000","last_peered":"2024-03-20T08:22:14.915365+0000","last_clean":"2024-03-20T08:22:14.915365+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:22:14.915365+0000","last_undegraded":"2024-03-20T08:22:14.915365+0000","last_fullsized":"2024-03-20T08:22:14.915365+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1132,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915290+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:22:14.915290+0000","last_peered":"2024-03-20T08:22:14.915290+0000","last_clean":"2024-03-20T08:22:14.915290+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:22:14.915290+0000","last_undegraded":"2024-03-20T08:22:14.915290+0000","last_fullsized":"2024-03-20T08:22:14.915290+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:42.883328+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:21:42.883328+0000","last_peered":"2024-03-20T08:21:42.883328+0000","last_clean":"2024-03-20T08:21:42.883328+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:21:42.883328+0000","last_undegraded":"2024-03-20T08:21:42.883328+0000","last_fullsized":"2024-03-20T08:21:42.883328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.778159+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:22:14.778159+0000","last_peered":"2024-03-20T08:22:14.778159+0000","last_clean":"2024-03-20T08:22:14.778159+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:22:14.778159+0000","last_undegraded":"2024-03-20T08:22:14.778159+0000","last_fullsized":"2024-03-20T08:22:14.778159+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444464+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:21:27.444464+0000","last_peered":"2024-03-20T08:21:27.444464+0000","last_clean":"2024-03-20T08:21:27.444464+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:21:27.444464+0000","last_undegraded":"2024-03-20T08:21:27.444464+0000","last_fullsized":"2024-03-20T08:21:27.444464+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866861+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:21:44.866861+0000","last_peered":"2024-03-20T08:21:44.866861+0000","last_clean":"2024-03-20T08:21:44.866861+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:21:44.866861+0000","last_undegraded":"2024-03-20T08:21:44.866861+0000","last_fullsized":"2024-03-20T08:21:44.866861+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866825+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:21:44.866825+0000","last_peered":"2024-03-20T08:21:44.866825+0000","last_clean":"2024-03-20T08:21:44.866825+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:21:44.866825+0000","last_undegraded":"2024-03-20T08:21:44.866825+0000","last_fullsized":"2024-03-20T08:21:44.866825+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:20.995500+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:22:20.995500+0000","last_peered":"2024-03-20T08:22:20.995500+0000","last_clean":"2024-03-20T08:22:20.995500+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:22:20.995500+0000","last_undegraded":"2024-03-20T08:22:20.995500+0000","last_fullsized":"2024-03-20T08:22:20.995500+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:54.732439+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:21:54.732439+0000","last_peered":"2024-03-20T08:21:54.732439+0000","last_clean":"2024-03-20T08:21:54.732439+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:21:54.732439+0000","last_undegraded":"2024-03-20T08:21:54.732439+0000","last_fullsized":"2024-03-20T08:21:54.732439+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912579+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:21:49.912579+0000","last_peered":"2024-03-20T08:21:49.912579+0000","last_clean":"2024-03-20T08:21:49.912579+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:21:49.912579+0000","last_undegraded":"2024-03-20T08:21:49.912579+0000","last_fullsized":"2024-03-20T08:21:49.912579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1391,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992645+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:21:45.992645+0000","last_peered":"2024-03-20T08:21:45.992645+0000","last_clean":"2024-03-20T08:21:45.992645+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:21:45.992645+0000","last_undegraded":"2024-03-20T08:21:45.992645+0000","last_fullsized":"2024-03-20T08:21:45.992645+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992629+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:21:45.992629+0000","last_peered":"2024-03-20T08:21:45.992629+0000","last_clean":"2024-03-20T08:21:45.992629+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:21:45.992629+0000","last_undegraded":"2024-03-20T08:21:45.992629+0000","last_fullsized":"2024-03-20T08:21:45.992629+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860764+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:22:19.860764+0000","last_peered":"2024-03-20T08:22:19.860764+0000","last_clean":"2024-03-20T08:22:19.860764+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:22:19.860764+0000","last_undegraded":"2024-03-20T08:22:19.860764+0000","last_fullsized":"2024-03-20T08:22:19.860764+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909806+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:21:24.909806+0000","last_peered":"2024-03-20T08:21:24.909806+0000","last_clean":"2024-03-20T08:21:24.909806+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:21:24.909806+0000","last_undegraded":"2024-03-20T08:21:24.909806+0000","last_fullsized":"2024-03-20T08:21:24.909806+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909777+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:21:24.909777+0000","last_peered":"2024-03-20T08:21:24.909777+0000","last_clean":"2024-03-20T08:21:24.909777+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:21:24.909777+0000","last_undegraded":"2024-03-20T08:21:24.909777+0000","last_fullsized":"2024-03-20T08:21:24.909777+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26674,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884162+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:21:52.884162+0000","last_peered":"2024-03-20T08:21:52.884162+0000","last_clean":"2024-03-20T08:21:52.884162+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:21:52.884162+0000","last_undegraded":"2024-03-20T08:21:52.884162+0000","last_fullsized":"2024-03-20T08:21:52.884162+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1261,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.728906+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:21:24.728906+0000","last_peered":"2024-03-20T08:21:24.728906+0000","last_clean":"2024-03-20T08:21:24.728906+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:21:24.728906+0000","last_undegraded":"2024-03-20T08:21:24.728906+0000","last_fullsized":"2024-03-20T08:21:24.728906+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11692,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994261+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:22:05.994261+0000","last_peered":"2024-03-20T08:22:05.994261+0000","last_clean":"2024-03-20T08:22:05.994261+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:22:05.994261+0000","last_undegraded":"2024-03-20T08:22:05.994261+0000","last_fullsized":"2024-03-20T08:22:05.994261+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309503+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:22:42.309503+0000","last_peered":"2024-03-20T08:22:42.309503+0000","last_clean":"2024-03-20T08:22:42.309503+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:22:42.309503+0000","last_undegraded":"2024-03-20T08:22:42.309503+0000","last_fullsized":"2024-03-20T08:22:42.309503+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309527+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:22:42.309527+0000","last_peered":"2024-03-20T08:22:42.309527+0000","last_clean":"2024-03-20T08:22:42.309527+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:22:42.309527+0000","last_undegraded":"2024-03-20T08:22:42.309527+0000","last_fullsized":"2024-03-20T08:22:42.309527+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850537+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:20:49.850537+0000","last_peered":"2024-03-20T08:20:49.850537+0000","last_clean":"2024-03-20T08:20:49.850537+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:20:49.850537+0000","last_undegraded":"2024-03-20T08:20:49.850537+0000","last_fullsized":"2024-03-20T08:20:49.850537+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5954,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994315+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:22:05.994315+0000","last_peered":"2024-03-20T08:22:05.994315+0000","last_clean":"2024-03-20T08:22:05.994315+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:22:05.994315+0000","last_undegraded":"2024-03-20T08:22:05.994315+0000","last_fullsized":"2024-03-20T08:22:05.994315+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309539+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:22:42.309539+0000","last_peered":"2024-03-20T08:22:42.309539+0000","last_clean":"2024-03-20T08:22:42.309539+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:22:42.309539+0000","last_undegraded":"2024-03-20T08:22:42.309539+0000","last_fullsized":"2024-03-20T08:22:42.309539+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850474+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:20:49.850474+0000","last_peered":"2024-03-20T08:20:49.850474+0000","last_clean":"2024-03-20T08:20:49.850474+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:20:49.850474+0000","last_undegraded":"2024-03-20T08:20:49.850474+0000","last_fullsized":"2024-03-20T08:20:49.850474+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:29.735999+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:22:29.735999+0000","last_peered":"2024-03-20T08:22:29.735999+0000","last_clean":"2024-03-20T08:22:29.735999+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:22:29.735999+0000","last_undegraded":"2024-03-20T08:22:29.735999+0000","last_fullsized":"2024-03-20T08:22:29.735999+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884275+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:21:52.884275+0000","last_peered":"2024-03-20T08:21:52.884275+0000","last_clean":"2024-03-20T08:21:52.884275+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:21:52.884275+0000","last_undegraded":"2024-03-20T08:21:52.884275+0000","last_fullsized":"2024-03-20T08:21:52.884275+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:02.306280+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:22:02.306280+0000","last_peered":"2024-03-20T08:22:02.306280+0000","last_clean":"2024-03-20T08:22:02.306280+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:22:02.306280+0000","last_undegraded":"2024-03-20T08:22:02.306280+0000","last_fullsized":"2024-03-20T08:22:02.306280+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3982,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:14.915318+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:22:14.915318+0000","last_peered":"2024-03-20T08:22:14.915318+0000","last_clean":"2024-03-20T08:22:14.915318+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:22:14.915318+0000","last_undegraded":"2024-03-20T08:22:14.915318+0000","last_fullsized":"2024-03-20T08:22:14.915318+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.867213+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:21:49.867213+0000","last_peered":"2024-03-20T08:21:49.867213+0000","last_clean":"2024-03-20T08:21:49.867213+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:21:49.867213+0000","last_undegraded":"2024-03-20T08:21:49.867213+0000","last_fullsized":"2024-03-20T08:21:49.867213+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850516+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:20:49.850516+0000","last_peered":"2024-03-20T08:20:49.850516+0000","last_clean":"2024-03-20T08:20:49.850516+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:20:49.850516+0000","last_undegraded":"2024-03-20T08:20:49.850516+0000","last_fullsized":"2024-03-20T08:20:49.850516+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1614,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912064+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:21:44.912064+0000","last_peered":"2024-03-20T08:21:44.912064+0000","last_clean":"2024-03-20T08:21:44.912064+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:21:44.912064+0000","last_undegraded":"2024-03-20T08:21:44.912064+0000","last_fullsized":"2024-03-20T08:21:44.912064+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:45.997313+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:22:45.997313+0000","last_peered":"2024-03-20T08:22:45.997313+0000","last_clean":"2024-03-20T08:22:45.997313+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:22:45.997313+0000","last_undegraded":"2024-03-20T08:22:45.997313+0000","last_fullsized":"2024-03-20T08:22:45.997313+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994228+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:22:05.994228+0000","last_peered":"2024-03-20T08:22:05.994228+0000","last_clean":"2024-03-20T08:22:05.994228+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:22:05.994228+0000","last_undegraded":"2024-03-20T08:22:05.994228+0000","last_fullsized":"2024-03-20T08:22:05.994228+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912041+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:21:44.912041+0000","last_peered":"2024-03-20T08:21:44.912041+0000","last_clean":"2024-03-20T08:21:44.912041+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:21:44.912041+0000","last_undegraded":"2024-03-20T08:21:44.912041+0000","last_fullsized":"2024-03-20T08:21:44.912041+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994287+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:22:05.994287+0000","last_peered":"2024-03-20T08:22:05.994287+0000","last_clean":"2024-03-20T08:22:05.994287+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:22:05.994287+0000","last_undegraded":"2024-03-20T08:22:05.994287+0000","last_fullsized":"2024-03-20T08:22:05.994287+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1145,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915352+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:22:14.915352+0000","last_peered":"2024-03-20T08:22:14.915352+0000","last_clean":"2024-03-20T08:22:14.915352+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:22:14.915352+0000","last_undegraded":"2024-03-20T08:22:14.915352+0000","last_fullsized":"2024-03-20T08:22:14.915352+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309547+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:22:42.309547+0000","last_peered":"2024-03-20T08:22:42.309547+0000","last_clean":"2024-03-20T08:22:42.309547+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:22:42.309547+0000","last_undegraded":"2024-03-20T08:22:42.309547+0000","last_fullsized":"2024-03-20T08:22:42.309547+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4608,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:29.735964+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:22:29.735964+0000","last_peered":"2024-03-20T08:22:29.735964+0000","last_clean":"2024-03-20T08:22:29.735964+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:22:29.735964+0000","last_undegraded":"2024-03-20T08:22:29.735964+0000","last_fullsized":"2024-03-20T08:22:29.735964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779096+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:22:24.779096+0000","last_peered":"2024-03-20T08:22:24.779096+0000","last_clean":"2024-03-20T08:22:24.779096+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:22:24.779096+0000","last_undegraded":"2024-03-20T08:22:24.779096+0000","last_fullsized":"2024-03-20T08:22:24.779096+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1219,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:29.735941+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:22:29.735941+0000","last_peered":"2024-03-20T08:22:29.735941+0000","last_clean":"2024-03-20T08:22:29.735941+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:22:29.735941+0000","last_undegraded":"2024-03-20T08:22:29.735941+0000","last_fullsized":"2024-03-20T08:22:29.735941+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1176,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447941+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:21:57.447941+0000","last_peered":"2024-03-20T08:21:57.447941+0000","last_clean":"2024-03-20T08:21:57.447941+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:21:57.447941+0000","last_undegraded":"2024-03-20T08:21:57.447941+0000","last_fullsized":"2024-03-20T08:21:57.447941+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:52.884264+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:21:52.884264+0000","last_peered":"2024-03-20T08:21:52.884264+0000","last_clean":"2024-03-20T08:21:52.884264+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:21:52.884264+0000","last_undegraded":"2024-03-20T08:21:52.884264+0000","last_fullsized":"2024-03-20T08:21:52.884264+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5194,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:49.857496+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:21:49.857496+0000","last_peered":"2024-03-20T08:21:49.857496+0000","last_clean":"2024-03-20T08:21:49.857496+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:21:49.857496+0000","last_undegraded":"2024-03-20T08:21:49.857496+0000","last_fullsized":"2024-03-20T08:21:49.857496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:44.918743+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:22:44.918743+0000","last_peered":"2024-03-20T08:22:44.918743+0000","last_clean":"2024-03-20T08:22:44.918743+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:22:44.918743+0000","last_undegraded":"2024-03-20T08:22:44.918743+0000","last_fullsized":"2024-03-20T08:22:44.918743+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68668,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:24.728852+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:21:24.728852+0000","last_peered":"2024-03-20T08:21:24.728852+0000","last_clean":"2024-03-20T08:21:24.728852+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:21:24.728852+0000","last_undegraded":"2024-03-20T08:21:24.728852+0000","last_fullsized":"2024-03-20T08:21:24.728852+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779055+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:22:24.779055+0000","last_peered":"2024-03-20T08:22:24.779055+0000","last_clean":"2024-03-20T08:22:24.779055+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:22:24.779055+0000","last_undegraded":"2024-03-20T08:22:24.779055+0000","last_fullsized":"2024-03-20T08:22:24.779055+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31735,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:27.886950+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:22:27.886950+0000","last_peered":"2024-03-20T08:22:27.886950+0000","last_clean":"2024-03-20T08:22:27.886950+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:22:27.886950+0000","last_undegraded":"2024-03-20T08:22:27.886950+0000","last_fullsized":"2024-03-20T08:22:27.886950+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29912,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:47.305131+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:21:47.305131+0000","last_peered":"2024-03-20T08:21:47.305131+0000","last_clean":"2024-03-20T08:21:47.305131+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:21:47.305131+0000","last_undegraded":"2024-03-20T08:21:47.305131+0000","last_fullsized":"2024-03-20T08:21:47.305131+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1311,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886931+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:22:27.886931+0000","last_peered":"2024-03-20T08:22:27.886931+0000","last_clean":"2024-03-20T08:22:27.886931+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:22:27.886931+0000","last_undegraded":"2024-03-20T08:22:27.886931+0000","last_fullsized":"2024-03-20T08:22:27.886931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447869+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:21:57.447869+0000","last_peered":"2024-03-20T08:21:57.447869+0000","last_clean":"2024-03-20T08:21:57.447869+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:21:57.447869+0000","last_undegraded":"2024-03-20T08:21:57.447869+0000","last_fullsized":"2024-03-20T08:21:57.447869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3380,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886897+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:22:27.886897+0000","last_peered":"2024-03-20T08:22:27.886897+0000","last_clean":"2024-03-20T08:22:27.886897+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:22:27.886897+0000","last_undegraded":"2024-03-20T08:22:27.886897+0000","last_fullsized":"2024-03-20T08:22:27.886897+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886942+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:22:27.886942+0000","last_peered":"2024-03-20T08:22:27.886942+0000","last_clean":"2024-03-20T08:22:27.886942+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:22:27.886942+0000","last_undegraded":"2024-03-20T08:22:27.886942+0000","last_fullsized":"2024-03-20T08:22:27.886942+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5701,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884206+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:21:52.884206+0000","last_peered":"2024-03-20T08:21:52.884206+0000","last_clean":"2024-03-20T08:21:52.884206+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:21:52.884206+0000","last_undegraded":"2024-03-20T08:21:52.884206+0000","last_fullsized":"2024-03-20T08:21:52.884206+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:47.305170+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:21:47.305170+0000","last_peered":"2024-03-20T08:21:47.305170+0000","last_clean":"2024-03-20T08:21:47.305170+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:21:47.305170+0000","last_undegraded":"2024-03-20T08:21:47.305170+0000","last_fullsized":"2024-03-20T08:21:47.305170+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857470+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:21:49.857470+0000","last_peered":"2024-03-20T08:21:49.857470+0000","last_clean":"2024-03-20T08:21:49.857470+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:21:49.857470+0000","last_undegraded":"2024-03-20T08:21:49.857470+0000","last_fullsized":"2024-03-20T08:21:49.857470+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4961,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:44.918770+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:22:44.918770+0000","last_peered":"2024-03-20T08:22:44.918770+0000","last_clean":"2024-03-20T08:22:44.918770+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:22:44.918770+0000","last_undegraded":"2024-03-20T08:22:44.918770+0000","last_fullsized":"2024-03-20T08:22:44.918770+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886979+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:22:27.886979+0000","last_peered":"2024-03-20T08:22:27.886979+0000","last_clean":"2024-03-20T08:22:27.886979+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:22:27.886979+0000","last_undegraded":"2024-03-20T08:22:27.886979+0000","last_fullsized":"2024-03-20T08:22:27.886979+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850497+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:20:49.850497+0000","last_peered":"2024-03-20T08:20:49.850497+0000","last_clean":"2024-03-20T08:20:49.850497+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:20:49.850497+0000","last_undegraded":"2024-03-20T08:20:49.850497+0000","last_fullsized":"2024-03-20T08:20:49.850497+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994252+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:22:05.994252+0000","last_peered":"2024-03-20T08:22:05.994252+0000","last_clean":"2024-03-20T08:22:05.994252+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:22:05.994252+0000","last_undegraded":"2024-03-20T08:22:05.994252+0000","last_fullsized":"2024-03-20T08:22:05.994252+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779085+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:22:24.779085+0000","last_peered":"2024-03-20T08:22:24.779085+0000","last_clean":"2024-03-20T08:22:24.779085+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:22:24.779085+0000","last_undegraded":"2024-03-20T08:22:24.779085+0000","last_fullsized":"2024-03-20T08:22:24.779085+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447930+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:21:57.447930+0000","last_peered":"2024-03-20T08:21:57.447930+0000","last_clean":"2024-03-20T08:21:57.447930+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:21:57.447930+0000","last_undegraded":"2024-03-20T08:21:57.447930+0000","last_fullsized":"2024-03-20T08:21:57.447930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779104+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:22:24.779104+0000","last_peered":"2024-03-20T08:22:24.779104+0000","last_clean":"2024-03-20T08:22:24.779104+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:22:24.779104+0000","last_undegraded":"2024-03-20T08:22:24.779104+0000","last_fullsized":"2024-03-20T08:22:24.779104+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857436+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:21:49.857436+0000","last_peered":"2024-03-20T08:21:49.857436+0000","last_clean":"2024-03-20T08:21:49.857436+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:21:49.857436+0000","last_undegraded":"2024-03-20T08:21:49.857436+0000","last_fullsized":"2024-03-20T08:21:49.857436+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.451137+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:22:27.451137+0000","last_peered":"2024-03-20T08:22:27.451137+0000","last_clean":"2024-03-20T08:22:27.451137+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:22:27.451137+0000","last_undegraded":"2024-03-20T08:22:27.451137+0000","last_fullsized":"2024-03-20T08:22:27.451137+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980279,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484672,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011683,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189035,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703515,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656981,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446925,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629320,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:24:58.986 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.986 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.986 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.986 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.986 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.986 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.986 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:24:58.987 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:24:59.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:24:59 smithi196 ceph-mon[87080]: from='client.44711 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:59.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:24:59 smithi118 ceph-mon[93121]: from='client.44711 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:24:59.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:24:59 smithi118 ceph-mon[95692]: from='client.44711 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:25:00.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:00 smithi196 ceph-mon[87080]: pgmap v1279: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:00.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:00 smithi118 ceph-mon[93121]: pgmap v1279: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:00.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:00 smithi118 ceph-mon[95692]: pgmap v1279: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:02.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:02 smithi196 ceph-mon[87080]: pgmap v1280: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:02.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:25:02 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:25:02.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:25:02.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:02 smithi118 ceph-mon[93121]: pgmap v1280: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:02.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:02 smithi118 ceph-mon[95692]: pgmap v1280: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:03.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:25:03] "GET /metrics HTTP/1.1" 200 36408 "" "Prometheus/2.43.0" 2024-03-20T08:25:04.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:04 smithi196 ceph-mon[87080]: pgmap v1281: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:04.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:04 smithi118 ceph-mon[93121]: pgmap v1281: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:04.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:04 smithi118 ceph-mon[95692]: pgmap v1281: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:06.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:06 smithi196 ceph-mon[87080]: pgmap v1282: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:06.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:06 smithi118 ceph-mon[93121]: pgmap v1282: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:06.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:06 smithi118 ceph-mon[95692]: pgmap v1282: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:08.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:08 smithi196 ceph-mon[87080]: pgmap v1283: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:08.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:08 smithi118 ceph-mon[93121]: pgmap v1283: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:08.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:08 smithi118 ceph-mon[95692]: pgmap v1283: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:08.988 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:25:09.208 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:25:10.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:10 smithi118 ceph-mon[93121]: pgmap v1284: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:10.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:10 smithi118 ceph-mon[95692]: pgmap v1284: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:10.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:10 smithi196 ceph-mon[87080]: pgmap v1284: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:11.181 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:25:11.183 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:25:11.933 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1285,"stamp":"2024-03-20T08:25:10.777407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003021"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915382+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:22:14.915382+0000","last_peered":"2024-03-20T08:22:14.915382+0000","last_clean":"2024-03-20T08:22:14.915382+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:22:14.915382+0000","last_undegraded":"2024-03-20T08:22:14.915382+0000","last_fullsized":"2024-03-20T08:22:14.915382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309560+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:22:42.309560+0000","last_peered":"2024-03-20T08:22:42.309560+0000","last_clean":"2024-03-20T08:22:42.309560+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:22:42.309560+0000","last_undegraded":"2024-03-20T08:22:42.309560+0000","last_fullsized":"2024-03-20T08:22:42.309560+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994305+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:22:05.994305+0000","last_peered":"2024-03-20T08:22:05.994305+0000","last_clean":"2024-03-20T08:22:05.994305+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:22:05.994305+0000","last_undegraded":"2024-03-20T08:22:05.994305+0000","last_fullsized":"2024-03-20T08:22:05.994305+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860780+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:22:19.860780+0000","last_peered":"2024-03-20T08:22:19.860780+0000","last_clean":"2024-03-20T08:22:19.860780+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:22:19.860780+0000","last_undegraded":"2024-03-20T08:22:19.860780+0000","last_fullsized":"2024-03-20T08:22:19.860780+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915278+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:22:14.915278+0000","last_peered":"2024-03-20T08:22:14.915278+0000","last_clean":"2024-03-20T08:22:14.915278+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:22:14.915278+0000","last_undegraded":"2024-03-20T08:22:14.915278+0000","last_fullsized":"2024-03-20T08:22:14.915278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915248+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:22:14.915248+0000","last_peered":"2024-03-20T08:22:14.915248+0000","last_clean":"2024-03-20T08:22:14.915248+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:22:14.915248+0000","last_undegraded":"2024-03-20T08:22:14.915248+0000","last_fullsized":"2024-03-20T08:22:14.915248+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.778123+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:22:14.778123+0000","last_peered":"2024-03-20T08:22:14.778123+0000","last_clean":"2024-03-20T08:22:14.778123+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:22:14.778123+0000","last_undegraded":"2024-03-20T08:22:14.778123+0000","last_fullsized":"2024-03-20T08:22:14.778123+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1127,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912609+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:21:49.912609+0000","last_peered":"2024-03-20T08:21:49.912609+0000","last_clean":"2024-03-20T08:21:49.912609+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:21:49.912609+0000","last_undegraded":"2024-03-20T08:21:49.912609+0000","last_fullsized":"2024-03-20T08:21:49.912609+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.735508+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:22:24.735508+0000","last_peered":"2024-03-20T08:22:24.735508+0000","last_clean":"2024-03-20T08:22:24.735508+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:22:24.735508+0000","last_undegraded":"2024-03-20T08:22:24.735508+0000","last_fullsized":"2024-03-20T08:22:24.735508+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866885+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:21:44.866885+0000","last_peered":"2024-03-20T08:21:44.866885+0000","last_clean":"2024-03-20T08:21:44.866885+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:21:44.866885+0000","last_undegraded":"2024-03-20T08:21:44.866885+0000","last_fullsized":"2024-03-20T08:21:44.866885+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992610+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:21:45.992610+0000","last_peered":"2024-03-20T08:21:45.992610+0000","last_clean":"2024-03-20T08:21:45.992610+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:21:45.992610+0000","last_undegraded":"2024-03-20T08:21:45.992610+0000","last_fullsized":"2024-03-20T08:21:45.992610+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:22.307947+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:22:22.307947+0000","last_peered":"2024-03-20T08:22:22.307947+0000","last_clean":"2024-03-20T08:22:22.307947+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:22:22.307947+0000","last_undegraded":"2024-03-20T08:22:22.307947+0000","last_fullsized":"2024-03-20T08:22:22.307947+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860734+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:22:19.860734+0000","last_peered":"2024-03-20T08:22:19.860734+0000","last_clean":"2024-03-20T08:22:19.860734+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:22:19.860734+0000","last_undegraded":"2024-03-20T08:22:19.860734+0000","last_fullsized":"2024-03-20T08:22:19.860734+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1225,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860800+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:22:19.860800+0000","last_peered":"2024-03-20T08:22:19.860800+0000","last_clean":"2024-03-20T08:22:19.860800+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:22:19.860800+0000","last_undegraded":"2024-03-20T08:22:19.860800+0000","last_fullsized":"2024-03-20T08:22:19.860800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1396,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994341+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:22:05.994341+0000","last_peered":"2024-03-20T08:22:05.994341+0000","last_clean":"2024-03-20T08:22:05.994341+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:22:05.994341+0000","last_undegraded":"2024-03-20T08:22:05.994341+0000","last_fullsized":"2024-03-20T08:22:05.994341+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864498+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:21:14.864498+0000","last_peered":"2024-03-20T08:21:14.864498+0000","last_clean":"2024-03-20T08:21:14.864498+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:21:14.864498+0000","last_undegraded":"2024-03-20T08:21:14.864498+0000","last_fullsized":"2024-03-20T08:21:14.864498+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1291,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864514+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:21:14.864514+0000","last_peered":"2024-03-20T08:21:14.864514+0000","last_clean":"2024-03-20T08:21:14.864514+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:21:14.864514+0000","last_undegraded":"2024-03-20T08:21:14.864514+0000","last_fullsized":"2024-03-20T08:21:14.864514+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994332+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:22:05.994332+0000","last_peered":"2024-03-20T08:22:05.994332+0000","last_clean":"2024-03-20T08:22:05.994332+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:22:05.994332+0000","last_undegraded":"2024-03-20T08:22:05.994332+0000","last_fullsized":"2024-03-20T08:22:05.994332+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915365+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:22:14.915365+0000","last_peered":"2024-03-20T08:22:14.915365+0000","last_clean":"2024-03-20T08:22:14.915365+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:22:14.915365+0000","last_undegraded":"2024-03-20T08:22:14.915365+0000","last_fullsized":"2024-03-20T08:22:14.915365+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1132,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915290+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:22:14.915290+0000","last_peered":"2024-03-20T08:22:14.915290+0000","last_clean":"2024-03-20T08:22:14.915290+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:22:14.915290+0000","last_undegraded":"2024-03-20T08:22:14.915290+0000","last_fullsized":"2024-03-20T08:22:14.915290+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:42.883328+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:21:42.883328+0000","last_peered":"2024-03-20T08:21:42.883328+0000","last_clean":"2024-03-20T08:21:42.883328+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:21:42.883328+0000","last_undegraded":"2024-03-20T08:21:42.883328+0000","last_fullsized":"2024-03-20T08:21:42.883328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.778159+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:22:14.778159+0000","last_peered":"2024-03-20T08:22:14.778159+0000","last_clean":"2024-03-20T08:22:14.778159+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:22:14.778159+0000","last_undegraded":"2024-03-20T08:22:14.778159+0000","last_fullsized":"2024-03-20T08:22:14.778159+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444464+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:21:27.444464+0000","last_peered":"2024-03-20T08:21:27.444464+0000","last_clean":"2024-03-20T08:21:27.444464+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:21:27.444464+0000","last_undegraded":"2024-03-20T08:21:27.444464+0000","last_fullsized":"2024-03-20T08:21:27.444464+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866861+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:21:44.866861+0000","last_peered":"2024-03-20T08:21:44.866861+0000","last_clean":"2024-03-20T08:21:44.866861+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:21:44.866861+0000","last_undegraded":"2024-03-20T08:21:44.866861+0000","last_fullsized":"2024-03-20T08:21:44.866861+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866825+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:21:44.866825+0000","last_peered":"2024-03-20T08:21:44.866825+0000","last_clean":"2024-03-20T08:21:44.866825+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:21:44.866825+0000","last_undegraded":"2024-03-20T08:21:44.866825+0000","last_fullsized":"2024-03-20T08:21:44.866825+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847510+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:20:24.847510+0000","last_peered":"2024-03-20T08:20:24.847510+0000","last_clean":"2024-03-20T08:20:24.847510+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:20:24.847510+0000","last_undegraded":"2024-03-20T08:20:24.847510+0000","last_fullsized":"2024-03-20T08:20:24.847510+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1343,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299358+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:20:32.299358+0000","last_peered":"2024-03-20T08:20:32.299358+0000","last_clean":"2024-03-20T08:20:32.299358+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:20:32.299358+0000","last_undegraded":"2024-03-20T08:20:32.299358+0000","last_fullsized":"2024-03-20T08:20:32.299358+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847561+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:20:24.847561+0000","last_peered":"2024-03-20T08:20:24.847561+0000","last_clean":"2024-03-20T08:20:24.847561+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:20:24.847561+0000","last_undegraded":"2024-03-20T08:20:24.847561+0000","last_fullsized":"2024-03-20T08:20:24.847561+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1197,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.847550+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:20:24.847550+0000","last_peered":"2024-03-20T08:20:24.847550+0000","last_clean":"2024-03-20T08:20:24.847550+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:20:24.847550+0000","last_undegraded":"2024-03-20T08:20:24.847550+0000","last_fullsized":"2024-03-20T08:20:24.847550+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:20.995500+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:22:20.995500+0000","last_peered":"2024-03-20T08:22:20.995500+0000","last_clean":"2024-03-20T08:22:20.995500+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:22:20.995500+0000","last_undegraded":"2024-03-20T08:22:20.995500+0000","last_fullsized":"2024-03-20T08:22:20.995500+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:32.299393+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:20:32.299393+0000","last_peered":"2024-03-20T08:20:32.299393+0000","last_clean":"2024-03-20T08:20:32.299393+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:20:32.299393+0000","last_undegraded":"2024-03-20T08:20:32.299393+0000","last_fullsized":"2024-03-20T08:20:32.299393+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:54.732439+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:21:54.732439+0000","last_peered":"2024-03-20T08:21:54.732439+0000","last_clean":"2024-03-20T08:21:54.732439+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:21:54.732439+0000","last_undegraded":"2024-03-20T08:21:54.732439+0000","last_fullsized":"2024-03-20T08:21:54.732439+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1223,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438791+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:20:37.438791+0000","last_peered":"2024-03-20T08:20:37.438791+0000","last_clean":"2024-03-20T08:20:37.438791+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:20:37.438791+0000","last_undegraded":"2024-03-20T08:20:37.438791+0000","last_fullsized":"2024-03-20T08:20:37.438791+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912579+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:21:49.912579+0000","last_peered":"2024-03-20T08:21:49.912579+0000","last_clean":"2024-03-20T08:21:49.912579+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:21:49.912579+0000","last_undegraded":"2024-03-20T08:21:49.912579+0000","last_fullsized":"2024-03-20T08:21:49.912579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1391,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992645+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:21:45.992645+0000","last_peered":"2024-03-20T08:21:45.992645+0000","last_clean":"2024-03-20T08:21:45.992645+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:21:45.992645+0000","last_undegraded":"2024-03-20T08:21:45.992645+0000","last_fullsized":"2024-03-20T08:21:45.992645+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992629+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:21:45.992629+0000","last_peered":"2024-03-20T08:21:45.992629+0000","last_clean":"2024-03-20T08:21:45.992629+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:21:45.992629+0000","last_undegraded":"2024-03-20T08:21:45.992629+0000","last_fullsized":"2024-03-20T08:21:45.992629+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860764+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:22:19.860764+0000","last_peered":"2024-03-20T08:22:19.860764+0000","last_clean":"2024-03-20T08:22:19.860764+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:22:19.860764+0000","last_undegraded":"2024-03-20T08:22:19.860764+0000","last_fullsized":"2024-03-20T08:22:19.860764+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909806+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:21:24.909806+0000","last_peered":"2024-03-20T08:21:24.909806+0000","last_clean":"2024-03-20T08:21:24.909806+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:21:24.909806+0000","last_undegraded":"2024-03-20T08:21:24.909806+0000","last_fullsized":"2024-03-20T08:21:24.909806+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438753+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:20:37.438753+0000","last_peered":"2024-03-20T08:20:37.438753+0000","last_clean":"2024-03-20T08:20:37.438753+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:20:37.438753+0000","last_undegraded":"2024-03-20T08:20:37.438753+0000","last_fullsized":"2024-03-20T08:20:37.438753+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:37.438729+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:20:37.438729+0000","last_peered":"2024-03-20T08:20:37.438729+0000","last_clean":"2024-03-20T08:20:37.438729+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:20:37.438729+0000","last_undegraded":"2024-03-20T08:20:37.438729+0000","last_fullsized":"2024-03-20T08:20:37.438729+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909777+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:21:24.909777+0000","last_peered":"2024-03-20T08:21:24.909777+0000","last_clean":"2024-03-20T08:21:24.909777+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:21:24.909777+0000","last_undegraded":"2024-03-20T08:21:24.909777+0000","last_fullsized":"2024-03-20T08:21:24.909777+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26674,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884162+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:21:52.884162+0000","last_peered":"2024-03-20T08:21:52.884162+0000","last_clean":"2024-03-20T08:21:52.884162+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:21:52.884162+0000","last_undegraded":"2024-03-20T08:21:52.884162+0000","last_fullsized":"2024-03-20T08:21:52.884162+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1261,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.728906+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:21:24.728906+0000","last_peered":"2024-03-20T08:21:24.728906+0000","last_clean":"2024-03-20T08:21:24.728906+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:21:24.728906+0000","last_undegraded":"2024-03-20T08:21:24.728906+0000","last_fullsized":"2024-03-20T08:21:24.728906+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11692,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994261+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:22:05.994261+0000","last_peered":"2024-03-20T08:22:05.994261+0000","last_clean":"2024-03-20T08:22:05.994261+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:22:05.994261+0000","last_undegraded":"2024-03-20T08:22:05.994261+0000","last_fullsized":"2024-03-20T08:22:05.994261+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309503+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:22:42.309503+0000","last_peered":"2024-03-20T08:22:42.309503+0000","last_clean":"2024-03-20T08:22:42.309503+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:22:42.309503+0000","last_undegraded":"2024-03-20T08:22:42.309503+0000","last_fullsized":"2024-03-20T08:22:42.309503+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309527+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:22:42.309527+0000","last_peered":"2024-03-20T08:22:42.309527+0000","last_clean":"2024-03-20T08:22:42.309527+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:22:42.309527+0000","last_undegraded":"2024-03-20T08:22:42.309527+0000","last_fullsized":"2024-03-20T08:22:42.309527+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1237,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:54.865182+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:22:54.865182+0000","last_peered":"2024-03-20T08:22:54.865182+0000","last_clean":"2024-03-20T08:22:54.865182+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:22:54.865182+0000","last_undegraded":"2024-03-20T08:22:54.865182+0000","last_fullsized":"2024-03-20T08:22:54.865182+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5954,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994315+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:22:05.994315+0000","last_peered":"2024-03-20T08:22:05.994315+0000","last_clean":"2024-03-20T08:22:05.994315+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:22:05.994315+0000","last_undegraded":"2024-03-20T08:22:05.994315+0000","last_fullsized":"2024-03-20T08:22:05.994315+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309539+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:22:42.309539+0000","last_peered":"2024-03-20T08:22:42.309539+0000","last_clean":"2024-03-20T08:22:42.309539+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:22:42.309539+0000","last_undegraded":"2024-03-20T08:22:42.309539+0000","last_fullsized":"2024-03-20T08:22:42.309539+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:54.865117+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:22:54.865117+0000","last_peered":"2024-03-20T08:22:54.865117+0000","last_clean":"2024-03-20T08:22:54.865117+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:22:54.865117+0000","last_undegraded":"2024-03-20T08:22:54.865117+0000","last_fullsized":"2024-03-20T08:22:54.865117+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:29.735999+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:22:29.735999+0000","last_peered":"2024-03-20T08:22:29.735999+0000","last_clean":"2024-03-20T08:22:29.735999+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:22:29.735999+0000","last_undegraded":"2024-03-20T08:22:29.735999+0000","last_fullsized":"2024-03-20T08:22:29.735999+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884275+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:21:52.884275+0000","last_peered":"2024-03-20T08:21:52.884275+0000","last_clean":"2024-03-20T08:21:52.884275+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:21:52.884275+0000","last_undegraded":"2024-03-20T08:21:52.884275+0000","last_fullsized":"2024-03-20T08:21:52.884275+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:02.306280+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:22:02.306280+0000","last_peered":"2024-03-20T08:22:02.306280+0000","last_clean":"2024-03-20T08:22:02.306280+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:22:02.306280+0000","last_undegraded":"2024-03-20T08:22:02.306280+0000","last_fullsized":"2024-03-20T08:22:02.306280+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3982,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:14.915318+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:22:14.915318+0000","last_peered":"2024-03-20T08:22:14.915318+0000","last_clean":"2024-03-20T08:22:14.915318+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:22:14.915318+0000","last_undegraded":"2024-03-20T08:22:14.915318+0000","last_fullsized":"2024-03-20T08:22:14.915318+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.867213+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:21:49.867213+0000","last_peered":"2024-03-20T08:21:49.867213+0000","last_clean":"2024-03-20T08:21:49.867213+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:21:49.867213+0000","last_undegraded":"2024-03-20T08:21:49.867213+0000","last_fullsized":"2024-03-20T08:21:49.867213+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:54.865165+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:22:54.865165+0000","last_peered":"2024-03-20T08:22:54.865165+0000","last_clean":"2024-03-20T08:22:54.865165+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:22:54.865165+0000","last_undegraded":"2024-03-20T08:22:54.865165+0000","last_fullsized":"2024-03-20T08:22:54.865165+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1614,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912064+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:21:44.912064+0000","last_peered":"2024-03-20T08:21:44.912064+0000","last_clean":"2024-03-20T08:21:44.912064+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:21:44.912064+0000","last_undegraded":"2024-03-20T08:21:44.912064+0000","last_fullsized":"2024-03-20T08:21:44.912064+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:45.997313+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:22:45.997313+0000","last_peered":"2024-03-20T08:22:45.997313+0000","last_clean":"2024-03-20T08:22:45.997313+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:22:45.997313+0000","last_undegraded":"2024-03-20T08:22:45.997313+0000","last_fullsized":"2024-03-20T08:22:45.997313+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994228+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:22:05.994228+0000","last_peered":"2024-03-20T08:22:05.994228+0000","last_clean":"2024-03-20T08:22:05.994228+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:22:05.994228+0000","last_undegraded":"2024-03-20T08:22:05.994228+0000","last_fullsized":"2024-03-20T08:22:05.994228+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912041+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:21:44.912041+0000","last_peered":"2024-03-20T08:21:44.912041+0000","last_clean":"2024-03-20T08:21:44.912041+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:21:44.912041+0000","last_undegraded":"2024-03-20T08:21:44.912041+0000","last_fullsized":"2024-03-20T08:21:44.912041+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994287+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:22:05.994287+0000","last_peered":"2024-03-20T08:22:05.994287+0000","last_clean":"2024-03-20T08:22:05.994287+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:22:05.994287+0000","last_undegraded":"2024-03-20T08:22:05.994287+0000","last_fullsized":"2024-03-20T08:22:05.994287+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1145,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915352+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:22:14.915352+0000","last_peered":"2024-03-20T08:22:14.915352+0000","last_clean":"2024-03-20T08:22:14.915352+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:22:14.915352+0000","last_undegraded":"2024-03-20T08:22:14.915352+0000","last_fullsized":"2024-03-20T08:22:14.915352+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309547+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:22:42.309547+0000","last_peered":"2024-03-20T08:22:42.309547+0000","last_clean":"2024-03-20T08:22:42.309547+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:22:42.309547+0000","last_undegraded":"2024-03-20T08:22:42.309547+0000","last_fullsized":"2024-03-20T08:22:42.309547+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4608,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:29.735964+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:22:29.735964+0000","last_peered":"2024-03-20T08:22:29.735964+0000","last_clean":"2024-03-20T08:22:29.735964+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:22:29.735964+0000","last_undegraded":"2024-03-20T08:22:29.735964+0000","last_fullsized":"2024-03-20T08:22:29.735964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:14.864469+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:21:14.864469+0000","last_peered":"2024-03-20T08:21:14.864469+0000","last_clean":"2024-03-20T08:21:14.864469+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:21:14.864469+0000","last_undegraded":"2024-03-20T08:21:14.864469+0000","last_fullsized":"2024-03-20T08:21:14.864469+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779096+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:22:24.779096+0000","last_peered":"2024-03-20T08:22:24.779096+0000","last_clean":"2024-03-20T08:22:24.779096+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:22:24.779096+0000","last_undegraded":"2024-03-20T08:22:24.779096+0000","last_fullsized":"2024-03-20T08:22:24.779096+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5631,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:14.864438+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:21:14.864438+0000","last_peered":"2024-03-20T08:21:14.864438+0000","last_clean":"2024-03-20T08:21:14.864438+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:21:14.864438+0000","last_undegraded":"2024-03-20T08:21:14.864438+0000","last_fullsized":"2024-03-20T08:21:14.864438+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1219,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:29.735941+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:22:29.735941+0000","last_peered":"2024-03-20T08:22:29.735941+0000","last_clean":"2024-03-20T08:22:29.735941+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:22:29.735941+0000","last_undegraded":"2024-03-20T08:22:29.735941+0000","last_fullsized":"2024-03-20T08:22:29.735941+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1176,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447941+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:21:57.447941+0000","last_peered":"2024-03-20T08:21:57.447941+0000","last_clean":"2024-03-20T08:21:57.447941+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:21:57.447941+0000","last_undegraded":"2024-03-20T08:21:57.447941+0000","last_fullsized":"2024-03-20T08:21:57.447941+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:52.884264+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:21:52.884264+0000","last_peered":"2024-03-20T08:21:52.884264+0000","last_clean":"2024-03-20T08:21:52.884264+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:21:52.884264+0000","last_undegraded":"2024-03-20T08:21:52.884264+0000","last_fullsized":"2024-03-20T08:21:52.884264+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5194,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:49.857496+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:21:49.857496+0000","last_peered":"2024-03-20T08:21:49.857496+0000","last_clean":"2024-03-20T08:21:49.857496+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:21:49.857496+0000","last_undegraded":"2024-03-20T08:21:49.857496+0000","last_fullsized":"2024-03-20T08:21:49.857496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:44.918743+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:22:44.918743+0000","last_peered":"2024-03-20T08:22:44.918743+0000","last_clean":"2024-03-20T08:22:44.918743+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:22:44.918743+0000","last_undegraded":"2024-03-20T08:22:44.918743+0000","last_fullsized":"2024-03-20T08:22:44.918743+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1253,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:54.725729+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:20:54.725729+0000","last_peered":"2024-03-20T08:20:54.725729+0000","last_clean":"2024-03-20T08:20:54.725729+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:20:54.725729+0000","last_undegraded":"2024-03-20T08:20:54.725729+0000","last_fullsized":"2024-03-20T08:20:54.725729+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68668,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:24.728852+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:21:24.728852+0000","last_peered":"2024-03-20T08:21:24.728852+0000","last_clean":"2024-03-20T08:21:24.728852+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:21:24.728852+0000","last_undegraded":"2024-03-20T08:21:24.728852+0000","last_fullsized":"2024-03-20T08:21:24.728852+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779055+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:22:24.779055+0000","last_peered":"2024-03-20T08:22:24.779055+0000","last_clean":"2024-03-20T08:22:24.779055+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:22:24.779055+0000","last_undegraded":"2024-03-20T08:22:24.779055+0000","last_fullsized":"2024-03-20T08:22:24.779055+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31735,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:27.886950+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:22:27.886950+0000","last_peered":"2024-03-20T08:22:27.886950+0000","last_clean":"2024-03-20T08:22:27.886950+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:22:27.886950+0000","last_undegraded":"2024-03-20T08:22:27.886950+0000","last_fullsized":"2024-03-20T08:22:27.886950+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1286,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862445+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:20:44.862445+0000","last_peered":"2024-03-20T08:20:44.862445+0000","last_clean":"2024-03-20T08:20:44.862445+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:20:44.862445+0000","last_undegraded":"2024-03-20T08:20:44.862445+0000","last_fullsized":"2024-03-20T08:20:44.862445+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29912,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:47.305131+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:21:47.305131+0000","last_peered":"2024-03-20T08:21:47.305131+0000","last_clean":"2024-03-20T08:21:47.305131+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:21:47.305131+0000","last_undegraded":"2024-03-20T08:21:47.305131+0000","last_fullsized":"2024-03-20T08:21:47.305131+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1311,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886931+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:22:27.886931+0000","last_peered":"2024-03-20T08:22:27.886931+0000","last_clean":"2024-03-20T08:22:27.886931+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:22:27.886931+0000","last_undegraded":"2024-03-20T08:22:27.886931+0000","last_fullsized":"2024-03-20T08:22:27.886931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447869+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:21:57.447869+0000","last_peered":"2024-03-20T08:21:57.447869+0000","last_clean":"2024-03-20T08:21:57.447869+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:21:57.447869+0000","last_undegraded":"2024-03-20T08:21:57.447869+0000","last_fullsized":"2024-03-20T08:21:57.447869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3380,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886897+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:22:27.886897+0000","last_peered":"2024-03-20T08:22:27.886897+0000","last_clean":"2024-03-20T08:22:27.886897+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:22:27.886897+0000","last_undegraded":"2024-03-20T08:22:27.886897+0000","last_fullsized":"2024-03-20T08:22:27.886897+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886942+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:22:27.886942+0000","last_peered":"2024-03-20T08:22:27.886942+0000","last_clean":"2024-03-20T08:22:27.886942+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:22:27.886942+0000","last_undegraded":"2024-03-20T08:22:27.886942+0000","last_fullsized":"2024-03-20T08:22:27.886942+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5701,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884206+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:21:52.884206+0000","last_peered":"2024-03-20T08:21:52.884206+0000","last_clean":"2024-03-20T08:21:52.884206+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:21:52.884206+0000","last_undegraded":"2024-03-20T08:21:52.884206+0000","last_fullsized":"2024-03-20T08:21:52.884206+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:47.305170+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:21:47.305170+0000","last_peered":"2024-03-20T08:21:47.305170+0000","last_clean":"2024-03-20T08:21:47.305170+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:21:47.305170+0000","last_undegraded":"2024-03-20T08:21:47.305170+0000","last_fullsized":"2024-03-20T08:21:47.305170+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857470+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:21:49.857470+0000","last_peered":"2024-03-20T08:21:49.857470+0000","last_clean":"2024-03-20T08:21:49.857470+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:21:49.857470+0000","last_undegraded":"2024-03-20T08:21:49.857470+0000","last_fullsized":"2024-03-20T08:21:49.857470+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4961,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:44.918770+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:22:44.918770+0000","last_peered":"2024-03-20T08:22:44.918770+0000","last_clean":"2024-03-20T08:22:44.918770+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:22:44.918770+0000","last_undegraded":"2024-03-20T08:22:44.918770+0000","last_fullsized":"2024-03-20T08:22:44.918770+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886979+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:22:27.886979+0000","last_peered":"2024-03-20T08:22:27.886979+0000","last_clean":"2024-03-20T08:22:27.886979+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:22:27.886979+0000","last_undegraded":"2024-03-20T08:22:27.886979+0000","last_fullsized":"2024-03-20T08:22:27.886979+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1237,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:54.865149+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:22:54.865149+0000","last_peered":"2024-03-20T08:22:54.865149+0000","last_clean":"2024-03-20T08:22:54.865149+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:22:54.865149+0000","last_undegraded":"2024-03-20T08:22:54.865149+0000","last_fullsized":"2024-03-20T08:22:54.865149+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994252+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:22:05.994252+0000","last_peered":"2024-03-20T08:22:05.994252+0000","last_clean":"2024-03-20T08:22:05.994252+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:22:05.994252+0000","last_undegraded":"2024-03-20T08:22:05.994252+0000","last_fullsized":"2024-03-20T08:22:05.994252+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779085+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:22:24.779085+0000","last_peered":"2024-03-20T08:22:24.779085+0000","last_clean":"2024-03-20T08:22:24.779085+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:22:24.779085+0000","last_undegraded":"2024-03-20T08:22:24.779085+0000","last_fullsized":"2024-03-20T08:22:24.779085+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447930+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:21:57.447930+0000","last_peered":"2024-03-20T08:21:57.447930+0000","last_clean":"2024-03-20T08:21:57.447930+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:21:57.447930+0000","last_undegraded":"2024-03-20T08:21:57.447930+0000","last_fullsized":"2024-03-20T08:21:57.447930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779104+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:22:24.779104+0000","last_peered":"2024-03-20T08:22:24.779104+0000","last_clean":"2024-03-20T08:22:24.779104+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:22:24.779104+0000","last_undegraded":"2024-03-20T08:22:24.779104+0000","last_fullsized":"2024-03-20T08:22:24.779104+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:44.862427+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:20:44.862427+0000","last_peered":"2024-03-20T08:20:44.862427+0000","last_clean":"2024-03-20T08:20:44.862427+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:20:44.862427+0000","last_undegraded":"2024-03-20T08:20:44.862427+0000","last_fullsized":"2024-03-20T08:20:44.862427+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857436+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:21:49.857436+0000","last_peered":"2024-03-20T08:21:49.857436+0000","last_clean":"2024-03-20T08:21:49.857436+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:21:49.857436+0000","last_undegraded":"2024-03-20T08:21:49.857436+0000","last_fullsized":"2024-03-20T08:21:49.857436+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.451137+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:22:27.451137+0000","last_peered":"2024-03-20T08:22:27.451137+0000","last_clean":"2024-03-20T08:22:27.451137+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:22:27.451137+0000","last_undegraded":"2024-03-20T08:22:27.451137+0000","last_fullsized":"2024-03-20T08:22:27.451137+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980282,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484675,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011686,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189037,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703518,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656984,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446928,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629323,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.939 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:11.940 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:25:12.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:12 smithi196 ceph-mon[87080]: pgmap v1285: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:12.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:12 smithi196 ceph-mon[87080]: from='client.44717 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:25:12.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:25:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:25:12.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:25:12.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:12 smithi118 ceph-mon[93121]: pgmap v1285: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:12.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:12 smithi118 ceph-mon[93121]: from='client.44717 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:25:12.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:12 smithi118 ceph-mon[95692]: pgmap v1285: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:12.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:12 smithi118 ceph-mon[95692]: from='client.44717 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:25:13.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:13 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:25:13.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:13 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:25:13.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:13 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:25:13.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:25:13] "GET /metrics HTTP/1.1" 200 36407 "" "Prometheus/2.43.0" 2024-03-20T08:25:14.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:14 smithi196 ceph-mon[87080]: pgmap v1286: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:14.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:14 smithi118 ceph-mon[93121]: pgmap v1286: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:14.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:14 smithi118 ceph-mon[95692]: pgmap v1286: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:16.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:16 smithi196 ceph-mon[87080]: pgmap v1287: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:16.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:16 smithi118 ceph-mon[93121]: pgmap v1287: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:16.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:16 smithi118 ceph-mon[95692]: pgmap v1287: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:18.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:18 smithi196 ceph-mon[87080]: pgmap v1288: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:18.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:18 smithi118 ceph-mon[93121]: pgmap v1288: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:18.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:18 smithi118 ceph-mon[95692]: pgmap v1288: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:20.642 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:20 smithi196 ceph-mon[87080]: pgmap v1289: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:20.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:20 smithi118 ceph-mon[93121]: pgmap v1289: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:20.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:20 smithi118 ceph-mon[95692]: pgmap v1289: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:21.941 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-20T08:25:22.165 INFO:teuthology.orchestra.run.smithi118.stderr:Inferring config /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/mon.a/config 2024-03-20T08:25:22.580 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:22 smithi118 ceph-mon[93121]: pgmap v1290: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:22.580 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:22 smithi118 ceph-mon[95692]: pgmap v1290: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:22.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:22 smithi196 ceph-mon[87080]: pgmap v1290: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:22.640 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:25:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:25:22.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:25:23.673 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:25:23] "GET /metrics HTTP/1.1" 200 36407 "" "Prometheus/2.43.0" 2024-03-20T08:25:24.169 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:25:24.171 INFO:teuthology.orchestra.run.smithi118.stderr:dumped all 2024-03-20T08:25:24.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:24 smithi118 ceph-mon[95692]: pgmap v1291: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:24.423 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:24 smithi118 ceph-mon[93121]: pgmap v1291: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:24.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:24 smithi196 ceph-mon[87080]: pgmap v1291: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:24.862 INFO:teuthology.orchestra.run.smithi118.stdout:{"pg_ready":true,"pg_map":{"version":1291,"stamp":"2024-03-20T08:25:22.780590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1868463100,"num_objects":1084,"num_object_clones":1010,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1084,"num_whiteouts":8,"num_read":4710,"num_read_kb":5703760,"num_write":8902,"num_write_kb":1970236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":493061728,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64999,"ondisk_log_size":64999,"up":363,"acting":363,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":363,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9161688,"kb_used_data":2511128,"kb_used_omap":605,"kb_used_meta":6649826,"kb_avail":740799528,"statfs":{"total":767960285184,"available":758578716672,"internally_reserved":0,"allocated":2571395072,"data_stored":6112713884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":620076,"internal_metadata":6809422292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003182"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1121,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915382+0000","last_change":"2024-03-20T08:09:46.777854+0000","last_active":"2024-03-20T08:22:14.915382+0000","last_peered":"2024-03-20T08:22:14.915382+0000","last_clean":"2024-03-20T08:22:14.915382+0000","last_became_active":"2024-03-20T08:09:46.777346+0000","last_became_peered":"2024-03-20T08:09:46.777346+0000","last_unstale":"2024-03-20T08:22:14.915382+0000","last_undegraded":"2024-03-20T08:22:14.915382+0000","last_fullsized":"2024-03-20T08:22:14.915382+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:41:06.873307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309560+0000","last_change":"2024-03-20T08:08:59.148851+0000","last_active":"2024-03-20T08:22:42.309560+0000","last_peered":"2024-03-20T08:22:42.309560+0000","last_clean":"2024-03-20T08:22:42.309560+0000","last_became_active":"2024-03-20T08:08:59.148546+0000","last_became_peered":"2024-03-20T08:08:59.148546+0000","last_unstale":"2024-03-20T08:22:42.309560+0000","last_undegraded":"2024-03-20T08:22:42.309560+0000","last_fullsized":"2024-03-20T08:22:42.309560+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:33.774967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994305+0000","last_change":"2024-03-20T08:06:44.497593+0000","last_active":"2024-03-20T08:22:05.994305+0000","last_peered":"2024-03-20T08:22:05.994305+0000","last_clean":"2024-03-20T08:22:05.994305+0000","last_became_active":"2024-03-20T08:06:44.497292+0000","last_became_peered":"2024-03-20T08:06:44.497292+0000","last_unstale":"2024-03-20T08:22:05.994305+0000","last_undegraded":"2024-03-20T08:22:05.994305+0000","last_fullsized":"2024-03-20T08:22:05.994305+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:27:26.394526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860780+0000","last_change":"2024-03-20T08:09:46.778893+0000","last_active":"2024-03-20T08:22:19.860780+0000","last_peered":"2024-03-20T08:22:19.860780+0000","last_clean":"2024-03-20T08:22:19.860780+0000","last_became_active":"2024-03-20T08:09:46.778585+0000","last_became_peered":"2024-03-20T08:09:46.778585+0000","last_unstale":"2024-03-20T08:22:19.860780+0000","last_undegraded":"2024-03-20T08:22:19.860780+0000","last_fullsized":"2024-03-20T08:22:19.860780+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:18.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915278+0000","last_change":"2024-03-20T08:09:46.789263+0000","last_active":"2024-03-20T08:22:14.915278+0000","last_peered":"2024-03-20T08:22:14.915278+0000","last_clean":"2024-03-20T08:22:14.915278+0000","last_became_active":"2024-03-20T08:09:46.788714+0000","last_became_peered":"2024-03-20T08:09:46.788714+0000","last_unstale":"2024-03-20T08:22:14.915278+0000","last_undegraded":"2024-03-20T08:22:14.915278+0000","last_fullsized":"2024-03-20T08:22:14.915278+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:20:02.545783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850507+0000","last_change":"2024-03-20T08:08:21.626943+0000","last_active":"2024-03-20T08:20:49.850507+0000","last_peered":"2024-03-20T08:20:49.850507+0000","last_clean":"2024-03-20T08:20:49.850507+0000","last_became_active":"2024-03-20T08:08:21.625631+0000","last_became_peered":"2024-03-20T08:08:21.625631+0000","last_unstale":"2024-03-20T08:20:49.850507+0000","last_undegraded":"2024-03-20T08:20:49.850507+0000","last_fullsized":"2024-03-20T08:20:49.850507+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:48:38.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915248+0000","last_change":"2024-03-20T08:09:46.782628+0000","last_active":"2024-03-20T08:22:14.915248+0000","last_peered":"2024-03-20T08:22:14.915248+0000","last_clean":"2024-03-20T08:22:14.915248+0000","last_became_active":"2024-03-20T08:09:46.782153+0000","last_became_peered":"2024-03-20T08:09:46.782153+0000","last_unstale":"2024-03-20T08:22:14.915248+0000","last_undegraded":"2024-03-20T08:22:14.915248+0000","last_fullsized":"2024-03-20T08:22:14.915248+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:43.253651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.778123+0000","last_change":"2024-03-20T08:08:59.151492+0000","last_active":"2024-03-20T08:22:14.778123+0000","last_peered":"2024-03-20T08:22:14.778123+0000","last_clean":"2024-03-20T08:22:14.778123+0000","last_became_active":"2024-03-20T08:08:59.151007+0000","last_became_peered":"2024-03-20T08:08:59.151007+0000","last_unstale":"2024-03-20T08:22:14.778123+0000","last_undegraded":"2024-03-20T08:22:14.778123+0000","last_fullsized":"2024-03-20T08:22:14.778123+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:05:17.173448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1127,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912609+0000","last_change":"2024-03-20T08:09:46.787775+0000","last_active":"2024-03-20T08:21:49.912609+0000","last_peered":"2024-03-20T08:21:49.912609+0000","last_clean":"2024-03-20T08:21:49.912609+0000","last_became_active":"2024-03-20T08:09:46.787501+0000","last_became_peered":"2024-03-20T08:09:46.787501+0000","last_unstale":"2024-03-20T08:21:49.912609+0000","last_undegraded":"2024-03-20T08:21:49.912609+0000","last_fullsized":"2024-03-20T08:21:49.912609+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:20:22.472465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441696+0000","last_change":"2024-03-20T08:08:59.166005+0000","last_active":"2024-03-20T08:21:02.441696+0000","last_peered":"2024-03-20T08:21:02.441696+0000","last_clean":"2024-03-20T08:21:02.441696+0000","last_became_active":"2024-03-20T08:08:59.165136+0000","last_became_peered":"2024-03-20T08:08:59.165136+0000","last_unstale":"2024-03-20T08:21:02.441696+0000","last_undegraded":"2024-03-20T08:21:02.441696+0000","last_fullsized":"2024-03-20T08:21:02.441696+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.381730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1187,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441665+0000","last_change":"2024-03-20T08:08:59.149091+0000","last_active":"2024-03-20T08:21:02.441665+0000","last_peered":"2024-03-20T08:21:02.441665+0000","last_clean":"2024-03-20T08:21:02.441665+0000","last_became_active":"2024-03-20T08:08:59.148686+0000","last_became_peered":"2024-03-20T08:08:59.148686+0000","last_unstale":"2024-03-20T08:21:02.441665+0000","last_undegraded":"2024-03-20T08:21:02.441665+0000","last_fullsized":"2024-03-20T08:21:02.441665+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:29:11.980980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1258,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.735508+0000","last_change":"2024-03-20T08:08:59.159838+0000","last_active":"2024-03-20T08:22:24.735508+0000","last_peered":"2024-03-20T08:22:24.735508+0000","last_clean":"2024-03-20T08:22:24.735508+0000","last_became_active":"2024-03-20T08:08:59.158755+0000","last_became_peered":"2024-03-20T08:08:59.158755+0000","last_unstale":"2024-03-20T08:22:24.735508+0000","last_undegraded":"2024-03-20T08:22:24.735508+0000","last_fullsized":"2024-03-20T08:22:24.735508+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:53:38.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1281,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866885+0000","last_change":"2024-03-20T08:08:59.145647+0000","last_active":"2024-03-20T08:21:44.866885+0000","last_peered":"2024-03-20T08:21:44.866885+0000","last_clean":"2024-03-20T08:21:44.866885+0000","last_became_active":"2024-03-20T08:08:59.145241+0000","last_became_peered":"2024-03-20T08:08:59.145241+0000","last_unstale":"2024-03-20T08:21:44.866885+0000","last_undegraded":"2024-03-20T08:21:44.866885+0000","last_fullsized":"2024-03-20T08:21:44.866885+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:18:29.922392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992610+0000","last_change":"2024-03-20T08:08:21.630855+0000","last_active":"2024-03-20T08:21:45.992610+0000","last_peered":"2024-03-20T08:21:45.992610+0000","last_clean":"2024-03-20T08:21:45.992610+0000","last_became_active":"2024-03-20T08:08:21.630490+0000","last_became_peered":"2024-03-20T08:08:21.630490+0000","last_unstale":"2024-03-20T08:21:45.992610+0000","last_undegraded":"2024-03-20T08:21:45.992610+0000","last_fullsized":"2024-03-20T08:21:45.992610+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:16:49.554565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:22.307947+0000","last_change":"2024-03-20T08:08:21.632447+0000","last_active":"2024-03-20T08:22:22.307947+0000","last_peered":"2024-03-20T08:22:22.307947+0000","last_clean":"2024-03-20T08:22:22.307947+0000","last_became_active":"2024-03-20T08:08:21.632138+0000","last_became_peered":"2024-03-20T08:08:21.632138+0000","last_unstale":"2024-03-20T08:22:22.307947+0000","last_undegraded":"2024-03-20T08:22:22.307947+0000","last_fullsized":"2024-03-20T08:22:22.307947+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:37:01.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1203,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860734+0000","last_change":"2024-03-20T08:09:46.778697+0000","last_active":"2024-03-20T08:22:19.860734+0000","last_peered":"2024-03-20T08:22:19.860734+0000","last_clean":"2024-03-20T08:22:19.860734+0000","last_became_active":"2024-03-20T08:09:46.778410+0000","last_became_peered":"2024-03-20T08:09:46.778410+0000","last_unstale":"2024-03-20T08:22:19.860734+0000","last_undegraded":"2024-03-20T08:22:19.860734+0000","last_fullsized":"2024-03-20T08:22:19.860734+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:25:27.147344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1225,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860800+0000","last_change":"2024-03-20T08:09:46.779486+0000","last_active":"2024-03-20T08:22:19.860800+0000","last_peered":"2024-03-20T08:22:19.860800+0000","last_clean":"2024-03-20T08:22:19.860800+0000","last_became_active":"2024-03-20T08:09:46.779206+0000","last_became_peered":"2024-03-20T08:09:46.779206+0000","last_unstale":"2024-03-20T08:22:19.860800+0000","last_undegraded":"2024-03-20T08:22:19.860800+0000","last_fullsized":"2024-03-20T08:22:19.860800+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:08:08.971275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1303,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878869+0000","last_change":"2024-03-20T08:09:46.778387+0000","last_active":"2024-03-20T08:20:42.878869+0000","last_peered":"2024-03-20T08:20:42.878869+0000","last_clean":"2024-03-20T08:20:42.878869+0000","last_became_active":"2024-03-20T08:09:46.777529+0000","last_became_peered":"2024-03-20T08:09:46.777529+0000","last_unstale":"2024-03-20T08:20:42.878869+0000","last_undegraded":"2024-03-20T08:20:42.878869+0000","last_fullsized":"2024-03-20T08:20:42.878869+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:12:51.057390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906676+0000","last_change":"2024-03-20T08:09:46.791070+0000","last_active":"2024-03-20T08:20:59.906676+0000","last_peered":"2024-03-20T08:20:59.906676+0000","last_clean":"2024-03-20T08:20:59.906676+0000","last_became_active":"2024-03-20T08:09:46.789880+0000","last_became_peered":"2024-03-20T08:09:46.789880+0000","last_unstale":"2024-03-20T08:20:59.906676+0000","last_undegraded":"2024-03-20T08:20:59.906676+0000","last_fullsized":"2024-03-20T08:20:59.906676+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:08:42.437829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"583'5","reported_seq":1396,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994341+0000","last_change":"2024-03-20T08:09:46.776155+0000","last_active":"2024-03-20T08:22:05.994341+0000","last_peered":"2024-03-20T08:22:05.994341+0000","last_clean":"2024-03-20T08:22:05.994341+0000","last_became_active":"2024-03-20T08:09:46.773802+0000","last_became_peered":"2024-03-20T08:09:46.773802+0000","last_unstale":"2024-03-20T08:22:05.994341+0000","last_undegraded":"2024-03-20T08:22:05.994341+0000","last_fullsized":"2024-03-20T08:22:05.994341+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:35:14.843152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:23:19.783635+0000","last_change":"2024-03-20T08:08:59.145626+0000","last_active":"2024-03-20T08:23:19.783635+0000","last_peered":"2024-03-20T08:23:19.783635+0000","last_clean":"2024-03-20T08:23:19.783635+0000","last_became_active":"2024-03-20T08:08:59.145195+0000","last_became_peered":"2024-03-20T08:08:59.145195+0000","last_unstale":"2024-03-20T08:23:19.783635+0000","last_undegraded":"2024-03-20T08:23:19.783635+0000","last_fullsized":"2024-03-20T08:23:19.783635+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:21:36.809831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"583'2","reported_seq":1292,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:23:19.783647+0000","last_change":"2024-03-20T08:08:21.624506+0000","last_active":"2024-03-20T08:23:19.783647+0000","last_peered":"2024-03-20T08:23:19.783647+0000","last_clean":"2024-03-20T08:23:19.783647+0000","last_became_active":"2024-03-20T08:08:21.624185+0000","last_became_peered":"2024-03-20T08:08:21.624185+0000","last_unstale":"2024-03-20T08:23:19.783647+0000","last_undegraded":"2024-03-20T08:23:19.783647+0000","last_fullsized":"2024-03-20T08:23:19.783647+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:02:25.219512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994332+0000","last_change":"2024-03-20T08:08:21.636693+0000","last_active":"2024-03-20T08:22:05.994332+0000","last_peered":"2024-03-20T08:22:05.994332+0000","last_clean":"2024-03-20T08:22:05.994332+0000","last_became_active":"2024-03-20T08:08:21.636288+0000","last_became_peered":"2024-03-20T08:08:21.636288+0000","last_unstale":"2024-03-20T08:22:05.994332+0000","last_undegraded":"2024-03-20T08:22:05.994332+0000","last_fullsized":"2024-03-20T08:22:05.994332+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:00:23.684407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"62'2","reported_seq":1124,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915365+0000","last_change":"2024-03-20T08:09:46.788149+0000","last_active":"2024-03-20T08:22:14.915365+0000","last_peered":"2024-03-20T08:22:14.915365+0000","last_clean":"2024-03-20T08:22:14.915365+0000","last_became_active":"2024-03-20T08:09:46.786480+0000","last_became_peered":"2024-03-20T08:09:46.786480+0000","last_unstale":"2024-03-20T08:22:14.915365+0000","last_undegraded":"2024-03-20T08:22:14.915365+0000","last_fullsized":"2024-03-20T08:22:14.915365+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:58:08.581690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1229,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:49.850526+0000","last_change":"2024-03-20T08:08:21.633982+0000","last_active":"2024-03-20T08:20:49.850526+0000","last_peered":"2024-03-20T08:20:49.850526+0000","last_clean":"2024-03-20T08:20:49.850526+0000","last_became_active":"2024-03-20T08:08:21.633659+0000","last_became_peered":"2024-03-20T08:08:21.633659+0000","last_unstale":"2024-03-20T08:20:49.850526+0000","last_undegraded":"2024-03-20T08:20:49.850526+0000","last_fullsized":"2024-03-20T08:20:49.850526+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:59:34.315929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"62'2","reported_seq":1132,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915290+0000","last_change":"2024-03-20T08:09:46.790262+0000","last_active":"2024-03-20T08:22:14.915290+0000","last_peered":"2024-03-20T08:22:14.915290+0000","last_clean":"2024-03-20T08:22:14.915290+0000","last_became_active":"2024-03-20T08:09:46.789584+0000","last_became_peered":"2024-03-20T08:09:46.789584+0000","last_unstale":"2024-03-20T08:22:14.915290+0000","last_undegraded":"2024-03-20T08:22:14.915290+0000","last_fullsized":"2024-03-20T08:22:14.915290+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:22.118370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1293,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:42.883328+0000","last_change":"2024-03-20T08:08:59.160491+0000","last_active":"2024-03-20T08:21:42.883328+0000","last_peered":"2024-03-20T08:21:42.883328+0000","last_clean":"2024-03-20T08:21:42.883328+0000","last_became_active":"2024-03-20T08:08:59.160137+0000","last_became_peered":"2024-03-20T08:08:59.160137+0000","last_unstale":"2024-03-20T08:21:42.883328+0000","last_undegraded":"2024-03-20T08:21:42.883328+0000","last_fullsized":"2024-03-20T08:21:42.883328+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:51:19.459943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"583'2","reported_seq":1290,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.778159+0000","last_change":"2024-03-20T08:09:46.769685+0000","last_active":"2024-03-20T08:22:14.778159+0000","last_peered":"2024-03-20T08:22:14.778159+0000","last_clean":"2024-03-20T08:22:14.778159+0000","last_became_active":"2024-03-20T08:09:46.769163+0000","last_became_peered":"2024-03-20T08:09:46.769163+0000","last_unstale":"2024-03-20T08:22:14.778159+0000","last_undegraded":"2024-03-20T08:22:14.778159+0000","last_fullsized":"2024-03-20T08:22:14.778159+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:03:35.769310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1257,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:24.722582+0000","last_change":"2024-03-20T08:07:56.433455+0000","last_active":"2024-03-20T08:20:24.722582+0000","last_peered":"2024-03-20T08:20:24.722582+0000","last_clean":"2024-03-20T08:20:24.722582+0000","last_became_active":"2024-03-20T08:07:56.433148+0000","last_became_peered":"2024-03-20T08:07:56.433148+0000","last_unstale":"2024-03-20T08:20:24.722582+0000","last_undegraded":"2024-03-20T08:20:24.722582+0000","last_fullsized":"2024-03-20T08:20:24.722582+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:40:20.100690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"583'2","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444464+0000","last_change":"2024-03-20T08:09:46.789395+0000","last_active":"2024-03-20T08:21:27.444464+0000","last_peered":"2024-03-20T08:21:27.444464+0000","last_clean":"2024-03-20T08:21:27.444464+0000","last_became_active":"2024-03-20T08:09:46.784907+0000","last_became_peered":"2024-03-20T08:09:46.784907+0000","last_unstale":"2024-03-20T08:21:27.444464+0000","last_undegraded":"2024-03-20T08:21:27.444464+0000","last_fullsized":"2024-03-20T08:21:27.444464+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:18:06.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1385,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:25.986058+0000","last_change":"2024-03-20T08:09:46.779066+0000","last_active":"2024-03-20T08:20:25.986058+0000","last_peered":"2024-03-20T08:20:25.986058+0000","last_clean":"2024-03-20T08:20:25.986058+0000","last_became_active":"2024-03-20T08:09:46.773391+0000","last_became_peered":"2024-03-20T08:09:46.773391+0000","last_unstale":"2024-03-20T08:20:25.986058+0000","last_undegraded":"2024-03-20T08:20:25.986058+0000","last_fullsized":"2024-03-20T08:20:25.986058+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:35:16.371134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866861+0000","last_change":"2024-03-20T08:08:59.145546+0000","last_active":"2024-03-20T08:21:44.866861+0000","last_peered":"2024-03-20T08:21:44.866861+0000","last_clean":"2024-03-20T08:21:44.866861+0000","last_became_active":"2024-03-20T08:08:59.145259+0000","last_became_peered":"2024-03-20T08:08:59.145259+0000","last_unstale":"2024-03-20T08:21:44.866861+0000","last_undegraded":"2024-03-20T08:21:44.866861+0000","last_fullsized":"2024-03-20T08:21:44.866861+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:05:54.769326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.866825+0000","last_change":"2024-03-20T08:07:16.796131+0000","last_active":"2024-03-20T08:21:44.866825+0000","last_peered":"2024-03-20T08:21:44.866825+0000","last_clean":"2024-03-20T08:21:44.866825+0000","last_became_active":"2024-03-20T08:07:16.795861+0000","last_became_peered":"2024-03-20T08:07:16.795861+0000","last_unstale":"2024-03-20T08:21:44.866825+0000","last_undegraded":"2024-03-20T08:21:44.866825+0000","last_fullsized":"2024-03-20T08:21:44.866825+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:24:36.574849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1225,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:49.864358+0000","last_change":"2024-03-20T08:09:46.776487+0000","last_active":"2024-03-20T08:22:49.864358+0000","last_peered":"2024-03-20T08:22:49.864358+0000","last_clean":"2024-03-20T08:22:49.864358+0000","last_became_active":"2024-03-20T08:09:46.776171+0000","last_became_peered":"2024-03-20T08:09:46.776171+0000","last_unstale":"2024-03-20T08:22:49.864358+0000","last_undegraded":"2024-03-20T08:22:49.864358+0000","last_fullsized":"2024-03-20T08:22:49.864358+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:19:34.026094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1344,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:57.310798+0000","last_change":"2024-03-20T08:07:56.433463+0000","last_active":"2024-03-20T08:22:57.310798+0000","last_peered":"2024-03-20T08:22:57.310798+0000","last_clean":"2024-03-20T08:22:57.310798+0000","last_became_active":"2024-03-20T08:07:56.433167+0000","last_became_peered":"2024-03-20T08:07:56.433167+0000","last_unstale":"2024-03-20T08:22:57.310798+0000","last_undegraded":"2024-03-20T08:22:57.310798+0000","last_fullsized":"2024-03-20T08:22:57.310798+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:35:51.764016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:02.441735+0000","last_change":"2024-03-20T08:08:59.167977+0000","last_active":"2024-03-20T08:21:02.441735+0000","last_peered":"2024-03-20T08:21:02.441735+0000","last_clean":"2024-03-20T08:21:02.441735+0000","last_became_active":"2024-03-20T08:08:59.165220+0000","last_became_peered":"2024-03-20T08:08:59.165220+0000","last_unstale":"2024-03-20T08:21:02.441735+0000","last_undegraded":"2024-03-20T08:21:02.441735+0000","last_fullsized":"2024-03-20T08:21:02.441735+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:09:38.762324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:49.864412+0000","last_change":"2024-03-20T08:09:46.776304+0000","last_active":"2024-03-20T08:22:49.864412+0000","last_peered":"2024-03-20T08:22:49.864412+0000","last_clean":"2024-03-20T08:22:49.864412+0000","last_became_active":"2024-03-20T08:09:46.775993+0000","last_became_peered":"2024-03-20T08:09:46.775993+0000","last_unstale":"2024-03-20T08:22:49.864412+0000","last_undegraded":"2024-03-20T08:22:49.864412+0000","last_fullsized":"2024-03-20T08:22:49.864412+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:09.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1198,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:49.864401+0000","last_change":"2024-03-20T08:08:59.155220+0000","last_active":"2024-03-20T08:22:49.864401+0000","last_peered":"2024-03-20T08:22:49.864401+0000","last_clean":"2024-03-20T08:22:49.864401+0000","last_became_active":"2024-03-20T08:08:59.154928+0000","last_became_peered":"2024-03-20T08:08:59.154928+0000","last_unstale":"2024-03-20T08:22:49.864401+0000","last_undegraded":"2024-03-20T08:22:49.864401+0000","last_fullsized":"2024-03-20T08:22:49.864401+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:02:37.555261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:20.995500+0000","last_change":"2024-03-20T08:08:21.627627+0000","last_active":"2024-03-20T08:22:20.995500+0000","last_peered":"2024-03-20T08:22:20.995500+0000","last_clean":"2024-03-20T08:22:20.995500+0000","last_became_active":"2024-03-20T08:08:21.625592+0000","last_became_peered":"2024-03-20T08:08:21.625592+0000","last_unstale":"2024-03-20T08:22:20.995500+0000","last_undegraded":"2024-03-20T08:22:20.995500+0000","last_fullsized":"2024-03-20T08:22:20.995500+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:45:28.677119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"62'3","reported_seq":1349,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:57.310832+0000","last_change":"2024-03-20T08:08:21.626540+0000","last_active":"2024-03-20T08:22:57.310832+0000","last_peered":"2024-03-20T08:22:57.310832+0000","last_clean":"2024-03-20T08:22:57.310832+0000","last_became_active":"2024-03-20T08:08:21.626188+0000","last_became_peered":"2024-03-20T08:08:21.626188+0000","last_unstale":"2024-03-20T08:22:57.310832+0000","last_undegraded":"2024-03-20T08:22:57.310832+0000","last_fullsized":"2024-03-20T08:22:57.310832+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:25:00.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1259,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:54.732439+0000","last_change":"2024-03-20T08:09:46.773925+0000","last_active":"2024-03-20T08:21:54.732439+0000","last_peered":"2024-03-20T08:21:54.732439+0000","last_clean":"2024-03-20T08:21:54.732439+0000","last_became_active":"2024-03-20T08:09:46.773643+0000","last_became_peered":"2024-03-20T08:09:46.773643+0000","last_unstale":"2024-03-20T08:21:54.732439+0000","last_undegraded":"2024-03-20T08:21:54.732439+0000","last_fullsized":"2024-03-20T08:21:54.732439+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:21:41.982250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"583'29","reported_seq":1224,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:57.454355+0000","last_change":"2024-03-20T08:08:59.150321+0000","last_active":"2024-03-20T08:22:57.454355+0000","last_peered":"2024-03-20T08:22:57.454355+0000","last_clean":"2024-03-20T08:22:57.454355+0000","last_became_active":"2024-03-20T08:08:59.149968+0000","last_became_peered":"2024-03-20T08:08:59.149968+0000","last_unstale":"2024-03-20T08:22:57.454355+0000","last_undegraded":"2024-03-20T08:22:57.454355+0000","last_fullsized":"2024-03-20T08:22:57.454355+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:45:07.736173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5822,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":14,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.912579+0000","last_change":"2024-03-20T08:09:46.783372+0000","last_active":"2024-03-20T08:21:49.912579+0000","last_peered":"2024-03-20T08:21:49.912579+0000","last_clean":"2024-03-20T08:21:49.912579+0000","last_became_active":"2024-03-20T08:09:46.782756+0000","last_became_peered":"2024-03-20T08:09:46.782756+0000","last_unstale":"2024-03-20T08:21:49.912579+0000","last_undegraded":"2024-03-20T08:21:49.912579+0000","last_fullsized":"2024-03-20T08:21:49.912579+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:09:43.749173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"62'2","reported_seq":1391,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992645+0000","last_change":"2024-03-20T08:07:56.426194+0000","last_active":"2024-03-20T08:21:45.992645+0000","last_peered":"2024-03-20T08:21:45.992645+0000","last_clean":"2024-03-20T08:21:45.992645+0000","last_became_active":"2024-03-20T08:07:56.425590+0000","last_became_peered":"2024-03-20T08:07:56.425590+0000","last_unstale":"2024-03-20T08:21:45.992645+0000","last_undegraded":"2024-03-20T08:21:45.992645+0000","last_fullsized":"2024-03-20T08:21:45.992645+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:28:16.644010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:45.992629+0000","last_change":"2024-03-20T08:09:46.774373+0000","last_active":"2024-03-20T08:21:45.992629+0000","last_peered":"2024-03-20T08:21:45.992629+0000","last_clean":"2024-03-20T08:21:45.992629+0000","last_became_active":"2024-03-20T08:09:46.774063+0000","last_became_peered":"2024-03-20T08:09:46.774063+0000","last_unstale":"2024-03-20T08:21:45.992629+0000","last_undegraded":"2024-03-20T08:21:45.992629+0000","last_fullsized":"2024-03-20T08:21:45.992629+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:27:18.708718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1231,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:19.860764+0000","last_change":"2024-03-20T08:09:46.777069+0000","last_active":"2024-03-20T08:22:19.860764+0000","last_peered":"2024-03-20T08:22:19.860764+0000","last_clean":"2024-03-20T08:22:19.860764+0000","last_became_active":"2024-03-20T08:09:46.776734+0000","last_became_peered":"2024-03-20T08:09:46.776734+0000","last_unstale":"2024-03-20T08:22:19.860764+0000","last_undegraded":"2024-03-20T08:22:19.860764+0000","last_fullsized":"2024-03-20T08:22:19.860764+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:46:17.530016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909806+0000","last_change":"2024-03-20T08:09:46.789010+0000","last_active":"2024-03-20T08:21:24.909806+0000","last_peered":"2024-03-20T08:21:24.909806+0000","last_clean":"2024-03-20T08:21:24.909806+0000","last_became_active":"2024-03-20T08:09:46.788176+0000","last_became_peered":"2024-03-20T08:09:46.788176+0000","last_unstale":"2024-03-20T08:21:24.909806+0000","last_undegraded":"2024-03-20T08:21:24.909806+0000","last_fullsized":"2024-03-20T08:21:24.909806+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:21:41.245478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1206,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:57.454331+0000","last_change":"2024-03-20T08:08:59.163048+0000","last_active":"2024-03-20T08:22:57.454331+0000","last_peered":"2024-03-20T08:22:57.454331+0000","last_clean":"2024-03-20T08:22:57.454331+0000","last_became_active":"2024-03-20T08:08:59.162728+0000","last_became_peered":"2024-03-20T08:08:59.162728+0000","last_unstale":"2024-03-20T08:22:57.454331+0000","last_undegraded":"2024-03-20T08:22:57.454331+0000","last_fullsized":"2024-03-20T08:22:57.454331+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:47:46.072649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:57.454303+0000","last_change":"2024-03-20T08:08:59.151973+0000","last_active":"2024-03-20T08:22:57.454303+0000","last_peered":"2024-03-20T08:22:57.454303+0000","last_clean":"2024-03-20T08:22:57.454303+0000","last_became_active":"2024-03-20T08:08:59.151552+0000","last_became_peered":"2024-03-20T08:08:59.151552+0000","last_unstale":"2024-03-20T08:22:57.454303+0000","last_undegraded":"2024-03-20T08:22:57.454303+0000","last_fullsized":"2024-03-20T08:22:57.454303+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:59:37.375208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"62'8","reported_seq":1319,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:42.878816+0000","last_change":"2024-03-20T08:06:44.493051+0000","last_active":"2024-03-20T08:20:42.878816+0000","last_peered":"2024-03-20T08:20:42.878816+0000","last_clean":"2024-03-20T08:20:42.878816+0000","last_became_active":"2024-03-20T08:06:44.492114+0000","last_became_peered":"2024-03-20T08:06:44.492114+0000","last_unstale":"2024-03-20T08:20:42.878816+0000","last_undegraded":"2024-03-20T08:20:42.878816+0000","last_fullsized":"2024-03-20T08:20:42.878816+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:16.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1120,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.909777+0000","last_change":"2024-03-20T08:09:46.782906+0000","last_active":"2024-03-20T08:21:24.909777+0000","last_peered":"2024-03-20T08:21:24.909777+0000","last_clean":"2024-03-20T08:21:24.909777+0000","last_became_active":"2024-03-20T08:09:46.782573+0000","last_became_peered":"2024-03-20T08:09:46.782573+0000","last_unstale":"2024-03-20T08:21:24.909777+0000","last_undegraded":"2024-03-20T08:21:24.909777+0000","last_fullsized":"2024-03-20T08:21:24.909777+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:12:09.193600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"863'30170","reported_seq":26674,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884162+0000","last_change":"2024-03-20T08:15:23.992533+0000","last_active":"2024-03-20T08:21:52.884162+0000","last_peered":"2024-03-20T08:21:52.884162+0000","last_clean":"2024-03-20T08:21:52.884162+0000","last_became_active":"2024-03-20T08:06:44.490277+0000","last_became_peered":"2024-03-20T08:06:44.490277+0000","last_unstale":"2024-03-20T08:21:52.884162+0000","last_undegraded":"2024-03-20T08:21:52.884162+0000","last_fullsized":"2024-03-20T08:21:52.884162+0000","mapping_epoch":376,"log_start":"466'22007","ondisk_log_start":"466'22007","created":73,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8163,"log_dups_size":0,"ondisk_log_size":8163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:47:12.179363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010726757,"stat_sum":{"num_bytes":117088256,"num_objects":61,"num_object_clones":58,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":216,"num_read_kb":239366,"num_write":413,"num_write_kb":110712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4571136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.f","version":"62'1","reported_seq":1261,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:24.728906+0000","last_change":"2024-03-20T08:09:46.774634+0000","last_active":"2024-03-20T08:21:24.728906+0000","last_peered":"2024-03-20T08:21:24.728906+0000","last_clean":"2024-03-20T08:21:24.728906+0000","last_became_active":"2024-03-20T08:09:46.774328+0000","last_became_peered":"2024-03-20T08:09:46.774328+0000","last_unstale":"2024-03-20T08:21:24.728906+0000","last_undegraded":"2024-03-20T08:21:24.728906+0000","last_fullsized":"2024-03-20T08:21:24.728906+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:47:28.104199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1143,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906693+0000","last_change":"2024-03-20T08:09:46.784658+0000","last_active":"2024-03-20T08:20:59.906693+0000","last_peered":"2024-03-20T08:20:59.906693+0000","last_clean":"2024-03-20T08:20:59.906693+0000","last_became_active":"2024-03-20T08:09:46.783562+0000","last_became_peered":"2024-03-20T08:09:46.783562+0000","last_unstale":"2024-03-20T08:20:59.906693+0000","last_undegraded":"2024-03-20T08:20:59.906693+0000","last_fullsized":"2024-03-20T08:20:59.906693+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:36:25.747822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"866'7201","reported_seq":11692,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994261+0000","last_change":"2024-03-20T08:15:24.083144+0000","last_active":"2024-03-20T08:22:05.994261+0000","last_peered":"2024-03-20T08:22:05.994261+0000","last_clean":"2024-03-20T08:22:05.994261+0000","last_became_active":"2024-03-20T08:08:59.161780+0000","last_became_peered":"2024-03-20T08:08:59.161780+0000","last_unstale":"2024-03-20T08:22:05.994261+0000","last_undegraded":"2024-03-20T08:22:05.994261+0000","last_fullsized":"2024-03-20T08:22:05.994261+0000","mapping_epoch":505,"log_start":"302'1500","ondisk_log_start":"302'1500","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":298,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:11:40.591402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072617309999999996,"stat_sum":{"num_bytes":71794688,"num_objects":58,"num_object_clones":56,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":126,"num_read_kb":72240,"num_write":399,"num_write_kb":70960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1581056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309503+0000","last_change":"2024-03-20T08:08:21.632876+0000","last_active":"2024-03-20T08:22:42.309503+0000","last_peered":"2024-03-20T08:22:42.309503+0000","last_clean":"2024-03-20T08:22:42.309503+0000","last_became_active":"2024-03-20T08:08:21.632577+0000","last_became_peered":"2024-03-20T08:08:21.632577+0000","last_unstale":"2024-03-20T08:22:42.309503+0000","last_undegraded":"2024-03-20T08:22:42.309503+0000","last_fullsized":"2024-03-20T08:22:42.309503+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:58:31.968474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309527+0000","last_change":"2024-03-20T08:09:46.771631+0000","last_active":"2024-03-20T08:22:42.309527+0000","last_peered":"2024-03-20T08:22:42.309527+0000","last_clean":"2024-03-20T08:22:42.309527+0000","last_became_active":"2024-03-20T08:09:46.771269+0000","last_became_peered":"2024-03-20T08:09:46.771269+0000","last_unstale":"2024-03-20T08:22:42.309527+0000","last_undegraded":"2024-03-20T08:22:42.309527+0000","last_fullsized":"2024-03-20T08:22:42.309527+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:19:50.274093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1237,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:54.865182+0000","last_change":"2024-03-20T08:08:21.640046+0000","last_active":"2024-03-20T08:22:54.865182+0000","last_peered":"2024-03-20T08:22:54.865182+0000","last_clean":"2024-03-20T08:22:54.865182+0000","last_became_active":"2024-03-20T08:08:21.639724+0000","last_became_peered":"2024-03-20T08:08:21.639724+0000","last_unstale":"2024-03-20T08:22:54.865182+0000","last_undegraded":"2024-03-20T08:22:54.865182+0000","last_fullsized":"2024-03-20T08:22:54.865182+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:50:50.254561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"865'1276","reported_seq":5954,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:05.994315+0000","last_change":"2024-03-20T08:15:24.083340+0000","last_active":"2024-03-20T08:22:05.994315+0000","last_peered":"2024-03-20T08:22:05.994315+0000","last_clean":"2024-03-20T08:22:05.994315+0000","last_became_active":"2024-03-20T08:07:56.436739+0000","last_became_peered":"2024-03-20T08:07:56.436739+0000","last_unstale":"2024-03-20T08:22:05.994315+0000","last_undegraded":"2024-03-20T08:22:05.994315+0000","last_fullsized":"2024-03-20T08:22:05.994315+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":297,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:47:29.034671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.026231873999999999,"stat_sum":{"num_bytes":230875136,"num_objects":158,"num_object_clones":152,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":2,"num_read":455,"num_read_kb":669693,"num_write":1047,"num_write_kb":231232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":59293696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"32","length":"1"}]},{"pgid":"4.4","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309539+0000","last_change":"2024-03-20T08:07:16.794620+0000","last_active":"2024-03-20T08:22:42.309539+0000","last_peered":"2024-03-20T08:22:42.309539+0000","last_clean":"2024-03-20T08:22:42.309539+0000","last_became_active":"2024-03-20T08:07:16.794351+0000","last_became_peered":"2024-03-20T08:07:16.794351+0000","last_unstale":"2024-03-20T08:22:42.309539+0000","last_undegraded":"2024-03-20T08:22:42.309539+0000","last_fullsized":"2024-03-20T08:22:42.309539+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:31:23.572298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1201,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:54.865117+0000","last_change":"2024-03-20T08:08:59.157171+0000","last_active":"2024-03-20T08:22:54.865117+0000","last_peered":"2024-03-20T08:22:54.865117+0000","last_clean":"2024-03-20T08:22:54.865117+0000","last_became_active":"2024-03-20T08:08:59.156514+0000","last_became_peered":"2024-03-20T08:08:59.156514+0000","last_unstale":"2024-03-20T08:22:54.865117+0000","last_undegraded":"2024-03-20T08:22:54.865117+0000","last_fullsized":"2024-03-20T08:22:54.865117+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:18:47.568563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1260,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:29.735999+0000","last_change":"2024-03-20T08:08:59.160207+0000","last_active":"2024-03-20T08:22:29.735999+0000","last_peered":"2024-03-20T08:22:29.735999+0000","last_clean":"2024-03-20T08:22:29.735999+0000","last_became_active":"2024-03-20T08:08:59.159910+0000","last_became_peered":"2024-03-20T08:08:59.159910+0000","last_unstale":"2024-03-20T08:22:29.735999+0000","last_undegraded":"2024-03-20T08:22:29.735999+0000","last_fullsized":"2024-03-20T08:22:29.735999+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:25:06.170348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.6","version":"837'13707","reported_seq":17603,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884275+0000","last_change":"2024-03-20T08:15:23.993106+0000","last_active":"2024-03-20T08:21:52.884275+0000","last_peered":"2024-03-20T08:21:52.884275+0000","last_clean":"2024-03-20T08:21:52.884275+0000","last_became_active":"2024-03-20T08:08:21.648839+0000","last_became_peered":"2024-03-20T08:08:21.648839+0000","last_unstale":"2024-03-20T08:21:52.884275+0000","last_undegraded":"2024-03-20T08:21:52.884275+0000","last_fullsized":"2024-03-20T08:21:52.884275+0000","mapping_epoch":470,"log_start":"387'5500","ondisk_log_start":"387'5500","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":255,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:57.757576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033910439000000001,"stat_sum":{"num_bytes":89571328,"num_objects":41,"num_object_clones":39,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":2,"num_read":168,"num_read_kb":388847,"num_write":326,"num_write_kb":78456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":37806080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"f"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"4.7","version":"0'0","reported_seq":1336,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302498+0000","last_change":"2024-03-20T08:08:21.621414+0000","last_active":"2024-03-20T08:21:12.302498+0000","last_peered":"2024-03-20T08:21:12.302498+0000","last_clean":"2024-03-20T08:21:12.302498+0000","last_became_active":"2024-03-20T08:08:21.621028+0000","last_became_peered":"2024-03-20T08:08:21.621028+0000","last_unstale":"2024-03-20T08:21:12.302498+0000","last_undegraded":"2024-03-20T08:21:12.302498+0000","last_fullsized":"2024-03-20T08:21:12.302498+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:40:43.143422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:02.306280+0000","last_change":"2024-03-20T08:08:59.151435+0000","last_active":"2024-03-20T08:22:02.306280+0000","last_peered":"2024-03-20T08:22:02.306280+0000","last_clean":"2024-03-20T08:22:02.306280+0000","last_became_active":"2024-03-20T08:08:59.150979+0000","last_became_peered":"2024-03-20T08:08:59.150979+0000","last_unstale":"2024-03-20T08:22:02.306280+0000","last_undegraded":"2024-03-20T08:22:02.306280+0000","last_fullsized":"2024-03-20T08:22:02.306280+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:33:00.046726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881424+0000","last_change":"2024-03-20T08:07:16.808322+0000","last_active":"2024-03-20T08:21:17.881424+0000","last_peered":"2024-03-20T08:21:17.881424+0000","last_clean":"2024-03-20T08:21:17.881424+0000","last_became_active":"2024-03-20T08:07:16.807994+0000","last_became_peered":"2024-03-20T08:07:16.807994+0000","last_unstale":"2024-03-20T08:21:17.881424+0000","last_undegraded":"2024-03-20T08:21:17.881424+0000","last_fullsized":"2024-03-20T08:21:17.881424+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:39:54.045064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"798'335","reported_seq":3982,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:14.915318+0000","last_change":"2024-03-20T08:15:23.999759+0000","last_active":"2024-03-20T08:22:14.915318+0000","last_peered":"2024-03-20T08:22:14.915318+0000","last_clean":"2024-03-20T08:22:14.915318+0000","last_became_active":"2024-03-20T08:09:46.788826+0000","last_became_peered":"2024-03-20T08:09:46.788826+0000","last_unstale":"2024-03-20T08:22:14.915318+0000","last_undegraded":"2024-03-20T08:22:14.915318+0000","last_fullsized":"2024-03-20T08:22:14.915318+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":167,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:54:21.328757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069865499999999996,"stat_sum":{"num_bytes":44867584,"num_objects":16,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":88,"num_read_kb":198271,"num_write":171,"num_write_kb":56440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":23584768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"45"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11a","length":"3"},{"start":"11f","length":"8"},{"start":"129","length":"1"},{"start":"12b","length":"1"},{"start":"131","length":"1"}]},{"pgid":"4.5","version":"0'0","reported_seq":1193,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444515+0000","last_change":"2024-03-20T08:09:46.773534+0000","last_active":"2024-03-20T08:21:27.444515+0000","last_peered":"2024-03-20T08:21:27.444515+0000","last_clean":"2024-03-20T08:21:27.444515+0000","last_became_active":"2024-03-20T08:09:46.773204+0000","last_became_peered":"2024-03-20T08:09:46.773204+0000","last_unstale":"2024-03-20T08:21:27.444515+0000","last_undegraded":"2024-03-20T08:21:27.444515+0000","last_fullsized":"2024-03-20T08:21:27.444515+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:45:19.573978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1289,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.867213+0000","last_change":"2024-03-20T08:08:59.146022+0000","last_active":"2024-03-20T08:21:49.867213+0000","last_peered":"2024-03-20T08:21:49.867213+0000","last_clean":"2024-03-20T08:21:49.867213+0000","last_became_active":"2024-03-20T08:08:59.145704+0000","last_became_peered":"2024-03-20T08:08:59.145704+0000","last_unstale":"2024-03-20T08:21:49.867213+0000","last_undegraded":"2024-03-20T08:21:49.867213+0000","last_fullsized":"2024-03-20T08:21:49.867213+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:05:47.944640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1200,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:54.865165+0000","last_change":"2024-03-20T08:09:46.777413+0000","last_active":"2024-03-20T08:22:54.865165+0000","last_peered":"2024-03-20T08:22:54.865165+0000","last_clean":"2024-03-20T08:22:54.865165+0000","last_became_active":"2024-03-20T08:09:46.777110+0000","last_became_peered":"2024-03-20T08:09:46.777110+0000","last_unstale":"2024-03-20T08:22:54.865165+0000","last_undegraded":"2024-03-20T08:22:54.865165+0000","last_fullsized":"2024-03-20T08:22:54.865165+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:28:31.526136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"223'197","reported_seq":1614,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912064+0000","last_change":"2024-03-20T08:09:46.778465+0000","last_active":"2024-03-20T08:21:44.912064+0000","last_peered":"2024-03-20T08:21:44.912064+0000","last_clean":"2024-03-20T08:21:44.912064+0000","last_became_active":"2024-03-20T08:09:46.777659+0000","last_became_peered":"2024-03-20T08:09:46.777659+0000","last_unstale":"2024-03-20T08:21:44.912064+0000","last_undegraded":"2024-03-20T08:21:44.912064+0000","last_fullsized":"2024-03-20T08:21:44.912064+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:56:04.165332+0000","last_clean_scrub_stamp":"2024-03-20T07:56:04.165332+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:39:45.398543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"861'7009","reported_seq":9299,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:19.854083+0000","last_change":"2024-03-20T08:15:24.083720+0000","last_active":"2024-03-20T08:21:19.854083+0000","last_peered":"2024-03-20T08:21:19.854083+0000","last_clean":"2024-03-20T08:21:19.854083+0000","last_became_active":"2024-03-20T08:08:21.620369+0000","last_became_peered":"2024-03-20T08:08:21.620369+0000","last_unstale":"2024-03-20T08:21:19.854083+0000","last_undegraded":"2024-03-20T08:21:19.854083+0000","last_fullsized":"2024-03-20T08:21:19.854083+0000","mapping_epoch":470,"log_start":"471'801","ondisk_log_start":"471'801","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":211,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:36:24.692569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021291955000000001,"stat_sum":{"num_bytes":42369024,"num_objects":30,"num_object_clones":28,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":116,"num_read_kb":98674,"num_write":269,"num_write_kb":56584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":25051136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"4.6","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:45.997313+0000","last_change":"2024-03-20T08:06:44.492774+0000","last_active":"2024-03-20T08:22:45.997313+0000","last_peered":"2024-03-20T08:22:45.997313+0000","last_clean":"2024-03-20T08:22:45.997313+0000","last_became_active":"2024-03-20T08:06:44.492458+0000","last_became_peered":"2024-03-20T08:06:44.492458+0000","last_unstale":"2024-03-20T08:22:45.997313+0000","last_undegraded":"2024-03-20T08:22:45.997313+0000","last_fullsized":"2024-03-20T08:22:45.997313+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:43:03.993436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994228+0000","last_change":"2024-03-20T08:07:56.427186+0000","last_active":"2024-03-20T08:22:05.994228+0000","last_peered":"2024-03-20T08:22:05.994228+0000","last_clean":"2024-03-20T08:22:05.994228+0000","last_became_active":"2024-03-20T08:07:56.426919+0000","last_became_peered":"2024-03-20T08:07:56.426919+0000","last_unstale":"2024-03-20T08:22:05.994228+0000","last_undegraded":"2024-03-20T08:22:05.994228+0000","last_fullsized":"2024-03-20T08:22:05.994228+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:02.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:44.912041+0000","last_change":"2024-03-20T08:09:46.781749+0000","last_active":"2024-03-20T08:21:44.912041+0000","last_peered":"2024-03-20T08:21:44.912041+0000","last_clean":"2024-03-20T08:21:44.912041+0000","last_became_active":"2024-03-20T08:09:46.781382+0000","last_became_peered":"2024-03-20T08:09:46.781382+0000","last_unstale":"2024-03-20T08:21:44.912041+0000","last_undegraded":"2024-03-20T08:21:44.912041+0000","last_fullsized":"2024-03-20T08:21:44.912041+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:48:23.576737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"867'61812","reported_seq":49872,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:27.444480+0000","last_change":"2024-03-20T08:15:23.992166+0000","last_active":"2024-03-20T08:21:27.444480+0000","last_peered":"2024-03-20T08:21:27.444480+0000","last_clean":"2024-03-20T08:21:27.444480+0000","last_became_active":"2024-03-20T08:08:59.150702+0000","last_became_peered":"2024-03-20T08:08:59.150702+0000","last_unstale":"2024-03-20T08:21:27.444480+0000","last_undegraded":"2024-03-20T08:21:27.444480+0000","last_fullsized":"2024-03-20T08:21:27.444480+0000","mapping_epoch":505,"log_start":"640'54810","ondisk_log_start":"640'54810","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":7002,"log_dups_size":0,"ondisk_log_size":7002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":194,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:49:10.829439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024451138000000001,"stat_sum":{"num_bytes":155295744,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":1,"num_read":268,"num_read_kb":389080,"num_write":604,"num_write_kb":191728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f3","length":"1"},{"start":"f5","length":"8"},{"start":"fe","length":"7"},{"start":"107","length":"1"},{"start":"109","length":"1"},{"start":"10b","length":"2"}]},{"pgid":"4.3","version":"0'0","reported_seq":1387,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994287+0000","last_change":"2024-03-20T08:09:46.780445+0000","last_active":"2024-03-20T08:22:05.994287+0000","last_peered":"2024-03-20T08:22:05.994287+0000","last_clean":"2024-03-20T08:22:05.994287+0000","last_became_active":"2024-03-20T08:09:46.780018+0000","last_became_peered":"2024-03-20T08:09:46.780018+0000","last_unstale":"2024-03-20T08:22:05.994287+0000","last_undegraded":"2024-03-20T08:22:05.994287+0000","last_fullsized":"2024-03-20T08:22:05.994287+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:51:37.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1145,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:14.915352+0000","last_change":"2024-03-20T08:09:46.783620+0000","last_active":"2024-03-20T08:22:14.915352+0000","last_peered":"2024-03-20T08:22:14.915352+0000","last_clean":"2024-03-20T08:22:14.915352+0000","last_became_active":"2024-03-20T08:09:46.783115+0000","last_became_peered":"2024-03-20T08:09:46.783115+0000","last_unstale":"2024-03-20T08:22:14.915352+0000","last_undegraded":"2024-03-20T08:22:14.915352+0000","last_fullsized":"2024-03-20T08:22:14.915352+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:41:21.422811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1346,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:42.309547+0000","last_change":"2024-03-20T08:08:21.637799+0000","last_active":"2024-03-20T08:22:42.309547+0000","last_peered":"2024-03-20T08:22:42.309547+0000","last_clean":"2024-03-20T08:22:42.309547+0000","last_became_active":"2024-03-20T08:08:21.637437+0000","last_became_peered":"2024-03-20T08:08:21.637437+0000","last_unstale":"2024-03-20T08:22:42.309547+0000","last_undegraded":"2024-03-20T08:22:42.309547+0000","last_fullsized":"2024-03-20T08:22:42.309547+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:08:18.288951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"850'565","reported_seq":4608,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:29.735964+0000","last_change":"2024-03-20T08:15:24.150485+0000","last_active":"2024-03-20T08:22:29.735964+0000","last_peered":"2024-03-20T08:22:29.735964+0000","last_clean":"2024-03-20T08:22:29.735964+0000","last_became_active":"2024-03-20T08:09:46.774926+0000","last_became_peered":"2024-03-20T08:09:46.774926+0000","last_unstale":"2024-03-20T08:22:29.735964+0000","last_undegraded":"2024-03-20T08:22:29.735964+0000","last_fullsized":"2024-03-20T08:22:29.735964+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:05:37.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002758632,"stat_sum":{"num_bytes":67452928,"num_objects":42,"num_object_clones":40,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":143,"num_read_kb":149331,"num_write":408,"num_write_kb":83544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":21569536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:23:19.783603+0000","last_change":"2024-03-20T08:09:46.770492+0000","last_active":"2024-03-20T08:23:19.783603+0000","last_peered":"2024-03-20T08:23:19.783603+0000","last_clean":"2024-03-20T08:23:19.783603+0000","last_became_active":"2024-03-20T08:09:46.770006+0000","last_became_peered":"2024-03-20T08:09:46.770006+0000","last_unstale":"2024-03-20T08:23:19.783603+0000","last_undegraded":"2024-03-20T08:23:19.783603+0000","last_fullsized":"2024-03-20T08:23:19.783603+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:37:13.199962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"586'2","reported_seq":1350,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302482+0000","last_change":"2024-03-20T08:08:59.148935+0000","last_active":"2024-03-20T08:21:12.302482+0000","last_peered":"2024-03-20T08:21:12.302482+0000","last_clean":"2024-03-20T08:21:12.302482+0000","last_became_active":"2024-03-20T08:08:59.148565+0000","last_became_peered":"2024-03-20T08:08:59.148565+0000","last_unstale":"2024-03-20T08:21:12.302482+0000","last_undegraded":"2024-03-20T08:21:12.302482+0000","last_fullsized":"2024-03-20T08:21:12.302482+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:49:45.923895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779096+0000","last_change":"2024-03-20T08:09:46.770123+0000","last_active":"2024-03-20T08:22:24.779096+0000","last_peered":"2024-03-20T08:22:24.779096+0000","last_clean":"2024-03-20T08:22:24.779096+0000","last_became_active":"2024-03-20T08:09:46.769837+0000","last_became_peered":"2024-03-20T08:09:46.769837+0000","last_unstale":"2024-03-20T08:22:24.779096+0000","last_undegraded":"2024-03-20T08:22:24.779096+0000","last_fullsized":"2024-03-20T08:22:24.779096+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:58:17.575398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"862'1462","reported_seq":5632,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:23:19.783564+0000","last_change":"2024-03-20T08:15:23.991990+0000","last_active":"2024-03-20T08:23:19.783564+0000","last_peered":"2024-03-20T08:23:19.783564+0000","last_clean":"2024-03-20T08:23:19.783564+0000","last_became_active":"2024-03-20T08:07:56.425509+0000","last_became_peered":"2024-03-20T08:07:56.425509+0000","last_unstale":"2024-03-20T08:23:19.783564+0000","last_undegraded":"2024-03-20T08:23:19.783564+0000","last_fullsized":"2024-03-20T08:23:19.783564+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1462,"log_dups_size":0,"ondisk_log_size":1462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":240,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:06:37.442451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010531155,"stat_sum":{"num_bytes":225714176,"num_objects":99,"num_object_clones":94,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":374,"num_read_kb":574761,"num_write":862,"num_write_kb":241304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":39927808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"13"},{"start":"c4","length":"1"},{"start":"c6","length":"2"},{"start":"c9","length":"4"},{"start":"ce","length":"9"},{"start":"d8","length":"2"},{"start":"db","length":"2"}]},{"pgid":"4.1","version":"0'0","reported_seq":1219,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:29.735941+0000","last_change":"2024-03-20T08:08:59.159667+0000","last_active":"2024-03-20T08:22:29.735941+0000","last_peered":"2024-03-20T08:22:29.735941+0000","last_clean":"2024-03-20T08:22:29.735941+0000","last_became_active":"2024-03-20T08:08:59.158922+0000","last_became_peered":"2024-03-20T08:08:59.158922+0000","last_unstale":"2024-03-20T08:22:29.735941+0000","last_undegraded":"2024-03-20T08:22:29.735941+0000","last_fullsized":"2024-03-20T08:22:29.735941+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:59:00.974759+0000","last_clean_scrub_stamp":"2024-03-20T07:59:00.974759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T07:59:48.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"62'2","reported_seq":1176,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447941+0000","last_change":"2024-03-20T08:09:46.771803+0000","last_active":"2024-03-20T08:21:57.447941+0000","last_peered":"2024-03-20T08:21:57.447941+0000","last_clean":"2024-03-20T08:21:57.447941+0000","last_became_active":"2024-03-20T08:09:46.771441+0000","last_became_peered":"2024-03-20T08:09:46.771441+0000","last_unstale":"2024-03-20T08:21:57.447941+0000","last_undegraded":"2024-03-20T08:21:57.447941+0000","last_fullsized":"2024-03-20T08:21:57.447941+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:23:48.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:52.884264+0000","last_change":"2024-03-20T08:07:56.439717+0000","last_active":"2024-03-20T08:21:52.884264+0000","last_peered":"2024-03-20T08:21:52.884264+0000","last_clean":"2024-03-20T08:21:52.884264+0000","last_became_active":"2024-03-20T08:07:56.439027+0000","last_became_peered":"2024-03-20T08:07:56.439027+0000","last_unstale":"2024-03-20T08:21:52.884264+0000","last_undegraded":"2024-03-20T08:21:52.884264+0000","last_fullsized":"2024-03-20T08:21:52.884264+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:28:52.943272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"863'1262","reported_seq":5194,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:49.857496+0000","last_change":"2024-03-20T08:15:24.083849+0000","last_active":"2024-03-20T08:21:49.857496+0000","last_peered":"2024-03-20T08:21:49.857496+0000","last_clean":"2024-03-20T08:21:49.857496+0000","last_became_active":"2024-03-20T08:08:59.155771+0000","last_became_peered":"2024-03-20T08:08:59.155771+0000","last_unstale":"2024-03-20T08:21:49.857496+0000","last_undegraded":"2024-03-20T08:21:49.857496+0000","last_fullsized":"2024-03-20T08:21:49.857496+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":212,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:30:35.912790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028605290000000001,"stat_sum":{"num_bytes":130310144,"num_objects":77,"num_object_clones":73,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":348,"num_read_kb":615010,"num_write":633,"num_write_kb":132128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":43720704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"31"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"5"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"},{"start":"ff","length":"2"}]},{"pgid":"2.8","version":"0'0","reported_seq":1118,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:44.918743+0000","last_change":"2024-03-20T08:09:46.778724+0000","last_active":"2024-03-20T08:22:44.918743+0000","last_peered":"2024-03-20T08:22:44.918743+0000","last_clean":"2024-03-20T08:22:44.918743+0000","last_became_active":"2024-03-20T08:09:46.778144+0000","last_became_peered":"2024-03-20T08:09:46.778144+0000","last_unstale":"2024-03-20T08:22:44.918743+0000","last_undegraded":"2024-03-20T08:22:44.918743+0000","last_fullsized":"2024-03-20T08:22:44.918743+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:09:12.885421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1254,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:23:04.740211+0000","last_change":"2024-03-20T08:09:46.774779+0000","last_active":"2024-03-20T08:23:04.740211+0000","last_peered":"2024-03-20T08:23:04.740211+0000","last_clean":"2024-03-20T08:23:04.740211+0000","last_became_active":"2024-03-20T08:09:46.774090+0000","last_became_peered":"2024-03-20T08:09:46.774090+0000","last_unstale":"2024-03-20T08:23:04.740211+0000","last_undegraded":"2024-03-20T08:23:04.740211+0000","last_fullsized":"2024-03-20T08:23:04.740211+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:57:24.473150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"866'65374","reported_seq":68668,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:24.728852+0000","last_change":"2024-03-20T08:15:24.150592+0000","last_active":"2024-03-20T08:21:24.728852+0000","last_peered":"2024-03-20T08:21:24.728852+0000","last_clean":"2024-03-20T08:21:24.728852+0000","last_became_active":"2024-03-20T08:08:21.641586+0000","last_became_peered":"2024-03-20T08:08:21.641586+0000","last_unstale":"2024-03-20T08:21:24.728852+0000","last_undegraded":"2024-03-20T08:21:24.728852+0000","last_fullsized":"2024-03-20T08:21:24.728852+0000","mapping_epoch":470,"log_start":"636'58815","ondisk_log_start":"636'58815","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":216,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:18:31.991863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057541459999999999,"stat_sum":{"num_bytes":163160064,"num_objects":83,"num_object_clones":79,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":318,"num_read_kb":482859,"num_write":747,"num_write_kb":182672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":28057600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"2b"},{"start":"db","length":"5"},{"start":"e1","length":"2"},{"start":"e4","length":"1"},{"start":"e6","length":"2"},{"start":"e9","length":"2"},{"start":"ec","length":"4"},{"start":"f6","length":"1"},{"start":"f9","length":"1"},{"start":"fb","length":"1"}]},{"pgid":"2.9","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302426+0000","last_change":"2024-03-20T08:09:46.772760+0000","last_active":"2024-03-20T08:21:12.302426+0000","last_peered":"2024-03-20T08:21:12.302426+0000","last_clean":"2024-03-20T08:21:12.302426+0000","last_became_active":"2024-03-20T08:09:46.772446+0000","last_became_peered":"2024-03-20T08:09:46.772446+0000","last_unstale":"2024-03-20T08:21:12.302426+0000","last_undegraded":"2024-03-20T08:21:12.302426+0000","last_fullsized":"2024-03-20T08:21:12.302426+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:08:13.490762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779055+0000","last_change":"2024-03-20T08:09:46.771075+0000","last_active":"2024-03-20T08:22:24.779055+0000","last_peered":"2024-03-20T08:22:24.779055+0000","last_clean":"2024-03-20T08:22:24.779055+0000","last_became_active":"2024-03-20T08:09:46.770445+0000","last_became_peered":"2024-03-20T08:09:46.770445+0000","last_unstale":"2024-03-20T08:22:24.779055+0000","last_undegraded":"2024-03-20T08:22:24.779055+0000","last_fullsized":"2024-03-20T08:22:24.779055+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:52:22.513386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"854'27932","reported_seq":31735,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:27.886950+0000","last_change":"2024-03-20T08:15:23.992886+0000","last_active":"2024-03-20T08:22:27.886950+0000","last_peered":"2024-03-20T08:22:27.886950+0000","last_clean":"2024-03-20T08:22:27.886950+0000","last_became_active":"2024-03-20T08:09:46.789956+0000","last_became_peered":"2024-03-20T08:09:46.789956+0000","last_unstale":"2024-03-20T08:22:27.886950+0000","last_undegraded":"2024-03-20T08:22:27.886950+0000","last_fullsized":"2024-03-20T08:22:27.886950+0000","mapping_epoch":551,"log_start":"442'19803","ondisk_log_start":"442'19803","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":8129,"log_dups_size":0,"ondisk_log_size":8129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:43:14.953521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010257334999999999,"stat_sum":{"num_bytes":121421824,"num_objects":61,"num_object_clones":59,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":1,"num_read":170,"num_read_kb":304787,"num_write":382,"num_write_kb":103192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":37183488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.a","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302446+0000","last_change":"2024-03-20T08:09:46.773115+0000","last_active":"2024-03-20T08:21:12.302446+0000","last_peered":"2024-03-20T08:21:12.302446+0000","last_clean":"2024-03-20T08:21:12.302446+0000","last_became_active":"2024-03-20T08:09:46.772777+0000","last_became_peered":"2024-03-20T08:09:46.772777+0000","last_unstale":"2024-03-20T08:21:12.302446+0000","last_undegraded":"2024-03-20T08:21:12.302446+0000","last_fullsized":"2024-03-20T08:21:12.302446+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:36:53.927562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1287,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:59.781844+0000","last_change":"2024-03-20T08:07:56.429001+0000","last_active":"2024-03-20T08:22:59.781844+0000","last_peered":"2024-03-20T08:22:59.781844+0000","last_clean":"2024-03-20T08:22:59.781844+0000","last_became_active":"2024-03-20T08:07:56.427617+0000","last_became_peered":"2024-03-20T08:07:56.427617+0000","last_unstale":"2024-03-20T08:22:59.781844+0000","last_undegraded":"2024-03-20T08:22:59.781844+0000","last_fullsized":"2024-03-20T08:22:59.781844+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:27:45.054465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"844'34144","reported_seq":29912,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:47.305131+0000","last_change":"2024-03-20T08:15:24.002686+0000","last_active":"2024-03-20T08:21:47.305131+0000","last_peered":"2024-03-20T08:21:47.305131+0000","last_clean":"2024-03-20T08:21:47.305131+0000","last_became_active":"2024-03-20T08:07:56.446000+0000","last_became_peered":"2024-03-20T08:07:56.446000+0000","last_unstale":"2024-03-20T08:21:47.305131+0000","last_undegraded":"2024-03-20T08:21:47.305131+0000","last_fullsized":"2024-03-20T08:21:47.305131+0000","mapping_epoch":445,"log_start":"381'27402","ondisk_log_start":"381'27402","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":280,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:24:29.939522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072065059999999997,"stat_sum":{"num_bytes":79020032,"num_objects":58,"num_object_clones":55,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":198,"num_read_kb":202686,"num_write":443,"num_write_kb":77248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":15736832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"a3","length":"2"},{"start":"a9","length":"1"},{"start":"ab","length":"1"},{"start":"af","length":"1"},{"start":"b1","length":"1"},{"start":"b5","length":"1"},{"start":"b8","length":"2"}]},{"pgid":"2.b","version":"583'3","reported_seq":1311,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886931+0000","last_change":"2024-03-20T08:08:21.633117+0000","last_active":"2024-03-20T08:22:27.886931+0000","last_peered":"2024-03-20T08:22:27.886931+0000","last_clean":"2024-03-20T08:22:27.886931+0000","last_became_active":"2024-03-20T08:08:21.632732+0000","last_became_peered":"2024-03-20T08:08:21.632732+0000","last_unstale":"2024-03-20T08:22:27.886931+0000","last_undegraded":"2024-03-20T08:22:27.886931+0000","last_fullsized":"2024-03-20T08:22:27.886931+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:40:42.082953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447869+0000","last_change":"2024-03-20T08:08:59.164527+0000","last_active":"2024-03-20T08:21:57.447869+0000","last_peered":"2024-03-20T08:21:57.447869+0000","last_clean":"2024-03-20T08:21:57.447869+0000","last_became_active":"2024-03-20T08:08:59.163588+0000","last_became_peered":"2024-03-20T08:08:59.163588+0000","last_unstale":"2024-03-20T08:21:57.447869+0000","last_undegraded":"2024-03-20T08:21:57.447869+0000","last_fullsized":"2024-03-20T08:21:57.447869+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:27:51.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":3380,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886897+0000","last_change":"2024-03-20T08:15:22.991071+0000","last_active":"2024-03-20T08:22:27.886897+0000","last_peered":"2024-03-20T08:22:27.886897+0000","last_clean":"2024-03-20T08:22:27.886897+0000","last_became_active":"2024-03-20T08:08:21.633952+0000","last_became_peered":"2024-03-20T08:08:21.633952+0000","last_unstale":"2024-03-20T08:22:27.886897+0000","last_undegraded":"2024-03-20T08:22:27.886897+0000","last_fullsized":"2024-03-20T08:22:27.886897+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:38:24.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011268669999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"ff"},{"start":"1af","length":"16"}]},{"pgid":"2.c","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886942+0000","last_change":"2024-03-20T08:08:21.632669+0000","last_active":"2024-03-20T08:22:27.886942+0000","last_peered":"2024-03-20T08:22:27.886942+0000","last_clean":"2024-03-20T08:22:27.886942+0000","last_became_active":"2024-03-20T08:08:21.632378+0000","last_became_peered":"2024-03-20T08:08:21.632378+0000","last_unstale":"2024-03-20T08:22:27.886942+0000","last_undegraded":"2024-03-20T08:22:27.886942+0000","last_fullsized":"2024-03-20T08:22:27.886942+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:30:34.236560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1119,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906627+0000","last_change":"2024-03-20T08:09:46.788856+0000","last_active":"2024-03-20T08:20:59.906627+0000","last_peered":"2024-03-20T08:20:59.906627+0000","last_clean":"2024-03-20T08:20:59.906627+0000","last_became_active":"2024-03-20T08:09:46.788523+0000","last_became_peered":"2024-03-20T08:09:46.788523+0000","last_unstale":"2024-03-20T08:20:59.906627+0000","last_undegraded":"2024-03-20T08:20:59.906627+0000","last_fullsized":"2024-03-20T08:20:59.906627+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T11:03:03.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.a","version":"860'1761","reported_seq":5701,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:21:52.884206+0000","last_change":"2024-03-20T08:15:23.992471+0000","last_active":"2024-03-20T08:21:52.884206+0000","last_peered":"2024-03-20T08:21:52.884206+0000","last_clean":"2024-03-20T08:21:52.884206+0000","last_became_active":"2024-03-20T08:07:56.440479+0000","last_became_peered":"2024-03-20T08:07:56.440479+0000","last_unstale":"2024-03-20T08:21:52.884206+0000","last_undegraded":"2024-03-20T08:21:52.884206+0000","last_fullsized":"2024-03-20T08:21:52.884206+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1761,"log_dups_size":0,"ondisk_log_size":1761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":260,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:32:01.124288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018160999000000001,"stat_sum":{"num_bytes":226099200,"num_objects":144,"num_object_clones":138,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":1,"num_read":508,"num_read_kb":795587,"num_write":1169,"num_write_kb":225448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":88350720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a9","length":"1"},{"start":"ab","length":"2"},{"start":"af","length":"4"},{"start":"b5","length":"9"},{"start":"c0","length":"1"},{"start":"c4","length":"1"},{"start":"c7","length":"1"},{"start":"cc","length":"1"},{"start":"d0","length":"1"},{"start":"d2","length":"1"},{"start":"d5","length":"1"}]},{"pgid":"2.d","version":"62'1","reported_seq":1348,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:47.305170+0000","last_change":"2024-03-20T08:07:56.433922+0000","last_active":"2024-03-20T08:21:47.305170+0000","last_peered":"2024-03-20T08:21:47.305170+0000","last_clean":"2024-03-20T08:21:47.305170+0000","last_became_active":"2024-03-20T08:07:56.433610+0000","last_became_peered":"2024-03-20T08:07:56.433610+0000","last_unstale":"2024-03-20T08:21:47.305170+0000","last_undegraded":"2024-03-20T08:21:47.305170+0000","last_fullsized":"2024-03-20T08:21:47.305170+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:49:03.233571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1230,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857470+0000","last_change":"2024-03-20T08:08:59.155771+0000","last_active":"2024-03-20T08:21:49.857470+0000","last_peered":"2024-03-20T08:21:49.857470+0000","last_clean":"2024-03-20T08:21:49.857470+0000","last_became_active":"2024-03-20T08:08:59.155324+0000","last_became_peered":"2024-03-20T08:08:59.155324+0000","last_unstale":"2024-03-20T08:21:49.857470+0000","last_undegraded":"2024-03-20T08:21:49.857470+0000","last_fullsized":"2024-03-20T08:21:49.857470+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T08:47:00.395649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"867'1362","reported_seq":4961,"reported_epoch":867,"state":"active+clean+snaptrim_error","last_fresh":"2024-03-20T08:22:44.918770+0000","last_change":"2024-03-20T08:15:24.004853+0000","last_active":"2024-03-20T08:22:44.918770+0000","last_peered":"2024-03-20T08:22:44.918770+0000","last_clean":"2024-03-20T08:22:44.918770+0000","last_became_active":"2024-03-20T08:09:46.775194+0000","last_became_peered":"2024-03-20T08:09:46.775194+0000","last_unstale":"2024-03-20T08:22:44.918770+0000","last_undegraded":"2024-03-20T08:22:44.918770+0000","last_fullsized":"2024-03-20T08:22:44.918770+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T08:01:25.163748+0000","last_clean_scrub_stamp":"2024-03-20T08:01:25.163748+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":177,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:26:51.663351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084378170000000002,"stat_sum":{"num_bytes":102825984,"num_objects":66,"num_object_clones":62,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":323,"num_read_kb":521171,"num_write":650,"num_write_kb":126136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":49676288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"32","length":"1"},{"start":"50","length":"1"},{"start":"65","length":"1"},{"start":"6b","length":"1"},{"start":"83","length":"1"},{"start":"89","length":"3"},{"start":"91","length":"1"},{"start":"95","length":"1"},{"start":"9d","length":"1"},{"start":"9f","length":"1"},{"start":"a2","length":"5"},{"start":"a8","length":"2"},{"start":"ab","length":"2"},{"start":"af","length":"36"},{"start":"e6","length":"e"},{"start":"f5","length":"11"},{"start":"107","length":"3"},{"start":"10b","length":"2"},{"start":"110","length":"1"},{"start":"112","length":"2"},{"start":"115","length":"1"},{"start":"118","length":"1"},{"start":"11b","length":"2"},{"start":"120","length":"2"},{"start":"123","length":"1"}]},{"pgid":"2.e","version":"62'2","reported_seq":1309,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.886979+0000","last_change":"2024-03-20T08:09:46.777955+0000","last_active":"2024-03-20T08:22:27.886979+0000","last_peered":"2024-03-20T08:22:27.886979+0000","last_clean":"2024-03-20T08:22:27.886979+0000","last_became_active":"2024-03-20T08:09:46.777002+0000","last_became_peered":"2024-03-20T08:09:46.777002+0000","last_unstale":"2024-03-20T08:22:27.886979+0000","last_undegraded":"2024-03-20T08:22:27.886979+0000","last_fullsized":"2024-03-20T08:22:27.886979+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:24.742760+0000","last_clean_scrub_stamp":"2024-03-20T07:58:24.742760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:21:40.146875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1142,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:20:59.906714+0000","last_change":"2024-03-20T08:09:46.782379+0000","last_active":"2024-03-20T08:20:59.906714+0000","last_peered":"2024-03-20T08:20:59.906714+0000","last_clean":"2024-03-20T08:20:59.906714+0000","last_became_active":"2024-03-20T08:09:46.782093+0000","last_became_peered":"2024-03-20T08:09:46.782093+0000","last_unstale":"2024-03-20T08:20:59.906714+0000","last_undegraded":"2024-03-20T08:20:59.906714+0000","last_fullsized":"2024-03-20T08:20:59.906714+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T14:40:21.212952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1188,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:27.444432+0000","last_change":"2024-03-20T08:08:59.149751+0000","last_active":"2024-03-20T08:21:27.444432+0000","last_peered":"2024-03-20T08:21:27.444432+0000","last_clean":"2024-03-20T08:21:27.444432+0000","last_became_active":"2024-03-20T08:08:59.147650+0000","last_became_peered":"2024-03-20T08:08:59.147650+0000","last_unstale":"2024-03-20T08:21:27.444432+0000","last_undegraded":"2024-03-20T08:21:27.444432+0000","last_fullsized":"2024-03-20T08:21:27.444432+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T09:41:19.495012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1304,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:17.881444+0000","last_change":"2024-03-20T08:08:59.160483+0000","last_active":"2024-03-20T08:21:17.881444+0000","last_peered":"2024-03-20T08:21:17.881444+0000","last_clean":"2024-03-20T08:21:17.881444+0000","last_became_active":"2024-03-20T08:08:59.160132+0000","last_became_peered":"2024-03-20T08:08:59.160132+0000","last_unstale":"2024-03-20T08:21:17.881444+0000","last_undegraded":"2024-03-20T08:21:17.881444+0000","last_fullsized":"2024-03-20T08:21:17.881444+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:12:25.203789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1237,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:54.865149+0000","last_change":"2024-03-20T08:08:21.626562+0000","last_active":"2024-03-20T08:22:54.865149+0000","last_peered":"2024-03-20T08:22:54.865149+0000","last_clean":"2024-03-20T08:22:54.865149+0000","last_became_active":"2024-03-20T08:08:21.622281+0000","last_became_peered":"2024-03-20T08:08:21.622281+0000","last_unstale":"2024-03-20T08:22:54.865149+0000","last_undegraded":"2024-03-20T08:22:54.865149+0000","last_fullsized":"2024-03-20T08:22:54.865149+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T10:31:16.167453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1388,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:05.994252+0000","last_change":"2024-03-20T08:07:56.425923+0000","last_active":"2024-03-20T08:22:05.994252+0000","last_peered":"2024-03-20T08:22:05.994252+0000","last_clean":"2024-03-20T08:22:05.994252+0000","last_became_active":"2024-03-20T08:07:56.425106+0000","last_became_peered":"2024-03-20T08:07:56.425106+0000","last_unstale":"2024-03-20T08:22:05.994252+0000","last_undegraded":"2024-03-20T08:22:05.994252+0000","last_fullsized":"2024-03-20T08:22:05.994252+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T12:32:33.305957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1295,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779085+0000","last_change":"2024-03-20T08:07:16.804259+0000","last_active":"2024-03-20T08:22:24.779085+0000","last_peered":"2024-03-20T08:22:24.779085+0000","last_clean":"2024-03-20T08:22:24.779085+0000","last_became_active":"2024-03-20T08:07:16.803984+0000","last_became_peered":"2024-03-20T08:07:16.803984+0000","last_unstale":"2024-03-20T08:22:24.779085+0000","last_undegraded":"2024-03-20T08:22:24.779085+0000","last_fullsized":"2024-03-20T08:22:24.779085+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T17:13:10.834050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1345,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:12.302466+0000","last_change":"2024-03-20T08:07:16.813405+0000","last_active":"2024-03-20T08:21:12.302466+0000","last_peered":"2024-03-20T08:21:12.302466+0000","last_clean":"2024-03-20T08:21:12.302466+0000","last_became_active":"2024-03-20T08:07:16.813046+0000","last_became_peered":"2024-03-20T08:07:16.813046+0000","last_unstale":"2024-03-20T08:21:12.302466+0000","last_undegraded":"2024-03-20T08:21:12.302466+0000","last_fullsized":"2024-03-20T08:21:12.302466+0000","mapping_epoch":408,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":409,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:35:54.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1199,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:57.447930+0000","last_change":"2024-03-20T08:09:46.772629+0000","last_active":"2024-03-20T08:21:57.447930+0000","last_peered":"2024-03-20T08:21:57.447930+0000","last_clean":"2024-03-20T08:21:57.447930+0000","last_became_active":"2024-03-20T08:09:46.772326+0000","last_became_peered":"2024-03-20T08:09:46.772326+0000","last_unstale":"2024-03-20T08:21:57.447930+0000","last_undegraded":"2024-03-20T08:21:57.447930+0000","last_fullsized":"2024-03-20T08:21:57.447930+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:17:04.598178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1280,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:24.779104+0000","last_change":"2024-03-20T08:07:56.428385+0000","last_active":"2024-03-20T08:22:24.779104+0000","last_peered":"2024-03-20T08:22:24.779104+0000","last_clean":"2024-03-20T08:22:24.779104+0000","last_became_active":"2024-03-20T08:07:56.428117+0000","last_became_peered":"2024-03-20T08:07:56.428117+0000","last_unstale":"2024-03-20T08:22:24.779104+0000","last_undegraded":"2024-03-20T08:22:24.779104+0000","last_fullsized":"2024-03-20T08:22:24.779104+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T19:12:52.292385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1236,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:19.854030+0000","last_change":"2024-03-20T08:08:21.630964+0000","last_active":"2024-03-20T08:21:19.854030+0000","last_peered":"2024-03-20T08:21:19.854030+0000","last_clean":"2024-03-20T08:21:19.854030+0000","last_became_active":"2024-03-20T08:08:21.630665+0000","last_became_peered":"2024-03-20T08:08:21.630665+0000","last_unstale":"2024-03-20T08:21:19.854030+0000","last_undegraded":"2024-03-20T08:21:19.854030+0000","last_fullsized":"2024-03-20T08:21:19.854030+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T16:08:24.240042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1282,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:59.781802+0000","last_change":"2024-03-20T08:07:56.428006+0000","last_active":"2024-03-20T08:22:59.781802+0000","last_peered":"2024-03-20T08:22:59.781802+0000","last_clean":"2024-03-20T08:22:59.781802+0000","last_became_active":"2024-03-20T08:07:56.427700+0000","last_became_peered":"2024-03-20T08:07:56.427700+0000","last_unstale":"2024-03-20T08:22:59.781802+0000","last_undegraded":"2024-03-20T08:22:59.781802+0000","last_fullsized":"2024-03-20T08:22:59.781802+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:01:16.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:49.857436+0000","last_change":"2024-03-20T08:09:46.794084+0000","last_active":"2024-03-20T08:21:49.857436+0000","last_peered":"2024-03-20T08:21:49.857436+0000","last_clean":"2024-03-20T08:21:49.857436+0000","last_became_active":"2024-03-20T08:09:46.793222+0000","last_became_peered":"2024-03-20T08:09:46.793222+0000","last_unstale":"2024-03-20T08:21:49.857436+0000","last_undegraded":"2024-03-20T08:21:49.857436+0000","last_fullsized":"2024-03-20T08:21:49.857436+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T13:31:10.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1386,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:21:10.989845+0000","last_change":"2024-03-20T08:09:46.778733+0000","last_active":"2024-03-20T08:21:10.989845+0000","last_peered":"2024-03-20T08:21:10.989845+0000","last_clean":"2024-03-20T08:21:10.989845+0000","last_became_active":"2024-03-20T08:09:46.775483+0000","last_became_peered":"2024-03-20T08:09:46.775483+0000","last_unstale":"2024-03-20T08:21:10.989845+0000","last_undegraded":"2024-03-20T08:21:10.989845+0000","last_fullsized":"2024-03-20T08:21:10.989845+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T15:37:16.360271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1205,"reported_epoch":867,"state":"active+clean","last_fresh":"2024-03-20T08:22:27.451137+0000","last_change":"2024-03-20T08:08:59.164852+0000","last_active":"2024-03-20T08:22:27.451137+0000","last_peered":"2024-03-20T08:22:27.451137+0000","last_clean":"2024-03-20T08:22:27.451137+0000","last_became_active":"2024-03-20T08:08:59.161836+0000","last_became_peered":"2024-03-20T08:08:59.161836+0000","last_unstale":"2024-03-20T08:22:27.451137+0000","last_undegraded":"2024-03-20T08:22:27.451137+0000","last_fullsized":"2024-03-20T08:22:27.451137+0000","mapping_epoch":505,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":506,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-20T07:58:25.744971+0000","last_clean_scrub_stamp":"2024-03-20T07:58:25.744971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-21T18:11:18.194577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":1867866112,"num_objects":1060,"num_object_clones":1010,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1060,"num_whiteouts":8,"num_read":3819,"num_read_kb":5702363,"num_write":8523,"num_write_kb":1967784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":491290624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2489618432,"data_stored":6064238592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64734,"ondisk_log_size":64734,"up":48,"acting":48,"num_store_stats":8},{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":812,"num_read_kb":1310,"num_write":311,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6620,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":79,"num_read_kb":87,"num_write":68,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":19860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41415,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":551,"seq":2366526980284,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":215060,"kb_used_data":155592,"kb_used_omap":38,"kb_used_meta":59417,"kb_avail":93530092,"statfs":{"total":95995035648,"available":95774814208,"internally_reserved":0,"allocated":159326208,"data_stored":423321712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39013,"internal_metadata":60843931},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":505,"seq":2168958484678,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1073720,"kb_used_data":202084,"kb_used_omap":75,"kb_used_meta":871540,"kb_avail":92671432,"statfs":{"total":95995035648,"available":94895546368,"internally_reserved":0,"allocated":206934016,"data_stored":527357138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77644,"internal_metadata":892457140},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":336,"seq":1443109011689,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1823648,"kb_used_data":387540,"kb_used_omap":50,"kb_used_meta":1436045,"kb_avail":91921504,"statfs":{"total":95995035648,"available":94127620096,"internally_reserved":0,"allocated":396840960,"data_stored":923847828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51740,"internal_metadata":1470510564},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":300,"seq":1288490189040,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2050824,"kb_used_data":373244,"kb_used_omap":147,"kb_used_meta":1677420,"kb_avail":91694328,"statfs":{"total":95995035648,"available":93894991872,"internally_reserved":0,"allocated":382201856,"data_stored":865915116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":150848,"internal_metadata":1717678784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":376,"seq":1614907703520,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1479524,"kb_used_data":261588,"kb_used_omap":50,"kb_used_meta":1217869,"kb_avail":92265628,"statfs":{"total":95995035648,"available":94480003072,"internally_reserved":0,"allocated":267866112,"data_stored":695141628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52002,"internal_metadata":1247098078},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":408,"seq":1752346656986,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":394476,"kb_used_data":273816,"kb_used_omap":42,"kb_used_meta":120597,"kb_avail":93350676,"statfs":{"total":95995035648,"available":95591092224,"internally_reserved":0,"allocated":280387584,"data_stored":663343898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43019,"internal_metadata":123492341},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":445,"seq":1911260446930,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1249044,"kb_used_data":523008,"kb_used_omap":85,"kb_used_meta":725930,"kb_avail":92496108,"statfs":{"total":95995035648,"available":94716014592,"internally_reserved":0,"allocated":535560192,"data_stored":1200582328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":87645,"internal_metadata":743352739},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":470,"seq":2018634629325,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":875392,"kb_used_data":334256,"kb_used_omap":115,"kb_used_meta":541004,"kb_avail":92869760,"statfs":{"total":95995035648,"available":95098634240,"internally_reserved":0,"allocated":342278144,"data_stored":813204236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":118165,"internal_metadata":553988715},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5467,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4222,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4801,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":5926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5379,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5735,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":5912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4942,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6233,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4636,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":371605504,"data_stored":859488256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":386846720,"data_stored":918011904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257884160,"data_stored":689307648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":657502208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":525557760,"data_stored":1194745856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":332271616,"data_stored":807362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196337664,"data_stored":520925184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":416894976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.8 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.3 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.5 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.6 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.4 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.7 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.2 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.1 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.0 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.871 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.f in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.872 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.e in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.872 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.d in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.872 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.c in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.872 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.a in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.872 INFO:tasks.cephadm.ceph_manager.ceph:pg 5.9 in trimming, state: active+clean+snaptrim_error 2024-03-20T08:25:24.872 INFO:tasks.cephadm.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-03-20T08:25:25.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:25 smithi196 ceph-mon[87080]: from='client.34803 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:25:25.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:25 smithi118 ceph-mon[93121]: from='client.34803 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:25:25.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:25 smithi118 ceph-mon[95692]: from='client.34803 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-20T08:25:26.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:26 smithi196 ceph-mon[87080]: pgmap v1292: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:26.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:26 smithi118 ceph-mon[93121]: pgmap v1292: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:26.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:26 smithi118 ceph-mon[95692]: pgmap v1292: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:25:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:27.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:27 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:27.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:25:27.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:27.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:27 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-20T08:25:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:27.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:27 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:28 smithi196 ceph-mon[87080]: pgmap v1293: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:25:28.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:25:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:28.640 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:28 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:25:28.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[93121]: pgmap v1293: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:28.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:25:28.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:25:28.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:28.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:25:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[95692]: pgmap v1293: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-20T08:25:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-20T08:25:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' 2024-03-20T08:25:28.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:28 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-20T08:25:30.639 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:30 smithi196 ceph-mon[87080]: pgmap v1294: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:30.672 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:30 smithi118 ceph-mon[93121]: pgmap v1294: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:30.673 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:30 smithi118 ceph-mon[95692]: pgmap v1294: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:31.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:31 smithi196 ceph-mon[87080]: pgmap v1295: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:31.922 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:31 smithi118 ceph-mon[93121]: pgmap v1295: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:31.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:31 smithi118 ceph-mon[95692]: pgmap v1295: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:32.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:25:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:25:32.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:25:33.923 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:33 smithi118 ceph-mon[93121]: pgmap v1296: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:33.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:33 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:25:33] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:25:33.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:33 smithi118 ceph-mon[95692]: pgmap v1296: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:34.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:33 smithi196 ceph-mon[87080]: pgmap v1296: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:34.873 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks/rados.py", line 290, in thread manager.wait_snap_trimming_complete(pool); File "/home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks/ceph_manager.py", line 2506, in wait_snap_trimming_complete assert (now - start) < FATAL_TIMEOUT, \ AssertionError: failed to complete snap trimming before timeout 2024-03-20T08:25:34.876 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/parallel.py", line 56, in task p.spawn(_run_spawned, ctx, confg, taskname) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/parallel.py", line 64, in _run_spawned mgr = run_tasks.run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/full_sequential.py", line 37, in task mgr.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks/rados.py", line 299, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks/rados.py", line 290, in thread manager.wait_snap_trimming_complete(pool); File "/home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks/ceph_manager.py", line 2506, in wait_snap_trimming_complete assert (now - start) < FATAL_TIMEOUT, \ AssertionError: failed to complete snap trimming before timeout 2024-03-20T08:25:35.189 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=50ae0d27dc824d589578cc8e5ba42604 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/parallel.py", line 56, in task p.spawn(_run_spawned, ctx, confg, taskname) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/parallel.py", line 64, in _run_spawned mgr = run_tasks.run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/full_sequential.py", line 37, in task mgr.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks/rados.py", line 299, in task running.get() File "src/gevent/greenlet.py", line 831, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 373, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/lib/python3.8/site-packages/gevent/_compat.py", line 49, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks/rados.py", line 290, in thread manager.wait_snap_trimming_complete(pool); File "/home/teuthworker/src/github.com_ceph_ceph-c_e7e80a6ac9849f0f23432d0d9832f096730308ae/qa/tasks/ceph_manager.py", line 2506, in wait_snap_trimming_complete assert (now - start) < FATAL_TIMEOUT, \ AssertionError: failed to complete snap trimming before timeout 2024-03-20T08:25:35.192 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-20T08:25:35.204 INFO:tasks.cephadm:Teardown begin 2024-03-20T08:25:35.204 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-20T08:25:35.205 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T08:25:35.543 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:25:35.543 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/320115093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":320115093},{"type":"v1","addr":"172.21.15.196:6833","nonce":320115093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-20T07:58:26.102802+0000","modified":"2024-03-20T08:10:20.035094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":583,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":44330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_44330":{"gid":44330,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6835/2279894254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2279894254},{"type":"v1","addr":"172.21.15.118:6835","nonce":2279894254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":44330,"qdb_cluster":[44330]},"id":1}]} 2024-03-20T08:25:35.544 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 14 2024-03-20T08:25:35.556 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-20T08:25:35.885 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:25:35.885 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/320115093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":320115093},{"type":"v1","addr":"172.21.15.196:6833","nonce":320115093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-20T07:58:26.102802+0000","modified":"2024-03-20T08:10:20.035094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":583,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":44330},"failed":[],"damaged":[],"stopped":[],"info":{"gid_44330":{"gid":44330,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":8,"addr":"172.21.15.118:6835/2279894254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6834","nonce":2279894254},{"type":"v1","addr":"172.21.15.118:6835","nonce":2279894254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":44330,"qdb_cluster":[44330]},"id":1}]} 2024-03-20T08:25:35.885 INFO:teuthology.orchestra.run.smithi118.stderr:dumped fsmap epoch 14 2024-03-20T08:25:35.900 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-20T08:25:36.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:35 smithi196 ceph-mon[87080]: pgmap v1297: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:36.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:35 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/3155439211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:25:36.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:35 smithi118 ceph-mon[93121]: pgmap v1297: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:36.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:35 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3155439211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:25:36.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:35 smithi118 ceph-mon[95692]: pgmap v1297: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:36.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:35 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/3155439211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:25:36.196 INFO:teuthology.orchestra.run.smithi118.stdout: 2024-03-20T08:25:36.196 INFO:teuthology.orchestra.run.smithi118.stdout:{"epoch":867,"fsid":"bc2566e8-e68e-11ee-95c9-87774f69a715","created":"2024-03-20T07:52:44.130170+0000","modified":"2024-03-20T08:15:23.987668+0000","last_up_change":"2024-03-20T08:09:45.750191+0000","last_in_change":"2024-03-20T07:57:49.141597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-20T07:56:03.504701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-20T07:58:24.105659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-20T07:58:25.289141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"rbd","create_time":"2024-03-20T07:59:00.573805+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"unique_pool_0","create_time":"2024-03-20T08:01:24.424653+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"867","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":455,"snap_epoch":867,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"teuthologyprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"88c23409-c8bb-4018-9071-b8a8f1d2eed8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":289,"up_from":300,"up_thru":551,"down_at":290,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6802","nonce":3093978881},{"type":"v1","addr":"172.21.15.118:6803","nonce":3093978881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6804","nonce":3093978881},{"type":"v1","addr":"172.21.15.118:6805","nonce":3093978881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6808","nonce":3093978881},{"type":"v1","addr":"172.21.15.118:6809","nonce":3093978881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6806","nonce":3093978881},{"type":"v1","addr":"172.21.15.118:6807","nonce":3093978881}]},"public_addr":"172.21.15.118:6803/3093978881","cluster_addr":"172.21.15.118:6805/3093978881","heartbeat_back_addr":"172.21.15.118:6809/3093978881","heartbeat_front_addr":"172.21.15.118:6807/3093978881","state":["exists","up"]},{"osd":1,"uuid":"3792a048-252f-44c3-ae87-fa7445418bc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":325,"up_from":336,"up_thru":551,"down_at":326,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6810","nonce":406990489},{"type":"v1","addr":"172.21.15.118:6811","nonce":406990489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6812","nonce":406990489},{"type":"v1","addr":"172.21.15.118:6813","nonce":406990489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6816","nonce":406990489},{"type":"v1","addr":"172.21.15.118:6817","nonce":406990489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6814","nonce":406990489},{"type":"v1","addr":"172.21.15.118:6815","nonce":406990489}]},"public_addr":"172.21.15.118:6811/406990489","cluster_addr":"172.21.15.118:6813/406990489","heartbeat_back_addr":"172.21.15.118:6817/406990489","heartbeat_front_addr":"172.21.15.118:6815/406990489","state":["exists","up"]},{"osd":2,"uuid":"54cacc96-d4a4-4c6a-b695-0edeb528c222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":363,"up_from":376,"up_thru":551,"down_at":364,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6818","nonce":3195797555},{"type":"v1","addr":"172.21.15.118:6819","nonce":3195797555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6820","nonce":3195797555},{"type":"v1","addr":"172.21.15.118:6821","nonce":3195797555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6824","nonce":3195797555},{"type":"v1","addr":"172.21.15.118:6825","nonce":3195797555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6822","nonce":3195797555},{"type":"v1","addr":"172.21.15.118:6823","nonce":3195797555}]},"public_addr":"172.21.15.118:6819/3195797555","cluster_addr":"172.21.15.118:6821/3195797555","heartbeat_back_addr":"172.21.15.118:6825/3195797555","heartbeat_front_addr":"172.21.15.118:6823/3195797555","state":["exists","up"]},{"osd":3,"uuid":"3b5ff3b5-b736-4c14-ac49-be59960e3a5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":396,"up_from":408,"up_thru":551,"down_at":397,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6826","nonce":2186452731},{"type":"v1","addr":"172.21.15.118:6827","nonce":2186452731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6828","nonce":2186452731},{"type":"v1","addr":"172.21.15.118:6829","nonce":2186452731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6832","nonce":2186452731},{"type":"v1","addr":"172.21.15.118:6833","nonce":2186452731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.118:6830","nonce":2186452731},{"type":"v1","addr":"172.21.15.118:6831","nonce":2186452731}]},"public_addr":"172.21.15.118:6827/2186452731","cluster_addr":"172.21.15.118:6829/2186452731","heartbeat_back_addr":"172.21.15.118:6833/2186452731","heartbeat_front_addr":"172.21.15.118:6831/2186452731","state":["exists","up"]},{"osd":4,"uuid":"7e6db394-b7e5-48c7-9aae-dd1b9469552e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":31,"last_clean_end":434,"up_from":445,"up_thru":551,"down_at":435,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":502714599},{"type":"v1","addr":"172.21.15.196:6801","nonce":502714599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":502714599},{"type":"v1","addr":"172.21.15.196:6803","nonce":502714599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":502714599},{"type":"v1","addr":"172.21.15.196:6807","nonce":502714599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":502714599},{"type":"v1","addr":"172.21.15.196:6805","nonce":502714599}]},"public_addr":"172.21.15.196:6801/502714599","cluster_addr":"172.21.15.196:6803/502714599","heartbeat_back_addr":"172.21.15.196:6807/502714599","heartbeat_front_addr":"172.21.15.196:6805/502714599","state":["exists","up"]},{"osd":5,"uuid":"ab8d481e-0604-4613-b8fa-d138564be31d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":456,"up_from":470,"up_thru":551,"down_at":457,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":4232618693},{"type":"v1","addr":"172.21.15.196:6809","nonce":4232618693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4232618693},{"type":"v1","addr":"172.21.15.196:6811","nonce":4232618693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4232618693},{"type":"v1","addr":"172.21.15.196:6815","nonce":4232618693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4232618693},{"type":"v1","addr":"172.21.15.196:6813","nonce":4232618693}]},"public_addr":"172.21.15.196:6809/4232618693","cluster_addr":"172.21.15.196:6811/4232618693","heartbeat_back_addr":"172.21.15.196:6815/4232618693","heartbeat_front_addr":"172.21.15.196:6813/4232618693","state":["exists","up"]},{"osd":6,"uuid":"69651e4b-3242-4177-9672-30e0979d33a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":47,"last_clean_end":492,"up_from":505,"up_thru":551,"down_at":493,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":812001496},{"type":"v1","addr":"172.21.15.196:6817","nonce":812001496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":812001496},{"type":"v1","addr":"172.21.15.196:6819","nonce":812001496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":812001496},{"type":"v1","addr":"172.21.15.196:6823","nonce":812001496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":812001496},{"type":"v1","addr":"172.21.15.196:6821","nonce":812001496}]},"public_addr":"172.21.15.196:6817/812001496","cluster_addr":"172.21.15.196:6819/812001496","heartbeat_back_addr":"172.21.15.196:6823/812001496","heartbeat_front_addr":"172.21.15.196:6821/812001496","state":["exists","up"]},{"osd":7,"uuid":"279db45c-13c3-4aaa-b9e2-f65751ae6beb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":55,"last_clean_end":534,"up_from":551,"up_thru":551,"down_at":535,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":4230059391},{"type":"v1","addr":"172.21.15.196:6825","nonce":4230059391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":4230059391},{"type":"v1","addr":"172.21.15.196:6827","nonce":4230059391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":4230059391},{"type":"v1","addr":"172.21.15.196:6831","nonce":4230059391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":4230059391},{"type":"v1","addr":"172.21.15.196:6829","nonce":4230059391}]},"public_addr":"172.21.15.196:6825/4230059391","cluster_addr":"172.21.15.196:6827/4230059391","heartbeat_back_addr":"172.21.15.196:6831/4230059391","heartbeat_front_addr":"172.21.15.196:6829/4230059391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:09.423574+0000","dead_epoch":289},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:55:34.656826+0000","dead_epoch":325},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:00.304111+0000","dead_epoch":363},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:26.604557+0000","dead_epoch":396},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:56:50.126630+0000","dead_epoch":434},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:14.694409+0000","dead_epoch":456},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:57:39.380324+0000","dead_epoch":492},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-20T07:58:04.973862+0000","dead_epoch":534}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":7,"to":2}]},{"pgid":"3.6","mappings":[{"from":0,"to":2}]},{"pgid":"3.1d","mappings":[{"from":4,"to":1},{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":0,"to":1},{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6832/5972345":"2024-03-21T08:10:17.017317+0000","172.21.15.118:6801/1685672110":"2024-03-21T08:03:58.165635+0000","172.21.15.118:6800/4166913641":"2024-03-21T08:02:37.593461+0000","172.21.15.118:0/1828201096":"2024-03-21T08:02:37.593461+0000","172.21.15.118:0/4210199143":"2024-03-21T07:53:20.171783+0000","172.21.15.118:6801/4166913641":"2024-03-21T08:02:37.593461+0000","172.21.15.196:6835/1340665682":"2024-03-21T08:03:13.331084+0000","172.21.15.118:0/3938132886":"2024-03-21T07:58:59.897028+0000","172.21.15.118:6800/1685672110":"2024-03-21T08:03:58.165635+0000","172.21.15.196:0/2412915018":"2024-03-21T08:03:13.331084+0000","172.21.15.196:6833/5972345":"2024-03-21T08:10:17.017317+0000","172.21.15.118:0/2426914918":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/3470993218":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2517893481":"2024-03-21T08:02:37.593461+0000","172.21.15.196:6834/1340665682":"2024-03-21T08:03:13.331084+0000","172.21.15.118:0/1838641107":"2024-03-21T08:03:58.165635+0000","172.21.15.118:6801/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/2649753508":"2024-03-21T07:58:59.897028+0000","172.21.15.196:0/3478301470":"2024-03-21T08:03:13.331084+0000","172.21.15.118:0/1149999385":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/237356808":"2024-03-21T08:02:37.593461+0000","172.21.15.118:0/2725213234":"2024-03-21T07:53:20.171783+0000","172.21.15.118:0/1237762586":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2856673731":"2024-03-21T08:03:58.165635+0000","172.21.15.118:0/2551714175":"2024-03-21T08:02:37.593461+0000","172.21.15.118:6800/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.196:0/3067939912":"2024-03-21T08:03:13.331084+0000","172.21.15.118:6801/1945765999":"2024-03-21T07:53:20.171783+0000","172.21.15.196:0/2774265323":"2024-03-21T08:03:13.331084+0000","172.21.15.118:6800/3775396232":"2024-03-21T07:53:45.184828+0000","172.21.15.118:6801/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/2309848212":"2024-03-21T08:02:37.593461+0000","172.21.15.118:0/184826175":"2024-03-21T07:58:59.897028+0000","172.21.15.196:0/4207256264":"2024-03-21T08:03:13.331084+0000","172.21.15.118:6800/815627917":"2024-03-21T07:58:59.897028+0000","172.21.15.118:0/495823095":"2024-03-21T07:53:45.184828+0000","172.21.15.118:0/1635512832":"2024-03-21T08:03:58.165635+0000","172.21.15.118:0/462969649":"2024-03-21T08:03:58.165635+0000","172.21.15.196:0/3872526590":"2024-03-21T08:03:13.331084+0000","172.21.15.118:0/2179849791":"2024-03-21T08:03:58.165635+0000","172.21.15.118:0/3072694613":"2024-03-21T08:03:58.165635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"teuthologyprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van","w":"8"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":50,"length":1},{"begin":80,"length":1},{"begin":101,"length":1},{"begin":107,"length":1},{"begin":131,"length":1},{"begin":137,"length":3},{"begin":145,"length":1},{"begin":149,"length":1},{"begin":157,"length":1},{"begin":159,"length":1},{"begin":162,"length":5},{"begin":168,"length":2},{"begin":171,"length":2},{"begin":175,"length":255},{"begin":431,"length":22}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-20T08:25:36.209 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-20T08:25:36.543 INFO:teuthology.orchestra.run.smithi118.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-20T08:25:36.560 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-20T08:25:36.839 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:25:36.542+0000 7f75111b8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-20T08:25:36.840 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:25:36.549+0000 7f59e6630700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:25:36.840 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:25:36.549+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:36.840 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:25:36.546+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:36.841 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[121438]: -5247> 2024-03-20T08:10:20.869+0000 7febac2d9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-20T08:25:36.841 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[121438]: ignoring --setuser ceph since I am not root 2024-03-20T08:25:36.841 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[121438]: ignoring --setgroup ceph since I am not root 2024-03-20T08:25:36.841 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[121438]: starting mds.a at 2024-03-20T08:25:36.842 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:25:36.567+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:25:36.842 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:25:36.550+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:36.842 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:25:36.550+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:36.843 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:25:36.548+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:25:36.843 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:25:36.548+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:36.843 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:25:36.548+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:37.139 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/734159250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:25:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/2697469062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:25:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/3706629467' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-20T08:25:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-mon[87080]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-20T08:25:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-mon[87080]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-20T08:25:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/3706629467' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-20T08:25:37.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-mon[87080]: osdmap e868: 8 total, 8 up, 8 in 2024-03-20T08:25:37.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:36 smithi196 ceph-mon[87080]: fsmap cephfs:0/1 1 up:standby, 1 failed 2024-03-20T08:25:37.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/734159250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:25:37.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/2697469062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:25:37.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/3706629467' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-20T08:25:37.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[95692]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-20T08:25:37.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[95692]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-20T08:25:37.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/3706629467' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-20T08:25:37.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[95692]: osdmap e868: 8 total, 8 up, 8 in 2024-03-20T08:25:37.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[95692]: fsmap cephfs:0/1 1 up:standby, 1 failed 2024-03-20T08:25:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/734159250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-20T08:25:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/2697469062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-20T08:25:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3706629467' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-20T08:25:37.174 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[93121]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-20T08:25:37.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[93121]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-20T08:25:37.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3706629467' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-20T08:25:37.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[93121]: osdmap e868: 8 total, 8 up, 8 in 2024-03-20T08:25:37.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:36 smithi118 ceph-mon[93121]: fsmap cephfs:0/1 1 up:standby, 1 failed 2024-03-20T08:25:37.563 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-20T08:25:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:38.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: pgmap v1299: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/745687935' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/745687935' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: mds.? [v2:172.21.15.118:6834/3795116364,v1:172.21.15.118:6835/3795116364] up:boot 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: fsmap 2 up:standby 2024-03-20T08:25:38.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:37 smithi196 ceph-mon[87080]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:25:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:38.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:38.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:38.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:25:38.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: pgmap v1299: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:38.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/745687935' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:25:38.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-20T08:25:38.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-20T08:25:38.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/745687935' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-20T08:25:38.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: mds.? [v2:172.21.15.118:6834/3795116364,v1:172.21.15.118:6835/3795116364] up:boot 2024-03-20T08:25:38.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: fsmap 2 up:standby 2024-03-20T08:25:38.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[95692]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:25:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:38.175 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:38.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:38.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:25:38.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: pgmap v1299: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:38.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/745687935' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-20T08:25:38.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-20T08:25:38.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-20T08:25:38.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/745687935' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-20T08:25:38.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: mds.? [v2:172.21.15.118:6834/3795116364,v1:172.21.15.118:6835/3795116364] up:boot 2024-03-20T08:25:38.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: fsmap 2 up:standby 2024-03-20T08:25:38.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:37 smithi118 ceph-mon[93121]: from='mgr.34107 172.21.15.118:0/1717727579' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-20T08:25:38.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:38.555+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:25:38.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:38.555+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:38.842 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:38.555+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:38.849 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_metadata' removed 2024-03-20T08:25:38.866 DEBUG:teuthology.orchestra.run.smithi118:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-20T08:25:39.139 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:39.021+0000 7fa5d1835700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:25:39.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:39.021+0000 7fa5d1835700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:39.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:39.022+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:39.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:25:38.851+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:39.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:25:38.851+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:39.141 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:25:39.021+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:25:39.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:25:38.849+0000 7ffb40715700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:39.141 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:25:38.850+0000 7ffb41f18700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/664945844' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:39.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:38 smithi196 ceph-mon[87080]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:39.172 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:38 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/664945844' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:39.173 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:38 smithi118 ceph-mon[93121]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:39.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:38 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/664945844' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:39.173 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:38 smithi118 ceph-mon[95692]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:39.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:25:39.024+0000 7f59e6630700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:39.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:25:39.024+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:25:39.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:25:39.024+0000 7f0cee45c700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 32 after trimming was started 2024-03-20T08:25:39.174 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:25:39.024+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:25:39.175 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:25:38 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:25:38.850+0000 7fc1f5631700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:39.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:39.022+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:25:39.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:39.028+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:25:39.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:39.028+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:25:39.175 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:39.029+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:25:39.851 INFO:teuthology.orchestra.run.smithi118.stderr:pool 'cephfs_data' removed 2024-03-20T08:25:39.876 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-20T08:25:39.905 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-20T08:25:39.939 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-20T08:25:39.939 DEBUG:teuthology.orchestra.run.smithi118:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-20T08:25:39.961 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-20T08:25:39.997 INFO:tasks.cephadm:Stopping all daemons... 2024-03-20T08:25:39.997 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-20T08:25:39.998 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a 2024-03-20T08:25:40.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:25:39.852+0000 7fafa6d8c700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1aa=[]:{fe=[fd,f4,f2,f1,eb],108=[108,106,105],113=[113,112,111,110,10f,10e,10d,10a],122=[122,121,120,11f],14a=[14a,149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b],156=[156,155],158=[158,157],163=[163,162,161,15f,15e,15d,15b,15a],16e=[16e,16d,16c,16b,16a,168,166,165,164],170=[170,16f],177=[177,176,175,174,173,172,171],185=[185,184,183],18c=[18c],18f=[18f,18e],1aa=[1aa,1a9,1a8,1a5,1a4,1a1,19e,19c,19a]} for object 5:44a43f74:::smithi11881747-38:ea 2024-03-20T08:25:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: pgmap v1300: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-20T08:25:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: osdmap e869: 8 total, 8 up, 8 in 2024-03-20T08:25:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:40.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:25:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:25:40.141 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: from='client.? 172.21.15.118:0/3131966524' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:40.142 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:39 smithi196 ceph-mon[87080]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:40.173 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:25:39.850+0000 7fc1f9639700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:40.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:25:39.855+0000 7f59ea638700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:25:40.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:25:39.855+0000 7f59ea638700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: pgmap v1300: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-20T08:25:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: osdmap e869: 8 total, 8 up, 8 in 2024-03-20T08:25:40.174 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:25:40.175 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:25:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: from='client.? 172.21.15.118:0/3131966524' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:40.176 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[95692]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:40.176 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: pgmap v1300: 121 pgs: 15 active+clean+snaptrim_error, 106 active+clean; 1.7 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail 2024-03-20T08:25:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-20T08:25:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: osdmap e869: 8 total, 8 up, 8 in 2024-03-20T08:25:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1b6=[]:{bb=[bb,ba,b7,b6,b4,b3,b2,b0],cb=[cb,ca,c9,c8,c6,c5,c4,c3,c2,c1,bf,be,bd,bc],cd=[cd,cc],cf=[cf,ce],e1=[e1,e0,df,de,dd,db,da,d7],e4=[e4,e3],f9=[f9,f8,f7,f5],ff=[ff,fe,fd,fc,fa],100=[100],118=[118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106,105],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119],154=[153],159=[159,158,157,156,155],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17b=[17b,17a,179],17e=[17e]} for object 5:36b533bd:::smithi11881747-14:a4 2024-03-20T08:25:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:40.177 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:40.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:40.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:40.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:25:40.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:25:40.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: No clone_snaps in snapset 1c4=[]:{b5=[a8],c2=[c2],d7=[d7,cd,c8,c5,c3],ea=[e9,e8,e5,e4,e3,e1,e0,dd],ef=[ef,ee,ed,eb],f9=[f9,f8,f7,f5,f4,f3,f2,f1,f0],fc=[fc,fa],110=[110,10f,10e,10d,10c,10a,108,107,106,105,102,101,fd],11d=[11d,11c,11b,11a,119,118,117,116],14d=[14d,14c,14b,149,148,147,146,145,144,143,142,141,140,13f],170=[170,16f,16e],17d=[17d,17b],17e=[17e],184=[184,183,181],1ae=[1ae,1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1ba=[1ba,1b8,1b7,1b4,1b0],1c3=[1c3,1c2]} for object 5:07779a33:::smithi11881747-18:9c 2024-03-20T08:25:40.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: from='client.? 172.21.15.118:0/3131966524' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:40.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:39 smithi118 ceph-mon[93121]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-20T08:25:40.178 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:40 smithi118 systemd[1]: Stopping Ceph mon.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:25:40.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:25:40.339+0000 7f75169c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:25:40.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a[93098]: 2024-03-20T08:25:40.339+0000 7f75169c3700 -1 mon.a@0(leader) e4 *** Got Signal Terminated *** 2024-03-20T08:25:40.673 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:40 smithi118 podman[147026]: 2024-03-20 08:25:40.370984345 +0000 UTC m=+0.185913125 container died 51c68c4d30af2a5c840ae69fed5980424145aa24b213b815a304fac875da3d8a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-20T08:25:40.674 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:40] ENGINE Bus STOPPING 2024-03-20T08:25:40.674 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:40] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:25:40.674 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:40] ENGINE Bus STOPPED 2024-03-20T08:25:40.674 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:40] ENGINE Bus STARTING 2024-03-20T08:25:40.674 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:40] ENGINE Serving on http://:::9283 2024-03-20T08:25:40.674 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:40] ENGINE Bus STARTED 2024-03-20T08:25:40.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:40.578+0000 7f831d6ac700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1b6=[]:{cb=[cb,ca,c9,c8,c6,c5,c3],d6=[d3,d1,cf,cd],e4=[e4],f6=[f6,f5,f4,f3,f2,f1,f0,eb],f8=[f8,f7],101=[101,100,ff,fe,fd,fc,fa],108=[108,107,106,105,104,103,102],113=[113,112,111],11e=[11e,11d,11a,119,117,116,114],126=[126,125,124,122,120,11f],15b=[15b,15a,159,158,157,156,155,153,152,151,150,14f,14e,14c,14b,148,147,146,145,143,141,13e,13d,13c,13b,136,135,134,130,12e,12a,127],166=[166,165,164,163,162,161,15e,15d],17d=[17b],17e=[17e],17f=[17f],194=[194,193],1a4=[1a4,1a1,1a0,19e,19d,19c,19a],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5],1b6=[1b4]} for object 5:12123ea8:::smithi11881747-45:ae 2024-03-20T08:25:40.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:40.578+0000 7f831c6aa700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bd=[]:{b6=[b4,b3,a8],c6=[c6,c5,c3,c2,c1,bf,be],cb=[cb,ca,c9,c8],ce=[ce,cd],d6=[d3,d1,cf],e3=[e3],f6=[f6,f5,f4,f3,f2,f1,f0,eb,e8,e5],124=[124,122,120,11f,11e,11d,11a,119,117,116,114,113,111,10f,10e,10d,10a,108,106],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127],16c=[16c,16b,16a,168,166,165],170=[170,16f,16e],176=[176,175,174,173,172,171],179=[179,178,177],18b=[18b,18a,188,187,184,183],19c=[19c,19b,19a,199,198],1a3=[1a3,1a1,1a0,19e,19d],1b6=[1b6,1b4,1b3,1b0,1af,1ae,1ad,1ac,1a9,1a5],1b7=[1b7],1bd=[1bd,1bc,1bb]} for object 5:5613ae06:::smithi11881747-16:9e 2024-03-20T08:25:40.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:40.579+0000 7f8316e9f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:beabf7e5:::smithi11881747-7:a0 repair needed (no obc) 2024-03-20T08:25:40.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:40.580+0000 7f831669e700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap be after trimming was started 2024-03-20T08:25:40.675 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:25:40.580+0000 7f831669e700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:66a7f7bc:::smithi11881747-33:c3 repair needed (no obc) 2024-03-20T08:25:41.115 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:25:40.698+0000 7f0cf1462700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:a551e079:::smithi11881747-5:6f repair needed (no obc) 2024-03-20T08:25:41.116 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:25:40.702+0000 7f0cee45c700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:c4fe5974:::smithi11881747-36:5a repair needed (no obc) 2024-03-20T08:25:41.116 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:40 smithi118 podman[147026]: 2024-03-20 08:25:40.881644667 +0000 UTC m=+0.696573493 container cleanup 51c68c4d30af2a5c840ae69fed5980424145aa24b213b815a304fac875da3d8a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-20T08:25:41.116 INFO:journalctl@ceph.mon.a.smithi118.stdout:Mar 20 08:25:40 smithi118 bash[147026]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-a 2024-03-20T08:25:41.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:25:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:25:40.722+0000 7ffa6b167700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1bf=[]:{fc=[fc,fa,f8,f7,f5,f4,f3,f2,f1,eb],fd=[fd],134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d,119,117,116,114,111,10f,10e,10d,10a,106],163=[163,162,161,160,15f,15e,15d,15b,15a,156,153],169=[169,168,167],178=[178,177,176,175,174,173,172,171,16f,16c,16b],17c=[17b],1a3=[1a3,1a1,1a0,19e,19d,19c,19b,19a,198],1aa=[1aa,1a9,1a8,1a7,1a5,1a4],1b2=[1b2,1b1,1b0,1af,1ae,1ad,1ac,1ab],1b3=[1b3],1b6=[1b6,1b4]} for object 5:e4198694:::smithi11881747-19:e3 2024-03-20T08:25:41.139 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:25:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:25:40.722+0000 7ffa6f16f700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:f8faee4c:::smithi11881747-1:9d repair needed (no obc) 2024-03-20T08:25:41.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:25:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:25:40.717+0000 7ffb3df10700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1c7=[]:{134=[134,133,132,131,130,12f,12e,12d,12c,12b,12a,129,128,127,11e,11d],14d=[14d,14c,14b],14e=[14e],151=[151,150,14f],15e=[15e,15d,15b,15a,159,156,153,152],178=[178,177,176,175,173,172,171,16f,16c,16b,162],17c=[17c,17b,17a],18d=[18c],1ad=[1ad,1ac,1ab,1aa,1a9,1a8,1a5,1a4,1a1,19c,19a],1b5=[1b4,1b3,1b0,1af,1ae],1c7=[1c7,1c6,1c5]} for object 5:902426e2:::smithi11881747-46:ea 2024-03-20T08:25:41.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:25:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:25:40.720+0000 7ffb3c70d700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap f4 after trimming was started 2024-03-20T08:25:41.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:25:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:25:40.720+0000 7ffb3c70d700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a7=[]:{107=[106],114=[114,111,10f,10e,10d,10a],126=[11e,11d,119,117,116],149=[149,148,147,146,145,144,143,142,141,140,13f,13e,13d,13c,13b,138,136,135,134,133,130,12e,12a,128,127],154=[153,152,151,150,14f,14e],158=[158,157,156,155],166=[166,165,164,163,162,161],16e=[16e,16d,16c,16b,16a,168],170=[170,16f],171=[171],17c=[17b,179,178,173],18b=[18b,18a,188,187,184,183],190=[18f],199=[199,198,197,196,195,194],1a3=[1a3,1a1,1a0,19e,19d,19c]} for object 5:27618338:::smithi11881747-17:f4 2024-03-20T08:25:41.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:40.718+0000 7fa5d583d700 -1 log_channel(cluster) log [ERR] : trim_object: Can not trim 5:89304740:::smithi11881747-4:9c repair needed (no obc) 2024-03-20T08:25:41.140 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:40.719+0000 7fa5cf030700 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 95 after trimming was started 2024-03-20T08:25:41.141 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:25:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:25:40.719+0000 7fa5cf030700 -1 log_channel(cluster) log [ERR] : No clone_snaps in snapset 1a1=[]:{e3=[e3,e0],f0=[f0,eb,e8,e5],100=[100,ff,fe,fd,fc,fa],11a=[11a,119,118,117,116,115,114,113,111,110,10f,10e,10d,10a,108,106],11c=[11c,11b],11e=[11e,11d],14d=[14d,14c,14b],152=[152,151,150,14f,14e],15b=[15b,15a,159,158,157,156,155,153],15e=[15e,15d],165=[165,164,163,162,161],175=[175,174,173,172,171,170,16f,16c,16b,16a],178=[178,177,176],17c=[17c,17b,17a,179],17e=[17e],186=[186,185,184,183],18a=[18a,188,187],1a1=[1a1,1a0,19f,19e,19d,19c,19b,19a,198]} for object 5:70e08578:::smithi11881747-24:9e 2024-03-20T08:25:41.241 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.a.service' 2024-03-20T08:25:41.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:25:41.291 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-20T08:25:41.291 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-03-20T08:25:41.292 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.c 2024-03-20T08:25:41.422 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:41 smithi118 systemd[1]: Stopping Ceph mon.c for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:25:41.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c[95669]: 2024-03-20T08:25:41.605+0000 7fd7cccf6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:25:41.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c[95669]: 2024-03-20T08:25:41.605+0000 7fd7cccf6700 -1 mon.c@2(peon) e4 *** Got Signal Terminated *** 2024-03-20T08:25:41.923 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:41 smithi118 podman[147171]: 2024-03-20 08:25:41.810551348 +0000 UTC m=+0.335770624 container died 1ea7cd20eda3de054e099c193a826ae99ba228da82b4f6d3e86d0966cf8e4242 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-20T08:25:41.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:41] ENGINE Bus STOPPING 2024-03-20T08:25:41.924 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:41] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-20T08:25:41.924 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:41] ENGINE Bus STOPPED 2024-03-20T08:25:41.924 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:41] ENGINE Bus STARTING 2024-03-20T08:25:41.924 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:41] ENGINE Serving on http://:::9283 2024-03-20T08:25:41.924 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:41 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: [20/Mar/2024:08:25:41] ENGINE Bus STARTED 2024-03-20T08:25:42.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:42 smithi118 podman[147171]: 2024-03-20 08:25:42.244485345 +0000 UTC m=+0.769704628 container cleanup 1ea7cd20eda3de054e099c193a826ae99ba228da82b4f6d3e86d0966cf8e4242 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-20T08:25:42.423 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:42 smithi118 bash[147171]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c 2024-03-20T08:25:42.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:25:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:25:42.315Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:25:42.662 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.c.service' 2024-03-20T08:25:42.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:42 smithi118 podman[147202]: 2024-03-20 08:25:42.429174473 +0000 UTC m=+0.615559080 container remove 1ea7cd20eda3de054e099c193a826ae99ba228da82b4f6d3e86d0966cf8e4242 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-c, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-20T08:25:42.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:42 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-20T08:25:42.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:42 smithi118 systemd[1]: Stopped Ceph mon.c for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:25:42.686 INFO:journalctl@ceph.mon.c.smithi118.stdout:Mar 20 08:25:42 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.c.service: Consumed 13.246s CPU time. 2024-03-20T08:25:42.698 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:25:42.754 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-03-20T08:25:42.755 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-03-20T08:25:42.755 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.b 2024-03-20T08:25:43.043 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:42 smithi196 systemd[1]: Stopping Ceph mon.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:25:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b[87057]: 2024-03-20T08:25:43.043+0000 7feff1784700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:25:43.389 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b[87057]: 2024-03-20T08:25:43.043+0000 7feff1784700 -1 mon.b@1(peon) e4 *** Got Signal Terminated *** 2024-03-20T08:25:43.390 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:43 smithi196 podman[130785]: 2024-03-20 08:25:43.28869686 +0000 UTC m=+0.339087081 container died bbde40d464eac0b6054de62ee2e0cbace7f4ededd480a05fd10988ac12846a25 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-20T08:25:43.889 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:43 smithi196 podman[130785]: 2024-03-20 08:25:43.790990409 +0000 UTC m=+0.841380639 container cleanup bbde40d464eac0b6054de62ee2e0cbace7f4ededd480a05fd10988ac12846a25 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:25:43.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Mar 20 08:25:43 smithi196 bash[130785]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mon-b 2024-03-20T08:25:43.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:43 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: ::ffff:172.21.15.196 - - [20/Mar/2024:08:25:43] "GET /metrics HTTP/1.1" 200 36409 "" "Prometheus/2.43.0" 2024-03-20T08:25:44.128 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mon.b.service' 2024-03-20T08:25:44.174 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:25:44.174 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-03-20T08:25:44.174 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-20T08:25:44.174 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y 2024-03-20T08:25:44.642 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:44 smithi118 systemd[1]: Stopping Ceph mgr.y for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:25:44.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y[84532]: 2024-03-20T08:25:44.642+0000 7ff9eda23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:25:44.923 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:44 smithi118 podman[147314]: 2024-03-20 08:25:44.674347716 +0000 UTC m=+0.313780809 container died 4dac06b91452454138244ea59c0207d6be22f4b7ece73c48b89d10ca2b044ef8 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-20T08:25:45.308 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:45 smithi118 podman[147314]: 2024-03-20 08:25:45.04967633 +0000 UTC m=+0.689109408 container cleanup 4dac06b91452454138244ea59c0207d6be22f4b7ece73c48b89d10ca2b044ef8 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-20T08:25:45.308 INFO:journalctl@ceph.mgr.y.smithi118.stdout:Mar 20 08:25:45 smithi118 bash[147314]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-y 2024-03-20T08:25:45.462 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-20T08:25:45.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:25:45.500 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-20T08:25:45.500 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-20T08:25:45.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x 2024-03-20T08:25:45.829 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:25:45 smithi196 systemd[1]: Stopping Ceph mgr.x for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:25:46.104 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:25:45 smithi196 podman[130920]: 2024-03-20 08:25:45.829685242 +0000 UTC m=+0.135151761 container died 043bae454c20ff29934ed7c84a96cbc983734e54231cce96c5b2d60355e191ce (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-20T08:25:46.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:25:46 smithi196 podman[130920]: 2024-03-20 08:25:46.195646253 +0000 UTC m=+0.501112780 container cleanup 043bae454c20ff29934ed7c84a96cbc983734e54231cce96c5b2d60355e191ce (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:25:46.389 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Mar 20 08:25:46 smithi196 bash[130920]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mgr-x 2024-03-20T08:25:46.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-20T08:25:46.652 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:25:46.652 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-20T08:25:46.653 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-20T08:25:46.653 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.0 2024-03-20T08:25:46.955 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:46 smithi118 systemd[1]: Stopping Ceph osd.0 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:25:47.213 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:25:46.955+0000 7f0d0eea2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:25:47.213 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:25:46.955+0000 7f0d0eea2700 -1 osd.0 870 *** Got signal Terminated *** 2024-03-20T08:25:47.214 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:46 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0[101791]: 2024-03-20T08:25:46.955+0000 7f0d0eea2700 -1 osd.0 870 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:25:52.639 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:25:52 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:25:52.314Z caller=manager.go:640 level=warn component="rule manager" file=/etc/prometheus/alerting/ceph_alerts.yml group=pools name=CephPoolGrowthWarning index=0 msg="Evaluating rule failed" rule="alert: CephPoolGrowthWarning\nexpr: (predict_linear(ceph_pool_percent_used[2d], 3600 * 24 * 5) * on (pool_id, instance)\n group_right () ceph_pool_metadata) >= 95\nlabels:\n oid: 1.3.6.1.4.1.50495.1.2.1.9.2\n severity: warning\n type: ceph_default\nannotations:\n description: Pool '{{ $labels.name }}' will be full in less than 5 days assuming\n the average fill-up rate of the past 48 hours.\n summary: Pool growth rate may soon exceed capacity\n" err="found duplicate series for the match group {instance=\"smithi118:9283\", pool_id=\"1\"} on the left hand-side of the operation: [{instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}, {cluster=\"bc2566e8-e68e-11ee-95c9-87774f69a715\", instance=\"smithi118:9283\", job=\"ceph\", pool_id=\"1\"}];many-to-many matching not allowed: matching labels must be unique on one side" 2024-03-20T08:25:53.087 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:52 smithi118 podman[147445]: 2024-03-20 08:25:52.801084216 +0000 UTC m=+5.953872239 container died 56e63ece2c91bc6bea1705d74f32caeb83ac15ef8a118c7add0b66a92cbf79b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-20T08:25:53.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:53 smithi118 podman[147445]: 2024-03-20 08:25:53.253713918 +0000 UTC m=+6.406501982 container cleanup 56e63ece2c91bc6bea1705d74f32caeb83ac15ef8a118c7add0b66a92cbf79b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-20T08:25:53.423 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:53 smithi118 bash[147445]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0 2024-03-20T08:25:53.865 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:53 smithi118 podman[147479]: 2024-03-20 08:25:53.521987056 +0000 UTC m=+0.725133502 container remove 56e63ece2c91bc6bea1705d74f32caeb83ac15ef8a118c7add0b66a92cbf79b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-20T08:25:54.116 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:53 smithi118 podman[147575]: 2024-03-20 08:25:53.769458028 +0000 UTC m=+0.018003685 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:25:54.116 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:53 smithi118 podman[147575]: 2024-03-20 08:25:53.873048956 +0000 UTC m=+0.121594648 container create 4b29f66fd0acdb7bc334203b5c448c0aa69400569e573e03d2db40efabeb90b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:25:54.403 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:54 smithi118 podman[147575]: 2024-03-20 08:25:54.117358383 +0000 UTC m=+0.365904038 container init 4b29f66fd0acdb7bc334203b5c448c0aa69400569e573e03d2db40efabeb90b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-20T08:25:54.404 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:54 smithi118 podman[147575]: 2024-03-20 08:25:54.122577502 +0000 UTC m=+0.371123157 container start 4b29f66fd0acdb7bc334203b5c448c0aa69400569e573e03d2db40efabeb90b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-20T08:25:54.404 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:54 smithi118 podman[147575]: 2024-03-20 08:25:54.217216822 +0000 UTC m=+0.465762483 container attach 4b29f66fd0acdb7bc334203b5c448c0aa69400569e573e03d2db40efabeb90b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-20T08:25:54.673 INFO:journalctl@ceph.osd.0.smithi118.stdout:Mar 20 08:25:54 smithi118 podman[147575]: 2024-03-20 08:25:54.404759341 +0000 UTC m=+0.653304985 container died 4b29f66fd0acdb7bc334203b5c448c0aa69400569e573e03d2db40efabeb90b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD) 2024-03-20T08:25:54.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:25:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:25:54.489Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:25:54.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:25:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:25:54.489Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:25:55.029 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.0.service' 2024-03-20T08:25:55.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:25:55.069 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-20T08:25:55.069 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-20T08:25:55.069 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.1 2024-03-20T08:25:55.418 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:25:55 smithi118 systemd[1]: Stopping Ceph osd.1 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:25:55.672 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:25:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:25:55.418+0000 7fc216878700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:25:55.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:25:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:25:55.418+0000 7fc216878700 -1 osd.1 870 *** Got signal Terminated *** 2024-03-20T08:25:55.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:25:55 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1[106589]: 2024-03-20T08:25:55.418+0000 7fc216878700 -1 osd.1 870 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:26:00.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:00.173Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:00.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:00.174Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:01.357 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:01 smithi118 podman[147708]: 2024-03-20 08:26:01.065006341 +0000 UTC m=+5.777218773 container died 3eae2fe0915aa02bdfb4c8617bf7ab488a2e8fb2911e5542967a421c871c3df7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-20T08:26:01.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:01 smithi118 podman[147708]: 2024-03-20 08:26:01.45702756 +0000 UTC m=+6.169240001 container cleanup 3eae2fe0915aa02bdfb4c8617bf7ab488a2e8fb2911e5542967a421c871c3df7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-20T08:26:01.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:01 smithi118 bash[147708]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1 2024-03-20T08:26:02.017 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:01 smithi118 podman[147732]: 2024-03-20 08:26:01.691559131 +0000 UTC m=+0.624602818 container remove 3eae2fe0915aa02bdfb4c8617bf7ab488a2e8fb2911e5542967a421c871c3df7 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-20T08:26:02.302 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:02 smithi118 podman[147831]: 2024-03-20 08:26:01.922357529 +0000 UTC m=+0.019059946 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:26:02.302 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:02 smithi118 podman[147831]: 2024-03-20 08:26:02.025819621 +0000 UTC m=+0.122522067 container create a15e5bffddaa4f360861d5f57a619273b08a2b2be1789a4bf2522ba32a4f693c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:26:02.302 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:02 smithi118 podman[147831]: 2024-03-20 08:26:02.211470698 +0000 UTC m=+0.308173114 container init a15e5bffddaa4f360861d5f57a619273b08a2b2be1789a4bf2522ba32a4f693c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:26:02.302 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:02 smithi118 podman[147831]: 2024-03-20 08:26:02.217340893 +0000 UTC m=+0.314043302 container start a15e5bffddaa4f360861d5f57a619273b08a2b2be1789a4bf2522ba32a4f693c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-20T08:26:02.302 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:02 smithi118 podman[147831]: 2024-03-20 08:26:02.303063716 +0000 UTC m=+0.399766138 container attach a15e5bffddaa4f360861d5f57a619273b08a2b2be1789a4bf2522ba32a4f693c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-20T08:26:02.673 INFO:journalctl@ceph.osd.1.smithi118.stdout:Mar 20 08:26:02 smithi118 podman[147831]: 2024-03-20 08:26:02.514751356 +0000 UTC m=+0.611453761 container died a15e5bffddaa4f360861d5f57a619273b08a2b2be1789a4bf2522ba32a4f693c (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-20T08:26:03.256 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.1.service' 2024-03-20T08:26:03.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:26:03.297 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-20T08:26:03.297 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-20T08:26:03.298 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.2 2024-03-20T08:26:03.560 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:03 smithi118 systemd[1]: Stopping Ceph osd.2 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:26:03.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:03.561Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:03.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:03.562Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:03.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:26:03.628+0000 7f83348df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:26:03.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:26:03.628+0000 7f83348df700 -1 osd.2 870 *** Got signal Terminated *** 2024-03-20T08:26:03.923 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2[111826]: 2024-03-20T08:26:03.628+0000 7f83348df700 -1 osd.2 870 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:26:04.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:04.489Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:04.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:04.489Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:04.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:04 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:04.490Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:09.726 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:09 smithi118 podman[147960]: 2024-03-20 08:26:09.455900781 +0000 UTC m=+5.938801612 container died 21c6276337212b3e3fad51869278169ddb764661de05663b44580832f4f49948 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS) 2024-03-20T08:26:10.102 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:09 smithi118 podman[147960]: 2024-03-20 08:26:09.893155143 +0000 UTC m=+6.376056043 container cleanup 21c6276337212b3e3fad51869278169ddb764661de05663b44580832f4f49948 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-20T08:26:10.103 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:09 smithi118 bash[147960]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2 2024-03-20T08:26:10.423 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:10 smithi118 podman[147982]: 2024-03-20 08:26:10.103280726 +0000 UTC m=+0.643460347 container remove 21c6276337212b3e3fad51869278169ddb764661de05663b44580832f4f49948 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-20T08:26:10.423 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:10.173Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:10.424 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:10.174Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:10.424 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:10 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:10.174Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:10.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:10 smithi118 podman[148077]: 2024-03-20 08:26:10.394608586 +0000 UTC m=+0.017893201 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:26:10.747 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:10 smithi118 podman[148077]: 2024-03-20 08:26:10.522319041 +0000 UTC m=+0.145603658 container create 1ec409d26befa20ae448c7b774c53a94d3f577fcd93e1a9a7e77a11eebdbddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-20T08:26:10.748 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:10 smithi118 podman[148077]: 2024-03-20 08:26:10.748452779 +0000 UTC m=+0.371737394 container init 1ec409d26befa20ae448c7b774c53a94d3f577fcd93e1a9a7e77a11eebdbddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-20T08:26:11.035 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:10 smithi118 podman[148077]: 2024-03-20 08:26:10.75398847 +0000 UTC m=+0.377273088 container start 1ec409d26befa20ae448c7b774c53a94d3f577fcd93e1a9a7e77a11eebdbddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:26:11.036 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:10 smithi118 podman[148077]: 2024-03-20 08:26:10.831654874 +0000 UTC m=+0.454939515 container attach 1ec409d26befa20ae448c7b774c53a94d3f577fcd93e1a9a7e77a11eebdbddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:26:11.325 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:11 smithi118 podman[148077]: 2024-03-20 08:26:11.037014968 +0000 UTC m=+0.660299574 container died 1ec409d26befa20ae448c7b774c53a94d3f577fcd93e1a9a7e77a11eebdbddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:26:11.578 INFO:journalctl@ceph.osd.2.smithi118.stdout:Mar 20 08:26:11 smithi118 podman[148124]: 2024-03-20 08:26:11.560128402 +0000 UTC m=+0.518477633 container remove 1ec409d26befa20ae448c7b774c53a94d3f577fcd93e1a9a7e77a11eebdbddcb (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:26:11.583 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.2.service' 2024-03-20T08:26:11.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:26:11.624 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-20T08:26:11.624 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-20T08:26:11.624 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.3 2024-03-20T08:26:11.923 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:11 smithi118 systemd[1]: Stopping Ceph osd.3 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:26:12.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:26:11.965+0000 7f5a05873700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:26:12.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:26:11.965+0000 7f5a05873700 -1 osd.3 870 *** Got signal Terminated *** 2024-03-20T08:26:12.423 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:11 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:26:11.965+0000 7f5a05873700 -1 osd.3 870 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:26:12.889 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:26:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:26:12.406Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-03-20T08:26:12.890 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:26:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:26:12.408Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-03-20T08:26:12.890 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:26:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:26:12.409Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-03-20T08:26:12.890 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:26:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:26:12.409Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-03-20T08:26:12.890 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:26:12 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:26:12.411Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.118:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.118:8765: connect: connection refused" 2024-03-20T08:26:13.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:13 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:13.067+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:13.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:13.561Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:13.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:13.561Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:13.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:13 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:13.562Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:14.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:14.173+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:14.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:14.046+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:14.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:14.554+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:14.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:14.587+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:14.922 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:14.490Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:14.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:14.490Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:14.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:14 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:14.491Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:15.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:15.215+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:15.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:14 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:14.996+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:15.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:15.565+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:15.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:15 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:15.512+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:15.922 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:15 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:26:15.433+0000 7f59fde5f700 -1 osd.3 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.812140+0000 front 2024-03-20T08:25:51.812242+0000 (oldest deadline 2024-03-20T08:26:15.312337+0000) 2024-03-20T08:26:16.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:16.181+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:16.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:16.023+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:16.824 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:16 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3[116557]: 2024-03-20T08:26:16.460+0000 7f59fde5f700 -1 osd.3 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.812140+0000 front 2024-03-20T08:25:51.812242+0000 (oldest deadline 2024-03-20T08:26:15.312337+0000) 2024-03-20T08:26:16.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:16.541+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:16.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:16.548+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:17.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:17.200+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:17.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:16 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:16.997+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:17.763 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:17 smithi118 podman[148208]: 2024-03-20 08:26:17.493778109 +0000 UTC m=+5.644968181 container died c7c15dc3876f3689a9d39e3aac9df13685f0319622c1dfddd5286289b23efc5e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:26:17.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:17.550+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:17.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:17 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:17.582+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:18.122 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:17 smithi118 podman[148208]: 2024-03-20 08:26:17.938202016 +0000 UTC m=+6.089392102 container cleanup c7c15dc3876f3689a9d39e3aac9df13685f0319622c1dfddd5286289b23efc5e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-20T08:26:18.122 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:17 smithi118 bash[148208]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3 2024-03-20T08:26:18.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:18.235+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:18.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:18.018+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:18.422 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:18 smithi118 podman[148230]: 2024-03-20 08:26:18.156395729 +0000 UTC m=+0.657421052 container remove c7c15dc3876f3689a9d39e3aac9df13685f0319622c1dfddd5286289b23efc5e (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-20T08:26:18.859 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:18 smithi118 podman[148334]: 2024-03-20 08:26:18.456348809 +0000 UTC m=+0.017818505 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:26:18.859 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:18 smithi118 podman[148334]: 2024-03-20 08:26:18.565495918 +0000 UTC m=+0.126965610 container create ea9ce58a876f174915c6d52e3124d43cac3128c8ccda942abf080f743d13366a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-20T08:26:18.859 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:18 smithi118 podman[148334]: 2024-03-20 08:26:18.776189037 +0000 UTC m=+0.337658742 container init ea9ce58a876f174915c6d52e3124d43cac3128c8ccda942abf080f743d13366a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-20T08:26:18.859 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:18 smithi118 podman[148334]: 2024-03-20 08:26:18.781884566 +0000 UTC m=+0.343354268 container start ea9ce58a876f174915c6d52e3124d43cac3128c8ccda942abf080f743d13366a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-03-20T08:26:18.860 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:18 smithi118 podman[148334]: 2024-03-20 08:26:18.859404275 +0000 UTC m=+0.420873981 container attach ea9ce58a876f174915c6d52e3124d43cac3128c8ccda942abf080f743d13366a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-20T08:26:18.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:18.513+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:18.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:18 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:18.581+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:19.173 INFO:journalctl@ceph.osd.3.smithi118.stdout:Mar 20 08:26:19 smithi118 podman[148334]: 2024-03-20 08:26:19.071209936 +0000 UTC m=+0.632679628 container died ea9ce58a876f174915c6d52e3124d43cac3128c8ccda942abf080f743d13366a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0) 2024-03-20T08:26:19.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:19.050+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:19.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:19.208+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:19.656 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.3.service' 2024-03-20T08:26:19.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:26:19.697 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-20T08:26:19.697 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-20T08:26:19.697 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.4 2024-03-20T08:26:19.731 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:19.479+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:19.731 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:19 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:19.568+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:20.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:19 smithi196 systemd[1]: Stopping Ceph osd.4 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:26:20.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:20.070+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:20.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:19.998+0000 7fa5f1279700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:26:20.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:19.998+0000 7fa5f1279700 -1 osd.4 870 *** Got signal Terminated *** 2024-03-20T08:26:20.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:19.998+0000 7fa5f1279700 -1 osd.4 870 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:26:20.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:20.228+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:20.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:20.174Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:20.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:20.175Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:20.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:20 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:20.176Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:20.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:20.486+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:20.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:20 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:20.579+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:21.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:21.188+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:21.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:21.062+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:21.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:21.507+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:21.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:21 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:21.610+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:22.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:22.071+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:22.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:22.166+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:22.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:22.166+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.557530+0000 front 2024-03-20T08:25:59.557083+0000 (oldest deadline 2024-03-20T08:26:21.256923+0000) 2024-03-20T08:26:22.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:22.547+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:22.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:22.607+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:22.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:22 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:22.607+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:23.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:23.158+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:23.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:23.158+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.557530+0000 front 2024-03-20T08:25:59.557083+0000 (oldest deadline 2024-03-20T08:26:21.256923+0000) 2024-03-20T08:26:23.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:23.117+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:23.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:23.117+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:23.677 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:23.625+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:23.677 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:23.625+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:23.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:23 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:23.541+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:23.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:23 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:23.561Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:24.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:24.078+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:24.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:24.078+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:24.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:24.159+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.356332+0000 front 2024-03-20T08:25:48.356416+0000 (oldest deadline 2024-03-20T08:26:13.656049+0000) 2024-03-20T08:26:24.390 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4[92806]: 2024-03-20T08:26:24.159+0000 7fa5e9865700 -1 osd.4 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.557530+0000 front 2024-03-20T08:25:59.557083+0000 (oldest deadline 2024-03-20T08:26:21.256923+0000) 2024-03-20T08:26:24.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:24.523+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:24.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:24.523+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:24.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:24.590+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:24.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:24 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:24.590+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:24.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:24.491Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:24.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:24.491Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:24.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:24 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:24.491Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:25.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:25.084+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:25.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:25.084+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:25.889 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:25 smithi196 podman[131067]: 2024-03-20 08:26:25.640061247 +0000 UTC m=+5.746009483 container died 4b7be73bd9c3abfafb87107c4d22d17a429c4dd24de26c7180be725804e04084 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-20T08:26:25.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:25.475+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:25.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:25.475+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:25.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:25.579+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:25.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:25 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:25.579+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:26 smithi196 podman[131067]: 2024-03-20 08:26:26.013915029 +0000 UTC m=+6.119863265 container cleanup 4b7be73bd9c3abfafb87107c4d22d17a429c4dd24de26c7180be725804e04084 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:26:26.247 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:26 smithi196 bash[131067]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4 2024-03-20T08:26:26.247 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:26.064+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:26.248 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:26.064+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:26.532 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:26 smithi196 podman[131102]: 2024-03-20 08:26:26.248333275 +0000 UTC m=+0.601042877 container remove 4b7be73bd9c3abfafb87107c4d22d17a429c4dd24de26c7180be725804e04084 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-03-20T08:26:26.533 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:26.449+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:26.533 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:26.449+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:26.809 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:26.532+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:26.810 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:26 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:26.532+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:26.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:26 smithi196 podman[131196]: 2024-03-20 08:26:26.473430964 +0000 UTC m=+0.018537394 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:26:26.810 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:26 smithi196 podman[131196]: 2024-03-20 08:26:26.582916424 +0000 UTC m=+0.128022852 container create 7b29df2a44d25085007e315e87d4d31936ba57734840c6fc8113b8e4ff6bfafc (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:26:27.101 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:26 smithi196 podman[131196]: 2024-03-20 08:26:26.810447694 +0000 UTC m=+0.355554125 container init 7b29df2a44d25085007e315e87d4d31936ba57734840c6fc8113b8e4ff6bfafc (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-20T08:26:27.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:26 smithi196 podman[131196]: 2024-03-20 08:26:26.816457606 +0000 UTC m=+0.361564033 container start 7b29df2a44d25085007e315e87d4d31936ba57734840c6fc8113b8e4ff6bfafc (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-20T08:26:27.102 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:26 smithi196 podman[131196]: 2024-03-20 08:26:26.89353792 +0000 UTC m=+0.438644362 container attach 7b29df2a44d25085007e315e87d4d31936ba57734840c6fc8113b8e4ff6bfafc (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-20T08:26:27.389 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 20 08:26:27 smithi196 podman[131196]: 2024-03-20 08:26:27.10273573 +0000 UTC m=+0.647842167 container died 7b29df2a44d25085007e315e87d4d31936ba57734840c6fc8113b8e4ff6bfafc (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-03-20T08:26:27.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:27.106+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:27.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:27.106+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:27.663 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:27.526+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:27.663 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:27.526+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:27.663 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:27.461+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:27.663 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:27 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:27.461+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:27.686 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.4.service' 2024-03-20T08:26:27.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:26:27.771 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-20T08:26:27.771 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-20T08:26:27.771 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.5 2024-03-20T08:26:28.102 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:27 smithi196 systemd[1]: Stopping Ceph osd.5 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:26:28.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:28.106+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:28.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:28.106+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:28.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:28.102+0000 7ffa89ba9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:26:28.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:28.102+0000 7ffa89ba9700 -1 osd.5 870 *** Got signal Terminated *** 2024-03-20T08:26:28.390 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:28.102+0000 7ffa89ba9700 -1 osd.5 870 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:26:28.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:28.454+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:28.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:28.454+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:28.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:28.569+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:28.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:28 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:28.569+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:29.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:29.104+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:29.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:29.104+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:29.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:29.531+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:29.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:29.531+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:29.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:29.460+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:29.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:29 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:29.460+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:30.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:30.068+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:30.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:30.068+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:30.674 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:30.175Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:30.674 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:30.175Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:30.675 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:30 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:30.176Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:30.889 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:30.446+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:30.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:30.446+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:30.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:30.556+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:30.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:30 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:30.556+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:31.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:31.045+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:31.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:31.045+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:31.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:31.605+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:31.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:31.605+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:31.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:31.605+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:31.891 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:31.418+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:31.891 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:31.418+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:31.891 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:31 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:31.418+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:04.846821+0000 front 2024-03-20T08:26:04.846696+0000 (oldest deadline 2024-03-20T08:26:30.647194+0000) 2024-03-20T08:26:32.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:32.033+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:32.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:32.033+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:32.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:32.578+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:32.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:32.578+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:32.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:32.578+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:32.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:32.436+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:48.544531+0000 front 2024-03-20T08:25:48.544325+0000 (oldest deadline 2024-03-20T08:26:14.444224+0000) 2024-03-20T08:26:32.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:32.436+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:59.645720+0000 front 2024-03-20T08:25:59.645694+0000 (oldest deadline 2024-03-20T08:26:23.745742+0000) 2024-03-20T08:26:32.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:32 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5[97976]: 2024-03-20T08:26:32.436+0000 7ffa8619d700 -1 osd.5 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:04.846821+0000 front 2024-03-20T08:26:04.846696+0000 (oldest deadline 2024-03-20T08:26:30.647194+0000) 2024-03-20T08:26:33.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:33.040+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:33.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:33.040+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:33.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:33.040+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:33.834 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:33.542+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:33.834 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:33.542+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:33.834 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:33 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:33.542+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:34.100 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:33 smithi196 podman[131325]: 2024-03-20 08:26:33.835508984 +0000 UTC m=+5.836743573 container died 72b569d10541a50f36b451da230a9333e61de5701646e3ac26a95121d021ecab (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-20T08:26:34.100 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:34.041+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:34.101 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:34.041+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:34.101 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:34.041+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:34.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:34 smithi196 podman[131325]: 2024-03-20 08:26:34.224850204 +0000 UTC m=+6.226084791 container cleanup 72b569d10541a50f36b451da230a9333e61de5701646e3ac26a95121d021ecab (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-20T08:26:34.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:34 smithi196 bash[131325]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5 2024-03-20T08:26:34.759 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:34 smithi196 podman[131349]: 2024-03-20 08:26:34.468308431 +0000 UTC m=+0.627422783 container remove 72b569d10541a50f36b451da230a9333e61de5701646e3ac26a95121d021ecab (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-20T08:26:34.760 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:34.498+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:34.760 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:34.498+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:34.760 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:34 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:34.498+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:34.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:34.491Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:34.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:34.491Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:34.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:34 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:34.492Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:35.062 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:34 smithi196 podman[131448]: 2024-03-20 08:26:34.663472813 +0000 UTC m=+0.018108699 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:26:35.063 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:34 smithi196 podman[131448]: 2024-03-20 08:26:34.769101388 +0000 UTC m=+0.123737269 container create dd307e8821f01067d75feb03b708d6412907a95f69dfcb2ecefa0d18bcab6cc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-20T08:26:35.063 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:34 smithi196 podman[131448]: 2024-03-20 08:26:34.980103957 +0000 UTC m=+0.334739839 container init dd307e8821f01067d75feb03b708d6412907a95f69dfcb2ecefa0d18bcab6cc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-20T08:26:35.063 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:34 smithi196 podman[131448]: 2024-03-20 08:26:34.985394628 +0000 UTC m=+0.340030509 container start dd307e8821f01067d75feb03b708d6412907a95f69dfcb2ecefa0d18bcab6cc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.license=GPLv2) 2024-03-20T08:26:35.063 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:35 smithi196 podman[131448]: 2024-03-20 08:26:35.063226943 +0000 UTC m=+0.417862828 container attach dd307e8821f01067d75feb03b708d6412907a95f69dfcb2ecefa0d18bcab6cc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-20T08:26:35.389 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 20 08:26:35 smithi196 podman[131448]: 2024-03-20 08:26:35.273848555 +0000 UTC m=+0.628484437 container died dd307e8821f01067d75feb03b708d6412907a95f69dfcb2ecefa0d18bcab6cc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-20T08:26:35.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:35.085+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:35.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:35.085+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:35.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:35.085+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:35.832 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:35.491+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:35.832 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:35.491+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:35.832 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:35 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:35.491+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:35.855 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.5.service' 2024-03-20T08:26:35.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:26:35.897 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-20T08:26:35.897 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-20T08:26:35.898 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.6 2024-03-20T08:26:36.139 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:35 smithi196 systemd[1]: Stopping Ceph osd.6 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:26:36.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:36.068+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:36.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:36.068+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:36.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:36.068+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:36.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:36.230+0000 7fafc2fc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:26:36.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:36.230+0000 7fafc2fc9700 -1 osd.6 870 *** Got signal Terminated *** 2024-03-20T08:26:36.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:36.230+0000 7fafc2fc9700 -1 osd.6 870 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:26:36.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:36.444+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:36.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:36.444+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:36.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:36 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:36.444+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:37.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:37.072+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:37.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:37.072+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:37.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:37.072+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:37.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:37.429+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:37.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:37.429+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:37.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:37 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:37.429+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:38.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:38.030+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:38.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:38.030+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:38.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:38.030+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:38.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:38.395+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:38.889 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:38.395+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:38.890 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:38 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:38.395+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:39.385 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:39.032+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:39.386 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:39.032+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:39.386 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:39.032+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:39.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:39.386+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:39.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:39.386+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:39.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:39 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:39.386+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:40.381 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:40.081+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:40.381 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:40.081+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:40.381 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:40.081+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:40.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:40.381+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:51.869426+0000 front 2024-03-20T08:25:51.869201+0000 (oldest deadline 2024-03-20T08:26:13.569170+0000) 2024-03-20T08:26:40.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:40.381+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:58.870265+0000 front 2024-03-20T08:25:58.870397+0000 (oldest deadline 2024-03-20T08:26:22.370084+0000) 2024-03-20T08:26:40.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:40 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6[103165]: 2024-03-20T08:26:40.381+0000 7fafbb5b5700 -1 osd.6 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:07.071090+0000 front 2024-03-20T08:26:07.071077+0000 (oldest deadline 2024-03-20T08:26:30.570917+0000) 2024-03-20T08:26:40.672 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:40.176Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:40.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:40.176Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:40.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:40 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:40.176Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:41.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:41.115+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:41.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:41.115+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:41.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:41 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:41.115+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:42.253 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:41 smithi196 podman[131576]: 2024-03-20 08:26:41.987457404 +0000 UTC m=+5.864181666 container died dca658056148404fad6de448d9f93df055d145dfd4dc6408106b6d61e2f7145a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-20T08:26:42.253 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:42.149+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:42.253 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:42.149+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:42.253 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:42.149+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:42.254 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:42 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:42.149+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-03-20T08:26:16.724372+0000 front 2024-03-20T08:26:16.724409+0000 (oldest deadline 2024-03-20T08:26:42.024191+0000) 2024-03-20T08:26:42.586 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:42 smithi196 podman[131576]: 2024-03-20 08:26:42.352557392 +0000 UTC m=+6.229281660 container cleanup dca658056148404fad6de448d9f93df055d145dfd4dc6408106b6d61e2f7145a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:26:42.586 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:42 smithi196 bash[131576]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6 2024-03-20T08:26:42.877 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:42 smithi196 podman[131599]: 2024-03-20 08:26:42.587121566 +0000 UTC m=+0.603109623 container remove dca658056148404fad6de448d9f93df055d145dfd4dc6408106b6d61e2f7145a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-20T08:26:43.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:43.102+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:43.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:43.102+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:43.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:43.102+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:43.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:43 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:43.102+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-03-20T08:26:16.724372+0000 front 2024-03-20T08:26:16.724409+0000 (oldest deadline 2024-03-20T08:26:42.024191+0000) 2024-03-20T08:26:43.140 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:42 smithi196 podman[131697]: 2024-03-20 08:26:42.78052339 +0000 UTC m=+0.018004852 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:26:43.140 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:42 smithi196 podman[131697]: 2024-03-20 08:26:42.887944933 +0000 UTC m=+0.125426384 container create 988f383e85151a4c2344248d0a8beb9807f8909421af47a9bc3867e49b4ba422 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:26:43.140 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:43 smithi196 podman[131697]: 2024-03-20 08:26:43.090882991 +0000 UTC m=+0.328364453 container init 988f383e85151a4c2344248d0a8beb9807f8909421af47a9bc3867e49b4ba422 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6) 2024-03-20T08:26:43.141 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:43 smithi196 podman[131697]: 2024-03-20 08:26:43.096710959 +0000 UTC m=+0.334192410 container start 988f383e85151a4c2344248d0a8beb9807f8909421af47a9bc3867e49b4ba422 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-20T08:26:43.639 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:43 smithi196 podman[131697]: 2024-03-20 08:26:43.207098105 +0000 UTC m=+0.444579560 container attach 988f383e85151a4c2344248d0a8beb9807f8909421af47a9bc3867e49b4ba422 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-20T08:26:43.640 INFO:journalctl@ceph.osd.6.smithi196.stdout:Mar 20 08:26:43 smithi196 podman[131697]: 2024-03-20 08:26:43.386521273 +0000 UTC m=+0.624002725 container died 988f383e85151a4c2344248d0a8beb9807f8909421af47a9bc3867e49b4ba422 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-20T08:26:43.993 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.6.service' 2024-03-20T08:26:44.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:26:44.034 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-20T08:26:44.034 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-20T08:26:44.034 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.7 2024-03-20T08:26:44.349 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:44.107+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:44.349 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:44.107+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:44.350 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:44.107+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:44.350 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:44.107+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-03-20T08:26:16.724372+0000 front 2024-03-20T08:26:16.724409+0000 (oldest deadline 2024-03-20T08:26:42.024191+0000) 2024-03-20T08:26:44.350 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:44 smithi196 systemd[1]: Stopping Ceph osd.7 for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:26:44.350 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:44.349+0000 7ffb5d954700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:26:44.351 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:44.349+0000 7ffb5d954700 -1 osd.7 870 *** Got signal Terminated *** 2024-03-20T08:26:44.351 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:44 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:44.349+0000 7ffb5d954700 -1 osd.7 870 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-20T08:26:44.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:44.491Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:44.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:44.492Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:44.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:44 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:44.492Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:45.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:45.070+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:45.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:45.070+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:45.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:45.070+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:45.391 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:45 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:45.070+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-03-20T08:26:16.724372+0000 front 2024-03-20T08:26:16.724409+0000 (oldest deadline 2024-03-20T08:26:42.024191+0000) 2024-03-20T08:26:46.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:46.116+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:46.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:46.116+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:46.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:46.116+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:46.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:46 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:46.116+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-03-20T08:26:16.724372+0000 front 2024-03-20T08:26:16.724409+0000 (oldest deadline 2024-03-20T08:26:42.024191+0000) 2024-03-20T08:26:47.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:47.101+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:47.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:47.101+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:47.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:47.101+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:47.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:47.101+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-03-20T08:26:16.724372+0000 front 2024-03-20T08:26:16.724409+0000 (oldest deadline 2024-03-20T08:26:42.024191+0000) 2024-03-20T08:26:47.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:47 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:47.101+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.196:6804 osd.4 since back 2024-03-20T08:26:24.925035+0000 front 2024-03-20T08:26:24.925154+0000 (oldest deadline 2024-03-20T08:26:46.625062+0000) 2024-03-20T08:26:48.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:48.099+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:48.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:48.099+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:48.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:48.099+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:48.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:48.099+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-03-20T08:26:16.724372+0000 front 2024-03-20T08:26:16.724409+0000 (oldest deadline 2024-03-20T08:26:42.024191+0000) 2024-03-20T08:26:48.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:48 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:48.099+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.196:6804 osd.4 since back 2024-03-20T08:26:24.925035+0000 front 2024-03-20T08:26:24.925154+0000 (oldest deadline 2024-03-20T08:26:46.625062+0000) 2024-03-20T08:26:49.389 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:49.073+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6806 osd.0 since back 2024-03-20T08:25:49.520461+0000 front 2024-03-20T08:25:49.520477+0000 (oldest deadline 2024-03-20T08:26:13.020199+0000) 2024-03-20T08:26:49.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:49.073+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6814 osd.1 since back 2024-03-20T08:25:57.522045+0000 front 2024-03-20T08:25:57.521930+0000 (oldest deadline 2024-03-20T08:26:22.221843+0000) 2024-03-20T08:26:49.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:49.073+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6822 osd.2 since back 2024-03-20T08:26:06.922920+0000 front 2024-03-20T08:26:06.922879+0000 (oldest deadline 2024-03-20T08:26:32.222704+0000) 2024-03-20T08:26:49.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:49.073+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.118:6830 osd.3 since back 2024-03-20T08:26:16.724372+0000 front 2024-03-20T08:26:16.724409+0000 (oldest deadline 2024-03-20T08:26:42.024191+0000) 2024-03-20T08:26:49.390 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:49 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7[107857]: 2024-03-20T08:26:49.073+0000 7ffb55f40700 -1 osd.7 870 heartbeat_check: no reply from 172.21.15.196:6804 osd.4 since back 2024-03-20T08:26:24.925035+0000 front 2024-03-20T08:26:24.925154+0000 (oldest deadline 2024-03-20T08:26:46.625062+0000) 2024-03-20T08:26:50.222 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:49 smithi196 podman[131827]: 2024-03-20 08:26:49.947698459 +0000 UTC m=+5.697130862 container died 52392403da44e54428ee449b4f9d69c2bf26bbb5c63f470720d5a8b566f1c480 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-20T08:26:50.554 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:50 smithi196 podman[131827]: 2024-03-20 08:26:50.312573835 +0000 UTC m=+6.062006236 container cleanup 52392403da44e54428ee449b4f9d69c2bf26bbb5c63f470720d5a8b566f1c480 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-20T08:26:50.554 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:50 smithi196 bash[131827]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7 2024-03-20T08:26:50.602 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:50.176Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:50.602 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:50.177Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:50.602 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:50 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:50.177Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:50.871 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:50 smithi196 podman[131849]: 2024-03-20 08:26:50.555639968 +0000 UTC m=+0.612021256 container remove 52392403da44e54428ee449b4f9d69c2bf26bbb5c63f470720d5a8b566f1c480 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-20T08:26:51.139 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:50 smithi196 podman[131943]: 2024-03-20 08:26:50.872946672 +0000 UTC m=+0.113879997 container create 43acb3e03e7740b0a5811eef95cf221c0db5b79548c69cdf70782f698a8ecc6d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-20T08:26:51.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:50 smithi196 podman[131943]: 2024-03-20 08:26:50.778376767 +0000 UTC m=+0.019310107 image pull 9ceba81fb2c8d8682cb159527017d591637b153b3aa4a390ffa8586db6dcda00 quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912 2024-03-20T08:26:51.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:51 smithi196 podman[131943]: 2024-03-20 08:26:51.075499331 +0000 UTC m=+0.316432662 container init 43acb3e03e7740b0a5811eef95cf221c0db5b79548c69cdf70782f698a8ecc6d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-20T08:26:51.140 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:51 smithi196 podman[131943]: 2024-03-20 08:26:51.081488304 +0000 UTC m=+0.322421632 container start 43acb3e03e7740b0a5811eef95cf221c0db5b79548c69cdf70782f698a8ecc6d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-20T08:26:51.639 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:51 smithi196 podman[131943]: 2024-03-20 08:26:51.166963951 +0000 UTC m=+0.407897279 container attach 43acb3e03e7740b0a5811eef95cf221c0db5b79548c69cdf70782f698a8ecc6d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-20T08:26:51.640 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:51 smithi196 podman[131943]: 2024-03-20 08:26:51.370684634 +0000 UTC m=+0.611617951 container died 43acb3e03e7740b0a5811eef95cf221c0db5b79548c69cdf70782f698a8ecc6d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-20T08:26:51.926 INFO:journalctl@ceph.osd.7.smithi196.stdout:Mar 20 08:26:51 smithi196 podman[131989]: 2024-03-20 08:26:51.911924102 +0000 UTC m=+0.536413987 container remove 43acb3e03e7740b0a5811eef95cf221c0db5b79548c69cdf70782f698a8ecc6d (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-20T08:26:51.931 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@osd.7.service' 2024-03-20T08:26:51.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:26:51.971 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-20T08:26:51.971 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-20T08:26:51.971 DEBUG:teuthology.orchestra.run.smithi118:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.a 2024-03-20T08:26:52.256 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:26:52 smithi118 systemd[1]: Stopping Ceph mds.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:26:52.673 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:26:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[121438]: 2024-03-20T08:26:52.256+0000 7fd66174e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:26:52.674 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:26:52 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a[121438]: 2024-03-20T08:26:52.256+0000 7fd66174e700 -1 mds.a *** got signal Terminated *** 2024-03-20T08:26:54.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:54.493Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:54.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:54.493Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:26:54.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:26:54 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:26:54.493Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:26:56.545 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:26:56 smithi118 podman[148480]: 2024-03-20 08:26:56.271949993 +0000 UTC m=+4.113395873 container died b9e9c53270c72ac57f1e92041ba3493d746b3db2af3c4940aca1423e9508908a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-20T08:26:56.923 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:26:56 smithi118 podman[148480]: 2024-03-20 08:26:56.720244995 +0000 UTC m=+4.561690873 container cleanup b9e9c53270c72ac57f1e92041ba3493d746b3db2af3c4940aca1423e9508908a (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-20T08:26:56.923 INFO:journalctl@ceph.mds.a.smithi118.stdout:Mar 20 08:26:56 smithi118 bash[148480]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-a 2024-03-20T08:26:57.190 DEBUG:teuthology.orchestra.run.smithi118:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.a.service' 2024-03-20T08:26:57.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:26:57.227 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-20T08:26:57.228 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-20T08:26:57.228 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.b 2024-03-20T08:26:57.585 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:26:57 smithi196 systemd[1]: Stopping Ceph mds.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:26:57.889 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:26:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b[112139]: 2024-03-20T08:26:57.583+0000 7f0535e86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-20T08:26:57.890 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:26:57 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b[112139]: 2024-03-20T08:26:57.583+0000 7f0535e86700 -1 mds.b *** got signal Terminated *** 2024-03-20T08:27:00.481 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:27:00 smithi196 podman[132089]: 2024-03-20 08:27:00.221138042 +0000 UTC m=+2.799289858 container died 237cfe0be47629abdd4e3b2f9e916702944b5bb23203447f2c05075543a70535 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, ceph=True) 2024-03-20T08:27:00.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:27:00.177Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:27:00.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:27:00.178Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi118.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.118:8443: connect: connection refused" 2024-03-20T08:27:00.673 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:00 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:27:00.178Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-03-20T08:27:00.864 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:27:00 smithi196 podman[132089]: 2024-03-20 08:27:00.580409008 +0000 UTC m=+3.158560820 container cleanup 237cfe0be47629abdd4e3b2f9e916702944b5bb23203447f2c05075543a70535 (image=quay.ceph.io/ceph-ci/ceph@sha256:976a8b2d06e3c6bfd12488297e8584dd704cc06c3e976c7e0f718c13e238e912, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-yuri5-testing-2024-03-18-1144-e7e80a6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-20T08:27:00.864 INFO:journalctl@ceph.mds.b.smithi196.stdout:Mar 20 08:27:00 smithi196 bash[132089]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-mds-b 2024-03-20T08:27:01.009 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@mds.b.service' 2024-03-20T08:27:01.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:27:01.046 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-20T08:27:01.047 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-03-20T08:27:01.047 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a 2024-03-20T08:27:01.389 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 systemd[1]: Stopping Ceph prometheus.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:27:01.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-20T08:27:01.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-20T08:27:01.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-20T08:27:01.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-20T08:27:01.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-20T08:27:01.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-20T08:27:01.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-20T08:27:01.390 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-20T08:27:01.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.362Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-20T08:27:01.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.365Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-20T08:27:01.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.365Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-20T08:27:01.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a[120179]: ts=2024-03-20T08:27:01.365Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-20T08:27:01.391 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 podman[132218]: 2024-03-20 08:27:01.372834309 +0000 UTC m=+0.111736787 container died bf82125604d28a2622d14af478df505067512cf315941aa4b44a7e5244e73721 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:02.090 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 podman[132218]: 2024-03-20 08:27:01.742600495 +0000 UTC m=+0.481502972 container cleanup bf82125604d28a2622d14af478df505067512cf315941aa4b44a7e5244e73721 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:02.090 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 bash[132218]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a 2024-03-20T08:27:02.090 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Mar 20 08:27:01 smithi196 podman[132238]: 2024-03-20 08:27:01.902062158 +0000 UTC m=+0.526351367 container remove bf82125604d28a2622d14af478df505067512cf315941aa4b44a7e5244e73721 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:02.091 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:01 smithi196 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a[124469]: logger=sqlstore.transactions t=2024-03-20T08:27:01.818821631Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-20T08:27:02.105 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@prometheus.a.service' 2024-03-20T08:27:02.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-20T08:27:02.141 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-03-20T08:27:02.141 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-20T08:27:02.295 INFO:teuthology.orchestra.run.smithi118.stdout:Deleting cluster with fsid: bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:27:03.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:03 smithi118 systemd[1]: Stopping Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:27:04.250 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:03 smithi118 ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a[129757]: ts=2024-03-20T08:27:03.953Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-20T08:27:04.250 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:03 smithi118 podman[148934]: 2024-03-20 08:27:03.994214443 +0000 UTC m=+0.145486468 container died 68a6e855b9aafc6c1e26ceb62524492a7f3e4be8aea4f202bbb1458d55f51f4a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:04.600 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:04 smithi118 podman[148934]: 2024-03-20 08:27:04.342026053 +0000 UTC m=+0.493298083 container cleanup 68a6e855b9aafc6c1e26ceb62524492a7f3e4be8aea4f202bbb1458d55f51f4a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:04.600 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:04 smithi118 bash[148934]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a 2024-03-20T08:27:04.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:04 smithi118 podman[148953]: 2024-03-20 08:27:04.601349647 +0000 UTC m=+0.605316848 container remove 68a6e855b9aafc6c1e26ceb62524492a7f3e4be8aea4f202bbb1458d55f51f4a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:04.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:04 smithi118 podman[148953]: 2024-03-20 08:27:04.668579969 +0000 UTC m=+0.672547177 volume remove 33f49e902662488d1a4f740318bae0621d42b6deebd0cb7728c90ba06b663908 2024-03-20T08:27:04.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:04 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-20T08:27:04.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:04 smithi118 systemd[1]: Stopped Ceph alertmanager.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:27:04.923 INFO:journalctl@ceph.alertmanager.a.smithi118.stdout:Mar 20 08:27:04 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@alertmanager.a.service: Consumed 1.510s CPU time. 2024-03-20T08:27:05.234 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:04 smithi118 systemd[1]: Stopping Ceph node-exporter.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:27:05.513 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:05 smithi118 podman[149068]: 2024-03-20 08:27:05.235082652 +0000 UTC m=+0.125320515 container died 768c22c1ab593caa72aa9c1634de305d43d2a7c25e2dbfc3e8c2f1729a763392 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:05.863 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:05 smithi118 podman[149068]: 2024-03-20 08:27:05.613173134 +0000 UTC m=+0.503410999 container cleanup 768c22c1ab593caa72aa9c1634de305d43d2a7c25e2dbfc3e8c2f1729a763392 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:05.863 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:05 smithi118 bash[149068]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a 2024-03-20T08:27:06.158 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:05 smithi118 podman[149085]: 2024-03-20 08:27:05.864298253 +0000 UTC m=+0.631434299 container remove 768c22c1ab593caa72aa9c1634de305d43d2a7c25e2dbfc3e8c2f1729a763392 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-20T08:27:06.158 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:05 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:27:06.158 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:06 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-20T08:27:06.158 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:06 smithi118 systemd[1]: Stopped Ceph node-exporter.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:27:06.159 INFO:journalctl@ceph.node-exporter.a.smithi118.stdout:Mar 20 08:27:06 smithi118 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.a.service: Consumed 2.384s CPU time. 2024-03-20T08:27:06.694 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-20T08:27:06.852 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:27:08.389 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:08 smithi196 systemd[1]: Stopping Ceph grafana.a for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:27:08.813 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:08 smithi196 podman[132678]: 2024-03-20 08:27:08.518925288 +0000 UTC m=+0.119395111 container died b2e2f67e151687ae234bdf0fe449763e56ca444d4838648e2fd74c3e8dbee07f (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, version=8.5, name=ubi8, architecture=x86_64, maintainer=Paul Cuzner , distribution-scope=public, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, io.buildah.version=1.32.0, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container) 2024-03-20T08:27:09.105 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:08 smithi196 podman[132678]: 2024-03-20 08:27:08.888543728 +0000 UTC m=+0.489013550 container cleanup b2e2f67e151687ae234bdf0fe449763e56ca444d4838648e2fd74c3e8dbee07f (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, io.openshift.expose-services=, io.openshift.tags=base rhel8, name=ubi8, architecture=x86_64, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner , description=Ceph Grafana Container, version=8.5, io.buildah.version=1.32.0) 2024-03-20T08:27:09.106 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:08 smithi196 bash[132678]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a 2024-03-20T08:27:09.384 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:09 smithi196 podman[132698]: 2024-03-20 08:27:09.106394669 +0000 UTC m=+0.589574381 container remove b2e2f67e151687ae234bdf0fe449763e56ca444d4838648e2fd74c3e8dbee07f (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-grafana-a, io.buildah.version=1.32.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., description=Ceph Grafana Container, architecture=x86_64, vcs-type=git, version=8.5, io.openshift.expose-services=, name=ubi8, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-03-20T08:27:09.385 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:09 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:27:09.385 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:09 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-03-20T08:27:09.385 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:09 smithi196 systemd[1]: Stopped Ceph grafana.a for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:27:09.385 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Mar 20 08:27:09 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@grafana.a.service: Consumed 8.405s CPU time. 2024-03-20T08:27:09.638 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:09 smithi196 systemd[1]: Stopping Ceph node-exporter.b for bc2566e8-e68e-11ee-95c9-87774f69a715... 2024-03-20T08:27:09.889 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:09 smithi196 podman[132808]: 2024-03-20 08:27:09.639381498 +0000 UTC m=+0.119008228 container died d2c82f232b48b2da2603e10bb87ec286a3be82b716620c08ea4c659de42af9b9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:27:10.167 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:09 smithi196 podman[132808]: 2024-03-20 08:27:09.975375574 +0000 UTC m=+0.455002302 container cleanup d2c82f232b48b2da2603e10bb87ec286a3be82b716620c08ea4c659de42af9b9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:27:10.167 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:09 smithi196 bash[132808]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b 2024-03-20T08:27:10.528 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:10 smithi196 podman[132828]: 2024-03-20 08:27:10.168254344 +0000 UTC m=+0.530142179 container remove d2c82f232b48b2da2603e10bb87ec286a3be82b716620c08ea4c659de42af9b9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-bc2566e8-e68e-11ee-95c9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-20T08:27:10.529 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:10 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-20T08:27:10.529 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:10 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-20T08:27:10.529 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:10 smithi196 systemd[1]: Stopped Ceph node-exporter.b for bc2566e8-e68e-11ee-95c9-87774f69a715. 2024-03-20T08:27:10.529 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Mar 20 08:27:10 smithi196 systemd[1]: ceph-bc2566e8-e68e-11ee-95c9-87774f69a715@node-exporter.b.service: Consumed 2.400s CPU time. 2024-03-20T08:27:11.038 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-20T08:27:11.073 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-20T08:27:11.102 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-20T08:27:11.104 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023/remote/smithi118/crash 2024-03-20T08:27:11.105 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/crash -- . 2024-03-20T08:27:11.153 INFO:teuthology.orchestra.run.smithi118.stderr:tar: /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-20T08:27:11.154 INFO:teuthology.orchestra.run.smithi118.stderr:tar: Error is not recoverable: exiting now 2024-03-20T08:27:11.156 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023/remote/smithi196/crash 2024-03-20T08:27:11.156 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/crash -- . 2024-03-20T08:27:11.193 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-20T08:27:11.194 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-03-20T08:27:11.195 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-20T08:27:11.195 DEBUG:teuthology.orchestra.run.smithi118:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-03-20T08:27:11.224 INFO:teuthology.orchestra.run.smithi118.stdout:2024-03-20T08:05:16.918854+0000 mon.a (mon.0) 265 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-03-20T08:27:11.226 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-20T08:27:11.226 INFO:tasks.cephadm:Compressing logs... 2024-03-20T08:27:11.226 DEBUG:teuthology.orchestra.run.smithi118:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-20T08:27:11.268 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-20T08:27:11.293 INFO:teuthology.orchestra.run.smithi118.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-20T08:27:11.293 INFO:teuthology.orchestra.run.smithi118.stderr:: No such file or directory 2024-03-20T08:27:11.293 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-20T08:27:11.296 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-20T08:27:11.296 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-20T08:27:11.296 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-20T08:27:11.296 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-20T08:27:11.296 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.1.log: /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-20T08:27:11.296 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-20T08:27:11.296 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.log 2024-03-20T08:27:11.300 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.cephadm.log: 90.6% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-20T08:27:11.301 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.2.log 92.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-20T08:27:11.301 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-03-20T08:27:11.301 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-20T08:27:11.303 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.2.log: /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.log: 94.4% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-20T08:27:11.304 INFO:teuthology.orchestra.run.smithi118.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-20T08:27:11.305 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-20T08:27:11.305 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-03-20T08:27:11.305 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-20T08:27:11.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-20T08:27:11.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-20T08:27:11.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-20T08:27:11.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-20T08:27:11.306 INFO:teuthology.orchestra.run.smithi196.stderr: 83.7% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-20T08:27:11.306 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-20T08:27:11.306 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-20T08:27:11.307 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.7.log: /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-20T08:27:11.307 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.log 2024-03-20T08:27:11.307 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-20T08:27:11.307 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-20T08:27:11.307 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.log: 90.8% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-20T08:27:11.307 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-20T08:27:11.307 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-volume.log: 83.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-20T08:27:11.316 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-20T08:27:11.325 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.b.log: 93.9% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-20T08:27:11.326 INFO:teuthology.orchestra.run.smithi196.stderr: 91.1% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-20T08:27:11.336 INFO:teuthology.orchestra.run.smithi118.stderr:/var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mds.a.log: /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.c.log: 91.7% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-20T08:27:11.337 INFO:teuthology.orchestra.run.smithi118.stderr: 91.1% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-20T08:27:11.341 INFO:teuthology.orchestra.run.smithi118.stderr: 96.2% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-20T08:27:11.346 INFO:teuthology.orchestra.run.smithi196.stderr: 89.3% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-20T08:27:11.857 INFO:teuthology.orchestra.run.smithi118.stderr: 89.2% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-20T08:27:11.879 INFO:teuthology.orchestra.run.smithi196.stderr: 91.6% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-20T08:27:12.005 INFO:teuthology.orchestra.run.smithi118.stderr: 91.2% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-20T08:27:13.645 INFO:teuthology.orchestra.run.smithi118.stderr: 90.2% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-20T08:27:14.847 INFO:teuthology.orchestra.run.smithi118.stderr: 94.1% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-20T08:27:16.302 INFO:teuthology.orchestra.run.smithi196.stderr: 94.7% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-20T08:27:25.682 INFO:teuthology.orchestra.run.smithi118.stderr: 94.7% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-20T08:27:26.553 INFO:teuthology.orchestra.run.smithi196.stderr: 95.0% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-20T08:27:26.915 INFO:teuthology.orchestra.run.smithi196.stderr: 94.7% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-20T08:27:28.239 INFO:teuthology.orchestra.run.smithi118.stderr: 95.1% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-20T08:27:31.401 INFO:teuthology.orchestra.run.smithi118.stderr: 94.1% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-20T08:27:31.403 INFO:teuthology.orchestra.run.smithi118.stderr: 2024-03-20T08:27:31.403 INFO:teuthology.orchestra.run.smithi118.stderr:real 0m20.123s 2024-03-20T08:27:31.404 INFO:teuthology.orchestra.run.smithi118.stderr:user 0m56.425s 2024-03-20T08:27:31.404 INFO:teuthology.orchestra.run.smithi118.stderr:sys 0m2.134s 2024-03-20T08:27:32.412 INFO:teuthology.orchestra.run.smithi196.stderr: 94.3% -- replaced with /var/log/ceph/bc2566e8-e68e-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-20T08:27:32.414 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-03-20T08:27:32.414 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m21.132s 2024-03-20T08:27:32.414 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m55.342s 2024-03-20T08:27:32.415 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m2.260s 2024-03-20T08:27:32.415 INFO:tasks.cephadm:Archiving logs... 2024-03-20T08:27:32.415 DEBUG:teuthology.misc:Transferring archived files from smithi118:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023/remote/smithi118/log 2024-03-20T08:27:32.417 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-20T08:27:39.667 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023/remote/smithi196/log 2024-03-20T08:27:39.668 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-20T08:27:46.815 INFO:tasks.cephadm:Removing cluster... 2024-03-20T08:27:46.815 DEBUG:teuthology.orchestra.run.smithi118:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 --force 2024-03-20T08:27:46.968 INFO:teuthology.orchestra.run.smithi118.stdout:Deleting cluster with fsid: bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:27:47.299 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bc2566e8-e68e-11ee-95c9-87774f69a715 --force 2024-03-20T08:27:47.455 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: bc2566e8-e68e-11ee-95c9-87774f69a715 2024-03-20T08:27:47.783 INFO:tasks.cephadm:Removing cephadm ... 2024-03-20T08:27:47.784 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-20T08:27:47.799 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-20T08:27:47.815 INFO:tasks.cephadm:Teardown complete 2024-03-20T08:27:47.815 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-20T08:27:47.827 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-20T08:27:47.827 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-20T08:27:47.842 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-20T08:27:47.935 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-20T08:27:47.945 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-20T08:27:47.945 DEBUG:teuthology.orchestra.run.smithi118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-20T08:27:47.948 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-20T08:27:47.959 INFO:teuthology.orchestra.run.smithi118.stderr:bash: line 1: ntpq: command not found 2024-03-20T08:27:47.962 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-03-20T08:27:47.963 INFO:teuthology.orchestra.run.smithi118.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-20T08:27:47.964 INFO:teuthology.orchestra.run.smithi118.stdout:=============================================================================== 2024-03-20T08:27:47.964 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 63 -2045us[-2043us] +/- 135ms 2024-03-20T08:27:47.964 INFO:teuthology.orchestra.run.smithi118.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 +246us[ +247us] +/- 61ms 2024-03-20T08:27:47.964 INFO:teuthology.orchestra.run.smithi118.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +730us[ +732us] +/- 74ms 2024-03-20T08:27:47.964 INFO:teuthology.orchestra.run.smithi118.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T08:27:47.966 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-20T08:27:47.967 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-03-20T08:27:47.967 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 64 -2033us[-2034us] +/- 135ms 2024-03-20T08:27:47.967 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 +255us[ +255us] +/- 61ms 2024-03-20T08:27:47.967 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 +751us[ +751us] +/- 75ms 2024-03-20T08:27:47.967 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-20T08:27:47.968 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-20T08:27:47.979 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-20T08:27:47.979 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-20T08:27:48.023 DEBUG:teuthology.orchestra.run.smithi118:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-20T08:27:48.115 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-20T08:27:48.198 DEBUG:teuthology.orchestra.run.smithi118:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-20T08:27:48.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T08:27:48.230 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-20T08:27:48.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T08:27:48.260 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-20T08:27:48.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-20T08:27:48.291 INFO:teuthology.task.internal:Duration was 2525.638697 seconds 2024-03-20T08:27:48.291 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-20T08:27:48.301 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-20T08:27:48.301 DEBUG:teuthology.orchestra.run.smithi118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-20T08:27:48.305 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-20T08:27:48.344 INFO:teuthology.orchestra.run.smithi118.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-20T08:27:48.348 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-20T08:27:48.718 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-20T08:27:48.718 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi118.front.sepia.ceph.com 2024-03-20T08:27:48.718 DEBUG:teuthology.orchestra.run.smithi118:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-20T08:27:48.778 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-03-20T08:27:48.778 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-20T08:27:48.801 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-20T08:27:48.801 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-20T08:27:48.821 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-20T08:27:48.875 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-20T08:27:48.875 DEBUG:teuthology.orchestra.run.smithi118:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-20T08:27:48.889 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-20T08:27:50.046 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-20T08:27:50.057 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-20T08:27:50.057 DEBUG:teuthology.orchestra.run.smithi118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-20T08:27:50.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-20T08:27:50.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-20T08:27:50.280 DEBUG:teuthology.orchestra.run.smithi118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-20T08:27:50.283 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-20T08:27:50.306 INFO:teuthology.orchestra.run.smithi118.stdout:kernel.core_pattern = core 2024-03-20T08:27:50.310 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-03-20T08:27:50.353 DEBUG:teuthology.orchestra.run.smithi118:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-20T08:27:50.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T08:27:50.376 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-20T08:27:50.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-20T08:27:50.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-20T08:27:50.420 INFO:teuthology.task.internal:Transferring archived files... 2024-03-20T08:27:50.421 DEBUG:teuthology.misc:Transferring archived files from smithi118:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023/remote/smithi118 2024-03-20T08:27:50.421 DEBUG:teuthology.orchestra.run.smithi118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-20T08:27:50.496 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023/remote/smithi196 2024-03-20T08:27:50.497 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-20T08:27:50.542 INFO:teuthology.task.internal:Removing archive directory... 2024-03-20T08:27:50.542 DEBUG:teuthology.orchestra.run.smithi118:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-20T08:27:50.545 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-20T08:27:50.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-20T08:27:50.604 INFO:teuthology.task.internal:Not uploading archives. 2024-03-20T08:27:50.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-20T08:27:50.643 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-20T08:27:50.643 DEBUG:teuthology.orchestra.run.smithi118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-20T08:27:50.646 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-20T08:27:50.658 INFO:teuthology.orchestra.run.smithi118.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 20 08:27 /home/ubuntu/cephtest 2024-03-20T08:27:50.660 INFO:teuthology.orchestra.run.smithi196.stdout: 266042 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 20 08:27 /home/ubuntu/cephtest 2024-03-20T08:27:50.662 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-20T08:27:50.709 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-20T08:27:50.797 INFO:teuthology.nuke:Checking targets against current locks 2024-03-20T08:27:50.824 DEBUG:teuthology.nuke:shortname: smithi118 2024-03-20T08:27:50.825 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-20T08:27:50.850 DEBUG:teuthology.nuke:shortname: smithi196 2024-03-20T08:27:50.850 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-20T08:27:50.854 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-20 07:36:40.677933', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbyBWiqm9K3vb0bEIwsorV4bqxBbWYER5MR15mpEnGkV035m+ro90i9c3nuZGwRXMZCpZJ0zrou/7PV2IohK0U='} 2024-03-20T08:27:50.867 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-19_00:09:45-rados-wip-yuri5-testing-2024-03-18-1144-distro-default-smithi/7610023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-20 07:36:40.679405', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIc2vb0XkU8YfXuf01b0sKg+BKbgcC0lr0xFiWuFUX/l/bXX7AbozlnxqWD2T1Urpsb4Jtwm3hjKnH9JrnfGt4='} 2024-03-20T08:27:50.906 INFO:teuthology.orchestra.console.smithi118:Power off 2024-03-20T08:27:50.907 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-20T08:27:50.922 INFO:teuthology.orchestra.console.smithi196:Power off 2024-03-20T08:27:50.922 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-20T08:27:50.932 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2024-03-20T08:27:50.932 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-20T08:27:50.945 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-03-20T08:27:50.946 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-20T08:27:50.956 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2024-03-20T08:27:50.972 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-20T08:27:54.957 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-20T08:27:54.973 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-20T08:27:55.070 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2024-03-20T08:27:55.083 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-20T08:27:59.074 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-20T08:27:59.084 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-20T08:27:59.185 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-03-20T08:27:59.193 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-20T08:27:59.286 INFO:teuthology.orchestra.console.smithi118:Power off completed 2024-03-20T08:28:03.197 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-20T08:28:03.308 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-03-20T08:28:03.409 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-03-20T08:28:03.513 INFO:teuthology.run:Summary data: description: rados/upgrade/parallel/{0-random-distro$/{centos_9.stream_runc} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} duration: 2525.6386971473694 failure_reason: failed to complete snap trimming before timeout flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=50ae0d27dc824d589578cc8e5ba42604 status: fail success: false 2024-03-20T08:28:03.514 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-20T08:28:03.609 INFO:teuthology.run:FAIL